Volume 56, Issue 2 e2019WR026694
Research Article
Free Access

Tailoring Infographics on Water Resources Through Iterative, User-Centered Design: A Case Study in the Peruvian Andes

Sam Grainger

Corresponding Author

Sam Grainger

Department of Civil and Environmental Engineering and Grantham Institute - Climate Change and the Environment, Imperial College London, London, UK

Irish Climate Analysis and Research UnitS(ICARUS), Department of Geography, Maynooth University, Kildare, Ireland

Correspondence to: S. Grainger,

[email protected]

Search for more papers by this author
Boris F. Ochoa-Tocachi

Boris F. Ochoa-Tocachi

Department of Civil and Environmental Engineering and Grantham Institute - Climate Change and the Environment, Imperial College London, London, UK

Regional Initiative for Hydrological Monitoring of Andean Ecosystems (iMHEA), Lima, Peru

Search for more papers by this author
Javier Antiporta

Javier Antiporta

Regional Initiative for Hydrological Monitoring of Andean Ecosystems (iMHEA), Lima, Peru

Consortium for the Sustainable Development of the Andean Ecoregion (CONDESAN), Lima, Peru

Search for more papers by this author
Art Dewulf

Art Dewulf

Public Administration and Policy Group, Wageningen University and Research, Wageningen, Netherlands

Search for more papers by this author
Wouter Buytaert

Wouter Buytaert

Department of Civil and Environmental Engineering and Grantham Institute - Climate Change and the Environment, Imperial College London, London, UK

Regional Initiative for Hydrological Monitoring of Andean Ecosystems (iMHEA), Lima, Peru

Search for more papers by this author
First published: 03 February 2020
Citations: 8

Abstract

Effective communication and knowledge sharing across stakeholder groups (e.g., science, government, business, civil society, farmers, and the general public) are essential for more informed water resource management. Visualizations and graphics are powerful tools to engage diverse groups with unfamiliar information. Despite this potential, the design of visuals within applied science settings often does not involve end-user interaction or explicit consideration of their existing knowledge systems, perspective, requirements, and context of use. As a result, products are often difficult for users to understand and contextualize. While user interaction and the development of tailored visualizations is increasingly promoted as a potential remedy, limited empirical evidence exists that shows the potential impact and can guide the development of specific approaches. We piloted an iterative and user-centered design methodology toward the tailoring of infographic-style posters in the context of Peruvian water governance. To test whether tailoring demonstrably improves the perceived effectiveness of products, we designed three products that conveyed similar information but were tailored to three different audiences (an Andean agricultural, urban professional, and urban general). We then compared the tailored posters to those tailored to other audiences by means of interviews and user grading. We found that end-users perceive products that have undergone tailoring as more interesting, clearer, and more useful than products designed without explicit user consideration. Our findings indicate that identifying groups with shared characteristics and requirements is key for effective tailoring. Our research provides empirical evidence to support the incorporation of user-centered design methods in water resource management contexts.

Key Points

  • An iterative and user-centered methodology for appropriate tailoring of infographics is tested in the context of Peruvian water governance
  • Visual products designed with explicit user consideration (tailoring) are perceived as more interesting, clearer, and more useful
  • A key challenge for tailoring is identifying groups with shared characteristics and requirements

Plain Language Summary

Communication of science-based information to nonscientists is essential to inform water resource management. Visualizations and graphics are powerful tools to engage people with unfamiliar information. Considering audiences explicitly, and directly consulting with them, may improve information uptake and understanding. Therefore, the production of tailored visuals developed for (and in consultation with) specific people or groups of people (e.g., engineers and policy makers) is increasingly being promoted to enhance knowledge sharing within environmental management. However, as yet, there is little scientific evidence to support its practical implementation. To address this, we trialed an iterative and user-focused approach to tailoring with the aim to provide three different posters, communicating similar information, to three different groups of people interested in Peruvian water resources management. At the end of the design process, we asked people from each group to evaluate all three posters to see if the poster tailored to them was more effective than the other two. We find evidence that target users perceive products that have undergone tailoring as more interesting, clearer, and more useful than products designed without consideration of their representative group. Our findings indicate that tailoring can be particularly effective when people within a target group share similar characteristics and requirements.

1 Introduction

Individuals and communities are required to make increasingly complex environmental decisions under high uncertainty. Effective communication of scientific information and knowledge can clarify options, open up alternative pathways, and enable environmental decision makers to better address sustainability challenges (McNie, 2007). With this in mind, scientists are increasingly trying to communicate with decision makers and lay audiences (Bruine de Bruin & Bostrom, 2013). There has been some success in recent decades with the introduction of more boundary organizations, knowledge systems, and scientific assessments (see Cash et al., 2003; Farrell & Jäger, 2006; Guston, 2001). However, despite efforts to reform science-decision-making models, a great deal of potentially useful knowledge being provided by scientists remains underutilized within environmental decision making (Clark et al., 2016; Kirchhoff et al., 2013). An extensive body of research on the use of scientific knowledge highlights a number of interconnected factors. Lemos et al. (2012) argue that usability is largely determined by the extent to which users perceive information meets their needs, the interplay between new and existing knowledge and, more broadly, the quality of provider-user interactions. High-quality interactions increase mutual understanding of each other's expectations and perspectives and build trust between scientists and users (Flagg & Kirchhoff, 2018). When designing specific products or tools, it is paramount that information providers also understand how users are likely to respond to different modes of communication and presentation styles (Vogel et al., 2007; von Winterfeldt, 2013). This is not always the case in practice as interactions are often superficial and end products are developed without explicit consideration of user needs and fundamental information design principles (Grainger et al., 2016).

Visuals (e.g., science-based posters, policy briefs, websites, and software) play a key role within science-decision-making interactions and have the potential to help engage audiences with unfamiliar information, accelerate information flows, and empower marginalized actors (Buytaert et al., 2014; McInerny et al., 2014). Such interventions can also trigger visceral responses and heighten concern for an issue, improving the likelihood of action (Kirchhoff et al., 2013; Weber, 2006). Infographics (short for information graphics) are a specific form of visualization that combine data, graphics, illustrations, images, and text to convey a stand-alone message (Krum, 2014). While data visualizations and graphics are rarely the goal in themselves, how they are designed and integrated within information services, products, and support tools (e.g., Environmental Virtual Observatories, Climate Services) is a crucial yet often under-resourced aspect of applied science and decision support projects (Karpouzoglou et al., 2016; Zulkafli et al., 2017). Scientists involved in such processes may not have received training in visually communicating scientific information to other scientists and even less likely with regard to nonacademic audiences (McInerny et al., 2014; Rapley et al., 2014). In addition to the lack of relevant expertise within project teams, interactions are often strongly driven by the information provider and lack the necessary conviction to conduct effective user engagement (McIntosh et al., 2011; Zulkafli et al., 2017). This often leads to the development of products and tools that fail to connect with user contexts and requirements, reducing the likelihood of comprehension and sustained uptake (McIntosh et al., 2011).

User-centered design (UCD) is a framework whereby user characteristics (needs, education, abilities, goals, behaviors, and motivation) and context of use are given explicit consideration during product or tool development (Beyer & Holtzblatt, 1998; McInerny et al., 2014). Ideally, the user would be situated at the center of a design process through a range of interview and observational approaches (Christel et al., 2018, Sanders, 2002, Sedlmair et al., 2012). Environmental studies are increasingly integrating user perspectives (see Arciniegas et al., 2013; Eide & Stølen, 2012; Michener et al., 2012; Robinson et al., 2012). Others go further by suggesting that the design of effective visualizations requires a highly iterative, user-centered and collaborative approach toward the development of tailor-made products (Spiegelhalter, 2011, Grainger et al., 2016; Lorenz et al., 2015). The inclusion of appropriate tailoring as a design principle and integration of user perspectives has the potential to produce communications that not only are perceived as credible and legitimate sources of information but also convey information that aligns with user requirements and epistemologies (Medin & Bang, 2014). Nevertheless, the lack of methodological experimentation, field studies, and user-driven evaluation of visualizations has been highlighted as a barrier to more effective visual communication of scientific knowledge (Spiegelhalter, 2011, Harold et al., 2016; Lorenz et al., 2015). Therefore, ways to facilitate appropriate tailoring and integrate user perspectives need to be explored and tested in real-world decision-making contexts to help generate empirically based methodologies.

This study aims to explore the value of appropriate tailoring and user-centered visual design in a water governance context, as well as contributing to broader practical and scholarly thinking on the contextualization of science within environmental decision making. To this end, we trial an iterative and UCD methodology toward the tailoring of infographic-style posters. Through interaction with target end users, we hope to connect science-informed visuals with the perspectives and knowledge systems of a diverse range of actors interested in the management of Lima water resources. Specifically, we focus on two objectives:
  • to test the hypothesis that appropriate tailoring demonstrably improves the perceived effectiveness of visuals and
  • to assess the added value of conducting an iterative and UCD process.

We start by outlining the case study, experimental setup, and design methodology, before reporting our results from each stage of the design process. Section 4 analyzes the obtained evidence in view of our hypothesis, lessons learned, and methodological recommendations. We conclude by synthesizing our findings and offering generalizable conclusions.

2 Methods

2.1 Study Region

The city of Lima, located in the arid Pacific coast of Peru, receives less than 10 mm of rain per year (Ochoa-Tocachi et al., 2019; SENAMHI, 2009). Its water supply relies predominantly on surface water extracted from highly seasonal rivers that descend from watersheds located in the high Andes (Ochoa-Tocachi et al., 2016). Climate change, urbanization, and highland degradation in these watersheds highlight the need for more sustainable water management approaches to avoid future shortages in Lima. The use of new approaches, such as nature-based solutions, has triggered activities to generate relevant hydrological evidence, which needs to be communicated to decision makers and other stakeholders (Grainger et al., 2019; Veiga et al., 2015).

2.2 Experimental Setup

We devised an experiment to design and evaluate the perceived effectiveness of three infographic-style posters on a predetermined water resource-related topic. Each poster conveyed similar information but was tailored to specific agricultural, professional, and general audiences. We identified three distinct user groups in Peru to represent these target groups all with a potential interest in Andean water management but without any formal training in hydrological science.

The agricultural user group comprised a sample of registered livestock farmers from the Andean community of Huamantanga (total population 117 registered farmers). Huamantanga is located at an elevation of 3,400 m above sea level, approximately 120 km from Lima. Local livelihoods depend on seasonal water resources for crop production and livestock grazing (Vila Benites, 2014).

The professional user group composed of individuals based in Lima with a professional interest in nature-based solutions. This included representatives from Lima's state-owned water and sewage company (SEDAPAL), the national water regulator (SUNASS), the Ministry of Environment (MINAM), the U.S. conservation NGO the Nature Conservancy (TNC), the U.S. Agency for International Development (USAID), and the Water Fund for Lima (Aquafondo) and Agrorural (an agency within Peru's Ministry of Agriculture and Irrigation).

The user group representing the general audience (hereafter referred to as the citizen user group) composed of citizens currently living in Lima with a general interest in environmental sustainability.

The active design process took place over the course of 9 months between October 2016 and June 2017, involving multiple stays in Huamantanga and Lima. During these interactions, individuals representing three user groups helped design and evaluate the effectiveness of the final products (Figure 1). Where possible, one-on-one interactions were used to mitigate for the inherent risk of bias from group sessions (Arcia et al., 2016). The design team was made up of a designer based outside Peru, who joined after obtaining user feedback on initial visual ideas, an intermediary, and two hydrologists (B. O. T. and W. B.) who provided scientific input throughout. A social scientist (S. G.) with the requisite skills and background adopted the role of intermediary, facilitating the process and counterbalancing these two perspectives while explicitly considering end-user requirements and characteristics.

Details are in the caption following the image
A timeline of the iterative and user-centered design process. The squares indicate moments at which different members of the design team gave input, and how that led to final products. Each square represents a contribution during the design process by the intermediary (blue), end users (green), designer (orange), or scientists (purple).

For the purpose of this experiment, effectiveness is defined in terms of user perception of the product's “appeal” (used interchangeably with the term “interest”), “clarity,” and, in the case of the professional and agricultural group, “usefulness.” We do not consider “usefulness” to be an appropriate criterion for the citizen group. This is based on the assumption that general audiences are unlikely to use any new knowledge gained from the posters for practical purposes (e.g., decision making). The selection of evaluation criteria was informed by general visualization principles (see Meirelles, 2013), context of use, and previous visualization user studies that assessed the effectiveness of visualizations (Bishop et al., 2013; Levontin et al., 2017; Lorenz et al., 2015).

2.3 Preparation

Based on previous studies from the Mountain-EVO project (see Grainger et al., 2019; Zulkafli et al., 2017), we selected a pre-Inca practice of artificial infiltration (locally known as “amunas” or “mamanteo”; Ochoa-Tocachi et al., 2019) as a water resource-related topic of confirmed interest to a wide range of actors. The practice had recently been promoted in the Lima region as a “greener” and more cost-effective alternative to “gray” water supply infrastructure, at both local and regional water governance scales (Grainger et al., 2019). This practice uses a community-scale network of stone canals that divert water from small streams to hand dug ditches and permeable slopes. Water infiltrates the soil and moves much more slowly underground than over the surface, resurfacing in springs and small lagoons downslope. This water harvesting method allows farmers to divert and store wet season rains underground for use during the dry season (Ochoa-Tocachi et al., 2019). Since the information conveyed had to be similar across all three posters for experimental reasons, the selection process was driven entirely by the research team. A fixed visual medium (A1 size posters) was also selected anticipating variation in the groups access to, and experience with, computer interfaces (Zulkafli et al., 2017).

Based on these parameters and the user information elicited from the governance analysis, baseline visualization aims and criteria were formulated. For the agricultural user group, the aim was to design a poster that conveyed locally relevant information and could be displayed in the local school, as a way to facilitate debate and inform community decisions relating to amunas. For the professional user group, the aim was to design a poster that conveyed decision-relevant information around the current scientific understanding of amuna hydrology, their overall functionality, and their significance in the context of Lima's water resources. At this stage, the poster aimed at Lima citizens could not be informed by direct interaction with end users due to time and cost constraints during the interview process. Instead, baseline assumptions were developed, in collaboration with local research partners, about the type of information and graphical approach that might particularly appeal to a citizen of Lima interested in environmental sustainability.

To begin the active design process, we described an amuna system as it is hydrologically understood and drafted up initial content narratives for each of the posters. Individual personas that represented the three end-user perspectives were written (see Hanington & Martin, 2012). Based on these profiles, we considered which aspects of the system were the most relevant and potentially useful to focus on for each user group.

2.4 Iterative and User-Centered Design

This phase involved the intermediary working directly with target end users and obtaining feedback on initial visual ideas and drafts from representatives of the three user groups. Implementing a UCD approach requires skilled researchers to engage just enough with users to elicit the communication problem and user requirements but not too much as to fatigue (Sedlmair et al., 2012). In addition to user group and context-specific information, the tailoring process was also informed by principles related to functional information design and the creation of visual clarity through the use of preattentive graphic features (such as color, size, orientation, and shape) and structural devices (such as narrative, titles, typography, annotation, repetition, and grid-based layouts) (Cairo 2013; Dahlstrom et al., 2014; Meirelles, 2013).

2.4.1 Participant Selection and Recruitment

The selection of participants to represent the three user groups was based on the following eligibility criteria. Agricultural participants had to be active farmers and have a livelihood-related interest in local water supply issues. Professional participants had to have a professional interest in amuna systems and Lima's water resources. Citizen participants had to live in Lima and demonstrate a general interest in environmental sustainability.

The intermediary conducted 14 user consultations with 10 farmers (1 focus group of 8 participants and 2 one-on-one interviews), 7 professionals (all one-on-one interviews), and 10 citizens (2 focus groups of 4 participants and 2 one-on-one interviews) (10 in English and 4 in Spanish). Ages within the agricultural sample ranged from approximately 30–80 and 7 of the 10 participants were male and three were female. The age of the professional sample ranged from approximately 20–60 and five out of the seven participants were male and two were female. Ages within the citizen sample ranged from approximately 20–40, and 8 of the 10 participants were female and 2 were male.

The intermediary approached potential participants for the agricultural user group during other research activities in Huamantanga. These participants were mainly composed of a group of farmers who had been collaborating with the project during the preceding year. Professional participants were recruited from CONDESAN's existing network of contacts within governmental (i.e., SUNASS, SEDAPAL, and MINAM) and nongovernmental (i.e., TNC, USAID, and Aquafondo) organizations. Citizens were recruited through the National Agrarian University, and our own personal contacts built up during repeated research stays in Lima.

2.4.2 Sharing Initial Visual Ideas

The intermediary developed a range of narratives and “visual ideas” for each user group based on an initial understanding of their needs (see Figure 2 for examples). While remaining rough, these sketches were developed enough for the user to imagine the end product. Some ideas were directly related to amunas, while others focused on demonstrating a particular graphical approach. Three sets of ideas were then printed and shared with recruited professionals, citizens, and farmers in Peru.

Details are in the caption following the image
The evolution of the agricultural, professional, and citizen posters from initial ideas to draft product to final product (English version). See Figure S1 for larger images of the final products.
During these interactions, the intermediary spent approximately 10 min discussing the participant's background, prior knowledge, and what they would consider useful information in relation to amunas. The participant was then given approximately 10 s to look at each sketch, before being asked:
  • In terms of the overall message/content, which topic/graphical idea catches your interest the most?
  • In terms of the overall visual appeal, which topic/graphical idea catches your interest the most?

Going through the designs one by one, a series of open questions were asked to elicit more detailed feedback (e.g., What do you see?). At the end of the discussion, the two comparative questions were repeated to check whether their preference had changed (see Text S1 in the supporting information for extended question guide). Where possible, the interactions were conducted as one-on-one interviews. However, due to limited participant availability and time, three focus groups were also used to increase sample size.

The recordings of these interactions were transcribed and reviewed. Participant responses from the three user groups were collated into corresponding tables to identify common trends between participants from the same user groups. The intermediary then reviewed initial assumptions and reconsidered any that were challenged.

2.4.3 Working Toward Three-Tailored Products

Informed by target end-user interactions, detailed design briefs were drawn up by the intermediary that included user characteristics, functionality objectives, design criteria, and an outline of the content we aimed to convey for each product. Having contacted creative agency (Soapbox communications Ltd), an “ideation” session was arranged with one of their infographic designers during which potential layouts and visual directions were discussed for each of the posters. At this stage, the intermediary also shared initial visual ideas, existing graphical representations of amunas, and sketches of the different hydrological processes we aimed to convey.

The designer shared conceptual mock-ups of the three products with the intermediary (Figure 2). Referring back to the user information gathered from end-user interactions, the intermediary provided feedback on the designs, focusing on retaining a balance between including the most salient information, retaining visual clarity, and presenting a contained narrative that guided users through the poster. The designer and intermediary agreed on the overall layout of the three posters before going into the art working stage. At the same time, textual and quantitative information, photographs, and map files were scientifically validated before being integrated into the designs.

The designer initially shared first drafts of the three products with the intermediary. After concerns were raised, a new concept was devised that adhered more to the requirements generated during the end-user interaction, in particular with improvements to the layout and overall flow. This yielded significant changes that on reflection were crucial in the design process. After further iteration between intermediary and designer, draft products for the professional and citizen user groups were produced, translated into Spanish and shared by email with the corresponding participants consulted with during end-user interaction. We also attached a link to an online survey asking for feedback relating to how interesting, clear, engaging, and (for the professional user group) useful the poster was (see Text S2 for survey questions). Due to time, cost, and technological constraints we were unable to share a draft product with farmers in Huamantanga and therefore could not integrate a second round of user feedback within the design process.

The intermediary collated all feedback from respondents and identified common strengths and weaknesses of the draft designs. Possible improvements were then discussed and agreed with the designer before producing the final products.

2.5 End-User Testing

2.5.1 Participant Selection and Recruitment

The intermediary tested the final posters on a new set of individuals adhering to the same eligibility criteria as the iterative and UCD phase. Testing on individuals that had been consulted during the design phase would have introduced bias, as they could have been predisposed to favoring the poster that they were involved in designing. To account for potential familiarity bias, we selected participants that had similar (low) levels of familiarity with user testing team members and previous project activities.

The intermediary conducted 31 one-on-one end-user testing sessions, that is, 10 farmers, 10 professionals, and 11 citizens (4 in English, 27 in Spanish). Ages within the agricultural sample ranged from approximately 30–80, and 7 of the 10 participants were male and three were female. The age of the professional sample ranged from approximately 20–60, and 8 out of the 10 participants were male and 2 were female. Ages within the citizen sample ranged from approximately 20–60, and 7 of the 11 participants were female and 4 were male.

The intermediary approached farmers during a week-long visit to Huamantanga. Professionals were recruited from our existing network of contacts within governmental and nongovernmental organizations. Citizens were recruited through our indirect contacts built up during repeated research stays in Lima.

2.5.2 Poster Evaluation Procedure

Without any knowledge of which group each poster was tailored to, participants were shown the posters individually. The display order of the posters was randomized to minimize learned effect and get a cross check of differently tailored designs.

At the beginning of each end-user testing session, the intermediary introduced personnel roles, study aims, and test protocol. The participants were then asked to imagine that they had come across these posters within a real-world context of use, depending on the user group they were representing. For example, the professional end users were asked to imagine a scenario related to their occupation. They were also, at this stage, asked not to compare the posters during the test.

The individual poster tests followed an identical protocol. The poster was laid out in front of the participant for 3 min. The intermediary then asked participants several open questions (e.g., What is it about? Which aspects, if any, catch your interest? How clear is the overall message?). Participants were subsequently asked to rate the posters on a scale from zero to five in terms of the appeal (used as a proxy for interest), clarity, and, in the case of the professionals and farmers, usefulness. These scores were used to identify relative differences between the posters. The intermediary repeated this process for the remaining two posters before participants were shown all three posters simultaneously and asked to rank them according to the same three criteria as above. Text S3 details the protocol for eliciting ratings and rankings. At the end of the interview, the intermediary explained the research design in more detail and elicited general feedback from participants regarding the effectiveness, novelty, and suitability of the posters. The whole interaction took between 40 and 60 min and was audiotaped with permission. This mixed method approach, encompassing ratings, rankings, and more open questions, is recommended by Bishop et al. (2013) because it allows for more confidence in the interpretation of the results arising from the analysis.

2.5.3 Qualitative, Quantitative, and Statistical Analysis

We collated qualitative poster feedback for each user group to understand which aspects of the designs users perceived to be especially strong or weak. Based on individual quantitative feedback reported, we calculated the mean ratings and mean ranking scores in terms of interest, clarity and, where applicable, usefulness, for all three posters and user groups. The ranking score was calculated by giving first placed posters three points, second placed two points, and third placed one point. Given that we were aiming to understand how relatively appealing, clear, and useful the posters were perceived to be (as opposed to objective understanding), we reasoned that the elicitation of subjective user preferences and opinions was a suitable analytical approach to take. The data were analyzed statistically to test for a significant difference between the mean poster ratings of each user group and for each of the criterion.

The Friedman test was used as the data were nonparametric, ordinal, repeated measures and there were groups across multiple conditions. If a statistically significant relationship (p = <0.05) was found, we then used the Wilcoxon signed-rank test as a post hoc analysis to see which two poster responses, and in which criterion, were significantly different. Similar statistical approaches have been used in comparable analyses to this one (Greis et al., 2015; McMahon et al., 2016; Skau et al., 2015).

3 Results

3.1 Iterative and UCD

3.1.1 User Group Feedback

Initial visual ideas were discussed with 10 farmers, 7 professionals, and 10 citizens (see Figure 2 for examples). These interactions helped to refine our understanding of the functionality and visual style required for effective tailoring. Here we summarize key differentiations between groups, while Text S4 provides a more detailed description of user group characteristics and graphical requirements.

Feedback from these initial interactions suggested that professionals and citizens from Lima were more likely to be educated at university level and familiar with basic graphical conventions, compared to farmers from Huamantanga. All participants expressed enthusiasm for viewing the final product, but the farmers were the only participants to explicitly state that a visual aid would inform their decision making. The professionals and farmers reported very specific information needs, while citizens emphasized that the message conveyed in the poster would need to be sufficiently optimistic or noteworthy to hold their attention. In line with Zulkafli et al. (2017), most of the farmers responded positively to photographs or images of places in their community and were occasionally influenced by particular words they considered meaningful in the textual information. Conversely, professionals expressed a preference for clear, concise chunks of information rather than visually cluttered displays. It should also be noted that throughout interactions with farmers in Huamantanga, it was unclear whether a positive response to a visual idea was driven by the visual representation itself or because amunas (due to its significance to the end user) provoked a favorable response.

Draft products were shown to 5 of the 7 professionals and 7 of the 10 citizens engaged during the first interaction (see Figure 2 for images). While respondents from both user groups reported the draft to be interesting and visually attractive, other comments clearly distinguished the two groups. While the citizens were exclusively concerned with the clarity of graphical elements and message, professionals requested more detail regarding data sources, any hydrological assumptions that were made, and how the underlying data and estimated infrastructure costs were derived.

3.1.2 Evolution of the Poster Design and Content

All final products (Figure 2; see Figure S1 for larger images) were informed by our original design criteria, information design theory, designer knowledge, and user group feedback from the iterative and UCD phase. For the purpose of the experiment, the aim of the posters was to convey similar information. However, the tailoring process resulted in very different emphases and approaches.

As explained in section 2.4.4, a draft product was not shared with farmers in Huamantanga. Consequently, the user-centered rationale stemmed entirely from our preparatory governance analysis (see Grainger et al., 2019) and feedback on initial visual ideas (see section 3.1.1). Interview findings indicated that farmers would not necessarily spend time reading a lot of text and quantitative information without any immediate reward for that investment. As a result, the hydrological information conveyed in the final product is largely qualitative and visual. Informed by infographic design theory, we also used connecting elements (e.g., arrows, colors, and numbering) to create a visual path or relationship from one section of the poster to another (Segel & Heer, 2010).

The top right section situates the community of Huamantanga geographically in relation to the headwater catchments. Ordinarily, the designer (guided by theory and experience) would have used a conventional geographic map to convey geospatial information to a general audience. However, previous studies (Zulkafli et al., 2017) and feedback on initial ideas indicated that the use of images that resemble familiar features of the community, such as photographs of their restored amuna, was strongly favored over more abstract content. Therefore, in consultation with the designer, we decided to map the location of water-related features on top of a 3D aerial image of the community (Figure S2, top). This allowed farmers to make a direct connection between elements of the amuna infiltration process and the local landscape they know, such as canals and reservoirs. For similar reasons, we also decided to encode nonhydrological reference points, such as roads and the village. Given the farmers' lack of familiarity with mapping conventions, we also had to carefully consider the inclusion and design of symbols and choose colors that were highly representative.

The poster tailored to water professionals in Lima was the most technical and quantitative in nature (Figure S1). It makes the most direct link to the temporal dynamics of river flow, its consequences for downstream water availability and relationship with water demand. The poster focuses on the seasonal deficit and the potential of the amuna technique to alleviate this issue, and its financial implications. Professional user group feedback on their draft product informed UCD decisions prior to end-user testing (Figure S2, middle). Amendments included adding symbols to represent farmer water use between stages four and five so as not to give the impression that water flows directly from the amuna to Lima's reservoir. We also reconsidered how we would convey the “unrestored” and “restored” scenarios, adding labels either side of the central river graphic and above the seasonal boxes. The pipe illustration in the bottom right of the poster was revised, adding taps to the end of the pipes to make them easier to understand when water is available and when it is not. Several other minor edits to the text, headings, graphics, and data source information were also made to improve overall visual clarity.

Lastly, the poster tailored to the urban citizens displayed many of the same graphical elements as the poster tailored to the farmers, but they are not arranged to invoke a direct connection to the surrounding landscape (Figure S1). Instead, the elements are linked through a more generic landscape drawing that is not tied to a specific location but that urban citizens will recognize as a typical Andean landscape, including land use practices (e.g., livestock grazing) and environmental issues (e.g., soil degradation). The poster makes a stronger link to the ecosystem service (water supply) on which Lima citizens depend. Citizen user group feedback on their draft product informed UCD decisions prior to end-user testing (Figure S2, bottom). Revisions included reducing and rearranging the text so that there was a clearer visual structure and fixed narrative to guide the reader. We addressed some feedback relating to overall comprehension by including a definition of amunas in bold at the beginning of the text. Several other minor edits to headings and graphics were also made to improve overall visual clarity.

3.2 End-User Testing

3.2.1 Qualitative User Feedback

During the end-user testing phase, participants gave qualitative noncomparative feedback of each poster prior to rating and ranking the three products. This gave an indication of the extent to which the posters were meaningful to the end users. Participant feedback relating to the posters that were specifically tailored to each user group is summarized below, while Text S5 provides a more detailed description.

Eight out of 10 farmers reported that they were interested in the overall message of their poster, particularly the diagram outlining how the amunas function and the estimated timings given for mountain water to reach their community. In terms of clarity, 7 out of 10 participants reported that the overall message of the agricultural poster was clear, referencing the amuna process to be particularly easy to understand. In terms of the usefulness of the information conveyed by the poster, participant responses were less conclusive. While 9 out of 10 participants scored the poster highly in terms of usefulness, only 2 of those 9 could explain how such a poster might be useful for their daily activities. Comments were of a more general nature, referring to the usefulness of water or the amuna system (itself), so it is difficult to say with certainty that the information conveyed by the product would inform future activities.

Six out of the 10 professionals reported that they were interested in the overall message of their poster, particularly the cost-benefit bar chart comparing green with gray infrastructure. Several participants reported to be less interested in the scenario boxes in the bottom right of the poster. In terms of clarity, 5 of 10 participants reported that the overall message of the professional poster was clear, referencing the conceptual representation of an unrestored and restored watershed scenario. However, several participants found the long-term supply/demand time series graph confusing. In terms of the usefulness of the information conveyed by the poster, 6 out of 10 participants reported that it would be directly relevant and useful within their decision making.

Nine out of the 11 citizens reported that they were interested in the overall message of their poster, particularly the central graphic showing mountain water resources management under a restored and unrestored scenario. In terms of clarity, nine of the participants reported that the overall message of the professional poster was clear, again referencing the central graphic as the element that was the easiest to understand. While we did not test for usefulness, two of the participants did report that given recent floods in the region, it is important to raise awareness in Lima about the management of the highlands and the rediscovery of these ancestral techniques.

3.2.2 Quantitative (Observed) User Feedback

Each participant reported both noncomparative ratings and comparative rankings for the three posters in regard to interest, clarity, and (where applicable) usefulness from which mean ratings (Figure 3) and mean ranking scores (Figure 4) were calculated.

Details are in the caption following the image
Mean ratings of the three posters reported by the agricultural (n = 10), professional (n = 10), and citizen (n = 11) user groups in regard to interest, clarity, and (where applicable) usefulness. As explained in section 2.2, we do not consider “usefulness” for the citizen group. Blue indicates when the tailored poster was rated highest by the target group both compared to the other two posters and compared to the other two target groups. Yellow indicates that the tailored poster was either rated the highest by the target group compared to the other two posters or compared to the other two target groups; red indicates that the tailored poster was neither rated the highest by the target group compared to the other posters nor compared to the other two target groups.
Details are in the caption following the image
Mean ranking scores for each poster reported by the agricultural, professional, and citizen user groups in regard to effectiveness criterion (interest, clarity, and usefulness). As explained in section 2.2, we do not consider the criterion “usefulness” for the citizen group. The higher the score, the higher the rankings were within that user group. Color indicates whether or not the poster was tailored to that specific group.

3.2.3 Statistical Analysis

The statistical significance of the difference in the mean ratings for the three posters in regard to the three criteria was calculated (Figure 5). For the agricultural user group, we found a significant difference between the three posters in terms of interest, clarity, and usefulness. This was driven by farmers rating their interest in their poster significantly higher than the professional poster, the clarity of the citizen poster significantly higher than the professional poster, and the usefulness of their poster significantly higher than the citizen poster and professional poster. For the professional user group, there was no significant difference between the three posters in terms of interest, clarity, and usefulness. For the citizen user group, a statistically significant difference exists between the three posters in terms of clarity. This was driven by citizens rating the clarity of the citizen poster significantly higher than the agricultural poster. They had no significant differential reaction to the professional poster.

Details are in the caption following the image
Summary of the statistical results. Statistically significant relationships (p = <0.05) are indicated with asterisks (* = <0.05; ** = <0.01). If a statistically significant relationship was found for the Friedman test, the Wilcoxon signed rank was used as a post hoc test to analyze which two poster responses, and in which criteria, were significantly different. Significant interactions that involved the tailored poster for that specific user group are also highlighted (shaded in green).

4 Discussion

4.1 Identifying Demonstrable Improvements

The end-user testing results indicate that the iterative and user-centered approach did demonstrably improve the perceived effectiveness of infographic-style products tailored to the agricultural and, to some extent, citizen user group. Although some of the expected differences were not observed or not significant, all the significant interactions support the hypothesis (Figures 5 and 6, boxes shaded in green). In line with insights from previous user studies (Lorenz et al., 2015; Taylor et al., 2015), it is important to keep in mind that preference and perceived ease of understanding may not necessarily correspond with objective understanding.

Details are in the caption following the image
Instances where the tailored poster for that specific user group was rated significantly higher for the Wilcoxon signed rank test (p = <0.05) than posters tailored to a different user group (shaded in green).

Reflecting on the mean ratings (Figure 3), Huamantanga farmers responded most favorably to their poster in terms of interest and potential usefulness, while citizens regarded their poster to be the most interesting of the three. These results demonstrate that in both cases the products conveyed information relating to amunas that was of a highly relevant nature. The professionals responded most favorably to the professional poster in terms of usefulness demonstrating that, compared to the other two posters, the professional poster offered information that matched more directly with the professionals' decision-making processes. The professionals' less favorable response in terms of interest and clarity could be partly explained by the fact that the professional user group, although distinct from the other two groups, was internally still relatively heterogeneous in terms of background, prior knowledge, framings, and information needs. As a result, designing a product that would be of interest to everyone within this group was always going to be challenging.

It is also interesting to note that professional participants representing the national water regulator (SUNASS) and Lima water company (SEDAPAL) responded more positively to the tailored product than those from other institutions. This is not surprising given that they have shared priorities and regularly collaborate. However, this may also indicate a certain bias in the design process toward these two perspectives given that three of the seven participants that provided feedback on initial visual ideas, and two out of the five participants that provided feedback on the draft product, were from these two institutions. Clearly, this would be less of a consideration when developing a product with a specific institution or individual end user. If there were to be another round of iteration, we would have tried to improve clarity by dividing the professional user group up into separate institutions and tailoring narratives even more. This could have potentially led to collaborative development of a computer-based decision support tool that would allow users explore a broad range of information, creating their own tailored story (Lee et al., 2015).

For the citizens, tailoring led to the development of a poster that was considerably clearer than the agricultural poster (mean rating = 4.14 and 3.36, respectively; Figure 3). This result and their comments during end-user testing indicate that the specific references/terms and community-level framing of the Google map may have disoriented citizens. Although somewhat surprising, the relatively positive response of citizens to the professional poster may have been due to the participants' high level of interest in the topic (Lima's water supply) and familiarity with quantitative information due to their university education and occupation. As a result, the professionals and citizens interests may have overlapped somewhat throughout the design process. If we had shown the professional poster to Lima citizens without any scientific background or interest in environmental sustainability, they may have responded less positively. However, differences in information and real-world product needs between the user groups were sufficiently distinct to avoid compromising the validity of the study.

The consistently positive responses to the citizen poster by all user groups, with a mean rating > 4 in all three categories (Figure 3), and the highest or joint highest mean ranking score in six of the eight instances (Figure 4), could be partly due to conceptual overlap between certain categories of end users within the sample. For instance, while farmers and professionals are also citizens, farmers are not professionals and professionals are not farmers. Furthermore, the poster was aimed toward the broadest group of people, with the least amount of prior knowledge and produced by a designer that was predisposed to engaging general audiences. Consequently, while it may not have conveyed the most directly useful information for the professionals and farmers, it was often considered a close second.

4.2 Main Insights From the End-User Interactions

The Mountain-EVO project had conducted multiple field studies of the community and its relationship with governmental and international actors in Lima (Grainger et al., 2019; Zulkafli et al., 2017). Prolonged engagement with farmers in Huamantanga and a range of water-related actors in Lima helped us establish a certain amount of credibility and legitimacy during the subsequent consultations. These interactions with target end users helped to crystallize our understanding of each user group's decision-making context, graphical preferences, and information needs. However, when sharing initial visual ideas, it was unclear in some cases whether participant opinions were genuine responses to a particular design or a function of the incomplete and esthetically unpolished nature of the sketches shared.

Building on the findings of Zulkafli et al. (2017), eliciting feedback on initial visual ideas confirmed our initial hypothesis that real-world or photographic images would be a more effective visual direction than abstract visualizations for the agricultural user group. For the professional user group, we were able to confirm our initial assumption that target users would be familiar with a wide range of graphical approaches. In terms of the type of information, we could confirm a professional interest in receiving information about different policy scenarios. We also learnt that these professionals primarily want to be presented with information that is relevant to their occupation, rather than feeling like they are being persuaded or emotionally manipulated in some way. We also discovered that end users responded positively in terms of the credibility and legitimacy of the designs, not only to transparency relating to where the information came from but also to visuals that appear to be professionally designed.

For the citizen user group, interactions broadly confirmed our theoretical assumption that general audiences find visualizations with clear narratives more engaging and easier to comprehend (Dahlstrom, 2014). As expected, they did not respond well to being presented with lots of information that distracted them from the main message. End-user interaction indicated that they were keen to learn something of benefit to themselves or a group of people that they empathized with. However, for some, it was hard to see themselves as related to the story and therefore direct beneficiaries.

4.3 Main Insights From the Design Process

Enlisting a transdisciplinary design team to communicate complex scientific concepts to multiple levels of Peruvian society is challenging. There are a number of lessons learned and improvements that could be made in future studies.

Throughout the whole design process, we observed that even though each user group was distinct from each other, there still remained diverse end-user characteristics within user groups. This was particularly apparent within the professional user group due to the range of expertise, policy interests, and professional contexts being represented. While these insights are consistent with previous tailoring studies that highlight the futility of trying to design an optimal visualization when lacking within-group homogeneity (Lorenz et al., 2015), we argue that revealing perspectives visually within participatory design processes could help water management actors be more reflexive and sensitive to more marginalized interests and knowledge systems.

An inherent limitation of group consultations is the possibility of acquiescence (agreement) bias as group responses converge around the most outspoken participants (Arcia et al., 2016). In this case, the intermediary's ability (as the facilitator of these meetings) to assess responses may have been compromised when more dominant participants influenced the responses of other participants, and in some cases even prevented them from expressing their opinion of the designs. For example, due to a power imbalance within the agricultural focus group, most of the feedback came from two senior farmers with whom the research team had already established relationships.

When interacting with both Lima groups, participants would occasionally focus on how other users might respond rather than seeing themselves as end users. This could have been because they saw themselves more as consultants rather than beneficiaries of the design process. For example, during end-user testing, ministerial participants talked about using the posters as tools to convince other governmental agencies of the value of amunas, rather than something that they found informative and useful for their decision making (see literature on different kinds of knowledge use, Caplan, 2013; Daviter, 2015; Knorr, 1977). This effect is, to a large extent, unavoidable given that test participants were required (for experimental purposes) to have no prior experience with earlier designs up until that point in the process. However, if they had been consulted during the design process and had a more explicit interest in the outcome, their perception of their role may have been different.

During the end-user testing phase, some farmers seemed to use the terms interest, importance, and usefulness interchangeably. Throughout the design process, it was also difficult to disentangle whether some farmers were responding to the visual representation, the information being conveyed, the underlying topic, or a combination of the three. For example, they would often give their opinion of the usefulness of water, rather than the particular type of water-related information being conveyed. These complications could have been due to a lack of clarity in the question wording or the overall purpose of the activity. Alternatively, asking for comment on design elements, rather than the poster content or physical entity itself, was perhaps too far removed or abstract an idea for some farmers to understand immediately. We would have liked to have trialed terminology and protocols on other farmers in Huamantanga but were constrained by the limited number of suitable and available participants during our visits. Despite taking measures to prevent familiarity bias (see section 2.5.1), some professional participants were recruited through our existing network of contacts and, therefore, may have been more familiar with the Mountain-EVO project and our intentions than others. However, such variation is unlikely to have significantly impacted our study as participants did not have a personal connection with any members of the testing team.

Reflecting on the iterative and user-centered process as a whole, we learned that it is vital for the intermediary to be familiar with the scientific background and local government context while also being a skilled enough social scientist and facilitator to plan and manage group and individual interactions. They must be able to bridge both scientific and end-user perspectives, ensuring that the scientific integrity of the product is not compromised while removing their own and any other nonuser preferences from the design process.

5 Conclusions

With the aim of tailoring three infographic-style products in the context of Peruvian water governance, we piloted an iterative and user-centered approach to poster design comprising two stages of direct interaction with potential end users. We used this approach in an attempt to develop visual products appropriately tailored to three user groups within the case study. We find evidence that specific tailoring can create products that users perceive as more interesting, clearer, and more useful. This evidence is strongest in the case of the farmers in Huamantanga. As this seems the most homogeneous of the three identified target groups, this suggests that identifying groups with shared characteristics and requirements is key for effective tailoring. On the other hand, interacting with broader user groups (e.g., water-related professionals or citizens) that compose of diverse framings, epistemologies, and professional requirements may offer limited added value given the extra resources and time that would be required.

However, the current study is strongly context dependent, and the small number of samples and often small differences between the results make it difficult to draw generalized conclusions. We identify several aspects on which the methodology can be improved. First, a larger number of participants may yield more solid statistical results. To improve the robustness of results, we strongly recommend future studies minimize group consultations and trial terminology and interview protocols on external representatives of user groups. Lastly, repeating the methodology for different contexts and audiences will further refine user-centered methodologies in a water governance context.

With these caveats in mind, our work contributes to developing and reflecting on a tailoring methodology that can indeed be enlisted in other contexts. Our research provides much needed empirical evidence for greater incorporation of UCD methods within science-informed decision-making contexts to increase trust and mutual understanding between scientists and potential information users (Flagg & Kirchhoff, 2018; Lemos et al., 2012). Quantifying the value of appropriate tailoring and user-centered visual design is a first step to addressing calls for more user-centered visual design and increased empirical testing of visualizations (Grainger et al., 2016; Harold et al., 2016; Lorenz et al., 2015; Spiegelhalter, 2011). We also demonstrate the value of conducting exploratory methodological research and critically reflecting on the role of scientists as information producers and providers and how those, and the information they communicate, are perceived by end users.

Acknowledgments

The authors gratefully acknowledge the community of Huamantanga, the city of Lima, and their authorities who have provided important and constant consent and support to our fieldwork. We would also like to thank Žiga Kropivšek and colleagues at Soapbox communications Ltd for contributing their professional expertise during the poster design process. The authors also acknowledge the valuable support received from Bert De Bièvre, Katarzyna Cieslik, Katya Perez, and Luisa Sarapura Romero while conducting research activities in Peru. This study was supported by the UK Research and Innovation (UKRI) and the UK Department for International Development (grant NE/K010239/1). S. G. and B. O. T. were part of the “Science and Solutions for a Changing Planet” Doctoral Training Partnership (NERC grant NE/L002515/1). B. O. T. also acknowledges funding from an Imperial College President's PhD Scholarship.