Infographics are one of the growing trends in presenting data and information. The idea is to take data and information and make it bright, colorful and pictorial. Often it is trying to bridge the fields of data analysts, marketing and visual graphics. By bridging these fields, they try to produce products that catch people’s attention, make them want to engage with the information and often help them make decisions or come to conclusions. Overall, they want to make the data approachable and usable.

In many respects, the goals of these infographics are the same as what we try to do as evaluators. We too are trying to produce data products that our funders, clients and stakeholders can use. We want our evaluation findings to catch their attention. We want them to engage with the evaluation results. We want them make decisions and changes based on the information that come from our work. Overall, we too want to make the evaluation data approachable and usable.

I think that in many respects the use of infographics, if done well, can further extend the reach and influence of our evaluations. For instance, I have been in charge of doing a large scale student survey in Wyoming for several years now. In 2008, I created for each county and school district a two-page, graphical fact sheet along with a larger and more comprehensive survey report. After releasing these products, I had the opportunity to interact with community members, prevention specialist and stakeholders about the survey results. Almost invariably when I did so, the data product that the community members pulled out were the factsheets and not the report. They liked the factsheets because they were short, and simple. They also had graphs that were relatively easy to understand. I cannot say that they were as fancy and graphical as most infographics, but it did show the power of accessible summaries.

The problem comes in the fact that infographics often want to emphasize the attention getting features of their presentation over and above what is considered good practices of displaying data. For instance, someone obviously spent a large amount of time creating this infographic:

The silhouettes of the different monuments and the amount of data packed into this chart are impressive. I know for myself it draws me in and makes me want to learn more.

However, as soon as I start looking at the graph I get lost. The graph presents the percentage rise and percentage fall in prices. It also displays the average hotel room cost in each city. When I try to track a given “ray” back to its “axis,” I invariably get lost and cannot tell what value the ray actually represents. It also raises questions like, how do I interpret the fact that the width of each ray gets bigger the farther it is away from the center point? Is there any interpretation of the height the different monuments? How did they choose the values to emphasize at the bottom of the graphic?

It quickly occurred to me that splitting this information into two bar graphs, one for the percentage change and one for the prices would vastly simplify and clarify the message. It would also help, in an electronic version to link to a table containing the values of this graph so that user can find the exact values. If it was a print version, the table could be printed on the back of the paper.

In the preparation of this article, I looked at many different infographics. I found that there some infographics that are better than others. However, what I also noticed is that it often took me much longer to understand many of the infographics than it would if the data were presented in a simple graph or table. A particularly notable trend is the use of circles to portray the data. This is despite the fact that Pie Charts and Ray Charts are so often misunderstood and misinterpreted. Almost invariably, if something could be portrayed as a bar chart, line chart or rectangular area chart, they would change it to something that is curved or even triangular shaped.

There may be many other concerns about infographics. For instance, many infographics are biased and are simply being used for marketing purposes. They also often do not identify their sources and it is next to impossible to make a judgment about how reliable and valid are the data.

Setting those legitimate concerns behind, I must admit that I also have my concerns about how usable they may appear at first glance against how usable they actually are. At first they seem to be a major step forward in making our evaluation results accessible and usable. However in practice, I am afraid that the most current practices obscure and distort a lot more than they illuminate and explain.

Let me know what you think of infographics? Are there any techniques, and presentations that your find particularly effective and straightforward when trying to create infographics or other similar summary materials? How do you balance the need to grab people’s attention with the need to present the data in a clear and unambiguous form?

Our post-conference poll was responded to by 75% of attendees, with 100% reporting as “satisfied” or “very satisfied” overall as well as with the panel session.  The breakout session yielded some dissatisfaction, and comments suggested it may have been improved with a better plan for input as well as a more tangible product or outcome to share with larger group – possibly via website.High satisfaction with the facility (University of Nebraska Omaha’s Thompson Alumni Center) and the conference location (Omaha) stand out.  However, comments suggest EN4MRB find ways to collaborate with the Research and Evaluation Exchange launched in Kansas City this past year.

100% of attendees were very likely or likely to attend again, and 25 out of 27 were just as likely to recommend membership to a colleague.  It should be noted that postconference, 41% were only somewhat likely to participate on the website.  Networking with people and learning of practice in other locations and disciplines were cited as “take aways” from the conference.  Finally, a number of comments noted the challenges to sustain personal interaction beyond the conference and to maintain an “intimate nature” as size of conference grows.  Good luck to new Program Committee.  I hope to see many of you at EN4MRB 2012!

As an evaluation specialist for CFCA, an international non-profit, Skype and webinar platforms have become my new way of interacting between periodic travels to project sites. This summer I launched two separate 4-session webinar series, Program Evaluation 101 for development practitioners. The content echoed what I have covered in two-day, on-site workshops, but I learned a lot about how my delivery strategies must always adapt. Planning an engaging webinar is not just a way for me to solicit good discussion that eases my insecurities about the message coming across, it is necessary to truly help participants build the confidence they need to take their learning into their work.

Here are just a few of my lessons learned:

• Keep training cohorts small. 6 – 12 participants is a good window.
• Require homework… and be sure to offer feedback. Homework submitted through email is one of the best ways to assess whether participants are grasping the materials and the corresponding feedback is an important tool for participants to get confirmation that they are on the right track.
• Utilize the ‘Change Role’ feature of webinars. Ask participants to present examples from homework or applicable personal examples in short segments.
• Make it practical. Produce usable outputs for future work through groupwork or homework, i.e. put together an evaluation plan, create a program theory of change with stakeholders, or build draft data collection tools.
• Get creative about face time. Although video conference might not be a sustainable option for more than a few people, or for slow internet connections, consider video introductions or ask participants to submit a photo and short bio to be shared with the group.
• Follow-up with individual participants. After a significant learning experience online, schedule an individual Skype call with each participant where you can discuss pending questions, get feedback on the experience from his or her perspective, and identify next steps for applying new knowledge in local contexts.
• And if possible… skip Skype and do the follow-up in person. Nothing can replace the opportunity to chat in person about the ideas discussed during the webinars. I recently met with one cohort of 10 participants from the Philippines after a webinar series and the conversation was more productive than I could have imagined. I had planned to address in more depth a few topics and to facilitate an analysis exercise. We did that, but more importantly, the session dramatically changed the demeanor and the confidence of the participants. They concluded the session sharing that they had felt a little scared to apply their knowledge in their communities and unsure if they were getting the materials right because they couldn’t see my face when they made a comment in a webinar. After the in-person session, they were confident in their new skills (relieved that this evaluation thing was not as complicated as they once thought) and genuinely excited to begin implementing their evaluation plans!

Online learning environments are a growing and wonderful opportunity to train and build the capacity of partners and stakeholders, but the use of these environments needs continued exploration. What tips do you have for making the best of the web world for evaluation capacity building?

Often when we measure things on surveys and other forms we don’t think how the order of the questions might affect the answers we receive. However, a recent article in the Washington Post demonstrates that the order of the questions can matter a lot.

After the 2008 financial collapse, Gallup integrated some questions about politics into a continuing survey, right before some questions asking about overall life satisfaction. After this change, there was a remarkable decrease in people’s satisfaction ratings. At the time, this effect on life satisfaction was attributed to the financial crisis. However, later Gallup removed the politics questions and the satisfaction ratings shot back up to pre-financial crisis levels. They also tried asking a transition question between the political questions and the satisfaction questions. This also eliminated the downward affect on satisfaction.

This is just one more reason we need to be thoughtful as we put together evaluation instruments. We never know how one question might affect the answer to other questions. This just came up on a survey instrument that I was working on. My team wanted to ask a few new questions about mental distress, suicidal ideation and suicide attempts. We decided to split up these sets of questions, kind of like Gallup did with the transition questions so that acknowledging suicidal ideation or suicide attempts did not inadvertently increase the ratings of mental distress, or vice versa, the mental distress ratings did not affect the suicide questions. Whether we are successful or not in avoiding this problem, we will probably never know, but at least we thought about it before we implemented it.

The EN₄MRB Conference is only a week away and we want to remind all those that have registered and are planning on coming to the conference to bring a abstract or project description with you to the meeting. Both of the breakout sessions will be chances to network and discuss your evaluation experiences with the other evaluators at the conference. The project descriptions will help as we converse with each other to help us network for innovative solutions to our challenges. The overarching goal of this conference is to establish a vibrant and productive working group of evaluators that can help each other improve and expand evaluation in our region. Your participation is the only way to make that happen.

Click to view full size.

We’re excited about the agenda that’s been crafted for this year’s annual conference. The conference will open with a panel of experienced regional evaluators, including: Neal Grandgenett of the University of Nebraska, Jerry Schultz of the University of Kansas, and Beverly Triana-Tremain of Public Health Consulting, LLC in Missouri.  Breakout sessions will focus on Challenges in Evaluation Sectors and Challenges in Evaluation Practice, where conference participants will have an opportunity to share in smaller groups of their chosen affinity.

Click the image of the agenda table to see what we have planned.  Please note that the link in this post has been updated 10/1/11 to yield the .pdf agenda that includes food items being offered at the October 7 conference.

Best regards, Bob Pawloski

Consider joining us for a very interactive format, where we will ask you to share your ideas as well as listen to and network with experienced evaluators from within our states.  Help us refine our vision and build an agenda to make this network valuable towards building evaluation capacity for this region.

Even if you can’t join us in person in Omaha in October, please consider becoming a member.  Membership will gain you exclusive access to interactive features that we will be considering building into our website, such as an option to share information about your work within the network. Our new website is built on a framework of blogging (WordPress) and social networking (BuddyPress), enabling our network to flourish online – whether or not members are able to attend the annual conference.