Key elements in Communication Evaluation and Success Criteria
Setting up Success Criteria and subsequently evaluating your communication can help answer several questions:
Is the communication effective?
Each question can be answered in various ways, but important is that expectations as to each question is addressed before the communication starts. For each question we describe key elements and provide examples.
Is the communication feasible and usable?
When developing a communication it is important to check with your target group if the communication is feasible and usable.
Involving the target group
Irrespective whether the communication channel is digital, written, video, oral or personal it should be tested whether:
It is important that the communication will indeed used by the target group. For instance, using “thinking aloud” procedures when navigating a website or reading a brochure. The instruction is to verbalize all your thoughts while using the communication.
Another method is to interview members individually or in a group (e.g., focus groups) regarding their experiences with using the communication. One could also use eye trackers to see which part of a screen or text people focus at, with a website use the log data to examine how people navigate, or use quantitative questionnaires to assess the opinion of the target group regarding the communication. Be aware to involve members of the target group that vary on important dimensions that may affect their usability judgments, like health literacy, digital experience, age, or culture.
Look at the communication environment
For example, if you are about to embark on a campaign to increase people’s perceptions of
Online information about users and national information usage like offered from Google Insights is particularly useful source for understanding the prominence of certain news topics over time in your particular country or region.
Is the communication process successful?
A process evaluation usually focuses at two points: (a) the amount of the communication that goes to the intended target group and (b) if the communication is delivered as intended (fidelity). A process evaluation will tell you if your communication was successfully delivered. Next step is if your communication was effective (question 3). However, if a communication is not effective, it could be because it did not reach the intended target group. So if no one read your brochures on how to treat athlete’s foot, then no effect is to be expected from your communication.
Some key process evaluation components are (Linnan & Steckler, 2002):
Is the communication effective?
Your communication is effective if you reached your goals in your target group with your communication. This implies that you should be able to assess change in your target group on instruments/indicators that measure your goals.Change
One way to assess change is to have a pre-posttest design. This implies that you need to know how your target group scores on instruments/indicators before the communication started. You have to repeat this measurement at least once after a specified time slot with the same instruments/indicators. Arguments regarding the time slot are when you expect change in the instruments/indicators. For instance, changes in knowledge are usually quite fast, changes in behavior or health take more time.
Example: Online information about users and national information usage like offered from Google Insights is particularly useful source for understanding the prominence of certain news topics over time in your particular country or region.
Usually you will not be able to collect data from all the persons in your target group. Therefore, you have to determine how you can approach a representative sample of your target group. The main point is to avoid selection bias. For instance, only inviting target group members that visited the website, while also non-visiting persons were exposed to the communication.
Types of outcomes
If we can say that behavior would be more people using your online self-care guide, then discussions on social media might give you an idea about changes in attitudes, increase in time used reading / listening / viewing relevant information at your site might give you an idea about change in comprehension, and more users at the site in general might be a measurement of increased awareness.
But not all communication efforts can be measured online. Some might be measured by monitoring increase in calls to your self-care helpdesk, number of leaflets removed from stands, number of questions about specific topics at the GPs (so measure by asking at least some of these as well), others by the increase in requests / questions asked at local pharmacies or other health care professionals. Therefore a wider view as to effect and evaluation should be taken and would benefit you in your work to show the different outcomes of your efforts.
Instruments at individual level