Examining the Validity of Message Evaluation Measures in the Context of Youth E-Cigarette Prevention

Abstract

Despite the routine use of a variety of message evaluation (ME) measures (e.g., perceived message effectiveness) in formative research for tobacco education campaigns, studies assessing the validity of these measures for predicting actual message effectiveness (AME) (e.g., intention to use e-cigarettes) are needed. A within-subjects experiment with adolescents (N = 2,120) was conducted with youth-focused e-cigarette prevention ads to examine and compare the validity of 10 ME measures often used in message pretesting studies. Aggregate message-level ME scores were created and their relationship with outcomes commonly targeted by tobacco prevention campaigns was assessed, while controlling for individual characteristics and baseline measures of the outcomes. More ME measures predicted intention to use e-cigarettes than other e-cigarette-related outcomes. Perceived message effectiveness (PME), assessed as message perceptions and as effects perceptions, psychological reactance, and perceived argument strength were the strongest predictors of intention to use e-cigarettes. Other ME measures, such as information seeking and positive emotional reactions, predicted outcomes that PME measures did not. However, due to the use of aggregate message-level ME scores, none of the ME measures were particularly strong predictors of AME. The lack of variation in the association between ME measures and intention to use e-cigarettes by e-cigarette use status suggests these ME measures are broadly applicable. These findings support the validity of ME measures and suggest that to better anticipate potential message effects on multiple tobacco education campaign outcomes, it may be worthwhile to include other ME measures in addition to message or effects perceptions in message pretesting.