Health Care Reform Re-formulated: Why our performance may be selling us short

The United States Citizenship test isn’t the only assessment I’ve taken recently. Yesterday, I took the Kaiser Family Foundation Health Reform Quiz:

99.6 percentile is a disappointingly poor performance.

That said, I took heart in the knowledge that the quiz is not simply a quiz: it is also an attempt to educate the public about the law’s effect. To this end, once scores are tallied, Kaiser displays your answer to each question, the correct answer, and an explanation, like so:

Overall, the news from this quiz isn’t good: After nearly four years of continuous news coverage and ongoing debate, the people who took this quiz – who one might imagine to be more interested in Health Care than the general population, and therefore more informed – correctly answered 52% of the questions presented to them.

In other words, hardly better than a coin flip. Thank you, FOX News.

But the outlook isn’t quite as bleak as these results might lead you to believe.

For one, anyone linked straight to the ‘results’ page is awarded 0 out of 10, with an indication that he or she skipped all 10 questions. I know this has happened at least once (since I did it), and each additional instance would marginally lower the nation’s collective score.

But accounting for that bias tells only part of the story. You see, the average rate of correct answers is 52%, but lumping together every question is somewhat less than informative; there is significant variability in performance based on the most elementary difference in question type.

You may have noticed above that the quiz’s questions were formulated ‘Yes/No’. In fact, the ten correct answers were evenly split between ‘Yes’ and ‘No’, 5-5. Given an overall split of 52% correct-48% incorrect, one might imagine that the rate of correct answers would be similar for both ‘Yes’ and ‘No’ questions.

You would be wrong.

The average number of correct answers to ‘Yes’ questions was 66% – hardly stellar, but a huge improvement over random chance. Meanwhile, correct answers to ‘No’ questions (in case you can’t do the math)? 38%.

Even more remarkably, not a single ‘Yes’ question scored below 62% (64, 62, 72, 67, 65), and not one ‘No’ question scored above 45% (45, 40, 25, 27, 42).

I find it difficult to imagine that this result is the outcome of ‘Yes’ questions being more difficult – rephrase any question with the exact same information and the correct answer could easily flip from a ‘Yes’ to a ‘No’.

I believe that there are two things going on here. The first – as alluded to above – is misinformation spread by FOX News, other media outlets, and certain future candidates for President of the United States. Frequent assertions that Obamacare encompasses measures it does not – like ‘death panels’ – make respondents more likely to find them familiar, and so answer ‘Yes’. This is a deliberate construct of the questions – Kaiser asked about issues around which disinformation swirls – meant to encourage failure, and eventually promote education.

Additionally, I suspect it’s worth acknowledging the effect of ‘Yes bias’, the term used to describe the propensity to answer ‘Yes or No’ questions with ‘Yes’ more often than not.

To learn more about ‘Yes bias’, I turned to Google. Here is a sample of first page of Google results for ‘Yes bias’ (results actually unrelated to ‘Yes bias’ have been omitted):

When do children exhibit a “yes” bias?

Young children’s yes bias: How does it relate to verbal ability 

The “Yes Bias” – Brain Injury Blog – Scarlett Law Group

Do bilingual children exhibit a yes bias to yes-no questions 

Access to Language and Cognitive Development – Google Books Result

The second page adds two new links:

Differences in Response Bias Among Younger and Older Preschoolers

Children in Asian cultures say yes to yes-no questions: Common 

Page 3:

Do Young Children Always Say Yes to Yes–No Questions? A 

Do young and old preschoolers exhibit response bias due to – NCBI

Young children’s yes bias: How does it relate to verbal ability 

As you may have gathered, there is a significant body of literature devoted to the study of ‘Yes bias’ in children in general, individuals who have suffered brain injuries, and Asian children.

Perhaps it’s time to start investigating its effect on full-grown adults.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s