Many surveys use “thermometer scores” to measure feelings of warmness/coldness (i.e. positive/negative) toward political candidates and other public figures. There is an often-overlooked problem, however: not everyone understands temperatures in the same way. From the aptly titled “Some Like It Hot” (Wilcox, Sigelman, and Cook, 1989):
“Respondents in the 1972 ANES used thermometer scales to indicate how the felt about themselves, 14 potential presidential candidates, and 29 groups in American society….
“The grand mean of the thermometer scores was 58, somehwat toward the warm end of the scale….
“We reasoned that some respondents might be affected by the nature of the thermometer metaphor: those respondents in chilly climates might think of 60 as relatively warm, while those in some balmy climes might view that temperature as fairly chilly.” (p. 247 – 8)
I would be interested to see (read: “am thinking about doing”) an update on this project. One possible confounder is that Southern states happen to be both warmer and more conservative. The more general point is that comparing survey results–even from a question that is nominally the same–across time and space requires an important assumption that the question is being understood in the same way by all respondents (or at least that the distribution of the misunderstandings behaves in such a way that average responses can be compared). Of course, like everything else in empirical political science, Gary King has already thought about this problem and has one possible solution on his website.