Conclusion
Our review of the literature and our own empirical study lead us to
conclude that the DK label and labels used for the middle position of
the Likert scale are interpreted similarly; this interpretation is
affected by whether or not both the DK and mid-point anchor are used in
the rating scale and the positions of these two anchors either alone or
together.
If respondents lack knowledge or interest about attitudes assessed in a
survey or are incapable of understanding or responding to a survey item,
forcing a response increases the likelihood of random error and
consequently decreases reliability and validity of the survey results.
It may be easier for respondents to be aware of a lack of necessary
knowledge or information to respond to a factual question than to an
attitude question.26 Interpretation of a DK response
to factual questions is straightforward and providing the DK response
option to factual questions should enhance validity of the survey
results. In factual surveys, higher frequency of DK responses reflects
less familiarity with the topic area, and less ability to respond. Low
involvement in the target issues (Durand & Lambert, 1988), and less
exposure to the target issues and low degrees of topical knowledge
(Falkenberry & Mason, 1978) also affect use of DK. Consequently,
inclusion of the DK option in surveys of factual knowledge should
improve validity of responses. Our review suggests that this
recommendation is generalizable to attitude items.
Those without knowledge about an attitudinal item or issue prefer to
acknowledge their lack of evaluative response when allowed to do so.
Therefore, in order to ensure that uninformed respondents provide the
most accurate response possible, we recommend that a DK option be
provided to all Likert rating scale formats, especially if a 5-point or
any other uneven-numbered format is used. Ideally, this should be placed
at the end of the rating scale with formatting (like extra spaces
separating the substantive scale from DK) to indicate to the respondents
that the DK response option is not part of the rating scale.
We also advise that a middle option should be provided to respondents,
and it should be labeled as Neutral or neutral-sounding (such as
“Neither Satisfied nor Dissatisfied”). First, this allows for a
substantive response that can be interpreted as a middle state of
endorsement, intensity, or simply a logical middle ground between two
polar anchors (such as “Moderate” between “Liberal” and
“Conservative”). As we have noted, Likert’s intention was that this
middle point be third in a five-point scale; but can be generalized to
any odd-numbered response scale. Also, when provided with a separate DK
or other non-substantive response option (such as “Not Applicable”),
it ensures that apparently substantive responses are just that, that the
Neutral response option is available for those whose “true”
attitudinal position is a middle one, and that the middle point does not
have to serve multiple functions. This will be especially true when
scale points are clearly labeled as they are intended to be used. We
hope we have contributed to the list of guidelines for avoiding misuse
of the Likert scale. In closing, we want to suggest that other rating
scale anchors such as frequency (e.g., often, seldom) and evaluative
(e.g., poor, good) labels should be explored as they might function more
effectively than Likert scale format for certain content to elicit
reliable and valid responses.
References
Alwin, D., & Krosnick, J. (1991). The reliability of survey attitude
measurement: The influence of question and respondent attributes.Sociological Methods and Research, 20 , 139-181.
Armstrong, R. L. (1987). The mid-point on a Likert-type scale.Perceptual and Motor Skills, 64, 359-362.
Baka, A., Figgou, L., & Triga, V. (2012). “Neither agree, nor
disagree”: A critical analysis of the middle answer category in Voting
Advice Applications. International Journal of Electronic
Governance , 5 (3), 244–263.
Bauer, M. (1996). Socio-demographic correlates of DK-responses in
knowledge surveys: Self-attributed ignorance of science. Social
Science Information, 35, 39-68.
Carifio, J. & Perla, R. (2008). Resolving the 50-year debate around
using and misusing Likert scales. Medical Education, 42,1150-1152.
Carifio, J. & Perla, R. (2007). Ten common misunderstanding,
misconceptions, persist myths and urban legends about Likert scales and
Likert response formats and their antidotes. Journal of Social
Sciences, 3 (3), 106-116.
DuBois, B., & Burns, J. A. (1975). An analysis of the meaning of the
question mark response category in attitude scales. Educational
and Psychological Measurement, 35, 869-884.
Durand, R. M., & Lambert, Z. L. (1988). Don’t know responses in
surveys: Analyses and interpersonal consequences. Journal of
Business Research, 16, 169-188.
Falkenberry, G. D., & Mason, R. (1978). Characteristics of nonopinion
and no opinion response groups. Public Opinion Quarterly, 42,533-545.
Feick, L. M. (1989). Latent class analysis of survey questions that
include “don’t know” responses. Public Opinion Quarterly, 53,525-547.
Francis, J. D., & Busch, L. (1975). What we don’t know about “I don’t
knows”. Public Opinion Quarterly, 34 , 207-218.
Gilljam, M., & Granberg, D. (1993). Should we take “don’t know” for
an answer? Public Opinion Quarterly, 57, 348-357.
Jamieson, S. (2004). Likert scales: How to (ab)use them. Medical
Education, 38, 1212- 1218.
Jamieson, S. (2005). Uses and misuses of Likert scales/Author’s reply.Medical Education, 39-97.
Kalton, G., Roberts, J., & Holt, D. (1980). The effects of offering a
middle response option with opinion questions. The Statistician,
29, 11-24.
Klopfer, F. J., & Madden, T. M. (1980). The middlemost choice on
attitude items: Ambivalence, Neutrality or Uncertainty?Personality and Social Psychology Bulletin, 6, 97-101.
Krosnick, J. A. (1999). Survey Research. Annual Review of
Psychology, 50, 537-567.
Kuzon, W.M., Urbanchek, M.G., & MaCabe, S. (1996). The seven deadly
sins of statistical analysis. Annals of Plastic Surgery, 37,265-272.
Lam, T.C.M., & Klockars, A.J. (1982, Winter). Effect of choice of
anchor points on the equivalence of questionnaire items. Journal
of Educational Measurement, 19( 4), 317-322.
Lam, T.C.M., Allen, G. & Green, K.E. (2010. April). Is
“Neutral” on a Likert Scale The