Conclusion
Our review of the literature and our own empirical study lead us to conclude that the DK label and labels used for the middle position of the Likert scale are interpreted similarly; this interpretation is affected by whether or not both the DK and mid-point anchor are used in the rating scale and the positions of these two anchors either alone or together.
If respondents lack knowledge or interest about attitudes assessed in a survey or are incapable of understanding or responding to a survey item, forcing a response increases the likelihood of random error and consequently decreases reliability and validity of the survey results. It may be easier for respondents to be aware of a lack of necessary knowledge or information to respond to a factual question than to an attitude question.26 Interpretation of a DK response to factual questions is straightforward and providing the DK response option to factual questions should enhance validity of the survey results. In factual surveys, higher frequency of DK responses reflects less familiarity with the topic area, and less ability to respond. Low involvement in the target issues (Durand & Lambert, 1988), and less exposure to the target issues and low degrees of topical knowledge (Falkenberry & Mason, 1978) also affect use of DK. Consequently, inclusion of the DK option in surveys of factual knowledge should improve validity of responses. Our review suggests that this recommendation is generalizable to attitude items.
Those without knowledge about an attitudinal item or issue prefer to acknowledge their lack of evaluative response when allowed to do so. Therefore, in order to ensure that uninformed respondents provide the most accurate response possible, we recommend that a DK option be provided to all Likert rating scale formats, especially if a 5-point or any other uneven-numbered format is used. Ideally, this should be placed at the end of the rating scale with formatting (like extra spaces separating the substantive scale from DK) to indicate to the respondents that the DK response option is not part of the rating scale.
We also advise that a middle option should be provided to respondents, and it should be labeled as Neutral or neutral-sounding (such as “Neither Satisfied nor Dissatisfied”). First, this allows for a substantive response that can be interpreted as a middle state of endorsement, intensity, or simply a logical middle ground between two polar anchors (such as “Moderate” between “Liberal” and “Conservative”). As we have noted, Likert’s intention was that this middle point be third in a five-point scale; but can be generalized to any odd-numbered response scale. Also, when provided with a separate DK or other non-substantive response option (such as “Not Applicable”), it ensures that apparently substantive responses are just that, that the Neutral response option is available for those whose “true” attitudinal position is a middle one, and that the middle point does not have to serve multiple functions. This will be especially true when scale points are clearly labeled as they are intended to be used. We hope we have contributed to the list of guidelines for avoiding misuse of the Likert scale. In closing, we want to suggest that other rating scale anchors such as frequency (e.g., often, seldom) and evaluative (e.g., poor, good) labels should be explored as they might function more effectively than Likert scale format for certain content to elicit reliable and valid responses.
References
Alwin, D., & Krosnick, J. (1991). The reliability of survey attitude measurement: The influence of question and respondent attributes.Sociological Methods and Research, 20 , 139-181.
Armstrong, R. L. (1987). The mid-point on a Likert-type scale.Perceptual and Motor Skills, 64, 359-362.
Baka, A., Figgou, L., & Triga, V. (2012). “Neither agree, nor disagree”: A critical analysis of the middle answer category in Voting Advice Applications. International Journal of Electronic Governance , 5 (3), 244–263.
Bauer, M. (1996). Socio-demographic correlates of DK-responses in knowledge surveys: Self-attributed ignorance of science. Social Science Information, 35, 39-68.
Carifio, J. & Perla, R. (2008). Resolving the 50-year debate around using and misusing Likert scales. Medical Education, 42,1150-1152.
Carifio, J. & Perla, R. (2007). Ten common misunderstanding, misconceptions, persist myths and urban legends about Likert scales and Likert response formats and their antidotes. Journal of Social Sciences, 3 (3), 106-116.
DuBois, B., & Burns, J. A. (1975). An analysis of the meaning of the question mark response category in attitude scales. Educational and Psychological Measurement, 35, 869-884.
Durand, R. M., & Lambert, Z. L. (1988). Don’t know responses in surveys: Analyses and interpersonal consequences. Journal of Business Research, 16, 169-188.
Falkenberry, G. D., & Mason, R. (1978). Characteristics of nonopinion and no opinion response groups. Public Opinion Quarterly, 42,533-545.
Feick, L. M. (1989). Latent class analysis of survey questions that include “don’t know” responses. Public Opinion Quarterly, 53,525-547.
Francis, J. D., & Busch, L. (1975). What we don’t know about “I don’t knows”. Public Opinion Quarterly, 34 , 207-218.
Gilljam, M., & Granberg, D. (1993). Should we take “don’t know” for an answer? Public Opinion Quarterly, 57, 348-357.
Jamieson, S. (2004). Likert scales: How to (ab)use them. Medical Education, 38, 1212- 1218.
Jamieson, S. (2005). Uses and misuses of Likert scales/Author’s reply.Medical Education, 39-97.
Kalton, G., Roberts, J., & Holt, D. (1980). The effects of offering a middle response option with opinion questions. The Statistician, 29, 11-24.
Klopfer, F. J., & Madden, T. M. (1980). The middlemost choice on attitude items: Ambivalence, Neutrality or Uncertainty?Personality and Social Psychology Bulletin, 6, 97-101.
Krosnick, J. A. (1999). Survey Research. Annual Review of Psychology, 50, 537-567.
Kuzon, W.M., Urbanchek, M.G., & MaCabe, S. (1996). The seven deadly sins of statistical analysis. Annals of Plastic Surgery, 37,265-272.
Lam, T.C.M., & Klockars, A.J. (1982, Winter). Effect of choice of anchor points on the equivalence of questionnaire items. Journal of Educational Measurement, 19( 4), 317-322.
Lam, T.C.M., Allen, G. & Green, K.E. (2010. April). Is “Neutral” on a Likert Scale The