Publication Date

5-2021

Date of Final Oral Examination (Defense)

1-21-2021

Type of Culminating Activity

Thesis

Degree Title

Master of Science in Organizational Performance and Workplace Learning

Department

Organizational Performance and Workplace Learning

Major Advisor

Seung Youn (Yonnie) Chyung, Ed.D.

Advisor

Soo Jeoung “Crystal” Han, Ph.D.

Advisor

In Gu Kang, Ph.D.

Abstract

Performance improvement practitioners value evidence-based practices, which include data-driven decisions. Data can be obtained through survey questionnaires designed with closed-ended questions and response scales. The Likert scale (Strongly Disagree, Disagree, Neutral, Agree, and Strongly Agree) is one of the most commonly used response scales. Whether the 5-point Likert scale, as a verbal descriptor scale, should be treated as an ordinal or interval scale is an on-going debate, as different types of statistical analyses are applied to ordinal and interval data. I conducted this study to examine if survey participants would perceive a 5-point Likert scale close to an interval level measurement when an adverb such as Moderately, Somewhat, or Slightly is added in front of Agree and Disagree. This information could be used by researchers who wish to construct an interval type Likert scale.

I conducted this study using a convenient sample of performance improvement practitioners, including master’s degree and graduate certificate seeking students, recent alumni, and faculty in the Organizational Performance and Workplace Learning department at Boise State University. For this study, I developed a web-based survey instrument using a horizontal slider format. The first screen of the survey instrument contained eight partially-labeled Likert scale sliders, each of which presented three anchors in ascending order (Strongly Disagree on the far-left side, Neutral in the middle, and Strongly Agree on the far-right side) along with their numerical values (-2, 0, and +2, respectively). The slider bar was initially placed on Neutral (0). Participants were instructed to move the slider bar to locate each of the following eight anchors on the Likert scale slider; Disagree, Moderately Disagree, Somewhat Disagree, Slightly Disagree, Agree, Moderately Agree, Somewhat Agree, and Slightly Agree. To test the response order effect, the second screen of the instrument asked the participants to repeat the above procedure using another set of eight Likert scale sliders presented in descending order. The third screen of the instrument asked for participants’ gender, age group, and native English speaker status.

The data was collected in October of 2020. The web-based survey system (Qualtrics) recorded data rounding to two decimal points and provided summary report data including mean, standard deviation, variance, and minimums and maximum response scores for each item. A survey invitation was sent to 327 practitioners, and a total of 109 of them submitted the survey. However, the initial data screening detected 37 datasets with responses where any responses were incomplete or used the incorrect side of the slider continuum, which were excluded. Two additional responses from non-native English speakers were also excluded due to the linguistic aspect of the study. This left 70 responses available for analysis (51 females, 18 males, 1 “do not want to report”).

The anchor being tested would be considered useful for constructing an interval measurement if its corresponding confidence interval included the value -1 or +1. To test this, 95% confidence intervals were constructed for each of the 16 items. Response order effects were investigated by performing paired sample t-tests comparing the average scores of the 8 response options when presented in ascending versus descending order.

The results showed that, Moderately Disagree and Moderately Agree were closely aligned with -1 or +1 on the continuum, respectively, regardless of the response orders used. Agree was aligned with +1 only when presented in ascending order, but not when presented in descending order. Adding other adverbs Somewhat and Slightly to Agree and Disagree made the 5-point Likert scales to be clearly ordinal scales in both response orders used. Therefore, the study concluded that when one needs to collect interval data from a 5-point Likert scale, Moderately Agree and Moderately Disagree can be used in either ascending or descending order of the scale.

Although Somewhat would not be a good adverb to be added to Disagree and Agree when the 5-point Likert scale is expected to generate interval data, an unexpected interesting finding was that Strongly Agree, Somewhat Agree, Somewhat Disagree, and Strongly Disagree in descending order can be used as an interval-level 4-point Likert scale.

This study was conducted with several limitations including the use of a convenience sample, and the generalization of the findings may be limited.

DOI

https://doi.org/10.18122/td.1778.boisestate

Download

Share

COinS