Total views : 972

A Framework for Examining the Utility of Technology-Enhanced Items

Affiliations

  • Boston College

Abstract


Interest in and use of technology-enhanced items has increased over the past decade. Given the additional time required to administer many technology-enhanced items and the increased expense required to develop them, it is important for testing programs to consider the utility of technology-enhanced items. The Technology-Enhanced Item Utility Framework presented in this paper identifies three characteristics of a technology-enhanced item that may affect its measurement utility. These include: a) the fidelity with which the response interaction space employed by a technology-enhanced item reflects the targeted construct; b) the usability of the interaction space; and c) the accessibility of the interaction space for students who are blind, have low vision, and/or have fine or gross motor skill needs. The framework was developed to assist testing programs and test developers in thinking critically about when the use of a technology-enhanced item is warranted from a measurement utility perspective.

Keywords

Accessibility, Digital Assessments, Item Development, Technology-Enhanced Items, Test Validity, Usability

Full Text:

 |  (PDF views: 380)

References


  • Davey, T. & Pitoniak, M. (2006). Designing computerized adaptive tests. In S.M. Downing & T.M. Haladyna (Eds.) Handbook of Test Development (pp. 543–574). New York, NY: Routledge.
  • Drasgow, F. & Olson-Buchanan, J.B. (1999). Innovations in Computerized Assessment. Mahwah, NJ: Lawrence Earlbaum Associates, Inc.
  • Florida Department of Educaiton (2010). Race to the Top Assessment Program Application for New Grants. Retrieved September 5, 2013 from hhttp://www2.ed.gov/ programs/racetothetop-assessment/rtta2010parcc.pdf Gove, P.B. (1986). Webster’s Third new International Dictionary of the English Language Unabridged. Springfield, MA: Merriam-Webster, Inc.
  • Haladyna, T.A. & Rodriguez, M.C. (2013). Developing and Validating Test Items. New York, NY: Routledge.
  • IMS Global Learning Consortium. (2002). IMS Question and Test Interoperability: An Overview Final Specification Version 1.2. Retrieved June 25, 2015 from http://www.imsglobal.org/question/qtiv1p2/imsqti_oviewv1p2.html.
  • IMS Global Learning Consortium. (2002). IMS Question and Test Interoperability: An Overview Version 2.1 Final. Retrieved June 25, 2015 from http://www.imsglobal.org/ question/qtiv2p1/imsqti_oviewv2p1.html.
  • Lane, S. & Stone, C.A. (2006) Performance assessment. In R.L. Brennan (Ed.) Educational Measurement (4th ed. pp. 387431). Westport, CT: American Psychological Association.
  • Marshall, A. (1920). Principles of Economics: An Introductory Volume (8th Edition). London: Mcmillan.
  • Measured Progress/ETS Collaborative. (2012). Smarter Balanced Assessment Consortium: Technology Enhanced Items. Retrieved June 23, 2015 from https://www.measuredprogress.org/wp-content/uploads/2015/08/ SBAC-Technology-Enhanced-Items-Guidelines.pdf.
  • Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement (3rd ed., pp.13-103). New York: American Council on Education/Macmillan.
  • Mislevy, R.J., Steinberg, L.S., & Almond, R.G. (2003). On the Structure of Educational Assessments, CSE Technical Report 597. Los Angles, CA: Center for the Study of Evaluation.
  • Russell, M. (2006). Technology and Assessment: The Tale of Two Perspectives. Greenwich, CT: Information Age Publishing.
  • Russell, M. (2011a). Accessible Test Design. In M. Russell & M. Kavanaugh, Assessing Students in the Margin: Challenges, Strategies, and Techniques. Charlotte, NC: Information Age Publishing.
  • Russell, M. (2011b). Digital Test Delivery: Empowering Accessible Test Design to Increase Test Validity for All Students. A Monograph Commissioned by the Arbella Advisors.
  • Scalise, K. & Gifford, B. (2006). Computer-Based Assessment in E-Learning: A Framework for Constructing “Intermediate Constraint” Questions and Tasks for Technology Platforms. Journal of Technology, Learning, and Assessment, 4(6). Retrieved June 23, 2015 from http://ejournals.bc.edu/ojs/ index.php/jtla/article/view/1653/1495.
  • Sireci, S.G. & Zenisky, A.L. (2006). Innovative item formats in computer-based testing: in pursuit of improved construct representation. In S.M. Downing & T.M. Haladyna (Eds.) Handbook of Test Development (pp. 329-348). New York, NY: Routledge.
  • Thompson, S. J., Johnstone, C. J., & Thurlow, M. L. (2002). Universal design applied to large scale assessments (Synthesis Report 44). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved June 24, 2015 from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/Synthesis44.html.
  • Washington State. (2010). Race to the Top Assessment Program Application for New Grants. Retrieved September 5, 2013 from http://www2.ed.gov/programs/racetothetop-assessment/ rtta2010smarterbalanced.pdf

Refbacks

  • There are currently no refbacks.