Total views : 1630

Evidence-Centered Design: Recommendations for Implementation and Practice

Abstract


Evidence-centered design (ECD) is an orientation towards assessment development. It differs from conventional practice in several ways and consists of multiple activities. Each of these activities results in a set of useful documentation: domain analysis, domain modeling, construction of the assessment framework, and assessment implementation/delivery (Mislevy & Haertel, 2006). In this article we focus on the four primary challenges we have encountered in our work with ECD in the context of large-scale educational assessment. For each challenge, we identify potential mitigation strategies as well as research studies or other endeavors that we think are helpful in advancing the science of ECD. The challenges discussed are: integrating learning theory into assessment design; identifying the appropriate levels of specificity with which to document the claims and evidence; developing and evaluating task models; and strategically incorporating iteration into the design process.

Keywords

Evidence - center ed design, Large - scale assessment, Assessment design

Full Text:

 |  (PDF views: 670)

References


  • College Board. (2011). AP French Language and Culture Course and Exam Description. http://apcentral.collegeboard.com/apc/public/repository/AP_FrenchLangCED_Effective_ Fall_2011.pdf
  • College Board. (2012). AP Biology Course and Exam Description. http://apcentral.collegeboard.com/apc/public/repository/AP_BiologyCED_Effective_Fall _2012_lkd.pdf
  • Behrens, J. T., Mislevy, R. J., Bauer, M., Williamson, D. M., & Levy, R. (2004). Introduction to evidence centered design and lessons learned from its application in a global E-learning program. The International Journal of Testing, 4, 295-301.
  • Ewing, M., Packman, S., Hamen, C., & Thurber, A. (2010). Representing targets of measurement within evidence-centered design. Applied Measurement in Education, 23(4), 325-341.
  • Hendrickson, A., Huff, K., & Luecht, R. (2010). Claims, evidence, and achievement-level descriptors as a foundation for item design and test specifications. Applied Measurement in Education, 23(4), 358-377.
  • Huff, K., Alves, C., Pellegrino, J., & Kaliski P. (in press). Using evidence centered design task models in automatic item generation. In M. Gierl & T. Haladyna (Eds.), Automatic item generation. New York, NY: Informa UK Limited.
  • Huff, K., & Plake, B. S. (2010). Innovations in setting performance standards for K-12 test-based accountability. Measurement: Interdisciplinary Research & Perspective, 8(2), 130-144.
  • Huff, K., Steinberg, L., & Matts, T. (2010). The promises and challenges of implementing evidence-centered design in large-scale assessment. Applied Measurement in Education, 23(4), 310-324.
  • Kaliski, P., Huff, K., & Barry, C. (2011, April). Aligning items and achievement levels: A study comparing expert judgments. Paper presented at the meeting of the National Council on Measurement in Education, New Orleans, LA.
  • Mislevy, R. J., & Haertel, G. (2006). Implications for evidence-centered design for educational assessment. Educational Measurement: Issues and Practice, 25, 6–20.
  • Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3–67.
  • National Research Council. (2000). How people learn: Mind, brain, experience and school. Washington, DC: National Academy Press.
  • National Research Council. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.
  • The Partnership for Assessment of Readiness for College and Careers (PARCC). (2012, June). PARCC Assessments in the Making: A Principled Assessment Design Approach. 2012 National Conference on Student Assessment, Minneapolis, MN.
  • Schmeiser, C. B., & Welch, C. J. (2006). Test development. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 307–353). Washington, DC: American Council on Education.
  • Schneider, M. C., Huff, K. L., Egan, K. L, Tully, M., & Ferrara, S. (2010, May). Aligning achievement level descriptors to mapped item demands to enhance valid interpretations of scale scores and inform item development. Paper presented at the annual meeting of the American Educational Research Association, Denver, CO.

Refbacks

  • There are currently no refbacks.