Print Page   |   Contact Us   |   Report Abuse   |   Sign In   |   Join or Create a Guest Account
LTRC 2018: Symposia (2)
Share |

Re-conceptualizing, challenging, and expanding principles of test validation

Spiros Papageorgiou (Chair); Michael T. Kane, Richard J. Tannenbaum, Ikkyu Choi, Jonathan Schmidgall


Argument-based approaches to validation have been influential in the field of language testing and have guided the development, revision, and evaluation of language tests (Chapelle, Enright, & Jamieson, 2008) or have informed validity frameworks that have become influential in language testing (Bachman, 2005; Bachman & Palmer, 2010; Fulcher & Davidson, 2009). Such approaches to test validation draw on Toulmin’s argument structure (Toulmin, 2003), which consists of making and supporting claims based on data (i.e., information or facts on which the claim is based). The Assessment Use Argument (AUA) framework (Bachman, 2005; Bachman & Palmer, 2010) has received particular attention in the language testing field, as it expands upon the basic foundations of earlier argument-based approaches to validation, in particular Kane (1992, 2006), by including criteria for the utility of assessment scores and the intended consequences as a result of using such assessments (Tannenbaum & Cho, 2014).

The argument-based approach facilitated an expansive and coherent approach to validation in which traditional concepts (e.g., reliability, validity, fairness) are concretely elaborated while aspects of their essential interdependence (e.g., reliability is a prerequisite to validity) are specified. Although initially conceived as a framework for justifying the use of a particular assessment for a particular purpose, the comprehensiveness of argument-based frameworks, such as the AUA, may facilitate more wide-ranging discussions of the issues associated with important topics in language assessment. Such issues are the focus of this symposium, which brings together four papers that touch upon the theme of conceptualizing, challenging, and expanding principles of argument-based approaches to test validation. Each presenter brings a unique perspective to the symposium, through years of engagement with argument-based validation, either from a theoretical standpoint or a practical one. This combination of theoretical and practical perspectives is then summarized by the moderator with implications for LTRC attendees.

More specifically, the first paper, given by a leading figure in test validation theory, presents an innovative way of thinking about test validation through the metaphorical lens of distinct properties of language. The second presenter focuses on what one could argue is the most important aspect of a language assessment for test users, that is, score reporting and in particular how reports can convey useful information about test performance and support intended interpretations and uses. The third presenter deals with the topic of evaluating validity arguments using statistical models and challenges current practices that distort such an evaluation, through illustrative examples. The fourth paper centers around the challenges for developing an AUA for language assessments in workplace contexts, and highlights the unique issues for construct definition for English in the workplace. Finally, the symposium moderator will summarize the main points raised by all papers and open the floor for discussion regarding the most important takeaways from the symposium. Taken together, these presentations provide suggestions for conceptualizing and implementing validity arguments based on the opportunities and challenges inherent in defining the interpretation and use of the scores and in developing the relevant evidence.

Association Management Software Powered by YourMembership  ::  Legal