Print Page   |   Contact Us   |   Report Abuse   |   Sign In   |   Join or Create a Guest Account
LTRC2020 Keynote Speakers
 Messick Lecture      |      Davies Lecture      |     DAA

Samuel J. Messick Memorial Lecture
Sponsored by Educational Testing Service

Validity worlds and values


Glenn Fulcher

Professor, University of Leicester, UK

The validity consensus created by the work of Messick has fragmented. Some have argued that the Messick model is simply impractical; others that it abandons any claim to truth; or does not account adequately for score use as a tool in power politics.  A multiplicity of largely incommensurable models now compete for primacy. Following Messick’s recommendation for the use of a dialectical approach to evaluate rival theories, I compare four popular models by their overt distinctive epistemology and the concealed value system that each represents. I do the latter with reference to the little discussed “value implications” in the Messick progressive matrix, which I believe has been misunderstood and disregarded in subsequent literature. I conclude with an articulation of “effect-driven testing” with values at the core, and make the claim that recently proposed validity models are little more than a footnote to Messick.


Glenn Fulcher is Professor of Applied Linguistics and Language Assessment at the University of Leicester. He studied Applied Linguistics and Language Testing (Birmingham and Lancaster) after an early career in philosophy, theology and the manuscript history of the Greek New Testament (King’s College London), followed by Education (Cambridge). He has served as President of the International Language Testing Association, and edited the journal Language Testing for 9 years. His books include Language Testing Revisited: A Philosophical and Social Inquiry, Practical Language Testing, and Language Testing and Assessment. He is currently working on a second edition of The Routledge Handbook of Language Testing. He has been a recipient of the ILTA best article award (2019), and the SAGE/ILTA best book award (2016). In 2014 he was awarded a Higher Education Academy National Teaching Fellowship for work in assessment literacy, including the design of innovative language testing paths in the Leicester MA, and creating electronic learning resources such as those available at his website: http://languagetesting.info.



Alan Davies Lecture
Sponsored by British Council

Describing the language construct (How are we doing?)


Dr. Fred Davidson

Professor Emeritus,
University of Illinois at Urbana-Champaign, USA

In 1994, Herrnstein and Murray published The Bell Curve, a book that reaffirmed principles of large-scale normative testing. I reviewed it for Language Testing in 1996; a copy of that review is included here. I saw a flaw in the book: the authors did not do a good job at describing their target of measurement. I concluded that language testers were doing better at explaining our construct than they did of theirs. Normative versus criterion-referenced assessment featured heavily in my reaction to their book. It had been a central contrast in my own PhD training, and as I wrote that review in late 1995, I felt that the norm-criterion tension was very much alive. Operationalization of language for testing happens on either side of that putative divide, in any case, and so I suppose that Herrnstein and Murray defined their construct as well as they needed to do, given their purpose and world view.

All these years later, how are we doing at operationalizing language across the complex and often pluricentric mandates of our tests? While I am certainly fascinated by the details of descriptions of language as a human ability, I remain more intrigued by how we craft such descriptions. During my talk, we will all have an opportunity to exercise our language analysis skills as we describe a sample language task, with this caveat: it is not the task itself that is of interest, but rather our actions – our ‘doing’ – as we refine it.


Fred Davidson was born in Chicago, Illinois, USA.  He still considers Chicago to hold his roots.  He attended Bryn Mawr Elementary School (now Bouchet Math and Science Academy), which is the same school where Michelle Obama attended some years later.  It was a strong-willed early educational experience, full of challenge, energy and “penner”, a schoolyard sport unique to the idiolect of that school’s playground.  (Ask him, later.)  He comes from a family of musicians.  Later in life, he joined the U.S. Peace Corps and served as a junior high teacher of English and science in Grand Gedeh County in Liberia.   Various different things happened, such as an MA in TESL, time as an intensive ESL teacher in the stunningly beautiful Athens Ohio area, and eventually reaching the PhD program at UCLA.  By that point, language testing had firmly settled as a permanent intellectual condition, for good or for ill.  His dissertation was at the height of the unitary/divisible trait wars.  After UCLA, he had a productive year at what was then called the University of Cambridge Local Examinations Syndicate (now Cambridge Assessment) and an equally fascinating year with the State of Illinois as a researcher on bilingual and ESL programs.  In 1990, he began as a faculty member at the University of Illinois at Urbana-Champaign, where he retired in 2014.  Now, he and his wife live gently in Lompoc, California.  His interests include bicycling and bicycle restoration, stereo gear and high fidelity music, coin collecting, and doing whatever he can to maintain a curious attitude toward the world.



Cambridge/ILTA Distinguished Achievement Award
Sponsored by Cambridge Assessment English/ILTA

Accommodating diversity in language assessment: Swings and roundabouts

Dr. Catherine Elder
University of Melbourne, Australia

The best part of my career in language testing has been spent at the Language Testing Research Centre in Melbourne which is celebrating its 30th anniversary this year. In this talk I will characterize some of the Centre’s work, focussing on the theme of diversity, and the tensions encountered in our efforts to accommodate a diversity of languages, of language learning backgrounds and of language and test use contexts on the one hand and, on the other, the constraints of uniformity, dictated by requirements for common standards or a level playing field.  Making reference to a number of projects which I have been directly or indirectly involved in, I will show how the emphasis of our work in language assessment has shifted over time, reflecting different understandings of the test construct and broader trends in the policy arena. In so doing I will offer a personal take on the development of language assessment literacy and the demands of professionalism in our field.

Cathie Elder completed her PhD in Applied Linguistics at the University of Melbourne in 1997.  In addition to serving as Director of the Language Testing Research Centre at the University of Melbourne she is also a Principal Fellow in the School of Languages and Linguistics at Melbourne and has worked at a number of other universities including Monash University and the University of Auckland.

 

Cathie has authored or co-authored three books, nine edited collections, 28 book chapters, 72 peer-reviewed articles, 45 reports and several book and test reviews. She has also received numerous awards for her work, including winning the ILTA Award for the best published paper on language testing three times (1996, 2002, and 2005), the Robert Lado Award for Best Graduate Student Presentation at the 1997 Language Testing Research Colloquium, the ALTAANZ best paper award in 2016, and she was invited to give the Davies lecture at the LTRC conference in 2019.  Her work in the area of the assessment of language for specific purposes has demonstrated great depth and breadth. She has published highly influential papers on the assessment of language teachers, health professionals and pilots and air traffic controllers (among other professions).  She has also published prolifically and made significant contributions to the assessment of languages other than English, particularly in the school system. Another area of language testing which she pioneered is the area of post-entry language assessment (PELA), where she was one of the early contributors to this area when she started the Diagnostic English Language Needs Assessment (DELNA) at the University of Auckland. She has also published on the assessment as English as a lingua franca.  Cathie’s work has been supported through many international and national competitive grants and contracts totaling more than AUD $4,000,000 (approximately USD $3,000,000 at current rates).

 

Cathie has provided outstanding leadership to the language testing community internationally over the years. For the period 2006-2012 she was the Director of the Language Testing Research Centre at Melbourne, where she consolidated and enhanced its international reputation as a leading independent research centre in language testing. She has served in all the leading roles in our profession: she was the ILTA President from 2017 to 2018; she served as the co-editor of the key journal in our field, Language Testing from 2007 to 2011; she served as the Chair of the Educational Testing Service TOEFL Committee of Examiners from 2004 to 2008. She also co-chaired the Language Testing Research Colloquium in Melbourne in 2006 and is a founding member and past Co-President of the Association for Language Testing and Assessment of Australia and New Zealand (ALTAANZ).

 

In her recent role as ILTA President, Cathie was instrumental in instigating and reforming a number of ILTA awards and practices. For example, she chaired the public engagement sub-committee and proposed the establishment of the ILTA Advocacy/Public Engagement award, which has recently been advertised for the first time. This award will offer recognition for an advocacy or public engagement initiative in which an individual or group has intervened in some way with a party outside of academia and drawn attention to inappropriate test use in a particular context. She also developed the LTRC venue rotation policy, set the ground for  restarted the ILTA Newsletter, which was dormant for many years, and also proposed the newly established ILTA Graduate Student Assembly. prepared guidelines for the creation of Special Interest Groups.

 

In Australia and New Zealand, where she has made her career, she has played a leading role nationally in setting policy direction in language assessment in schools, in universities and in employment. She has influenced policies in relation to the testing of language proficiency of language teachers in various languages. She has also set the research agenda of the Occupational English Test (OET) over many years, in collaboration with the OET Centre. In her work at tertiary institutions in Australia and New Zealand, she has influenced the design and implementation of post-entry language assessments and she is still highly influential in providing thought leadership in this area. In her role as Director of the Language Testing Research Centre, she played an instrumental part in ensuring the international character and quality of work emerging from the Centre.

 

Cathie has had an extensive role in developing language tests throughout her career. This has included both large-scale tests and classroom assessments. She was the principal developer of the Diagnostic English Language Needs Assessment (DELNA), as mentioned above. She was also involved in the development of the Academic English Screening Test (AEST), which is currently being used at six Australian universities to screen students new to the university to ensure that there are provided with the necessary support. She developed a number of proficiency tests for language teachers of English and other languages, and, as noted above, is highly regarded for this work. These tests included the Proficiency Test for Language Teachers: Japanese and Italian. She has also worked with colleagues in other countries to developed on tests of implicit and explicit knowledge, and on computer-adaptive vocabulary size and strength tests. Through these projects she has provided thought leadership in the development and ongoing revision and validation of the tests. Cathie also steered the development work on the Occupational English Test during a time when a number of innovations were introduced to the test.

 

Finally, Cathie has played a very active and committed role in mentoring students and early career researchers in the field of language testing. She has supervised numerous PhD students to completion, both in New Zealand and in Australia. Some of these are now either prominent scholars in the field or highly active in other capacities in language test development and administration.

 

In summary, Cathie Elder’s contributions to the field have been extensive and varied and have made a major impact on the wider field of language testing. Therefore, the Award Committee has reached a unanimous decision in granting her the 2020 Distinguished Achievement Award.

 

Association Management Software Powered by YourMembership  ::  Legal