Sunday, April 26 - Tuesday, April 28, 1987
Marco Polo Resort
Hotel
Miami Beach, Florida
Sunday, April 26
Monday, April 27
Winton H. Manning
"Development of Cloze-Elide Tests of English as a
Second Language"
(Part II is on Tuesday.)
Grant Henning and Young Gee
"Specialization Bias"
Eduardo Cascallar
"Syntactic and Semantic Activation Measures in
Language Proficiency Testing"
Allene Grognet and Rebecca Oxford
"Validity and Reliability Issues in
Developing a Hands-On Test of Work- Oriented English Language Skills for
Refugees"
David Nevo and Elana Shohamy
"Beyond Reliability and Validity:
Evaluating a National Oral Proficiency Test"
Brendan J. Carroll
"The English-Speaking Union (ESU) Framework Project"
Fred Davidson
"The Dimensionality of ESL Proficiency Testing"
Russanne Hozayin
"Investigating the Interdependence of Language Ability
Items: A Comparative Study of Three Analytic Methods"
Thom Hudson
"Norm-Referenced, Criterion-Referenced, and Item Response
Theory"
Charles Alderson, Grant Henning, Yasmeen Lukmeni
"Levels of
Understanding in Reading Comprehension Tests"
Pardee Lowe and Tom Parry
"Is There a Reading Offset?"
Elana Shohamy and Claire Gordon
"What Are We Really Testing on Reading
Comprehension Tests?
Esin Kaya-Carton and Aaron S. Carton
"Issues of Validity in Judging
Passages for Testing Reading Proficiency"
Lyle Bachman and John Clark (Chairs)
Charles Alderson, Pat Dandonoli,
Thom Hudson, Charles Stansfield, John Upshur (Respondents)
"Forum on a
Program of Research and Development of Measures of Communicative Language
Proficiency"
Tuesday, April 28
Topics: Dates, location, and coordinators of 1988 Language Testing Research Colloquium; publication of 1987 Proceedings; improved communications (mailing list, etc.).
Charles B. Friedman
"Multiple Choice Cloze: Application of Latent Trait
Theory"
Marilyn Chasan and Aurora Marron
"Content or Function Word Deletions as
a Factor in Cloze Test Difficulty"
John Clark and John Thain
"Computer-Adaptive Testing at the Defense
Language Institute"
Harold Madsen and Diane Strong-Krause
"Development of a Multi-Modality
ESL Computer-Adaptive Test"
Liz Hamp-Lyons
"A General Performance Profile for Testing Specific
Academic Writing"
Lynn Sawyer-Thompson and Ray Graham
"Testing of Language Skill
Attrition"
Sylvia Rothfarb and Maria Ariza
"Developing Multiple Criteria to
Mainstream English-as-a-Second-or-Other- Language (ESOL) Students"
The Colloquium Coordinators (Mary Mack, Rebecca Oxford, and Neil Anderson) wish to thank the following individuals and institutions for their contributions to this year's meeting:
Eduardo Cascallar
David Crookall
Fred Davidson
Liz
Hamp-Lyons
Sylvia Rothfarb
The Center for Applied Linguistics,
Washington, DC
Continuing Education, Florida International University,
Miami
The Diplomat Hotel, Miami
**********************************************************************
Pardee Lowe, Tom Parry, and Rebecca Oxford (Chairs). "What Are the Key Testing Questions in Each of the Four Language Skill Areas, and How Can They Be Addressed?"
Presents the major arguments raised in discussions on each of four key language testing questions, one for each skill area (for example, general versus special-purpose speaking assessment and guidelines for participative versus non-participative listening comprehension tests). Participation both by experts in each skill area and by the audience is expected.
**********************************************************************
David Nevo and Elana Shohamy. "Beyond Reliability and Validity: Evaluating a National Oral Proficiency Test."
Discusses a new EFL oral proficiency test in Israel and explains the need for using a more comprehensive set of criteria for judging test quality, with factors to include utility, accuracy, feasibility, and fairness, as well as reliability and validity.
**********************************************************************
Alan Davies. "Operationalizing Uncertainty in Language Testing."
Discusses the positive aspects of uncertainty variables (language, native speaker, cut-off score, and criterion score) and argues that their lack of reliability may be a guarantee of validity in terms of the "truth value" of the test. Re-examines the classical relationship between validity and reliability.
**********************************************************************
Eduardo Cascallar. "Syntactic and Semantic Activation Measures in Language Proficiency Testing."
Describes a method to test language proficiency that involves two levels of linguistic function (syntactic and semantic) and varied activation cues in a computerized context. Especially examines validity-related aspects of the method.
**********************************************************************
Grant Henning and Young Gee. "Specialization Bias."
Discusses empirical construct validity of ESL tests with an ESP focus by describing a study of specialization bias among 200 students in different university specialties. Argues in favor of specialized ESL proficiency testing.
**********************************************************************
Allene Grognet and Rebecca Oxford. "Validity and Reliability Issues in Developing a Hands-on Test of Work-oriented English Language Skills of Refugees."
Describes the development of a test of work-based English language proficiency for newly arrived refugees in the U.S. Especially emphasizes the often difficult logistics of creating a hands-on language test that is valid and reliable for a special population.
**********************************************************************
Brendan J. Carroll. "The English-Speaking Union (ESU) Framework Project"
Explains the major ESU project designed to describe the wide range of English language examinations, with later phases of the project planned to indicate gaps and overlaps in existing examinations and to produce a coherent and easily understood "User's Guide" to such examinations.
**********************************************************************
Fred Davidson. "The Dimensionality of ESL Proficiency Testing."
Compares exploratory factor analysis and multidimensional scaling as methods to identify the dimensionality of two self-rating scales for ESL difficulty and strength.
**********************************************************************
Russanne Hozayin. "Investigating the Interdependence of Language Ability Items: A Comparative Study of Three Analytic Methods."
Describes network analysis, path analysis, and spatial point pattern analysis as approaches for investigating the interdependence of language ability items used in two standard language tests for adult learners (with Rasch and multidimensional scaling employed to analyze the data preliminarily before using the three approaches).
**********************************************************************
Thom Hudson. "Norm-referenced, Criterion-referenced, and Item Response Theory."
Addresses two concerns in language testing: first, the role of functional or communicative competence in language testing, and second, the appropriacy of criterion-referenced measurement, norm-referenced measurement, and item response theory as approaches to functional language testing.
**********************************************************************
J. Charles Alderson, Grant Henning, and Yasmeen Lukmeni. "Levels of Understanding in Reading Comprehension Tests."
Argues that the frequent assumption of a hierarchy of language skills may not be valid in reading comprehension tests, and presents results of multi-method ESL research related to the question of reading-skill hierarchies.
**********************************************************************
Pardee Lowe and Tom Parry. "Is There a Reading Offset?"
Discusses the relationship among Interagency Language Roundtable ratings of listening comprehension, speaking, and reading, with a special focus on how other skills affect the assessment of reading. Presents practical and theoretical implications.
**********************************************************************
Elana Shohamy and Claire Gordon. "What Are We Really Testing on Reading Comprehension Tests?"
Compares trait versus method in reading comprehension tests, and argues that the testing method strongly influences students' scores, especially with low level students. Discusses the best ways for testing reading comprehension with the minimum interference from the testing method.
**********************************************************************
Esin Kaya-Carton and Aaron S. Carton. "Issues of Validity in Judging Passages for Testing Reading Proficiency."
Discusses a study which tested three hypotheses about validity of French reading proficiency passages as rated by ACTFL-trained expert judges and by students. Findings suggest that judgments about difficulty level and about other validity-related elements may depend on a number of considerations, such as passage length, use of cloze procedure, and so on.
**********************************************************************
Lyle Bachman and John L.D. Clark (Chairs). "Forum on a Program of Research and Development of Measures of Communicative Language Proficiency."
Includes a panel presentation and an open discussion considering specific applications to language testing research and development of a framework of communicative language proficiency. A program of research and development activities is also proposed and discussed. Panelists include: J. Charles Alderson, Pat Dandonoli, Thom Hudson, Charles Stansfield, and John Upshur.
**********************************************************************
Charles B. Friedman. "Multiple Choice Cloze: Application of Latent Trait Theory."
Applies latent trait theory to an integrative cloze language task and examines the robustness of the Rasch model in this application. Specifically examines assumptions of unidimensionality and independence and compares classical measurement theory to latent trait theory in the context of the cloze procedure.
**********************************************************************
Marilyn Chasan and Aurora Marron. "Content or Function Word Deletions as a Factor in Cloze Test Difficulty."
Compares content and function word deletions as factors in the difficulty of cloze tests in two studies of reading comprehension. Gives implications for design of future cloze tests.
**********************************************************************
Winton H. Manning. "Development of Cloze-Elide Tests of English as a Second Language"
Describes cloze-elide tests, which are developed by inserting words into a running text at random points and which require students to extract meaning from the text by finding and eliminating barriers to comprehension that are presented by the extraneous words. Describes research on the validity of this new approach to testing language proficiency.
**********************************************************************
John L. D. Clark and John Thain. "Computer-Adaptive Language Testing at the Defense Language Institute."
Describes work at DLI on a major project to develop a full-scale prototype computer- adaptive listening comprehension and reading test in Russian, based on Interagency Language Roundtable proficiency guidelines. Presents question formats, measurement rationales, item selection algorithms, presentation process, and procedures for scoring and determining proficiency levels."
**********************************************************************
Harold S. Madsen and Diane Strong-Krause. "Development of a Multi-Modality ESL Computer-Adaptive Test.
Describes and evaluates an innovative ESL computer-adaptive test, developed by means of Rasch item response theory procedures. Evaluation includes statistical and affective aspects.
**********************************************************************
Liz Hamp-Lyons. "A General Performance Profile for Testing Specific Academic Writing"
Describes work done by the British Council's English Language Testing Service to assess specific academic writing. Explains the scoring method, which offers both quantitative and qualitative judgments.
**********************************************************************
Lynn Sawyer-Thompson. "Diagnostic Testing of Language Skill Attrition."
Describes a project involving the development of discrete-point diagnostic tests in three languages to be used to assess specific language skill losses after the end of formal language training.
**********************************************************************
Sylvia H. Rothfarb and Maria J. Ariza. "Developing Multiple Criteria to Mainstream English-as-a-second-or-other language (ESOL) students."
Describes two studies leading to the development of multiple criteria used to mainstream ESOL students in a large public school system. Focuses on research about acceptable language proficiency measurement procedures for special populations in public school settings.
**********************************************************************