Print Page   |   Contact Us   |   Report Abuse   |   Sign In   |   Join or Create a Guest Account
R & RStudio for Reproducible Language Test Analysis, Research, and Reporting
Share |

R & RStudio for Reproducible Language Test Analysis, Research, and Reporting
Leaders:
Geoffrey T. LaFlair, Duolingo
Daniel R. Isbell, Michigan State University

Day 1: 8:00-4:00
Day 2: 8:00-4:00
Maximum number of participants: 30

Intro/Rationale: When R and RStudio are used together (with other tools internal and external to the
RStudio environment) they provide an excellent toolkit for carrying out analyses relevant to research in
the field of language assessment as well as the practice of evaluating language assessments.
Additionally, this toolkit allows for data processing and analysis that is reproducible, helping to ensure
quality control in practical settings and transparency in research. Combining basic data and statistical
functions with packages specifically for working with test data, we will provide attendees with a
foundation in using R for language testing.

Prerequisites: For Day 1, participants are expected to have knowledge of norm-referenced and criterion-
referenced test theory. No prior experience with R is expected of participants who attend Day 1. For Day
2, participants are expected to have a working knowledge of R (ability to read in, subset, and summarize
data or having attended Day 1 of the workshop). In Day 2, basic familiarity with item-response theory,
and (confirmatory) factor analysis will be helpful but not necessary.

Participants should bring their own computer with R, RStudio. Participants should also install the
following packages by running this script in their R console: install.packages(c("tidyverse", "rcrtan",
"CTT", "psych", "rmarkdown", "tinytex", "eRm", "lme4", "lavaan", "knitr", "devtools", "equate")). Data
for Parts 1, 2, and 3 will be provided by the workshop leaders. Participants are encouraged to bring data
from their own projects for the Part 4 “office hour” discussion at the end of each day.

Goals:

  • Participants will attain basic working familiarity with R and RStudio, including how to read in,
    subset/filter, summarize, and produce basic visualizations of data.
  • Participants will be able to use R to carry out common test data processing and reporting tasks,
    including reliability analyses, item analyses, and score reporting.
  • Participants will be able to use R for analyzing test data for validation/research purposes, including
    test equating, Rasch/IRT and CFA analyses.

Workshop Agenda:

Day 1

Part 1: Getting your data into R, and Getting to Know it. To begin the workshop, we will familiarize
participants with the R Studio interface and base R commands. The tidyverse family of R packages will
be used in learning how to read in data files, subset and filter data, summarize data, and produce basic
data visualizations.

Part 2: Routine Analyses of Testing Data. This part of the workshop will cover reliability and item
analyses in classical test theory (e.g., for placement tests), using the CTT package, and criterion
referenced test theory (e.g., for achievement tests), using the rcrtan package. Basic reliability analyses
for the subjective scoring of performance assessments will also be introduced, mainly using the psych
package. Participants will also learn how to equate forms of tests using the equate package.

Lunch: 1 hour

Part 3: Reporting (Reproducible) Results with R Markdown: Slides and Reports. In this part of the
workshop we will focus on using R Markdown to generate slide decks and score reports for presenting
the results of routine analyses of testing data to a variety of stakeholders.

Part 4: Problem Set & Office hours: Apply what you learned today with a new dataset. You are also
welcome to work with some of your own data in R using the techniques covered. Troubleshoot your
ideas and data with the presenters and your co-participants.

Day 2

Part 1: Using R for Item Response Modeling. To start Day 2, we demonstrate how R can be used for
common item response theory (IRT) analyses (with psych and eRm). We will then introduce how R can
be used for explanatory item response modeling (e.g., linear logistic test model (LLTM)) with eRm and
lme4.

Part 2: Advanced Analyses of Testing Data. To close out the analysis portion of the workshop, we will
introduce CFA and SEM in R (with lavaan).

Lunch: 1 hour

Part 3: Reporting (Reproducible) Results with R Markdown. In this part of the workshop we focus on
using R Markdown to generate research reports. The packages tinytex and knitr will be introduced to
create high-quality PDF reports. We will also demonstrate how an existing script can be used with “new”
data to reproduce analyses and reports.

Part 4: Problem Set & Office hours: Apply what you learned today with a new dataset. You are also
welcome to work with some of your own data in R using the techniques covered. Troubleshoot your
ideas and data with the presenters and your co-participants.

more Calendar

3/4/2019 » 3/8/2019
LTRC 2019

Association Management Software Powered by YourMembership  ::  Legal