Pre-Summit Workshop
Get ready to take your ECPS summit experience to the next level. We are thrilled to announce our pre-summit workshop! Mark your calendars for June 3rd from 11:00 to 1:00 PM EDT. This is one workshop you won't want to miss! All attendees will receive a certificate of participation, once the workshop has been completed.
Automated Scoring and Feedback using Transformer Models
Cambium Assessment, Inc. (CAI) has been using transformer models, which are the basis for generative AI, in its automated scoring engine since 2020. This approach has led to high-quality scoring for our state-level clients, particularly when combined with human scoring. This approach also led to two grand prize awards on the NAEP reading comprehension automated scoring challenge. Our work in transformer architectures for automated scoring has laid the foundation for the development of a writing feedback tool that highlights for students the structure of the argumentative essays and provides feedback aligned with k-12 standards in conventions. This work has also led to instruction fine-tuning and reinforcement learning of large language models such as LLAMA 2, Mistral, and Claude models for scoring feedback, generic scoring, and item passage development. In this session we will provide an overview of the architecture of our engines, describe the benefits and challenges in its development and use, present the promising and less promising results of our work, and recommendations around the skills and resources needed to build, deploy, and use these complex systems.
Speaker: Sue Lottridge, Chief Scientist, NL Applications, Cambium Assessment
Sue Lottridge, Ph.D. is a Chief Scientist of Natural Language Applications at the Cambium Assessment, Inc. (CAI). In this role, she leads CAI’s machine learning team on the research, development, and operation of CAI’s software that process text and speech. This software includes automated essay scoring, short answer scoring, speech scoring, writing feedback, and an engine that detects disturbing content in student responses. Dr. Lottridge has worked in automated scoring for eighteen years and has contributed to the design, research, and use of multiple automated scoring engines including equation scoring, essay scoring, short answer scoring, crisis alert detection, and feedback. She earned her Ph.D. from James Madison University in assessment and measurement (2006) and holds Masters’ degrees in Mathematics and in Computer Science from the University of Wisconsin-Madison (1997). Most recently, Dr. Lottridge has published on the psychometric considerations around the use of deep learning in automated scoring and feedback, and is the author of set of guidelines for automated scoring use in the Guidelines for Technology Based Assessment.
To attend, just simply add the workshop to your registration when registering for the Summit.
Member: $30.00 | Non-Member: $35.00
If you have any questions about registering or about this workshop, please email atp@canvasmeetings.com.
Register now!