AI-Based Assessment and Evaluation

Description

Teachers design assessment tasks all the time: quizzes, tests, writing prompts, comprehension questions, and classroom activities used to evaluate learning.

Yet very few educators have ever been properly trained on how to do it well, that is to say, how to write assessment tasks that are truly clear, reliable, and meaningful.

As a result, assessments often become confusing or unintentionally biased: questions that fool students, tasks that are either too easy or too difficult, and items that are unwittingly biased against particular groups.

When this happens, the data that comes out of an assessment is much less useful to the people who need it most: the teachers trying to make sense of how their students are doing.

This course intends to directly address this “gap”. The participants will learn the basic principles of effective assessment design and good item writing, in the hope of helping them to create assessments that are clear, fair and really mean something.

Being hands-on, practice-based, this course will allow participants to learn how to align assessment tasks with learning objectives, write clear and accessible prompts, adjust difficulty levels appropriately, avoid common item-writing mistakes, and design assessments that are fair, valid, and useful for both teachers and learners.

Throughout the course, participants will use, analyse, and improve real examples of assessment tasks (multiple-choice questions; short-answer items; extended responses; performance-based tasks), looking closely at what makes some items work well and others fall flat.

The course also explores how AI tools can support the assessment design process.

Participants will experiment with AI-assisted item generation and learn how to critically evaluate, adapt, and improve AI-generated content while maintaining quality, fairness, and pedagogical value.

By the end, participants will definitely feel more confident in writing assessment items that will stand up to scrutiny and produce results worth acting on.

They will leave with practical strategies to use immediately and an increased ability to interpret student performance and determine next steps.

Concept by Cansu Akan

Learning outcomes

The course will help participants to:

• Explain the key principles of validity, reliability, and fairness as they apply to classroom assessment;
• Write high-quality items across multiple formats: selected response, constructed response, and performance tasks;
• Align assessment items precisely to learning objectives using established frameworks;
• Identify and correct common item-writing flaws that reduce validity and fairness;
• Design clearer and more accessible prompts for learners;
• Develop performance tasks and simple assessment rubrics;
• Use AI tools to support assessment design and item generation;
• Critically evaluate and improve AI-generated assessment materials;
• Design coherent and practical assessments adapted to their own teaching context.

Tentative schedule

Day 1 – Introduction to the course

• Introduction to the course and the Academy;
• Introduction to the host city and the week’s cultural activities;
• Icebreaker activities;
• Presentations of the participants’ schools and institutions.

Foundation of assessment quality

• What makes an assessment effective?;
• Understanding the importance of validity, reliability, and fairness;
• Common item-writing problems and their impact on student performance;
• Review and critique of sample assessments from participants’ own subjects.

Day 2 – Designing effective selected-response items

• Principles of multiple-choice item design;
• Common flaws and how to avoid them;
• True/false, matching, and ordering tasks;
• Aligning questions with Bloom’s Taxonomy and learning objectives;
• Workshop: writing and reviewing selected-response items.

Day 3 – Constructed response and performance tasks

• Designing short-answer and extended-response questions;
• Scaffolding, clarity, and marking consistency;
• Introduction to performance-based assessment;
• Developing simple rubrics and assessment criteria;
• Reflection: reducing bias and improving comparability in assessment.

Day 4 – AI as an item writing assistant

• Using AI to generate item drafts;
• Prompting strategies for AI-assisted item writing;
• Evaluating AI-generated questions using quality criteria;
• Adapting and refining AI outputs for classroom use;
• Workshop: building a practical item bank with AI support.

Day 5 – Test design and action planning

• Structuring and sequencing assessment tasks effectively;
• Weighting, formatting, and test blueprints;
• Workshop: designing a complete classroom assessment;
• Peer feedback and group reflection;
• Action planning for future classroom implementation;
• Course evaluation: round-up of acquired competencies, feedback, and discussions;
• Awarding of the course Certificate of Attendance.

Day 6 – Structured cultural activities

• Structured cultural activity(ies) according to the hosting location;
• Learning journal: individual written reflection on the course experience;
• Course reflection: self-reflection and review of course materials and resources.

Download or share

Get weekly recaps on confirmed sessions!