production
Skip to Content

CS5063: EVALUATION OF AI SYSTEMS (2017-2018)

Last modified: 27 Feb 2018 18:42


Course Overview

Artificial intelligence has helped solve complex practical problems such as driving a car, translating text from/to different languages, understanding and answering questions, and playing games such as chess and Go. This course will provide students of our MSc in AI with knowledge of core evaluation concepts, approaches, tools, techniques and technologies.

Course Details

Study Type Postgraduate Level 5
Session First Sub Session Credit Points 15 credits (7.5 ECTS credits)
Campus Old Aberdeen Sustained Study No
Co-ordinators
  • Professor Judith Masthoff
  • Professor Ehud Reiter

Qualification Prerequisites

None.

What courses & programmes must have been taken before this course?

  • Any Postgraduate Programme (Studied)

What other courses must be taken with this course?

None.

What courses cannot be taken with this course?

None.

Are there a limited number of places available?

No

Course Description

The course will cover concepts, methods, techniques and tools/technologies for evaluating AI systems. Students will be equipped with knowledge on statistical analysis (e.g., variance, correlations and regression) and learn to use software/tools for statistical analysis. The course will introduce criteria for the evaluation of AI systems (e.g., usability, accessibility and learnability), and the theoretical evaluation of AI systems (e.g., guarantees regarding correctness, completeness, complexity, admissibility of heuristics, and so on). The course will provide a comprehensive exposition to issues pertaining to the empirical evaluation of AI Systems, including the design of experiments (to address specific criteria/issues), human-driven experiments (including the design of forms and questionnaires, interviews, “talk-aloud” experiments, logging/filming, etc.), systems with optimal behaviours vs. (sub-optimal) human-like behaviour, crowd-sourcing of experiments (including Amazon’s “Mechanic Turk” and others), evaluation through gaming, and other related topics.


Contact Teaching Time

Information on contact teaching time is available from the course guide.

Teaching Breakdown

More Information about Week Numbers


Details, including assessments, may be subject to change until 31 August 2023 for 1st half-session courses and 22 December 2023 for 2nd half-session courses.

Summative Assessments

Continuous In-course Assessment (100%).

Resit: Where a student fails the course overall they will be afforded the opportunity to resit those parts of the course that they failed.

Formative Assessment

There are no assessments for this course.

Feedback

Formative feedback for in-course assessments will be provided in written form. Additionally, formative feedback on performance will be provided informally during practical sessions.

Course Learning Outcomes

None.

Compatibility Mode

We have detected that you are have compatibility mode enabled or are using an old version of Internet Explorer. You either need to switch off compatibility mode for this site or upgrade your browser.