Consistency Check
The consistency check is an AI-assisted feature in Hyperion that detects cross-artifact inconsistencies in programming exercises before they are published to students.
A programming exercise consists of four interconnected artifacts: the problem statement, the template repository, the solution repository, and the test repository.
When one artifact is changed without updating the others, students may face exercises they cannot solve — for example, the problem statement describes a method named calculateAverage while the template declares computeMean.
For the formal definition of the underlying issue types and ontology, see: Dietrich et al., "Automated Consistency Checking of Multi-Artifact Programming Exercises", Koli Calling 2025.
Triggering a Consistency Check
Instructors with at least Editor role can trigger a consistency check in the online code editor. Open the exercise details, scroll to Repositories and builds, and choose Code → Edit in editor in the Template or Solution panel. In the editor, choose Artemis Intelligence → Check Consistency.
The Check Consistency control beside the Repositories and builds heading, and the corresponding exercise list action, perform a different check: they verify that the exercise's repositories and build plans exist on the version control and CI systems. The AI consistency check runs synchronously and typically completes within 20–50 seconds.
Understanding the Report
Each detected issue includes:
| Field | Description |
|---|---|
| Severity | LOW, MEDIUM, or HIGH — indicates how likely the issue is to block students |
| Category | The type of inconsistency (see issue types below) |
| Description | Human-readable explanation of what is inconsistent and where |
| Suggested fix | Concrete recommendation for resolving the issue |
| Locations | Affected artifact, file path, and line range for each side of the inconsistency |
Issue Types
| Category | What it detects |
|---|---|
| Method return type mismatch | A method's return type differs between artifacts |
| Method parameter mismatch | A method's parameter list (names, types, or count) differs between artifacts |
| Constructor parameter mismatch | A constructor's parameter list differs between artifacts |
| Attribute type mismatch | A field's declared type differs between artifacts |
| Visibility mismatch | A member's access modifier (public, private, …) differs between artifacts |
| Identifier naming inconsistency | An identifier is spelled differently across artifacts |
What the Check Does Not Flag
The checker deliberately ignores intentional pedagogical gaps:
- Empty method stubs and
TODOcomments in the template repository - Abstract method bodies
- Issues already covered by existing review threads on the exercise