Skip to main content

Add Exercises to an Exam

An exam is built from exercise groups rather than from exercises directly. Each group is one slot on the paper, and each slot can hold several alternatives, so that students do not all receive the same exam.

Exercise Groups​

When Artemis generates the individual exams it draws one exercise at random from every group. Three groups holding two exercises each therefore produce a three-exercise exam in eight possible combinations.

A group is either Mandatory or Optional. Every mandatory group appears in every individual exam. Optional groups are only drawn on if there are more groups than the Number of exercises in exam set in the exam configuration — which is how you offer a choice of topics without lengthening the exam. Groups are mandatory by default; Edit group changes it later.

Exercise groups, each holding several exercise variants
Exercise groups with alternative exercises

Each group lists its exercises as rows. Drag a row's handle to move it into another group — exercises have no manual order within a group — and use the row's overflow menu (…) for the remaining actions, including generating a variant with AI.

Import Exercise Groups​

Import Exercise Group copies groups out of an existing exam, in two steps.

First choose the exam to copy from. Select Exercise Groups opens it.

Choosing the exam to import exercise groups from
Step one: choose the source exam

Then choose what comes across. You can deselect individual exercises, and rename both the groups and the exercises — including the short name of a programming exercise, which together with its title has to be unique. Importing into the same course therefore always needs a rename first; Artemis checks uniqueness when you start the import and tells you which ones clash.

Choosing which exercises to import alongside the groups
Step two: choose the exercises

Add Exercises​

Add Exercise at the foot of a group opens Manage Exercises. Its Create tab offers the five exercise types — Programming, Quiz, Modeling, Text and File Upload — and takes you to the creation form for the one you pick. Its Import tab offers the same five types and copies an existing exercise into the group; its search covers the courses and exams you hold at least editor rights in, and matches on exercise title, course title or exercise id.

Manage Exercises dialog offering the five exercise types
Adding an exercise to a group

Text and modeling exercises can also carry example submissions with example assessments. Assessors work through those first, which is how a correction team arrives at the real submissions already agreed on what the exercise is worth.

Programming Exercises​

Two settings on a programming exercise need attention in an exam.

Enable manual assessment decides whether your assessors can grade the submissions by hand at all. Leave it off and the assessment is whatever the tests produced.

Test visibility decides what a student learns while writing. Configure Grading on the exercise lets you set the Weight, Bonus multiplier and Bonus points of each test case, and its Visibility. In an exam the useful setting is After Release Date of Results: the test still runs and still scores, but the student sees neither its feedback nor its points until results are published. Artemis applies it to every test case of a programming exercise created inside an exam.

Configure Grading page showing test case weights and visibility
Test case configuration for an exam exercise

Hiding every test leaves the student knowing only whether their code compiled, which is often what an exam wants. Controlling when the tests run is a separate question from when their results become visible, and it depends on the setup. With Integrated Code Lifecycle it is part of the build configuration, described next. With Jenkins the exercise instead offers Rerun tests after due date, which runs the suite once over every submission after the date you set.

Run the Tests After the Exam​

With Integrated Code Lifecycle a build is a list of phases, each with an Inclusion condition of Always or After due date. Before a build starts, Artemis drops the phases whose condition is not met and runs the rest. A programming exercise created inside an exam starts with After due date on every phase that collects test results, and Always on the others. Most default templates have a compile, build or syntax-check phase that runs on every push and a test phase that waits for the end of the exam (see the exceptions below). This gives the exam two steps without any manual set-up:

  1. During the exam, each push to a student repository runs only the phases that are not deferred, which in most templates is the compile, build or syntax-check step. The student sees whether the code compiles, and no test runs, so there is no need to replace the real tests in the test repository with a placeholder test and revert that commit afterwards.
  2. After the exam, Artemis rebuilds the latest submission of every student with all phases active, and the results of that run are graded. The rebuild starts 15 minutes after the latest individual exam end including the grace period, so a student with extra working time is not cut short. You can change this date in the timeline of the exercise's edit form, where it is labelled Date for running tests after due date. When you change the exam's times, Artemis moves the date along by the same offset.

Two default templates do not follow this pattern:

  • MATLAB has a single phase, the one that runs the tests, so after the exam defaults no phase runs during the exam. The student gets no compilation or syntax feedback until the tests run after the exam. If you want such feedback, add a phase with the Always condition that checks the code.
  • Empty has no compile phase. Its placeholder phase run_tests does not collect results itself, because the results are moved in a later phase, so it keeps Always. Once you replace the placeholder with the real test command, those tests would run on every push during the exam. Set that phase to After due date yourself.

In the other default templates, including their static code analysis and sequential test run variants, the Always phase is a compile or build step (for Python it byte-compiles the code, for Dart it runs the analyzer), except for JavaScript, R, Ruby and Bash, where it is a syntax check. A syntax check reports syntax errors only.

The template and solution builds ignore the conditions and always run every phase. They are how you check an exercise before the exam: the solution build should pass every test, and the template build shows how the tests judge the code students start from. Both statuses appear next to the phases in the build plan editor.

To inspect or change the phases:

  1. Open the exam and go to Exercise Groups.
  2. Click the exercise's title to open its detail page. The phases are listed there, and a warning appears on a phase that expects tests but is included before the due date.
  3. Click Edit build plan, which is offered to editors and instructors.
  4. In the build plan editor, set the Inclusion condition of each phase and edit its script. Add Phase splits the build into further steps, for example a separate compile phase for the test code.

Artemis joins the scripts of the active phases in order into one build, so a phase that fails stops the ones after it. A phase with Execute at end regardless of previous failures switched on is the exception: Artemis moves it behind all the other phases and runs it even if one of them failed. The switch applies to the phase it is set on and does not keep later phases running, so switch it on for each phase that must run despite an earlier failure. See Adapt Build Phases for every option of a phase and for how Artemis decides whether a build succeeded.

Search documentation