Skip to main content

Cleanup Service

Artemis provides a feature to delete data from older courses.

Cleanup view

As shown in the image, administrators can delete the following data types:

  • Plagiarism results with an undecided outcome
  • Orphaned data
  • Non-rated results from older courses
  • Rated results from older courses

Since orphaned data has no connections to other data by nature, it is deleted without considering specific dates. For other types, administrators can track the related exercises and courses. When a cleanup operation is performed with specified "from" and "to" dates, all data associated with that type and related to courses that started after the "from" date and ended before the "to" date is deleted.

In addition to these manual, date-range operations, Artemis provides age-based data-privacy retention operations that delete data a configurable time after a course ends, and that can optionally run automatically on a schedule. These are described in detail in Data-Privacy Retention (age-based cleanup and automation) below.

caution

Every cleanup operation is irreversible. Deleted data cannot be recovered from within Artemis. For the age-based operations, always use the count/preview shown on the page to review how much data would be affected before you confirm, and make sure a course archive exists for any course whose student data you are about to reset.


Data Deletion by Operation Type

1. Orphaned Data

  • Long Feedback Text with feedback that has no results
  • Text Block with feedback that has no results
  • Feedback records without results
  • Student and team scores where either a student or a team is specified
  • Long Feedback Text where both participation and submission are missing
  • Text Block where the referenced feedback has no associated participation or submission
  • Feedback with no associated participation or submission
  • All Ratings where the related result has no associated participation or submission
  • Results without associated participation or submission

2. Plagiarism Results with an Undecided Outcome

  • All plagiarism comparisons related to courses within the specified dates and marked as undecided.

3. Rated and Non-rated Results

  • Both types follow the same logic, except for the rating status of results (rated vs. non-rated).
  • For each type, only the latest valid result within a participation is retained, while all others are deleted.
  • Because direct result deletion is restricted due to data integrity reasons, Artemis first removes associated data for results scheduled for deletion, including Long Feedback Text, Text Block, Feedback, Student Score, and Team Score.

Artemis also records the date of the last cleanup operation, as seen in the last column of the table shown in the image.


Data-Privacy Retention (age-based cleanup and automation)

The operations above are driven by explicit "from"/"to" dates. Artemis additionally provides age-based data-privacy operations that act on data a configurable period after the owning course ends, plus optional automation that runs them on a schedule. These implement the storage-limitation and erasure principles of the GDPR (Art. 5(1)(e) and Art. 17): personal student data is removed once it is no longer needed, while the reusable course material is preserved.

All of these operations are admin-only, expose a count/preview before execution, and are off by default for automation. Their cutoffs and schedules are configured centrally (see Configuration reference).

Age-based operations

OperationWhat it deletesWhat it keepsDefault cutoff
Reset student data of old courses (two phases)All student data of the course (participations, submissions, results, feedback, posts, enrollments, student/tutor/editor group memberships)The course itself, its exercises, lectures, exams, competencies and tutorial-group configuration, and the instructors5 years after end (grade-relevant), 1 year (non-grade-relevant), plus a grace period
Delete feedback of non-latest resultsFeedback, long feedback texts and text blocks of results that are not the latest per participationThe results themselves, and the feedback of the latest rated and latest non-rated result per participation8 weeks after course end
Delete old submission versionsSubmission versions (the editor keystroke/version history of a submission)The submission itself and its latest stored content8 weeks after course end
Warn + delete not-enrolled users (two phases)Phase 1 emails users enrolled in no course and inactive that their account will be deleted after a grace period; phase 2 soft-deletes and anonymizes those still not-enrolled who did not log in since the warningEnrolled users, recently active users, users who log in during the grace period, administrators, and the Iris bot6 months of inactivity + a grace period
Delete plagiarism cases of old coursesPlagiarism cases of courses that ended long ago, together with their notification post and answer postsEverything else; the case's plagiarism submissions are kept (only their reference to the deleted case is cleared)5 years after course end (the grade-relevant retention)
info

Courses are never deleted by these operations. The old-course operation only resets the student data and keeps all course material so the course can be reused. Exam submission versions are not affected by the submission-version cleanup because exam exercises are not attached to a course.

Grade-relevance and retention periods

Each course carries a grade-relevance flag (stored in a lazily-loaded CourseConfiguration, so it does not widen the course record and is only loaded when needed). Instructors set it in the course settings; it defaults to grade-relevant so a course is always treated with the longer retention unless someone explicitly opts out.

  • Grade-relevant courses hold examination data (Prüfungsunterlagen). The default retention is 5 years after the course ends.
  • Non-grade-relevant courses carry no exam-retention obligation, so their student data is minimized after 1 year.

Legal basis (defaults chosen for TUM / Bavaria; adjust per your institution). Bavarian universities must keep examination documents for 5 years, then destroy them (Bavarian State Ministry circular). The frequently-cited 2-year period from the RaPO applies to Fachhochschulen, not universities. The authoritative long-term grade/degree record (the reduced exam file, kept for 50 years) lives in the campus-management system such as TUMonline, not in Artemis. See the deployed privacy statement (BayHSchG Art. 61, TUM APSO, BayFEV).

:::caution Legal holds A pending objection or legal proceeding legally extends the retention period until it concludes. Set Data-retention hold in the course settings of every affected course for as long as the proceeding runs. A course on hold is excluded from both phases: it is never warned and never reset, and a hold set after the warning withdraws that warning, so lifting it later starts a fresh grace period. The operations themselves are aggregate, an administrator cannot skip individual courses when confirming them, which is why the hold has to be set on the course rather than decided at confirmation time. :::

The two-phase old-course reset lifecycle

Resetting an old course is destructive, so it is split into two phases with a grace period in between. A course moves through a strict one-shot lifecycle recorded on its CourseConfiguration:

not warned ──(phase 1: warn + archive)──▶ warned ──(grace elapsed, phase 2)──▶ reset
▲ │
└──────(course left the scope again)───────┘

Phase 1: Warn and archive (Warn about upcoming student-data reset). For every course whose retention period has elapsed and that has not yet been warned or reset, Artemis:

  1. Ensures a course archive (a downloadable ZIP backup) exists, creating it if necessary.
  2. Emails every eligible instructor a warning with the archive download link and the notice that the student data will be deleted after the grace period.
  3. Stamps the warning timestamp on the course, but only if at least one instructor was actually notified. If email is not configured, or the course has no eligible instructor, or archiving fails, the course is left untouched and retried on the next run. This guarantees a course is never scheduled for reset without its instructors having been warned and given a backup.

Test courses, courses without an end date, and courses on a data-retention hold are always skipped.

Phase 2: Reset (Reset student data of old courses). Once the grace period (measured from the actual warning event, not from the course end date) has elapsed, Artemis resets the student data of each warned course and stamps the reset timestamp so it is never reset again. Because the grace is anchored to the real warning, the full grace is always honored regardless of scheduling delays.

Eligibility is re-evaluated at reset time instead of being trusted from the warning. A warned course whose end date moved into the future, which became grade-relevant (a longer retention period), which became a test course, or which was put on a data-retention hold, has its warning withdrawn and returns to the start of the lifecycle. It is therefore never reset before its current retention deadline, and it needs a new warning plus a full grace period before it can be reset later.

What guarantees no data is lost by accident

  • Two independent conditions gate a reset: the retention period must have elapsed and the instructors must have been warned (with an archive) and the grace period must have passed. A course that was never warned is never reset.
  • One-shot lifecycle: the warning and reset timestamps make re-warning or re-resetting an already-processed course impossible, even if a schedule fires repeatedly.
  • Eligibility is re-checked before deleting: a course that left the scope after it was warned has its warning withdrawn instead of being reset, so a change made during the grace period always takes effect.
  • Data-retention hold: a course under a legal hold is excluded from both phases for as long as the hold is set, and an update that omits the flag can never lift it.
  • Feedback cleanup keeps results: only feedback entries of non-latest results are removed, never the results, and always keeping the latest rated and latest non-rated feedback per participation.
  • Not-enrolled user cleanup is two-phase: an inactive, course-less account is first emailed a warning and only soft-deleted (and anonymized, not hard-deleted) after a grace period, and only if it is still not-enrolled and has not logged in since the warning. Logging in cancels the deletion. Administrators and the Iris bot are never affected. Inactivity is measured by the user's last login (falling back to the account creation date), not by the last modification of the user row.
  • Preview first: every operation has a count endpoint so an admin can see exactly how many entities would be affected before confirming.

These guarantees are covered by server integration tests that run the real deletion (and the real scheduled entry point) against a database, asserting both that the intended data is deleted and that everything outside the gate survives. See Testing and assurance.

Automation and scheduling

Each age-based operation has a matching scheduled job that runs on the scheduling node. Every job is guarded by its own kill switch (artemis.cleanup.*-schedule-enabled), which defaults to false. The cron still fires on the configured schedule, but the job body does nothing while its switch is off. Nothing is deleted automatically until an administrator explicitly enables the corresponding switch.

The default cron schedules (configurable under artemis.scheduling) are:

JobDefault scheduleKill switch
Warn + archive old coursesSundays 02:00old-courses-warning-schedule-enabled
Reset old courses' student dataSundays 03:00old-courses-reset-schedule-enabled
Delete feedback of non-latest resultsSundays 04:30old-feedback-schedule-enabled
Delete old submission versionsSundays 04:45old-submission-versions-schedule-enabled
Warn not-enrolled users1st of month 05:00not-enrolled-users-warning-schedule-enabled
Delete not-enrolled users1st of month 06:00not-enrolled-users-schedule-enabled
Delete plagiarism cases of old coursesSundays 03:30plagiarism-cases-schedule-enabled

The warning job is scheduled before the reset job so that, once both are enabled, a course is warned and archived in one run and only reset in a later run after its grace period.

Configuration reference

All cutoffs and kill switches live under artemis.cleanup, and the cron expressions under artemis.scheduling, in application-artemis.yml (override per environment in your deployment configuration). A restart is required to change them.

artemis:
scheduling:
old-courses-warning-time: 0 0 2 * * SUN
old-courses-reset-time: 0 0 3 * * SUN
old-feedback-cleanup-time: 0 30 4 * * SUN
old-submission-versions-cleanup-time: 0 45 4 * * SUN
not-enrolled-users-warning-time: 0 0 5 1 * *
not-enrolled-users-cleanup-time: 0 0 6 1 * *
plagiarism-cases-cleanup-time: 0 30 3 * * SUN
cleanup:
# Retention cutoffs
grade-relevant-retention-years: 5 # reset grade-relevant courses 5 years after they end
non-grade-relevant-retention-years: 1 # reset non-grade-relevant courses 1 year after they end
reset-warning-grace-period-days: 30 # days between warning instructors and the reset
old-feedback-cutoff-weeks: 8 # purge feedback of non-latest results 8 weeks after course end
old-submission-versions-cutoff-weeks: 8 # purge submission versions 8 weeks after course end
not-enrolled-users-inactivity-months: 6 # inactivity (by last login) before a course-less user is warned
not-enrolled-users-warning-grace-period-days: 30 # days between warning a user and deleting the account (login cancels it)
# Schedule kill switches (all off by default)
old-courses-warning-schedule-enabled: false
old-courses-reset-schedule-enabled: false
old-feedback-schedule-enabled: false
old-submission-versions-schedule-enabled: false
not-enrolled-users-warning-schedule-enabled: false
not-enrolled-users-schedule-enabled: false
plagiarism-cases-schedule-enabled: false

The plagiarism-case cleanup reuses the grade-relevant retention (grade-relevant-retention-years, 5 years by default): plagiarism cases are examination records and share the same retention obligation as other grade-relevant data.

Enabling automation in production

The switches are off by default deliberately, because the operations delete personal data. To fulfill the storage-limitation obligation in practice, an institution must turn the relevant jobs on. A safe rollout:

  1. Review the cutoffs against your institution's regulations and adjust the artemis.cleanup.* values if needed.
  2. Dry-run manually first. Open the admin Data Cleanup page and use the count/preview for each operation to see how much data would be affected on your production data.
  3. Enable the housekeeping jobs first (old-feedback-schedule-enabled, old-submission-versions-schedule-enabled), which only trim redundant history and keep all results.
  4. Enable the two-phase course reset by turning on the warning job and the reset job together (old-courses-warning-schedule-enabled and old-courses-reset-schedule-enabled). Make sure email is configured, otherwise the warning phase cannot advance and no reset will ever happen (by design).
  5. Enable the two-phase not-enrolled-user cleanup by turning on the warning job and the deletion job together (not-enrolled-users-warning-schedule-enabled and not-enrolled-users-schedule-enabled), once you have confirmed via the preview that only the expected accounts are selected. As with the course reset, email must be configured, otherwise the warning phase cannot advance and no account is ever deleted (by design).
  6. Confirm the scheduling node. The jobs run only on the node with the scheduling profile active.
caution

Inactivity for the not-enrolled-user cleanup is measured by the user's last login (lastLoginDate, recorded on every successful authentication), falling back to the account creation date for accounts that never logged in. This is a real activity signal, unlike the user row's last modification date (which is bumped by, for example, group synchronization). For existing accounts at the time of the upgrade, lastLoginDate is backfilled to the migration time, so no existing account is immediately eligible; the inactivity clock starts at deployment. The warning phase and the "logging in cancels the deletion" rule provide an additional safeguard against deleting an account the user still wants. Always review the count/preview before enabling these jobs.

Audit event retention

The audit table (jhi_persistent_audit_event) is pruned by its own nightly job, independently of the operations above and with no kill switch: it holds no personal data beyond the acting user's login, and letting it grow without bound is not a useful default.

Two retention periods apply, because the table mixes records with opposite characteristics:

WhatEvent typesDefaultProperty
The login recordAUTHENTICATION_SUCCESS, AUTHENTICATION_FAILURE, AUTHENTICATION_PASSKEY_SUCCESS, SAML2_AUTHENTICATION_SUCCESS, LOGOUT_SUCCESS365 daysartemis.audit-events.general-retention-period
Everything elseDeliberate actions such as DELETE_EXERCISE or RESET_EXAM, plus any unrecognised or missing type1825 daysartemis.audit-events.application-retention-period

One row is written per login attempt, so the login record is the bulk of the table and tells you little a few weeks later. The remaining rows are rare and are what has to be reconstructed when a question is raised long after the fact, an exam dispute in particular. Anything not recognised as a login gets the long retention, so an event type added in a later version is over-retained rather than silently dropped.

Both values must be positive and are validated when the configuration is bound, so a typo such as 0 or a negative number fails startup instead of being applied. This job has no kill switch and its deletions are irreversible, so a cutoff in the future would delete records written minutes ago.

The job runs at 03:10 every night on the node with the scheduling profile active. It deletes in batches of 5000 rows, up to 200 batches per retention period per night, so a backlog that has accumulated over years is drained over several nights rather than in one very long transaction. The log line reports how many rows each period removed, and says explicitly when the per-run cap was reached and rows remain.

Testing and assurance

The correctness of these deletions is protected by server integration tests that run against a real database with no mocking of the deletion logic:

  • The selection filters are tested exhaustively (for example, the not-enrolled-user warn query excludes enrolled, recently active, administrator, super-administrator, already-deleted and already-warned users; the delete query selects only warned, past-grace, still-inactive accounts that did not log in since the warning).
  • The destructive operations are tested end to end: a warned, past-grace course really has its student data deleted while the course, its exercises and its configuration survive; the not-enrolled-user cleanup only warns inactive course-less accounts and only deletes them after the grace period (sparing anyone who re-enrolls or logs in), soft-deleting and anonymizing exactly the right accounts; the feedback cleanup removes only non-latest feedback and keeps the results; the submission-version cleanup removes only versions of ended courses.
  • The scheduled entry point is tested through the real AutomaticDataCleanupScheduleService (flag gate and system-context wrapper included): with every kill switch off, running all jobs deletes nothing; with a switch on, the corresponding job performs exactly the intended deletion.
Search documentation