System Design

Architectural Drivers

The architecture of Artemis follows from five properties of the environment it runs in. The rationale given below for each design decision refers back to them.

1. Self-hosted by universities. Every instance is operated by the institution that uses it, often by a small team. Artemis has to be installable and operable with few moving parts.
2. Peaks of load. Many students submit at the same deadline, and exams start and end at the same moment. Capacity has to grow by adding nodes, without downtime.
3. Untrusted code. Student submissions are executed automatically. A faulty or malicious submission must not endanger the host or other submissions.
4. Variety between institutions. Institutions differ in how users authenticate, in the features they want and in the AI services they may use. Features and integrations that an institution does not need must be switchable.
5. Many contributors over many years. Artemis is open source and developed by many people. The structure has to keep changes local, and architectural rules have to be checked automatically.

Top-Level Design

The following UML component diagram shows the top-level design of Artemis. Artemis is decomposed into application clients (an Angular single-page application in the browser, the Android and iOS apps, and any Git client) and an application server (one Spring Boot application). The server is the only place where data, rules and builds live. The clients present them, and the systems around the server (database, user management, optional AI services, distributed data) are replaceable behind an interface.

Interfaces are drawn as ball-and-socket connectors: the ball is the provided interface, the socket is the required interface. The mobile apps are drawn with an interface of their own, although the server offers one REST and WebSocket API to all clients. The apps are separate products with their own code bases and release cycle, see Mobile applications, so their dependency on the API is a dependency of its own that has to stay compatible.

Top-Level Design (UML component diagram)

Component	Responsibility	Interfaces
Artemis Client	Presents all user interfaces of the web application and talks to the server.	Requires REST and WebSocket
Mobile Apps	The native Android and iOS apps, which cover the activities of students on the move, see Mobile applications. They use the same server API as the web client.	Require REST and WebSocket
Git Client	The IDE or the command line of a student, an instructor or a tutor.	Requires Git
Artemis Server	Contains all feature modules, LocalVC and LocalCI.	Provides REST + WebSocket and Git (HTTPS, SSH); requires Hibernate (JDBC), Distributed Data, Authentication and Service APIs
LocalVC	The integrated Git server. Stores the repositories on the file system and serves fetch and push over HTTPS and SSH.	Provides Git
LocalCI	Keeps the queue of build jobs, receives the results of the build jobs, grades them and notifies the user.	Requires Distributed Data
Build Agent	Takes build jobs from the queue, clones the repositories from LocalVC and runs each job in a Docker container. Can run inside a core node or as a separate node.	Requires Git and Distributed Data
Distributed Data Provider	Holds the state that all nodes of an installation share, for example the build job queue. Hazelcast, Redis (or Valkey), or a local implementation for a single node.	Provides Distributed Data
PostgreSQL	Persistent data, accessed with Hibernate. MySQL is still supported until Artemis 11, see the callout below.	Provides JDBC
User Management	Optional external user management such as LDAP, SAML2 or OIDC.	Provides Authentication
Optional AI Services	Iris, Athena, Hyperion, Weaviate and further services that extend the core.	Provide Service APIs

⚠️

Warning:

Planned removals in Artemis 11 (March 2027). Jenkins, the only remaining external CI system, is deprecated and will be removed in Artemis 11. MySQL support will be removed in Artemis 11 as well, which leaves PostgreSQL as the only database. See Planned removals in Artemis 11 for the timeline, and the guides Jenkins to LocalCI and MySQL to PostgreSQL for the migration steps.

Rationale

The following table lists the main design decisions with the driver they address and what they cost. It describes the intent behind the architecture. The decisions are not independent: the first three together make the Integrated Code Lifecycle possible.

Decision	Why	Trade-off
One server application with feature modules instead of separate services (drivers 1, 5)	One artifact to build, deploy and configure keeps the operation simple for a small team. The feature modules keep the code base decomposable: each module reaches other modules only through its api package, and architecture tests enforce this.	The server scales as a unit and is one failure domain per node. Running several core nodes mitigates both.
Integrated version control (LocalVC) and continuous integration (LocalCI) instead of external systems (drivers 1, 3)	Administrators need no separate Git hosting or CI server. Repository operations such as copying the template into a new repository are file system operations, which is faster than calls to an external system. LocalVC reuses the authorization rules of the online editor, and build agents authenticate with per-job clone tokens or SSH keys instead of a shared password.	Artemis maintains a Git server and a build system itself. Supporting Jenkins as an alternative cost test time and code paths for every feature, which is why Jenkins is removed once LocalCI covers all features.
Build agents decoupled from the core nodes by a shared job queue (drivers 2, 3)	Builds are bursty and CPU intensive and must not compete with serving the user interface. Agents can be added where capacity is available, run inside the university or in a cloud, and take a job only when they have capacity.	Needs a reliable shared state, see below. Jobs of a lost agent have to be detected, which LocalCIEventListenerService does.
One Docker container per build job (driver 3)	The container isolates the untrusted code, is created for the job, and is removed with its data when the job ends.	Images have to be provided on the agents, and starting a container adds latency to each build.
A DistributedDataProvider abstraction instead of using Hazelcast or Redis directly (drivers 2, 5)	The nodes share the job queue, feature toggles, caches and more. Behind one interface the provider can be Hazelcast (embedded in each node, no extra service), Redis (external, persistent) or local (one node, for development). An architecture test fails the build if production code outside the provider adapters uses a provider directly.	Hazelcast loses its data when the cluster stops, and Redis needs a versioned data schema. The maintainers are considering to deprecate Hazelcast, which the abstraction makes possible.
An external STOMP broker for several core nodes (driver 2)	A websocket message has to reach the clients that are connected to any node. A single node uses an in-memory broker.	One more component to run, only needed with several core nodes.
One relational database, migrated with Liquibase at startup (drivers 1, 5)	There is one source of truth, and an upgrade needs no separate migration step. Supporting only PostgreSQL after Artemis 11 removes the need to write queries that behave the same on two engines, and one database from the test matrix.	Existing MySQL installations have to migrate.
Optional modules and integrations behind configuration (driver 4)	An institution enables what it needs, for example with artemis.iris.enabled, and delegates authentication to LDAP, SAML2 or OIDC.	Every combination is a configuration that has to be tested.
A single-page client that talks to the server through REST and WebSocket (driver 5)	The server API is the only coupling between client and server. For DTO-only endpoints, the client models are generated from OpenAPI specifications, see OpenAPI.	Two code bases that evolve together.
Native mobile apps next to the web client, on the same API (drivers 4, 5)	Students expect to read learning material and follow communication on their phone, and they expect push notifications and the look of their platform. A native app delivers that, and it reuses the API, so the server has no second interface to maintain. The apps live in their own repositories and are released independently. The server can demand a minimum app version, so an old app is asked to update when the API has moved on.	An API change has to be compatible with the app versions that are in use. The apps do not cover every feature of the web application.
Hibernate with Spring Data repositories as the only access to the database (drivers 1, 5)	An object-relational mapper keeps the domain model in code and the schema in Liquibase changelogs, so one model serves all feature modules. Repositories are the only place where queries and transactions live. A transaction in a service would hold its locks across every repository call, remote request and file write inside it and block other users, so the guidelines place transactions in repositories. Lazy fetching is the rule, because loading the relationships of an entity costs far more than selecting the entity.	The mapper hides what a query costs. Fetching and transaction rules are needed, and checked by architecture tests, see the database guidelines.

Deployment

The following UML deployment diagram shows a typical deployment of an Artemis installation. A box with a shaded side is a node, a server or a device. The boxes inside it are the software that runs on it. The number of nodes depends on the installation: there are one or more core nodes, any number of build agent nodes, and the distributed data server and the broker node only exist when the installation needs them. Student, instructor and teaching assistant devices are all equipped equally with the Artemis client in the browser or the mobile app, and a Git client.

Deployment Overview (UML deployment diagram)

• Reverse proxy. nginx receives the HTTPS requests of the users and forwards HTTP and websocket requests to the core nodes.
• Core nodes run the Artemis server artifact with LocalVC and LocalCI. They keep their state in the database, in the shared file system (the repositories) and in the distributed data provider, so a load balancer can distribute requests without sticky sessions, and a node can be added without downtime. Several small nodes usually beat one large one at the same cost, because they fail independently, see Scaling.
• Exactly one core node has the scheduling profile. The server runs scheduled tasks, for example to lock repositories on the due date or to clean up unused resources. They must not run on every node. This node also sends the startup telemetry.
• Build agent nodes run the same artifact with the buildagent profile. They connect to the distributed data provider, clone repositories from LocalVC and run each build job in a Docker container. They can run on servers of the university or in a cloud, and are scaled independently of the core nodes. A core node can also build.
• Database server. PostgreSQL, which the core nodes access with Hibernate. MySQL is still supported until Artemis 11.
• Distributed data server. Only needed with the Redis provider. Hazelcast runs embedded in the core and build agent nodes.
• Broker node. Only needed with several core nodes, to relay websocket messages between them.

Rationale. The deployment separates what has a different load, a different risk and a different owner. The reverse proxy is the single entry point for browsers and apps, so the core nodes can be added and removed behind it without the users noticing. The database server, the distributed data server and the broker are separate nodes because each is operated, backed up and scaled by different means than the Artemis server. An institution can use a managed database or an existing Redis service instead of running its own. Build agents are separate from the core nodes because they run untrusted student code in containers and need access to the Docker daemon of their host. Keeping them off the nodes that serve the users limits what a faulty or malicious submission can reach, and the builds cannot take capacity from the user interface at a deadline. The Git clients connect to the core nodes, because LocalVC is part of the server, and the build agents clone from there as well. Everything that is optional in the diagram is left out in a small installation: one core node can run the scheduled tasks, build in Docker, use the local provider and the in-memory websocket broker. The distributed data server and the broker appear when the installation grows to several core nodes, which keeps the minimal installation to one application and one database (driver 1).

Data Model

The Artemis application server uses the following (simplified) data model in its relational database (PostgreSQL). It supports multiple courses with multiple exercises. Each student in the participating student group can participate in the exercise by clicking the Start Exercise button. Then a repository for the student (User) will be created and configured. With the deprecated Jenkins integration, Artemis additionally creates a build plan for each student. The initialization state helps to track the progress of this complex operation and allows recovering from errors. A student can submit multiple solutions by committing and pushing the source code changes to a given example code into the version control system or using the user interface. LocalCI automatically tests each submission and hands the result to the application server for grading. (An external CI system such as Jenkins notifies the application server when a new result exists.) In addition, teaching assistants can assess student solutions and "manually" create results.

Data Model

ℹ️

Information:

Please note, that the actual database model is more complex. The UML class diagram above omits some details for readability (e.g. lectures, student questions, exercise details, static code analysis, quiz questions, exam sessions, submission subclasses, etc.)

Server Architecture

The following UML component diagram shows the structure of the Artemis server. It is a layered architecture whose application layer is divided into feature modules.

Server Architecture (UML component diagram)

• The web layer exposes the server to the client. REST resources check the role of the user, and websocket topics are declared in code and sent through one messaging service.
• The application layer holds the logic. It is divided into feature modules such as exercise, programming, exam, assessment, communication and lecture, and into the Integrated Code Lifecycle with the modules localvc and localci.
• The data layer consists of Spring Data repositories. Transactions are declared in the repositories only, so that a service or a controller never decides about a transaction boundary.

Dependencies point downwards: resources use the application layer, which uses the repositories. The application layer reaches the database and the shared state of the installation (Distributed Data) through the data layer and the distributed data provider.

Rationale.

• Layers separate what changes at a different pace: the REST API, the domain logic and the persistence. Each layer can be understood, tested and changed without knowing the details of the others.
• The web layer is the border of trust. Every REST resource declares who may call it, and every websocket destination is a declared topic with an access rule that the sender uses. Nothing reaches the logic without passing a check, which matters because an institution exposes the server to all of its students.
• Feature modules keep a change local to one feature. Another module reaches a module only through its api package and never through its repositories, so a module can be changed internally and its dependencies are visible. Optional modules are reached through Optional<*Api>, so a module that an institution switches off does not break the others (driver 4).
• Repositories are the only access to the database. Queries and transactions live there and nowhere else, see the rationale for Hibernate in the table above. Services therefore cannot hold a transaction open across a remote call, and the cost of a query shows up in one place.
• Dependencies point downwards. Resources use the application layer, which uses the repositories. The application layer reaches the database and the shared state of the installation (Distributed Data) through the data layer and the distributed data provider, so a feature module never depends on Hibernate, Hazelcast or Redis directly.
• The rules are checked, not only written. The architecture tests (ArchUnit) check them on every change, because with many contributors over many years a convention that is not checked does not last (driver 5). See the server development guidelines for the details.

Integrated Code Lifecycle

The Integrated Code Lifecycle is the default and recommended way to run programming exercises. It consists of the integrated version control system (LocalVC) and the integrated continuous integration system (LocalCI). The only external CI system that Artemis still supports is Jenkins (combined with LocalVC), which is deprecated and removed in Artemis 11. An exemplary deployment with Integrated Code Lifecycle (without using an external user management system), consisting of one main application server and three build agent servers, looks like this:

Integrated Code Lifecycle Deployment

Employing the Integrated Code Lifecycle, administrators and developers can set the Artemis application up without the need for dedicated VCS and CIS installations. This architecture simplifies the setup process, reduces dependencies on external systems, and streamlines maintenance for both developers and administrators. Developers have fewer applications to run in parallel, which translates into decreased system requirements. See Integrated Code Lifecycle Setup Description on how to set up a single-node environment for developing purposes.

Version Control Subsystem

The following diagram shows an overview of the components in the version control subsystem:

Version Control Subsystem

The Local VC Service implements the VersionControlService interface and thus contains methods that the exercise management subsystem and the exercise participation subsystem need to interact with the VC system. E.g. the createRepository() method creates a repository on the file system. For users to be able to access the repositories using their integrated Git client, the integrated VC subsystem contains a Git Server component. It responds to fetch and push requests from Git clients, enabling instructors and students to interact with their repositories the way they are used to. It encompasses all the logic for implementing the Git HTTP protocol server-side. This includes extracting the command and parameters from the client request and executing the Git commands on the server-side repository, provided the repository exists, and the user has the requisite permissions. It reads objects and refs from the repository, updates the repository for push requests, and formats the results of the Git commands it executes into a response that it sends back to the client. This could involve sending objects and refs to the client in a packfile, or transmitting error messages. The Git Server delegates all logic connected to Artemis to the Local VC Servlet Service. This service resolves the repository from the file system depending on the repository URI. It also handles user authentication (only Basic Auth for now) and authorization. For authorization (e.g. "is the requesting user the owner of the repository?", "has the due date already passed?"), it uses the logic outsourced to the RepositoryAccessService that the existing online editor also uses. For push requests, the Local VC Servlet Service calls the processNewProgrammingSubmission() method of the Programming Submission Service to create a new submission and finally calls the integrated CI subsystem to trigger a new build.

Integrating the VC system into the Artemis server application improves performance. For instance, when an instructor creates a new programming exercise, Artemis needs to copy the template source code to the template repository. Using the integrated VCS, Artemis merely needs to communicate with the host file system, copying the files from one location in the file system to another, which is faster than communicating with the external VCS through the network.

Continuous Integration Subsystem

The following diagram shows an overview of the components in the integrated continuous integration subsystem:

Continuous Integration Subsystem

The integrated CIS consists of two further subsystems: the CI Management and the Build Agent. Both systems are decoupled and can be deployed on separate servers if necessary (not obligatory). This allows for a flexible scaling of the system, as we can deploy multiple build agent instances to handle a high number of build jobs.

CI Management

The following diagram shows an overview of the components in the CI Management subsystem:

CI Management Subsystem

The CI Management prepares information for build jobs and add them to the distributed build job queue, which lives in the distributed data provider (Hazelcast, Redis or local). It has complete access to the distributed data structures related to the CI system. It provides endpoints so users can interact with these data structures, such as viewing and cancelling build jobs. It also receives the build job results, grades them, and notifies the user. The CI Management has access to the database and the file system.

The CI Management subsystem implements the ContinuousIntegrationTriggerService interface, the LocalCITriggerService which provides the triggerBuild method. This method gets called whenever a repository needs to be tested, i.e. after creating a programming exercise or when a student submits code. When the triggerBuild method is called, all information necessary to execute the build job is prepared and used to create a BuildJobQueueItem object. The object contains, among other things, repository URIs, the build configuration, a user-defined build script (prepared by the BuildScriptProviderService) and a priority value (more details in the Priority Calculation section). The exercise due date is also taken into account. This object is then added to the job queue where it will be retrieved by a build agent to execute the build job. The following diagram shows the structure of the BuildJobQueueItem:

BuildJobQueueItem

The CI Management subsystem consists of two additional services: The SharedQueueManagementService and the Local CI Result Processing Service. The SharedQueueManagementService has direct access to the job queue as well as to other Hazelcast data structures, a map for currently running build jobs, a map for build agent information and a topic for cancelled build jobs. The service provides the functionality for an Artemis user to interact with build jobs and build agents. Build jobs can be viewed and cancelled. The user can access this functionality using the UI over a set of endpoints provided by a REST API. The LocalCIResultProcessingService retrieves the build job results which were generated by the build agents from the result queue. It is responsible for grading the build job results, notifying the user and persisting information on the build job execution in the database.

Additionally, the LocalCIEventListenerService is responsible for monitoring changes in build jobs and build agents to ensure job states remain accurate and users receive timely notifications. It registers event listeners for both the job queue and ongoing job processing, handling state transitions such as when a job starts or finishes. The service periodically checks for lost or stuck jobs, marking them as missing if necessary. This mechanism helps recover from issues like build agent crashes, network failures, or application restarts that could lead to inconsistencies in job tracking. To provide real-time feedback, the service triggers WebSocket updates. New event listeners should be registered within this service to maintain consistency in handling CI-related events. For simplicity, this service is not included in the diagram above.

Priority Calculation

Build jobs are assigned a priority value when they are added to the job queue. This value is used to determine the order in which build jobs are executed. The higher the priority value, the sooner the build job is executed.

The priority value is determined based on several factors related to the programming exercise and its context. Exercises that are part of an ongoing exam are given the highest priority of 1 to ensure quick feedback. Exam exercises that are part of a test exam or a test run within an exam are assigned a priority of 2. Submissions made after the due date, such as those in practice mode or finished exams, receive a lower priority of 3. Optional exercises, which are not included in the overall score, are also assigned a priority of 3. If the build is part of a batch, for example, if an instructor triggers a re-build of all submissions of an exercise, it is assigned the lowest priority of 4. If none of the above conditions apply, for example, exercises that are part of a real course and not part of an exam, the priority is set to 2. Additionally, if the exercise is part of a test course, a penalty is added to the priority, making it less urgent. The penalty is implemented by adding a value of 5 to the priority. The priority calculation is implemented in the LocalCITriggerService.

Build Agent

The following diagram shows an overview of the components in the Build Agent subsystem:

Build Agent Subsystem

The build agent is a separate subsystem that is responsible for executing build jobs. It can be run as a standalone application or as part of the main Artemis application. The build agent implements multiple services responsible for retrieving queued build jobs and executing them.

The SharedQueueProcessingService has direct access to the job queue and detects newly added build jobs. The job is then taken from the queue if the build agent currently has the capacity to execute the job. The service then makes an asynchronous method call to the BuildJobManagementService that eventually results in either a BuildJobResult or an exception if something went wrong during the build job processing. Either way, a ResultQueueItem object containing all necessary information about the build job execution is created and added to the result queue. The BuildJobManagementService contains the logic for managing build jobs. It prepares a build task as a lambda function and submits it to the ThreadPoolExecutor for execution. The task execution is handled by the BuildJobExecutionService, which provides the runBuildJob method containing the core logic for executing a build job. Once the build job completes, the result is returned to the BuildJobManagementService for further processing. If any error happens during the build job execution, it is propagated to the BuildJobManagementService, so it can handle all errors in one spot.

The SharedQueueProcessingService utilizes the BuildAgentInformationService to update build agent information and gather relevant monitoring data. This ensures that build agents are accurately tracked, including their status, number of running jobs, and performance metrics. The collected data is used to maintain an up-to-date view of the system.

To improve the reliability of the system, the BuildJobManagementService implements a timeout mechanism. Administrators can configure a maximum amount of time that build jobs can run by setting artemis.continuous-integration.build-timeout-seconds.max. The default value is 240 seconds. If a build job times out, the BuildJobManagementService interrupts the build job. This is crucial to prevent jobs that require an abnormally high amount of time from clogging up the system and reducing overall system performance.

A basic build job for the purpose of providing automated assessment in Artemis consists of the following steps:

• Check out the relevant repositories.
• Configure Docker container.
• Start a Docker container for the build job.
• Copy repositories into container
• Execute the build script in the container.
• Retrieve the test results from the container.
• Stop the container.
• Parse the test results.

To address potential security risks associated with executing student code during automated assessment, we run the build job in a container, that the BuildJobContainerService creates and starts just for this purpose. This container functions as an isolated environment. If a student submits potentially malicious code, the container confines its execution, preventing it from directly affecting the host system or other containers.

The ephemeral nature of Docker containers allows the BuildJobExecutionService to quickly remove them and the data they produced during the build when a build job finishes.

Finally, when the build ran through successfully, the SharedQueueProcessingService puts the build result into the result queue so it can then be processed by the CI Management. If there were any errors, the BuildJobManagementService stops the Docker container and SharedQueueProcessingService relays the exception message to the CI Management via the result queue.

Distributed Data Structures

Local CI keeps the build job queue, the running jobs and the build agent information in shared data structures that all core nodes and build agents access. Artemis does not use Hazelcast or Redis directly. It accesses them through the DistributedDataProvider abstraction, and the provider is selected with artemis.distributed-data.provider:

Provider	Use	Notes
Hazelcast	Single or multiple nodes	The default. Artemis nodes form a Hazelcast cluster and discover each other through the service registry. Data is kept in memory and is lost when the cluster stops.
Redis	Single or multiple nodes	Uses Redis or Valkey as an external store. Needs no service registry. Data persists across restarts, so the stored schema is versioned, see Distributed Data with Redis or Valkey.
Local	A single node only	Keeps the data in the memory of the node.

ℹ️

Information:

The maintainers are considering to deprecate the Hazelcast provider. No decision has been made yet. Code outside the provider implementations must not depend on Hazelcast, so that the provider can be replaced. The structures below use the Hazelcast type names; the Redis provider implements the same structures with versioned keys (artemis:v1:*).

The integration employs various distributed data structures to streamline the processing of build jobs across the network. These structures include the buildJobQueue, an IQueue used for queuing all build jobs awaiting processing; the processingJobs, an IMap tracking build jobs currently under processing; and the canceledBuildJobsTopic, a DistributedTopic utilized to signal build agents to cancel specific build jobs.

Here we can see an overview of all distributed data structures used in Local CI and their purpose (named after their Hazelcast types):

Name	Type	Description
buildJobQueue	IQueue	Queue for all build jobs that are waiting to be processed.
processingJobs	IMap	Map including all build jobs that are currently being processed.
buildAgentInformation	IMap	Map containing information on each build agent, such as its status and metrics.
canceledBuildJobsTopic	ITopic	Topic to order build agents to cancel a specific build job.
buildJobResultQueue	IQueue	Queue containing the results produced by the build job execution.
dockerImageCleanupInfo	IMap	Map containing information about the last date of use of Docker images.
pauseBuildAgentTopic	ITopic	Topic used to notify which build agent should be paused.
resumeBuildAgentTopic	ITopic	Topic used to notify which build agent should be resumed.

This approach ensures the efficient synchronization of build job processing. The CI Management also provides REST endpoints and supports WebSocket connections. These endpoints and real-time connections allow users to monitor and manage both queued and running jobs along with the build agents themselves, enhancing the system's operability and user interaction capabilities. Additionally, the processingJobs, buildJobQueue and canceledBuildJobsTopic are observed with listeners of the distributed data provider, enabling immediate detection of changes such as additions, removals, and updates. As previously mentioned, most of the listeners are added in LocalCIEventListenerService.

Build Job Cancellation

The build job cancellation mechanism is a crucial feature of the CI system. It allows users to cancel build jobs that are taking too long or are no longer needed. The following diagram shows an overview of the components in the build job cancellation mechanism:

Build Job Cancellation Class Diagram

The build job cancellation mechanism is implemented using the distributed data structures of the DistributedDataProvider, whichever provider is configured. As shown in the Build Job Cancellation Class Diagram we have services such as the SharedQueueManagementService that run on the core Artemis node and are responsible for managing these distributed data types. However, the real processing of each build job runs independently on each build agent. Therefore we have separate services such as the SharedQueueProcessingService in a build agent package.

The processing of build jobs gets triggered by the insertion or removal of jobs from the buildJobQueue. The queue listener in the SharedQueueProcessingService is then responsible for invoking the checkAvailabilityAndProcessNextBuild() method. If a build agent has available threads, the job is moved from the buildJobQueue to the processingJobs. Subsequently, the processBuild(BuildJobQueueItem buildJob) method is called, which in turn invokes executeBuildJob(BuildJobQueueItem buildJob) in the BuildJobManagementService. This method is tasked with asynchronously executing the build job in the BuildJobExecutionService, by creating a future. Upon creating the new future, it is added to a local Map<String, Future<LocalCIBuildResult>> runningFutures, which is unique to each build agent and contains only the futures of that agent. This future is then processed, with outcomes such as completion, failure, or cancellation being sent back and managed within the processBuild(...) method. This includes removing the build job from the processingJobs and initiating a new cycle by calling checkAvailabilityAndProcessNextBuild().

Build Job Cancellation Communication Diagram for a Processing Build Job

The above Communication Diagram sets the stage for understanding the cancellation mechanism. Upon receiving a cancellation request through the @DeleteMapping("build-jobs/{buildJobId}/cancel") endpoint in either the AdminBuildJobQueueResource or BuildJobQueueResource, the cancelBuildJob(String buildJobId) method within the SharedQueueManagementService is activated. Initially, it checks whether the build job is queued but not yet in process. If so, it is simply removed from the buildJobQueue. The process becomes more complex for jobs already in process, involving the canceledBuildJobsTopic DistributedTopic. A cancellation message for the build job ID is broadcast to all build agents, with the BuildJobManagementService listening for these messages through a listener on the topic of the distributed data provider. Every build agent then checks its runningFutures map to see if it holds the specific build job future. Should a build agent currently process the specified job, it executes cancelBuildJob(String buildJobId) within the BuildJobManagementService. This step involves cancelling the future and recording the job ID in Set<String> cancelledBuildJobs. The cancellation triggers an Exception which is caught and managed in processBuild(...) within the SharedQueueProcessingService. The build status is set to CANCELLED, the job is removed from processingJobs, and the process reinitialized with checkAvailabilityAndProcessNextBuild(). Modifications to processingJobs trigger WebSocket updates, consequently removing the cancelled job from the client's build overview display.