6.1 Queue (NxQueue)

NxQueue is responsible for analyzing all work added to the database via various other pieces of software (specified later). This is the ‘center’ of the archive system, controlling what media is moved - when and how. The queue is responsible for resources such as drive availability, disk status, remaining capacity, and group membership. Its logic is highly complex and not always obvious, it will require a great deal of thought to fully understand what it does.

NOTE: Within a clustered environment all services run (are executed by systemd) on all servers. It is for the services themselves to decide who is ‘master’ and who are ‘clients’, this is decided by the ordering within the servers table (detailed below).

NxN_Queue runs as a service on each node in the cluster, currently it loops (or polls) the database for new work added. This should be improved by using a message queuing service in the future such that it only performs work when needed. This will remove some of the stress on the database needlessly reading tables when no more work can be started.

The service is run from boot by adding a service contract to Linux. This is added via the setup procedure and adds the NxQueue.service file to /etc/systemd/system. At boot each service specified in this directory will be executed.

Once running, NxQueue checks currently ‘running’ processes from within the nxn_queue table to see that they are indeed still running or have since terminated from last boot. Next it loads the current list of all volumes, drives and servers to make calculations about what can be executed.

It reads the nxn_queue table to find jobs that are in the ‘Ready’ state and then calculates if a ready job could be executed, given the resource limitations of:

If the job is an

  • Archive
  • Restore
  • Format
  • Other process
    1. If the requisite volume would be available for such an action
    2. If a drive would be available (should the request involve tape)
    3. If a server is able to process the job
    4. If all resources are available, the job is set to ‘launched’ within the table.

This does not mean it will be executed immediately as the node on which the queue is ‘master’ may not be the node within the cluster that the job is destined to be run on. Another NxQueue on the ‘target’ server will pick this job up and execute it when it sees the job set for it.

- Related Tables in Database:

  • nxn_queue
  • nxn_details
  • nxn_volumns
  • nxn_drives
  • nxn_groups
  • nxn_servers

(see detail in the below Database Structure)