6.5 WORKFLOW (NxWorkflow)

The workflow service is loaded at system boot and simply scans the nxn_workflow table for enabled workflow jobs. Should any be within their allotted time (start time and run date etc.) then they will be executed in exactly the same way as a NxCore job would.

NxWorkflowEngine -Q <jobID>

This time the id refers to the workflow Id within its listing.

The workflow engine acts upon the rules of the workflow specified in the table entry. It creates a temporary table named nxn_mirror_<N> where N specifies the Id number of the workflow. Within this table is stored the objects that the workflow finds plus their state in the process.

In order to do its work, the engine performs MD5 checksums on objects it discovers and compares them with a list of known objects. This allows it to detect objects that have been altered since they were last examined.

The various types of workflows are described below:

  • Scan - This type of workflow will instruct the engine to ‘Scan’ over a filesystem periodically to ascertain if any new media has been added. If new objects are detected they are added to the mirror table for processing. Usually scanning a filesystem results in all objects found being sent to a ‘Destination’ of some sort, whether that be Tape, S3, ALTO disk or any other form of storage. Media can also be duplicated using the simply suffix mechanism described in groups. This is done automatically via the NxCore engine simply by creating multiple groups, there is no need to specify anything within the workflow other than the initial target group.
  • Watch - This workflow sets up a filesystem ‘Watch’ on some operating systems that support it. This flow is not supported on all sources but can be useful as it consumes less overhead than a Scan. However, it can be less accurate than a scan as Watches can be cancelled without files being detected.
  • Copy - This type of workflow copies media (including its metadata) from a source to a target. This type of workflow is useful when making copies of tapes which were not initially duplicated. A copy will examine the source ‘Group’ of media and copy everything that is not currently contained on the ‘Target’ group of media.
  • Mirror - A mirror is similar to a copy other than it will copy media in both directions to guarantee whatever is contained on source media is on target media and vice versa.
  • Migrate - This is effectively a move from one media to another. Useful for moving all assets from one type of media to another. The migrate also includes any metadata associated with the objects that are moved.

Workflows can be executed as many times as required. Some workflows are set to execute on a single occasion whilst others can be set to run multiple times at an interval. Workflows can also specify what type of media to look for ( *.mxf, *.mpg ) as well as how much data should be included into each dataset or job sent to archive.