15. Database Structure and Schema: MySQL
Table | Key Fields | Description |
nxn_archives | `id`, `archiveId`, `volumeId`, `volumeName`, `archiveNo`, `description`, `created`, `size`, `fileCount`, `coreVersion`, `footerPosition`, `absolutePosition`, `completed`, `product` | The archive table contains a list of archives, which contain assets and below which are stored objects. So, typical data storage would consist of: archive->asset->objects |
nxn_assets | `id`, `vid`, `archNo`, `auid`, `displayName`, `archivePosition`, `absolutePosition`, `rootPath`, `type`, `sourceSystem`, `totalSize`, `deleted`, `deletedBy`, `archiveId` | Each object (or file) is contained within an asset. Assets are usually specified by MAM systems such as Avid or Stratus as a container for related objects. |
nxn_authentications | `id`, `resourceName`, `username`, `accessKey`, `type`, `password`, `secretKey`, `location`, `class` | Authentications is a temp table which can be used by processes to read volume, NAS or filesystem authentications required. |
nxn_changer_requests | `id`, `volumeName`, `driveName`, `driveAddress`, `completionTime`, `rid`, `status`, `process`, `message`, `changer`, `fromLocation`, `toLocation`, `serverName`, `pid`, `mountVolume`, `mountPoint`, `jobID` | This table is a simple list of all requests for NxChanger to perform work on. It lists all requests to move media in the attached robotic devices. There may be up to 256 such devices connected to the system, although more than 4 is unusual. |
nxn_changers | `id`, `name`, `slotCount`, `driveCount`, `eePortCount`, `slotStart`, `driveStart`, `armAddress`, `eePortStart`, `vendor`, `product`, `revision`, `serialNo`, `deviceName`, `enabled` | This is a table specifying which changers have been detected and configured on the system. It is used by the changer process (NxChanger) to determine serial numbers and device allocations, and to determine a changers state. |
nxn_details | `id`, `jobID`, `uid`, `path`, `size`, `partialStart`, `partialEnd`, `partialType`, `type`, `assetID`, `displayName`, `status`, `message`, `volid`, `archNo`, `absolutePos` | This table is linked via the jobID to the nxn_queue. It details the files involved with the particular job. Any job is broken up into an archive, which contains assets (usually only 1 asset) and each asset contains objects (files). The details table lists these ‘objects’ and their status for each jobID processed. |
nxn_drives | `id`, `name`, `map`, `enabled`, `Inquiry`, `serialNo`, `changer`, `location`, `loadedVolume`, `status`, `locked`, `pid`, `serverName`, `jobID`, `mountPoint`, `isMounted`, `generation` | This table contains a list of all drives (tape or otherwise) within the system, the queue uses this table to determine the availability and type of drive which would be required to execute any given job. For example, an archive is required for a group: ‘NxMedia’. This group contains LTO8 media. For this job to succeed there will need to be an LTO8 tape drive within the robotics where the volumes are held, which is free (not in use by another job). |
nxn_groups | `id`, `name`, `readPriority`, `writePriority`, `readLimit`, `writeLimit`, `verification`, `autoAddVolume`, `targetSystem`, `changer` | This table lists the names of volume ‘grouping’. A group is simply a logical grouping of media to which data can be sent. It is merely a representation of the media under a single name. Archive jobs use group names to specify which media might be used. It is up to the queue to determine if within a group there is a volume, which is not in use yet, has enough capacity remaining and has not been set ‘disabled’ or ‘read-only’. |
nxn_license | `customerName`, `licenceXML`, `version`, `licenceDate` | This table contains licence information which all processes can look up in order to ascertain which product is licensed and thus able to run. |
nxn_locations | `id`, `path`, `server`, `user`, `passwd`, `accessType` | This is a list of disk / storage locations held. These are used by NxCore when attempting to access certain disk or S3 locations that might require a login information of some sort. For example, S3 locations require Secret Keys, usernames, and a ‘location’ identity. |
nxn_mirror_# | `id`, `stageID`, `object`, `objectType`, `objectStatus`, `checksum`, `sourceID`, `targetID`, `size`, `qid`, `ouid` | This is dynamically created by the NxWorkflowEngine when processing data on its particular job. When a workflow is created it is designated a row ID in the database, it is this ID that is used to name the mirror table and insert data into. NxWorkflowEngine processes requests and uses this dynamic table it creates to insert files and objects it friends and subsequently processes. (There many be many of these tabled in the database. nxn_mirror_ Followed by a number.) |
nxn_mount_points | `id`, `mountPoint`, `mountName`, `serverName`, `user`, `password` | This table stores the mount point information. |
nxn_objects | `id`, `ouid`, `auid`, `volid`, `archNo`, `archiveId`, `deleted`, `restoreCount`, `deletedBy`, `absolutePosition`, `archivePosition`, `modificationTime`, `accessTime`, `filename`, `size`, `checksum`, `originalPath`, `accessSpeed`, `linkedID`, `managementID`, `displayName` | Is the table containing all objects stored within the library and their metadata such as size, volume location, name, uid and so on. manage the objects which relate to archive. |
nxn_objects_metadata | `id`, `ouid`, `dataName`, `dataValue` | one object can have many metadata, each of meta-data include key-value. |
nxn_queue | `jobID`, `assetID`, `pid`, `driveID`, `machineID`, `displayName`, `operation`, `priority`, `startTime`, `status`, `size`, `sourceMedia`, `targetMedia`, `archiveNumber`, `bytesProcessed`, `errorMsg`, `sourceGroup`, `targetGroup`, `information`, `CompletionTime`, `targetSystem`, `targetAPI`, `dataRate`, `lastUpdate`, `rid`, `errorCode`, `jobState`, `sessionID`, `logMessage`, `workflowID`, `workflowStage` | Contains a row for each ‘job’ being processed. The state of each job is kept along with various other statistical information, including speed, job size, process id of the running core, drive in use, display name of the task, priority and so on. This included the information about workflow, Asset, … |
nxn_scsi_details | `id`, `server`, `scsiAddress`, `serialNo`, `deviceType`, `changer` | This table is joined with the drives file to specify on each node within a cluster how to access a particular device. It is a map as each node within a cluster may not necessarily see devices with the same name. For example, on node 1 the tape device LTO8 may be seen as /dev/sg15 where on Node 2 this may be accessed through /dev/sg4. |
nxn_scsi_devices | `id`, `serverName`, `deviceAddress`, `name`, `vendor`, `serialNo`, `revNo`, `type` | This is used as a temporary table when scanning the SCSI bus to store all devices found, NxScsi uses this information to update the main tables of the devices found on the bus. |
nxn_servers | `id`, `node_name`, `enabled`, `active`, `priority`, `changer_service`, `cs_master`, `queue_service`, `qs_master`, `rest_service`, `rs_master`, `workflow_service`, `wf_master`, `process_load`, `pause` | This table keeps the information pertinent to each server, an multiple servers if in a cluster. |
nxn_settings | `name`, `value` | This table simply keeps a list of useful settings for other utilities to take note of. An example of a useful setting would be the ‘duplicate’ or ‘triplicate’ suffix. This specifies to the REST API if a particular archive object should be duplicated and if so to which group. An example of this duplication (detailed later) would be having a duplication suffix of ‘B’. This indicates to NxREST that should a job be sent to archive material to group ‘NxMedia’ then should another group be present called ‘NxMedia_B’ then this media should also be sent to this group simultaneously. Another useful setting is the ‘PortNo’ which specifies which port the NxUI (User Interface) web service should listen for connections. By default, this is set to 8186. |
nxn_users | `id`, `name`, `password`, `description`, `role`, `lastLogin` | This table holds user authentication information for NxUI to utilize when allowing access to the system. This authentication system is yet to be written. |
nxn_volumes | `vid`, `name`, `groupName`, `description`, `enabled`, `originalLocation`, `currentLocation`, `inUse`, `jobID`, `capacity`, `generation`, `errors`, `spaceLeft`, `changer`, `modificationBy`, `serverName`, `pid`, `systemType`, `lastFormat`, `originalId`, `directAccess`, `uuid` | This table contains a list of all ‘volumes’ within the system, whether they be tape, disk, ALTO or S3 etc. The queue uses the table to ascertain whether a certain job might succeed if it were executed. In order to do this the queue looks at the locked status, capacity and group within which the volume is contained. All of which is available from this table. |
nxn_workflows | `id`, `description`, `enabled`, `sourceMedia`, `sourceMediaType`, `targetMedia`, `targetMediaType`, `taskType`, `execStart`, `spareColumn`, `frequency`, `mediaTypes`, `maximumJobSize`, `maximumQueueSize`, `jobPriority`, `displayName`, `lastExecDate`, `runCount`, `pid`, `status`, `machineID`, `totalData`, `dataProcessed`, `totalObjects`, `objectsProcessed`, `information`, `scanComplete`, `processComplete`, `workflowID`, `estimatedCompletionDate`, `dataRate` | This table stores the workflows set up by the user from the UI. It is polled by NxWorkflow, which is a Linux service. If a workflow is destined to be processed, then it creates a NxWorkflowEngine process which will examine that entry and perform work based upon it. |
Relationships: nxn_changer_requests.changer_id → nxn_drives.drive_id; nxn_assets.object_id → nxn_objects.object_id; nxn_objects.volume_id → nxn_drives.drive_id; nxn_objects_metadata.object_id → nxn_objects.object_id