Registry Manager
Overview
Registry Manager is a command-line tool which can perform following tasks:
- Create and delete registry indices in Elasticsearch.
- Load JSON data files generated by Harvest tool into Registry.
- Delete data from registry by lidvid, lid, package id.
- Export files from registry BLOB storage.
- Manage data dictionary.
Note: for elasticsearch available though defaults http ports (80, 443), explicit port is required whenever the -es option is used. e.g. `-es https//host:443`
Registry Manager executable scripts for Windows (registry-manager.bat) and Linux / Mac (registry-manager) are located in bin sub-folder of the installation directory (e.g., /home/pds/registry/).
To see a list of Registry Manager commands run registry-manager without any parameters.
To print command-specific usage information, pass -help parameter after any command. For example, to print usage information for delete-data command, run
registry-manager delete-data -help
Usage: registry-manager delete-data <options> Delete data from registry index Required parameters, one of: -lidvid <id> Delete data by lidvid -lid <id> Delete data by lid -packageId <id> Delete data by package id -all Delete all data Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry'
Registry Management Commands
Create Registry Index
To create registry index in Elasticsearch run create-registry command.
Usage: registry-manager create-registry <options> Create registry index Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry' -schema <path> Elasticsearch index schema (JSON file) Default value is $REGISTRY_MANAGER_HOME/elastic/registry.json -shards <number> Number of shards (partitions) for registry index. Default is 1 -replicas <number> Number of replicas (extra copies) of registry index. Default is 0
Delete Registry Index
To delete registry index from Elasticsearch run delete-registry command.
Usage: registry-manager delete-registry <options> Delete registry index and all its data Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry'
Update Registry schema
To update Elasticsearch schema from a PDS data dictionary, run update-schema command:
Usage: registry-manager update-schema <options> Update Elasticsearch schema Required parameters: -file <path> A file with a list of field names Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry'
This command is described in Registry Customization section.
Data Management Commands
Load Data
To load JSON data files generated by Harvest tool, use load-data command:
Usage: registry-manager load-data <options> Load data into registry index Required parameters: -dir <path> Harvest output directory to load Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry' -updateSchema <y/n> Update registry schema. Default is 'yes' -force Use 'keyword' ES datatype for missing field definitions
For example, to load data from /home/pds/harvest/out/ file use the following command.
registry-manager load-data -dir /home/pds/harvest/out/
Delete Data
You can delete documents from Registry by lidvid, lid or package id (Harvest run id). You can also delete all documents.
Usage: registry-manager delete-data <options> Delete data from registry index Required parameters, one of: -lidvid <id> Delete data by lidvid -lid <id> Delete data by lid -packageId <id> Delete data by package id -all Delete all data Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry'
Below are few examples.
registry-manager delete-data -lidvid urn:nasa:pds:context:target:asteroid.4_vesta::1.1 registry-manager delete-data -lid urn:nasa:pds:context:target:asteroid.4_vesta registry-manager delete-data -packageId 8d8ae96d-044e-473d-a278-62635b1c5977 registry-manager delete-data -all
Export Data
You can export documents from Registry by lidvid or package id (Harvest run id). You can also export all documents.
Usage: registry-manager export-data <options> Export data from registry index Required parameters: -file <path> Output file path -lidvid <id> Export data by lidvid -packageId <id> Export data by package id -all Export all data Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry'
Below are few examples.
registry-manager export-data -file /tmp/mydata.xml -lidvid urn:nasa:pds:context:target:asteroid.4_vesta::1.1 registry-manager export-data -file /tmp/mydata.xml -packageId 8d8ae96d-044e-473d-a278-62635b1c5977 registry-manager export-data -file /tmp/mydata.xml -all
Update Archive Status
When documents are loaded into the Registry, their archive status is not set. You can update archive status of one or more documents by lidvid or package id (Harvest run id).
Usage: registry-manager set-archive-status <options> Set product archive status Required parameters: -status <status> One of the following values: ARCHIVED ARCHIVED_ACCUMULATING IN_LIEN_RESOLUTION IN_LIEN_RESOLUTION_ACCUMULATING IN_PEER_REVIEW IN_PEER_REVIEW_ACCUMULATING IN_QUEUE IN_QUEUE_ACCUMULATING LOCALLY_ARCHIVED LOCALLY_ARCHIVED_ACCUMULATING PRE_PEER_REVIEW PRE_PEER_REVIEW_ACCUMULATING SAFED STAGED SUPERSEDED -lidvid <id> Update archive status of a document with given lidvid, or -packageId <id> Update archive status of all documents with given package id Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry'
Below are few examples.
registry-manager set-archive-status IN_QUEUE -lidvid urn:nasa:pds:context:target:asteroid.4_vesta::1.1 registry-manager set-archive-status ARCHIVED -packageId 8d8ae96d-044e-473d-a278-62635b1c5977
Export File (BLOB)
To export a file from embedded BLOB storage by lidvid, run export-file command:
Usage: registry-manager export-file <options> Export a file from blob storage Required parameters: -lidvid <id> Lidvid of a file to export from blob storage -file <path> Output file path Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry'
This command will only work, if file BLOBs were generated by Harvest. An example is shown below.
registry-manager export-file -lidvid urn:nasa:pds:context:target:asteroid.4_vesta::1.1 -file /tmp/4_vesta.xml
Data Dictionary Commands
List Data Dictionaries
To list data dictionaries loaded into Registry, use list-dd command:
Usage: registry-manager list-dd <options> List data dictionaries Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry' -ns <namespace> LDD namespace.
Example:
registry-manager list-dd
Namespace File Version Date ----------------------------------------------------------------------------------------------- cart PDS4_CART_1F00_1950.JSON 1.15.0.0 2020-12-21T21:48:19Z disp PDS4_DISP_1F00_1500.JSON 1.15.0.0 2020-12-15T22:09:58Z geom PDS4_GEOM_1F00_1910.JSON 1.15.0.0 2021-01-12T00:37:40Z img PDS4_IMG_1F00_1810.JSON 1.15.0.0 2020-10-14T02:55:04Z img_surface PDS4_IMG_SURFACE_1F00_1240.JSON 1.15.0.0 2021-01-12T00:56:39Z msn PDS4_MSN_1F00_1300.JSON 1.15.0.0 2020-10-14T02:55:21Z msn_surface PDS4_MSN_SURFACE_1F00_1200.JSON 1.15.0.0 2020-10-14T02:55:29Z particle PDS4_PARTICLE_1G00_2010.JSON 1.16.0.0 2021-08-05T21:40:47Z pds PDS4_PDS_1F00.JSON 1.15.0.0 2020-12-23T15:16:28Z proc PDS4_PROC_1F00_1210.JSON 1.15.0.0 2020-12-09T03:22:22Z rings PDS4_RINGS_1F00_1A00.JSON 1.15.0.0 2020-12-02T19:08:01Z sp PDS4_SP_1F00_1300.JSON 1.15.0.0 2020-11-03T19:47:46Z
Load Data Dictionary
To load data dictionary files, use load-dd command:
Usage: registry-manager load-dd <options> Load data dictionary Required parameters, one of: -dd <path> PDS4 LDD data dictionary file (JSON) -dump <path> Data dump created by 'export-dd' command (NJSON) -csv <path> Custom data dictionary file in CSV format Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry' -ns <namespaces> LDD namespace. Can be used with -dd parameter.
This command is described in Registry Customization section.
Delete Data Dictionary
You can delete documents from data dictionary by id or namespace. You can also delete all documents.
Usage: registry-manager delete-dd <options> Delete data from data dictionary index Required parameters, one of: -id <id> Delete data by ID (Full field name) -ns <namespace> Delete data by namespace -all Delete all data Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry'
Below are few examples.
registry-manager delete-dd -id proc/Parameter/proc/value registry-manager delete-dd -ns orex registry-manager delete-dd -all
Export Data Dictionary
You can export all documents from data dictionary.
Usage: registry-manager export-dd <options> Export data from registry data dictionary Required parameters: -file <path> Output file path Optional parameters: -auth <file> Authentication config file -es <url> Elasticsearch URL. Default is http://localhost:9200 -index <name> Elasticsearch index name. Default is 'registry'
Example:
registry-manager export-dd -file /tmp/my-dd.json