Registry Manager

Overview

Registry Manager is a command-line tool which can perform following tasks:

  • Create and delete registry indices in Elasticsearch.
  • Load JSON data files generated by Harvest tool into Registry.
  • Delete data from registry by lidvid, lid, package id.
  • Export files from registry BLOB storage.
  • Manage data dictionary.

Note: for elasticsearch available though defaults http ports (80, 443), explicit port is required whenever the -es option is used. e.g. `-es https//host:443`

Registry Manager executable scripts for Windows (registry-manager.bat) and Linux / Mac (registry-manager) are located in bin sub-folder of the installation directory (e.g., /home/pds/registry/).

To see a list of Registry Manager commands run registry-manager without any parameters.

To print command-specific usage information, pass -help parameter after any command. For example, to print usage information for delete-data command, run

registry-manager delete-data -help

Usage: registry-manager delete-data <options>

Delete data from registry index

Required parameters, one of:
  -lidvid <id>      Delete data by lidvid
  -lid <id>         Delete data by lid
  -packageId <id>   Delete data by package id
  -all              Delete all data
Optional parameters:
  -auth <file>      Authentication config file
  -es <url>         Elasticsearch URL. Default is http://localhost:9200
  -index <name>     Elasticsearch index name. Default is 'registry'

Registry Management Commands

Create Registry Index

To create registry index in Elasticsearch run create-registry command.

Usage: registry-manager create-registry <options>

Create registry index

Optional parameters:
  -auth <file>         Authentication config file
  -es <url>            Elasticsearch URL. Default is http://localhost:9200
  -index <name>        Elasticsearch index name. Default is 'registry'
  -schema <path>       Elasticsearch index schema (JSON file)
                       Default value is $REGISTRY_MANAGER_HOME/elastic/registry.json
  -shards <number>     Number of shards (partitions) for registry index. Default is 1
  -replicas <number>   Number of replicas (extra copies) of registry index. Default is 0

Delete Registry Index

To delete registry index from Elasticsearch run delete-registry command.

Usage: registry-manager delete-registry <options>

Delete registry index and all its data

Optional parameters:
  -auth <file>    Authentication config file
  -es <url>       Elasticsearch URL. Default is http://localhost:9200
  -index <name>   Elasticsearch index name. Default is 'registry'

Update Registry schema

To update Elasticsearch schema from a PDS data dictionary, run update-schema command:

Usage: registry-manager update-schema <options>

Update Elasticsearch schema

Required parameters:
  -file <path>     A file with a list of field names
Optional parameters:
  -auth <file>     Authentication config file
  -es <url>        Elasticsearch URL. Default is http://localhost:9200
  -index <name>    Elasticsearch index name. Default is 'registry'

This command is described in Registry Customization section.

Data Management Commands

Load Data

To load JSON data files generated by Harvest tool, use load-data command:

Usage: registry-manager load-data <options>

Load data into registry index

Required parameters:
  -dir <path>           Harvest output directory to load   
Optional parameters:
  -auth <file>          Authentication config file
  -es <url>             Elasticsearch URL. Default is http://localhost:9200
  -index <name>         Elasticsearch index name. Default is 'registry'
  -updateSchema <y/n>   Update registry schema. Default is 'yes'
  -force                Use 'keyword' ES datatype for missing field definitions

For example, to load data from /home/pds/harvest/out/ file use the following command.

registry-manager load-data -dir /home/pds/harvest/out/

Delete Data

You can delete documents from Registry by lidvid, lid or package id (Harvest run id). You can also delete all documents.

Usage: registry-manager delete-data <options>

Delete data from registry index

Required parameters, one of:
  -lidvid <id>      Delete data by lidvid
  -lid <id>         Delete data by lid
  -packageId <id>   Delete data by package id
  -all              Delete all data
Optional parameters:
  -auth <file>      Authentication config file
  -es <url>         Elasticsearch URL. Default is http://localhost:9200
  -index <name>     Elasticsearch index name. Default is 'registry'

Below are few examples.

registry-manager delete-data -lidvid urn:nasa:pds:context:target:asteroid.4_vesta::1.1
registry-manager delete-data -lid urn:nasa:pds:context:target:asteroid.4_vesta
registry-manager delete-data -packageId 8d8ae96d-044e-473d-a278-62635b1c5977
registry-manager delete-data -all

Export Data

You can export documents from Registry by lidvid or package id (Harvest run id). You can also export all documents.

Usage: registry-manager export-data <options>

Export data from registry index

Required parameters:
  -file <path>      Output file path
  -lidvid <id>      Export data by lidvid
  -packageId <id>   Export data by package id
  -all              Export all data
Optional parameters:
  -auth <file>      Authentication config file
  -es <url>         Elasticsearch URL. Default is http://localhost:9200
  -index <name>     Elasticsearch index name. Default is 'registry'

Below are few examples.

registry-manager export-data -file /tmp/mydata.xml -lidvid urn:nasa:pds:context:target:asteroid.4_vesta::1.1
registry-manager export-data -file /tmp/mydata.xml -packageId 8d8ae96d-044e-473d-a278-62635b1c5977
registry-manager export-data -file /tmp/mydata.xml -all

Update Archive Status

When documents are loaded into the Registry, their archive status is not set. You can update archive status of one or more documents by lidvid or package id (Harvest run id).

Usage: registry-manager set-archive-status <options>

Set product archive status

Required parameters:
  -status <status>   One of the following values:
     ARCHIVED
     ARCHIVED_ACCUMULATING
     IN_LIEN_RESOLUTION
     IN_LIEN_RESOLUTION_ACCUMULATING
     IN_PEER_REVIEW
     IN_PEER_REVIEW_ACCUMULATING
     IN_QUEUE
     IN_QUEUE_ACCUMULATING
     LOCALLY_ARCHIVED
     LOCALLY_ARCHIVED_ACCUMULATING
     PRE_PEER_REVIEW
     PRE_PEER_REVIEW_ACCUMULATING
     SAFED
     STAGED
     SUPERSEDED
  -lidvid <id>       Update archive status of a document with given lidvid, or
  -packageId <id>    Update archive status of all documents with given package id
Optional parameters:
  -auth <file>       Authentication config file
  -es <url>          Elasticsearch URL. Default is http://localhost:9200
  -index <name>      Elasticsearch index name. Default is 'registry'

Below are few examples.

registry-manager set-archive-status IN_QUEUE -lidvid urn:nasa:pds:context:target:asteroid.4_vesta::1.1
registry-manager set-archive-status ARCHIVED -packageId 8d8ae96d-044e-473d-a278-62635b1c5977

Export File (BLOB)

To export a file from embedded BLOB storage by lidvid, run export-file command:

Usage: registry-manager export-file <options>

Export a file from blob storage

Required parameters:
  -lidvid <id>    Lidvid of a file to export from blob storage
  -file <path>    Output file path
Optional parameters:
  -auth <file>    Authentication config file
  -es <url>       Elasticsearch URL. Default is http://localhost:9200
  -index <name>   Elasticsearch index name. Default is 'registry'

This command will only work, if file BLOBs were generated by Harvest. An example is shown below.

registry-manager export-file -lidvid urn:nasa:pds:context:target:asteroid.4_vesta::1.1 -file /tmp/4_vesta.xml

Data Dictionary Commands

List Data Dictionaries

To list data dictionaries loaded into Registry, use list-dd command:

Usage: registry-manager list-dd <options>

List data dictionaries

Optional parameters:
  -auth <file>      Authentication config file
  -es <url>         Elasticsearch URL. Default is http://localhost:9200
  -index <name>     Elasticsearch index name. Default is 'registry'
  -ns <namespace>   LDD namespace.

Example:

registry-manager list-dd
Namespace            File                                        Version   Date
-----------------------------------------------------------------------------------------------
cart                 PDS4_CART_1F00_1950.JSON                   1.15.0.0   2020-12-21T21:48:19Z
disp                 PDS4_DISP_1F00_1500.JSON                   1.15.0.0   2020-12-15T22:09:58Z
geom                 PDS4_GEOM_1F00_1910.JSON                   1.15.0.0   2021-01-12T00:37:40Z
img                  PDS4_IMG_1F00_1810.JSON                    1.15.0.0   2020-10-14T02:55:04Z
img_surface          PDS4_IMG_SURFACE_1F00_1240.JSON            1.15.0.0   2021-01-12T00:56:39Z
msn                  PDS4_MSN_1F00_1300.JSON                    1.15.0.0   2020-10-14T02:55:21Z
msn_surface          PDS4_MSN_SURFACE_1F00_1200.JSON            1.15.0.0   2020-10-14T02:55:29Z
particle             PDS4_PARTICLE_1G00_2010.JSON               1.16.0.0   2021-08-05T21:40:47Z
pds                  PDS4_PDS_1F00.JSON                         1.15.0.0   2020-12-23T15:16:28Z
proc                 PDS4_PROC_1F00_1210.JSON                   1.15.0.0   2020-12-09T03:22:22Z
rings                PDS4_RINGS_1F00_1A00.JSON                  1.15.0.0   2020-12-02T19:08:01Z
sp                   PDS4_SP_1F00_1300.JSON                     1.15.0.0   2020-11-03T19:47:46Z

Load Data Dictionary

To load data dictionary files, use load-dd command:

Usage: registry-manager load-dd <options>

Load data dictionary

Required parameters, one of:
  -dd <path>         PDS4 LDD data dictionary file (JSON)
  -dump <path>       Data dump created by 'export-dd' command (NJSON)
  -csv <path>        Custom data dictionary file in CSV format
Optional parameters:
  -auth <file>       Authentication config file
  -es <url>          Elasticsearch URL. Default is http://localhost:9200
  -index <name>      Elasticsearch index name. Default is 'registry'
  -ns <namespaces>   LDD namespace. Can be used with -dd parameter.

This command is described in Registry Customization section.

Delete Data Dictionary

You can delete documents from data dictionary by id or namespace. You can also delete all documents.

Usage: registry-manager delete-dd <options>

Delete data from data dictionary index

Required parameters, one of:
  -id <id>          Delete data by ID (Full field name)
  -ns <namespace>   Delete data by namespace
  -all              Delete all data
Optional parameters:
  -auth <file>      Authentication config file
  -es <url>         Elasticsearch URL. Default is http://localhost:9200
  -index <name>     Elasticsearch index name. Default is 'registry'

Below are few examples.

registry-manager delete-dd -id proc/Parameter/proc/value
registry-manager delete-dd -ns orex
registry-manager delete-dd -all

Export Data Dictionary

You can export all documents from data dictionary.

Usage: registry-manager export-dd <options>

Export data from registry data dictionary

Required parameters:
  -file <path>      Output file path
Optional parameters:
  -auth <file>      Authentication config file
  -es <url>         Elasticsearch URL. Default is http://localhost:9200
  -index <name>     Elasticsearch index name. Default is 'registry'

Example:

registry-manager export-dd -file /tmp/my-dd.json