Registry Manager Overview

Overview

Registry Manager is a command-line tool which can perform following tasks:

  • Create and delete registry indices in Elasticsearch.
  • Load JSON data files generated by Harvest tool into Registry.
  • Delete data from registry by lidvid, lid, package id.
  • Export files from registry BLOB storage.
  • Manage data dictionary.

Note: for elasticsearch available though defaults http ports (80, 443), explicit port is required whenever the -es option is used. e.g. `-es https//host:443`

Registry Manager executable scripts for Windows (registry-manager.bat) and Linux / Mac (registry-manager) are located in bin sub-folder of the installation directory (e.g., /home/pds/registry/).

To see a list of Registry Manager commands run registry-manager without any parameters.

To print command-specific usage information, pass -help parameter after any command. For example, to print usage information for delete-data command, run

registry-manager delete-data -help

Usage: registry-manager delete-data <options>

Delete data from registry index

Required parameters, one of:
  -lidvid <id>      Delete data by lidvid
  -lid <id>         Delete data by lid
  -packageId <id>   Delete data by package id
  -all              Delete all data
Optional parameters:
  -auth <file>      Authentication config file
  -es <url>         Elasticsearch URL. Default is http://localhost:9200
  -index <name>     Elasticsearch index name. Default is 'registry'

Registry Management Commands

Create Registry Index

To create registry index in Elasticsearch run create-registry command.

Usage: registry-manager create-registry <options>

Create registry index

Optional parameters:
  -auth <file>         Authentication config file
  -es <url>            Elasticsearch URL. Default is http://localhost:9200
  -index <name>        Elasticsearch index name. Default is 'registry'
  -schema <path>       Elasticsearch index schema (JSON file)
                       Default value is $REGISTRY_MANAGER_HOME/elastic/registry.json
  -shards <number>     Number of shards (partitions) for registry index. Default is 1
  -replicas <number>   Number of replicas (extra copies) of registry index. Default is 0

Delete Registry Index

To delete registry index from Elasticsearch run delete-registry command.

Usage: registry-manager delete-registry <options>

Delete registry index and all its data

Optional parameters:
  -auth <file>    Authentication config file
  -es <url>       Elasticsearch URL. Default is http://localhost:9200
  -index <name>   Elasticsearch index name. Default is 'registry'

Update Registry schema

To update Elasticsearch schema from a PDS data dictionary, run update-schema command:

Usage: registry-manager update-schema <options>

Update Elasticsearch schema

Required parameters:
  -file <path>     A file with a list of field names
Optional parameters:
  -auth <file>     Authentication config file
  -es <url>        Elasticsearch URL. Default is http://localhost:9200
  -index <name>    Elasticsearch index name. Default is 'registry'

This command is described in Registry Customization section.

Data Management Commands

Load Data

To load JSON data files generated by Harvest tool, use load-data command:

Usage: registry-manager load-data <options>

Load data into registry index

Required parameters:
  -dir <path>           Harvest output directory to load   
Optional parameters:
  -auth <file>          Authentication config file
  -es <url>             Elasticsearch URL. Default is http://localhost:9200
  -index <name>         Elasticsearch index name. Default is 'registry'
  -updateSchema <y/n>   Update registry schema. Default is 'yes'

For example, to load data from /home/pds/harvest/out/ file use the following command.

registry-manager load-data -dir /home/pds/harvest/out/

Delete Data

You can delete documents from Registry by lidvid, lid or package id (Harvest run id). You can also delete all documents.

Usage: registry-manager delete-data <options>

Delete data from registry index

Required parameters, one of:
  -lidvid <id>      Delete data by lidvid
  -lid <id>         Delete data by lid
  -packageId <id>   Delete data by package id
  -all              Delete all data
Optional parameters:
  -auth <file>      Authentication config file
  -es <url>         Elasticsearch URL. Default is http://localhost:9200
  -index <name>     Elasticsearch index name. Default is 'registry'

Below are few examples.

registry-manager delete-data -lidvid urn:nasa:pds:context:target:asteroid.4_vesta::1.1
registry-manager delete-data -lid urn:nasa:pds:context:target:asteroid.4_vesta
registry-manager delete-data -packageId 8d8ae96d-044e-473d-a278-62635b1c5977
registry-manager delete-data -all

Export Data

You can export documents from Registry by lidvid or package id (Harvest run id). You can also export all documents.

Usage: registry-manager export-data <options>

Export data from registry index

Required parameters:
  -file <path>      Output file path
  -lidvid <id>      Export data by lidvid
  -packageId <id>   Export data by package id
  -all              Export all data
Optional parameters:
  -auth <file>      Authentication config file
  -es <url>         Elasticsearch URL. Default is http://localhost:9200
  -index <name>     Elasticsearch index name. Default is 'registry'

Below are few examples.

registry-manager export-data -file /tmp/mydata.xml -lidvid urn:nasa:pds:context:target:asteroid.4_vesta::1.1
registry-manager export-data -file /tmp/mydata.xml -packageId 8d8ae96d-044e-473d-a278-62635b1c5977
registry-manager export-data -file /tmp/mydata.xml -all

Update Archive Status

When documents are loaded into the Registry, their archive status is not set. You can update archive status of one or more documents by lidvid or package id (Harvest run id).

Usage: registry-manager set-archive-status <options>

Set product archive status

Required parameters:
  -status <status>   One of the following values:
     ARCHIVED
     ARCHIVED_ACCUMULATING
     IN_LIEN_RESOLUTION
     IN_LIEN_RESOLUTION_ACCUMULATING
     IN_PEER_REVIEW
     IN_PEER_REVIEW_ACCUMULATING
     IN_QUEUE
     IN_QUEUE_ACCUMULATING
     LOCALLY_ARCHIVED
     LOCALLY_ARCHIVED_ACCUMULATING
     PRE_PEER_REVIEW
     PRE_PEER_REVIEW_ACCUMULATING
     SAFED
     STAGED
     SUPERSEDED
  -lidvid <id>       Update archive status of a document with given lidvid, or
  -packageId <id>    Update archive status of all documents with given package id
Optional parameters:
  -auth <file>       Authentication config file
  -es <url>          Elasticsearch URL. Default is http://localhost:9200
  -index <name>      Elasticsearch index name. Default is 'registry'

Below are few examples.

registry-manager set-archive-status IN_QUEUE -lidvid urn:nasa:pds:context:target:asteroid.4_vesta::1.1
registry-manager set-archive-status ARCHIVED -packageId 8d8ae96d-044e-473d-a278-62635b1c5977

Export File (BLOB)

To export a file from embedded BLOB storage by lidvid, run export-file command:

Usage: registry-manager export-file <options>

Export a file from blob storage

Required parameters:
  -lidvid <id>    Lidvid of a file to export from blob storage
  -file <path>    Output file path
Optional parameters:
  -auth <file>    Authentication config file
  -es <url>       Elasticsearch URL. Default is http://localhost:9200
  -index <name>   Elasticsearch index name. Default is 'registry'

This command will only work, if file BLOBs were generated by Harvest. An example is shown below.

registry-manager export-file -lidvid urn:nasa:pds:context:target:asteroid.4_vesta::1.1 -file /tmp/4_vesta.xml

Data Dictionary Commands

Load Data Dictionary

To load data dictionary files, use load-dd command:

Usage: registry-manager load-dd <options>

Load data dictionary

Required parameters, one of:
  -dd <path>         Standard PDS4 data dictionary file (JSON)
  -dump <path>       Data dump created by 'export-dd' command (NJSON)
  -csv <path>        Custom data dictionary file in CSV format
Optional parameters:
  -auth <file>       Authentication config file
  -es <url>          Elasticsearch URL. Default is http://localhost:9200
  -index <name>      Elasticsearch index name. Default is 'registry'
  -ns <namespaces>   Comma separated list of namespaces. Can be used with -dd parameter.

This command is described in Registry Customization section.

Delete Data

You can delete documents from data dictionary by id or namespace. You can also delete all documents.

Usage: registry-manager delete-dd <options>

Delete data from data dictionary index

Required parameters, one of:
  -id <id>          Delete data by ID (Full field name)
  -ns <namespace>   Delete data by namespace
  -all              Delete all data
Optional parameters:
  -auth <file>      Authentication config file
  -es <url>         Elasticsearch URL. Default is http://localhost:9200
  -index <name>     Elasticsearch index name. Default is 'registry'

Below are few examples.

registry-manager delete-dd -id proc/Parameter/proc/value
registry-manager delete-dd -ns orex
registry-manager delete-dd -all

Export Data

You can export all documents from data dictionary.

Usage: registry-manager export-dd <options>

Export data from registry data dictionary

Required parameters:
  -file <path>      Output file path
Optional parameters:
  -auth <file>      Authentication config file
  -es <url>         Elasticsearch URL. Default is http://localhost:9200
  -index <name>     Elasticsearch index name. Default is 'registry'

Example:

registry-manager export-dd -file /tmp/my-dd.json