org.apache.nifi.atlas

Class NiFiAtlasClient

java.lang.Object

org.apache.nifi.atlas.NiFiAtlasClient

All Implemented Interfaces:

AutoCloseable

public class NiFiAtlasClient
extends Object
implements AutoCloseable

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.atlas.AtlasClientV2	atlasClient
private Pattern	FLOW_PATH_URL_PATTERN
private static org.slf4j.Logger	logger

Constructor Summary

Constructors

Constructor and Description
NiFiAtlasClient(org.apache.atlas.AtlasClientV2 atlasClient)

Method Summary

Modifier and Type	Method and Description
void	close()
(package private) void	deleteTypeDefs(String... typeNames) This is an utility method to delete unused types.
private Map<org.apache.atlas.model.instance.AtlasObjectId,org.apache.atlas.model.instance.AtlasEntity>	fetchFlowComponents(String componentType, Map<String,org.apache.atlas.model.instance.AtlasEntity> referredEntities) Retrieves the flow components of type componentType from Atlas server.
NiFiFlow	fetchNiFiFlow(String rootProcessGroupId, String namespace) Fetch existing NiFiFlow entity from Atlas.
private org.apache.atlas.model.typedef.AtlasTypesDef	getTypeDefs(String... typeNames)
boolean	isNiFiTypeDefsRegistered()
private Map<String,List<org.apache.atlas.model.instance.AtlasEntity>>	registerDataSetEntities(NiFiFlow nifiFlow) Register DataSet within specified NiFiFlow.
private Set<org.apache.atlas.model.instance.AtlasObjectId>	registerFlowPathEntities(NiFiFlow nifiFlow)
void	registerNiFiFlow(NiFiFlow nifiFlow)
private org.apache.atlas.model.instance.AtlasEntity	registerNiFiFlowEntity(NiFiFlow nifiFlow)
void	registerNiFiTypeDefs(boolean update) Create or update NiFi types in Atlas type system.
org.apache.atlas.model.instance.AtlasEntity.AtlasEntityWithExtInfo	searchEntityDef(org.apache.atlas.model.instance.AtlasObjectId id)
private List<org.apache.atlas.model.instance.AtlasObjectId>	toAtlasObjectIds(Object _references)
private Map<org.apache.atlas.model.instance.AtlasObjectId,org.apache.atlas.model.instance.AtlasEntity>	toQualifiedNameIds(List<org.apache.atlas.model.instance.AtlasObjectId> ids) AtlasObjectIds returned from Atlas have GUID, but do not have qualifiedName, while ones created by the reporting task do not have GUID, but qualifiedName.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

private static final org.slf4j.Logger logger

atlasClient

private final org.apache.atlas.AtlasClientV2 atlasClient

FLOW_PATH_URL_PATTERN

private Pattern FLOW_PATH_URL_PATTERN

Constructor Detail

NiFiAtlasClient

public NiFiAtlasClient(org.apache.atlas.AtlasClientV2 atlasClient)

Method Detail

close

public void close()

Specified by:

close in interface AutoCloseable

deleteTypeDefs

void deleteTypeDefs(String... typeNames)
             throws org.apache.atlas.AtlasServiceException

This is an utility method to delete unused types. Should be used during development or testing only.

Parameters:

typeNames - to delete

Throws:

org.apache.atlas.AtlasServiceException

isNiFiTypeDefsRegistered

public boolean isNiFiTypeDefsRegistered()
                                 throws org.apache.atlas.AtlasServiceException

Returns:

True when required NiFi types are already created.

Throws:

org.apache.atlas.AtlasServiceException

registerNiFiTypeDefs

public void registerNiFiTypeDefs(boolean update)
                          throws org.apache.atlas.AtlasServiceException

Create or update NiFi types in Atlas type system.

Parameters:

update - If false, doesn't perform anything if there is existing type def for the name.

Throws:

org.apache.atlas.AtlasServiceException

getTypeDefs

private org.apache.atlas.model.typedef.AtlasTypesDef getTypeDefs(String... typeNames)
                                                          throws org.apache.atlas.AtlasServiceException

Throws:

org.apache.atlas.AtlasServiceException

fetchNiFiFlow

public NiFiFlow fetchNiFiFlow(String rootProcessGroupId,
                              String namespace)
                       throws org.apache.atlas.AtlasServiceException

Fetch existing NiFiFlow entity from Atlas.

Parameters:

rootProcessGroupId - The id of a NiFi flow root process group.

namespace - The namespace of a flow.

Returns:

A NiFiFlow instance filled with retrieved data from Atlas. Status objects are left blank, e.g. ProcessorStatus.

Throws:

org.apache.atlas.AtlasServiceException - Thrown if requesting to Atlas API failed, including when the flow is not found.

fetchFlowComponents

private Map<org.apache.atlas.model.instance.AtlasObjectId,org.apache.atlas.model.instance.AtlasEntity> fetchFlowComponents(String componentType,
                                                                                                                           Map<String,org.apache.atlas.model.instance.AtlasEntity> referredEntities)

Retrieves the flow components of type componentType from Atlas server. Deleted components will be filtered out before calling Atlas. Atlas object ids will be initialized with all the attributes (guid, type, unique attributes) in order to be able to match ids retrieved from Atlas (having guid) and ids created by the reporting task (not having guid yet).

Parameters:

componentType - Atlas type of the flow component (nifi_flow_path, nifi_queue, nifi_input_port, nifi_output_port)

referredEntities - referred entities of the flow entity (returned when the flow fetched) containing the basic data (id, status) of the flow components

Returns:

flow component entities mapped to their object ids

toAtlasObjectIds

private List<org.apache.atlas.model.instance.AtlasObjectId> toAtlasObjectIds(Object _references)

toQualifiedNameIds

private Map<org.apache.atlas.model.instance.AtlasObjectId,org.apache.atlas.model.instance.AtlasEntity> toQualifiedNameIds(List<org.apache.atlas.model.instance.AtlasObjectId> ids)

AtlasObjectIds returned from Atlas have GUID, but do not have qualifiedName, while ones created by the reporting task do not have GUID, but qualifiedName. AtlasObjectId.equals returns false for this combination. In order to match ids correctly, this method converts fetches actual entities from ids to get qualifiedName attribute.

Also, AtlasObjectIds returned from Atlas does not have entity state. If Atlas is configured to use soft-delete (default), deleted ids are still returned. Fetched entities are used to determine whether an AtlasObjectId is still active or deleted. Deleted entities will not be included in the result of this method.

Parameters:

ids - to convert

Returns:

AtlasObjectIds with qualifiedName

registerNiFiFlow

public void registerNiFiFlow(NiFiFlow nifiFlow)
                      throws org.apache.atlas.AtlasServiceException

Throws:

org.apache.atlas.AtlasServiceException

registerNiFiFlowEntity

private org.apache.atlas.model.instance.AtlasEntity registerNiFiFlowEntity(NiFiFlow nifiFlow)
                                                                    throws org.apache.atlas.AtlasServiceException

Throws:

org.apache.atlas.AtlasServiceException

registerDataSetEntities

private Map<String,List<org.apache.atlas.model.instance.AtlasEntity>> registerDataSetEntities(NiFiFlow nifiFlow)
                                                                                       throws org.apache.atlas.AtlasServiceException

Register DataSet within specified NiFiFlow.

Returns:

Set of registered Atlas type names and its remaining entities without deleted ones.

Throws:

org.apache.atlas.AtlasServiceException

registerFlowPathEntities

private Set<org.apache.atlas.model.instance.AtlasObjectId> registerFlowPathEntities(NiFiFlow nifiFlow)
                                                                             throws org.apache.atlas.AtlasServiceException

Throws:

org.apache.atlas.AtlasServiceException

searchEntityDef

public org.apache.atlas.model.instance.AtlasEntity.AtlasEntityWithExtInfo searchEntityDef(org.apache.atlas.model.instance.AtlasObjectId id)
                                                                                   throws org.apache.atlas.AtlasServiceException

Throws:

org.apache.atlas.AtlasServiceException