org.apache.nifi.atlas.hook

Class NotificationSender

java.lang.Object

org.apache.nifi.atlas.hook.NotificationSender

class NotificationSender
extends Object

This class implements Atlas hook notification message deduplication mechanism. Separated from NiFiAtlasHook for better testability.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private class	NotificationSender.Metrics

Field Summary

Fields

Modifier and Type	Field and Description
private NiFiAtlasClient	atlasClient
private Map<String,String>	guidToTypedQualifiedName An index to resolve a qualifiedName from a GUID.
private static org.slf4j.Logger	logger
private Map<String,org.apache.atlas.v1.model.instance.Referenceable>	typedQualifiedNameToRef An index to resolve a Referenceable from a typeName::qualifiedName.

Constructor Summary

Constructors

Constructor and Description
NotificationSender()

Method Summary

Modifier and Type	Method and Description
private static <K,V> Map<K,V>	createCache(int maxSize)
private Predicate<org.apache.atlas.v1.model.instance.Referenceable>	distinctReferenceable()
private Map<String,org.apache.atlas.v1.model.instance.Referenceable>	fromReferenceable(Object _refs, NotificationSender.Metrics metrics)
private boolean	isAssigned(org.apache.atlas.v1.model.instance.Id id)
private Collection<org.apache.atlas.v1.model.instance.Referenceable>	mergeRefs(Collection<org.apache.atlas.v1.model.instance.Referenceable> r1, Collection<org.apache.atlas.v1.model.instance.Referenceable> r2)
private void	mergeRefs(org.apache.atlas.v1.model.instance.Referenceable r1, org.apache.atlas.v1.model.instance.Referenceable r2)
private <K,V> List<V>	safeGet(Map<K,List<V>> map, K key)
(package private) void	send(List<org.apache.atlas.model.notification.HookNotification> messages, BiConsumer<List<org.apache.atlas.model.notification.HookNotification>,org.apache.hadoop.security.UserGroupInformation> notifier) Send hook notification messages.
(package private) void	setAtlasClient(NiFiAtlasClient atlasClient)
private Map<String,org.apache.atlas.v1.model.instance.Referenceable>	toReferenceables(Object _refs, NotificationSender.Metrics metrics) Convert nifi_flow_path inputs or outputs to a map of Referenceable keyed by type + qualifiedName.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

private static final org.slf4j.Logger logger

atlasClient

private NiFiAtlasClient atlasClient

guidToTypedQualifiedName

private final Map<String,String> guidToTypedQualifiedName

An index to resolve a qualifiedName from a GUID.

typedQualifiedNameToRef

private final Map<String,org.apache.atlas.v1.model.instance.Referenceable> typedQualifiedNameToRef

An index to resolve a Referenceable from a typeName::qualifiedName.

Constructor Detail

NotificationSender

NotificationSender()

Method Detail

createCache

private static <K,V> Map<K,V> createCache(int maxSize)

setAtlasClient

void setAtlasClient(NiFiAtlasClient atlasClient)

distinctReferenceable

private Predicate<org.apache.atlas.v1.model.instance.Referenceable> distinctReferenceable()

safeGet

private <K,V> List<V> safeGet(Map<K,List<V>> map,
                              K key)

mergeRefs

private void mergeRefs(org.apache.atlas.v1.model.instance.Referenceable r1,
                       org.apache.atlas.v1.model.instance.Referenceable r2)

mergeRefs

private Collection<org.apache.atlas.v1.model.instance.Referenceable> mergeRefs(Collection<org.apache.atlas.v1.model.instance.Referenceable> r1,
                                                                               Collection<org.apache.atlas.v1.model.instance.Referenceable> r2)

send

void send(List<org.apache.atlas.model.notification.HookNotification> messages,
          BiConsumer<List<org.apache.atlas.model.notification.HookNotification>,org.apache.hadoop.security.UserGroupInformation> notifier)

Send hook notification messages. In order to notify relationships between 'nifi_flow_path' and its inputs/outputs, this method sends messages in following order:

1. As a a single HookNotificationV1.EntityCreateRequest message:
   • New entities except 'nifi_flow_path', including DataSets such as 'nifi_queue', 'kafka_topic' or 'hive_table' ... etc, so that 'nifi_flow_path' can refer
   • New 'nifi_flow_path' entities, entities order is guaranteed in a single message
2. Update 'nifi_flow_path' messages, before notifying update messages, this method fetches existing 'nifi_flow_path' entity to merge new inputs/outputs element with existing ones, so that existing ones will not be removed.
3. Other messages except

Messages having the same type and qualified name will be de-duplicated before being sent.

Parameters:

messages - list of messages to be sent

notifier - responsible for sending notification messages, its accept method can be called multiple times

toReferenceables

private Map<String,org.apache.atlas.v1.model.instance.Referenceable> toReferenceables(Object _refs,
                                                                                      NotificationSender.Metrics metrics)

Convert nifi_flow_path inputs or outputs to a map of Referenceable keyed by type + qualifiedName.

Atlas removes existing references those are not specified when a collection attribute is updated. In order to preserve existing DataSet references, existing elements should be passed within a partial update message.

This method also populates entity cache for subsequent lookups.

Parameters:

_refs - Contains references from an existin nifi_flow_path entity inputs or outputs attribute.

Returns:

A map of Referenceables keyed by qualifiedName.

fromReferenceable

private Map<String,org.apache.atlas.v1.model.instance.Referenceable> fromReferenceable(Object _refs,
                                                                                       NotificationSender.Metrics metrics)

isAssigned

private boolean isAssigned(org.apache.atlas.v1.model.instance.Id id)