Package org.apache.nifi.controller.repository

Interface FlowFileRepository

All Superinterfaces:
AutoCloseable, Closeable
public interface FlowFileRepository extends Closeable
Implementations must be thread safe

  • Method Details

    • initialize

      void initialize(ResourceClaimManager claimManager) throws IOException
      Initializes the Content Repository, providing to it the ContentClaimManager that is to be used for interacting with Content Claims
      Parameters:
      claimManager - for handling claims
      Throws:
      IOException - if unable to initialize repository

    • getStorageCapacity

      long getStorageCapacity() throws IOException
      Returns:
      the maximum number of bytes that can be stored in the underlying storage mechanism
      Throws:
      IOException - if computing capacity fails

    • getUsableStorageSpace

      long getUsableStorageSpace() throws IOException
      Returns:
      the number of bytes currently available for use by the underlying storage mechanism
      Throws:
      IOException - if computing usable space fails

    • getFileStoreName

      String getFileStoreName()
      Returns the name of the FileStore that the repository is stored on, or null if not applicable or unable to determine the file store name
      Returns:
      the name of the FileStore

    • updateRepository

      void updateRepository(Collection<RepositoryRecord> records) throws IOException
      Updates the repository with the given RepositoryRecords.
      Parameters:
      records - the records to update the repository with
      Throws:
      IOException - if update fails

    • loadFlowFiles

      long loadFlowFiles(QueueProvider queueProvider) throws IOException
      Loads all flow files found within the repository, establishes the content claims and their reference count
      Parameters:
      queueProvider - the provider of FlowFile Queues into which the FlowFiles should be enqueued
      Returns:
      index of highest flow file identifier
      Throws:
      IOException - if load fails

    • findQueuesWithFlowFiles

      Set<String> findQueuesWithFlowFiles(FlowFileSwapManager flowFileSwapManager) throws IOException
      Searches through the repository to find the ID's of all FlowFile Queues that currently have data queued
      Returns:
      the set of all FlowFileQueue identifiers for which a FlowFile is queued
      Throws:
      IOException - if unable to read from the FlowFile Repository

    • isVolatile

      boolean isVolatile()
      Returns:
      true if the Repository is volatile (i.e., its data is lost upon application restart), false otherwise

    • getNextFlowFileSequence

      long getNextFlowFileSequence()
      Returns:
      the next ID in sequence for creating FlowFiles.

    • getMaxFlowFileIdentifier

      long getMaxFlowFileIdentifier() throws IOException
      Returns:
      the max ID of all FlowFiles that currently exist in the repository.
      Throws:
      IOException - if computing max identifier fails

    • updateMaxFlowFileIdentifier

      void updateMaxFlowFileIdentifier(long maxId)
      Notifies the FlowFile Repository that the given identifier has been identified as the maximum value that has been encountered for an 'external' (swapped out) FlowFile.
      Parameters:
      maxId - the max id of any FlowFile encountered

    • swapFlowFilesOut

      void swapFlowFilesOut(List<FlowFileRecord> swappedOut, FlowFileQueue flowFileQueue, String swapLocation) throws IOException
      Updates the Repository to indicate that the given FlowFileRecords were Swapped Out of memory
      Parameters:
      swappedOut - the FlowFiles that were swapped out of memory
      flowFileQueue - the queue that the FlowFiles belong to
      swapLocation - the location to which the FlowFiles were swapped
      Throws:
      IOException - if swap fails

    • swapFlowFilesIn

      void swapFlowFilesIn(String swapLocation, List<FlowFileRecord> flowFileRecords, FlowFileQueue flowFileQueue) throws IOException
      Updates the Repository to indicate that the given FlowFileRecords were Swapped In to memory
      Parameters:
      swapLocation - the location (e.g., a filename) from which FlowFiles were recovered
      flowFileRecords - the records that were swapped in
      flowFileQueue - the queue that the FlowFiles belong to
      Throws:
      IOException - if swap fails

    • isValidSwapLocationSuffix

      boolean isValidSwapLocationSuffix(String swapLocationSuffix)

      Determines whether or not the given swap location suffix is a valid, known location according to this FlowFileRepository. Note that while the swapFlowFilesIn(String, List, FlowFileQueue) and swapFlowFilesOut(List, FlowFileQueue, String) methods expect a full "swap location" this method expects only the "suffix" of a swap location. For example, if the location points to a file, this method would expect only the filename, not the full path.

      This method differs from the others because the other methods want to store the swap location or recover from a given location. However, this method is used to verify that the location is known. If for any reason, NiFi is stopped, its FlowFile Repository relocated to a new location (for example, a different disk partition), and restarted, the swap location would not match if we used the full location. Therefore, by using only the "suffix" (i.e. the filename for a file-based implementation), we can avoid worrying about relocation.

      Parameters:
      swapLocationSuffix - the suffix of the location to check
      Returns:
      true if the swap location is known and valid, false otherwise

    • findResourceClaimReferences

      default Map<ResourceClaim,Set<ResourceClaimReference>> findResourceClaimReferences(Set<ResourceClaim> resourceClaims, FlowFileSwapManager swapManager) throws IOException

      Scans the FlowFile Repository to locate any FlowFiles that reference the given Resource Claims. If the FlowFile Repository does not implement this capability, it will return null.

      Parameters:
      resourceClaims - the resource claims whose references should be found
      swapManager - the swap manager to use for scanning swap files
      Returns:
      a Mapping of Resource Claim to a representation of the FlowFiles/Swap Files that reference those Resource Claims
      Throws:
      IOException - if an IO failure occurs when attempting to find references

    • findOrphanedResourceClaims

      default Set<ResourceClaim> findOrphanedResourceClaims()
      Returns the set of Resource Claims that are referenced by FlowFiles that have been "orphaned" because they belong to FlowFile Queues/Connections that did not exist in the flow when NiFi started
      Returns:
      the set of orphaned Resource Claims