Data Files Monitor Integration Connector

Overview

The data files monitor integration connector monitors changes in a file directory (folder) and updates the open metadata repository/repositories to reflect the changes to both the files and folders underneath it.

Specifically, a DataFile asset is created and then maintained for each file in the folder (or any sub-folder). When a new file is created, a new DataFile asset is created. If a file is modified, the lastModified property of the corresponding DataFile asset is updated. When a file is deleted, its corresponding DataFile asset is either:

Figure 1 shows the set up for the data files monitor integration connector.

Figure 1

Figure 1: Operation of the data files monitor integration connector

A FileFolder metadata asset for the monitored folder is created when the first file is catalogued, if it does not already exist.

Configuration

This connector uses the Files Integrator OMIS running in the Integration Daemon.

This is its connection definition to use on the administration commands that configure the Files Integrator OMIS. Replace {folderName} with the path name of the folder where the files will be located. The configurationProperties are optional and are used to override the connector’s default behavior:

{
   "connection" : { 
                      "class" : "Connection",
                      "connectorType" : 
                      {
                           "class" : "ConnectorType",
                           "connectorProviderClassName" : "org.odpi.openmetadata.adapters.connectors.integration.basicfiles.DataFilesMonitorIntegrationProvider"           
                      },
                      "endpoint" :
                      {
                           "class" : "Endpoint",
                           "address" : "{folderName}"
                      },
                      "configurationProperties" : 
                      {
                           "templateQualifiedName" : "{templateQualifiedName}",
                           "allowCatalogDelete" : ""         
                      }
                  }
}

Examples of use



License: CC BY 4.0, Copyright Contributors to the ODPi Egeria project.