Developer Platform Functional Detail
Egeria’s Developer Platform contains the core Egeria implementation and provides support for:
- Integrating third party technology into the open metadata ecosystem;
- Extending Egeria to run in different environments or to use different infrastructure services.
Its use is described in the Developer’s Guide.
Figure 1 summarizes the capabilities it includes. They are described in the sections the follow.
Figure 1: Developer Platform Detail
Open Metadata and Governance (OMAG) Registered Services
The registered services are dynamically loaded in the OMAG Server Platform. This means they can be added and removed as needed to create a customized platform. This may include registered services written by the Egeria community and supplied by third parties.
Access Services
The Open Metadata Access Services (OMASs) provide specialist APIs/Events for different types of tools. They work with the pre-defined open metadata types described below and use the repository services to access metadata.
There are access services for asset catalogs, data users and curation tools; glossary tools; data managers and engines; data science/AI workbenches, Business Intelligence (BI) and reporting platforms; dev-ops pipelines and tools; information architecture and data engineering tools; digital service management; software development tools, governance programs, operational governance, privacy and security tools; design modeling tools, IT infrastructure management, automated metadata discovery tools, stewardship and workflow tools.
The access services run in the following types of cohort members: Metadata Access Point and Metadata Server.
Engine Services
The Open Metadata Engine Services (OMES) each provide the services that host a specific type of governance engine. The governance engines collectively provide active governance to the assets and their associated metadata.
Engine services are hosted in an Engine Host governance server.
The engine services are:
- Asset Analysis for automated metadata discovery covering the content of an Asset’s real-world counterpart.
- Governance Action for monitoring changes to open metadata, making decisions on which actions to take and running actions that update metadata to repair a situation.
Integration Services
The Open Metadata Integration Services (OMIS) each provide a specialized API to integration connectors. These are hosted in an integration daemon.
The purpose of the integration services is to simplify the implementation and management of connectors that integrate metadata exchange with third party technologies. Follow the link for the integration daemon to understand more.
These are the integration services provided by the Egeria Community today:
- Catalog Integrator - provides a two-way synchronization for data catalogs.
- Database Integrator - provides metadata extraction from relational databases.
- Files Integrator - collects metadata about files stored in a filesystem or file manager.
- Lineage Integrator - collects metadata about processes, their internal logic and the data assets they work with.
In addition, the following integration services will be added in the near future:
- Organization Integrator - imports details of an organization’s structure - such as teams and departments.
- Security Integrator - distributes security properties to access control enforcement points.
View Services
The Open Metadata View Services (OMVS) provide the services used by UIs. They are typically fine-grained services and they run in the view server. The use of the separate server (and server platform) enables an extra firewall to be erected between the view servers and the metadata servers and governance servers, hiding the internal systems from end users.
Open Metadata Types
The open metadata types provide common definitions for the different types of metadata needed by an organization. The open metadata type system is extendable. However, by providing a comprehensive starter set, and encouraging tools to use them, then Egeria ensures metadata can be seamlessly shared amongst them.
The predefined types are organized as follows: IT Infrastructure (Area 0); Collaboration (Area 1); Assets (Area 2); Glossary (Area 3); Governance (Area 4); Schemas, Reference Data, Models (Area 5); Automatic Metadata Discovery (Area 6); Lineage (Area 7).
Open Metadata and Governance (OMAG) Server Platform
The OMAG Server Platform provides a multi-tenant runtime platform for OMAG Servers. Each OMAG Server hosts the connectors along with the Egeria services to integrate third party technology.
Server Chassis
The server chassis uses Spring Boot to provide the web server and REST API support for the platform.
Administration Services
The admin services supports configuring and operating the OMAG Platform and Servers. Details of how to use the admin services are provided in the Administration Guide
Platform Services
The platform services provide the means to query the OMAG servers and services running on an OMAG Server Platform.
Multi-tenancy Management
The multi-tenant module supports multiple OMAG Servers running on an OMAG Server Platform.
Repository Services
The Open Metadata Repository Services (OMRS) provide the basic ability to share metadata between metadata repositories. The metadata repositories are organized into open metadata repository cohorts. These cohorts define the scope of the metadata sharing. There are two complementary mechanisms that are operating together to ensure metadata is available to all consumers in the cohort. The first is the ability for any member of a cohort to issue a federated query that includes all other members of the cohort. There is also optional metadata replication occurring across the cohort allowing a repository to selectively cache metadata from other members.
The repository services implementation includes the Type Definitions and Instance Structures that are used to support the open metadata types described above; Event Payloads for the asynchronous exchange of metadata between repositories in a cohort; Repository connector APIs to allow third party tools to “plug in” to open metadata and repository implementations to expand the metadata storage capability; Cohort Registration and Membership that broadcasts details of new members to automatically configure the repository services in other members of the cohort; Federated queries through the enterprise repository services; a Repository Conformance Workbench to test the ability of a cohort member to operate correctly in the cohort; Open Metadata Archives for managing metadata import/export as well as content packs of standard definitions; Audit Log Implementation with routing to multiple destinations as needed by the operations teams.
Metadata Security
The metadata-security module provides customizable authorization checks for calls to the OMAG Server Platform, OMAG Server and the open metadata instances themselves.
Governance Server Services
A governance server makes use of open metadata to actively manage an aspect of the digital landscape. The governance server services each provide the principle subsystem of a type of governance server. They include:
- data-engine-proxy-services for the Data Engine Proxy.
- engine-host-services for the Engine Host.
- integration-daemon-services for the Integration Daemon.
- open-lineage-services for the Open Lineage Server.
The strategy for governance servers is here.
Generic Handlers
The generic handlers provide support for the type specific maintenance and retrieval of metadata that follows the open metadata types. This includes managing visibility of metadata through the Governance Zones, calls to Open Metadata Security and metadata management using templates.
Open Metadata Frameworks for plug-in components
The open metadata frameworks define the interfaces implemented by components that “plug-in” to Egeria, either to integrate calls to third party technology or extend the function of Egeria. The frameworks are as follows:
- Open Connector Framework (OCF) - base framework for all types of plug-in components called connectors.
- Open Discovery Framework (ODF) - specialized connectors called discovery services that support automated metadata discovery,
- Governance Action Framework (GAF) - specialized connectors for the triage and remediation of issues found in the digital landscape.
- Audit Log Framework (ALF) - extensions for all types of connectors to enable natural language diagnostics such as exceptions and audit log messages.
Return to Status Overview
License: CC BY 4.0, Copyright Contributors to the ODPi Egeria project.