Developer Platform Functional Detail
Egeria’s Developer Platform contains the core Egeria implementation and provides support for:
- Integrating third party technology into the open metadata ecosystem;
- Extending Egeria to run in different environments or to use different infrastructure services.
Its use is described in the Developer’s Guide.
Figure 1 summarizes the capabilities it includes. They are described in the sections the follow.
Figure 1: Developer Platform Detail
Open Metadata and Governance (OMAG) Services
Access Services
The Open Metadata Access Services (OMASs) provide specialist APIs/Events for different types of tools. They work with the pre-defined open metadata types described below and use the repository services to access metadata.
There are access services for data catalogs and curation tools; glossary tools; data managers and engines; data science/AI workbenches, Business Intelligence (BI) and reporting platforms; dev-ops pipelines and tools; digital service management; software development tools, governance, privacy and security tools; design modeling tools, IT infrastructure management, automated metadata discovery tools, stewardship and workflow tools.
The access services run in the following types of cohort members: Metadata Access Point, Metadata Server and Repository Proxy.
Engine Services
The Open Metadata Engine Services (OMES) each provide the services that host a specific type of governance engine. They are hosted in an Engine Host governance server.
These services are still in design. There is more information described in its engine-services code module.
Integration Services
The Open Metadata Integration Services (OMIS) each provide a specialized API to integration connectors. These are hosted in an integration daemon.
The purpose of the integration services is to simplify the implementation and management of connectors that integrate metadata exchange with third party technologies. Follow the link for the integration daemon to understand more.
View Services
The Open Metadata View Services provide the services used by UIs. They are typically fine-grained services and they run in the view server. The use of the separate server (and server platform) enables an extra firewall to be erected between the view servers and the metadata servers and governance servers, hiding the internal systems from end users.
Repository Services
The Open Metadata Repository Services (OMRS) provide the basic ability to share metadata between metadata repositories. The metadata repositories are organized into open metadata repository cohorts. These cohorts define the scope of the metadata sharing. There are two complementary mechanisms that are operating together to ensure metadata is available to all consumers in the cohort. The first is the ability for any member of a cohort to issue a federated query that includes all other members of the cohort. There is also optional metadata replication occurring across the cohort allowing a repository to selectively cache metadata from other members.
The repository services implementation includes the Type Definitions and Instance Structures that are used to support the open metadata types described above; Event Payloads for the asynchronous exchange of metadata between repositories in a cohort; Repository connector APIs to allow third party tools to “plug in” to open metadata and repository implementations to expand the metadata storage capability; Cohort Registration and Membership that broadcasts details of new members to automatically configure the repository services in other members of the cohort; Federated queries through the enterprise repository services; a Repository Conformance Workbench to test the ability of a cohort member to operate correctly in the cohort; Open Metadata Archives for managing metadata import/export as well as content packs of standard definitions; Audit Log Implementation with routing to multiple destinations as needed by the operations teams.
Open Metadata Types
The open metadata types provide common definitions for the different types of metadata needed by an organization. The open metadata type system is extendable. However, by providing a comprehensive starter set, and encouraging tools to use them, then Egeria ensures metadata can be seamlessly shared amongst them.
The predefined types are organized as follows: IT Infrastructure (Area 0); Collaboration (Area 1); Assets (Area 2); Glossary (Area 3); Governance (Area 4); Schemas, Reference Data, Models (Area 5); Automatic Metadata Discovery (Area 6); Lineage (Area 7).
Open Metadata and Governance (OMAG) Server Platform
The OMAG Server Platform provides a multi-tenant runtime platform for OMAG Servers. Each OMAG Server hosts the connectors along with the Egeria services to integrate third party technology.
Server Chassis
The server chassis uses Spring Boot to provide the web server and REST API support for the platform.
Administration Services
The admin services supports configuring and operating the OMAG Platform and Servers. Details of how to use the admin services are provided in the Administration Guide
Platform Services
The platform services provide the means to query the OMAG servers and services running on an OMAG Server Platform.
Multi-tenancy Management
The multi-tenant module supports multiple OMAG Servers running on an OMAG Server Platform.
Metadata Security
The metadata-security module provides customizable authorization checks for calls to the OMAG Server Platform, OMAG Server and the open metadata instances themselves.
Governance Services
The governance services each provide the principle subsystem of a governance server. They include:
- data-engine-proxy-services for the Data Engine Proxy.
- discovery-engine-services for the Discovery Server.
- integration-daemon-services for the Integration Daemon.
- engine-host-services for the Engine Hosting Server.
- open-lineage-services for the Open Lineage Server.
- security-officer-services for the Security Server.
- security-sync-services for the Security Sync Server.
- stewardship-engine-services for the Stewardship Server.
Each governance server is making use of open metadata to actively manage an aspect of the digital landscape.
Note: as our experience with writing governance servers increases, there is planned consolidation in the governance services:
- The integration services that run in the integration daemon will be extended to provide the support now offered by the data engine proxy services and security sync services.
- The engine services that run in the engine hosting server will be implemented to support the function currently offered by the discovery-engine-services and the stewardship-engine-services.
Generic Handlers
The generic handlers provide support for the type specific maintenance and retrieval of metadata that follows the open metadata types. This includes managing visibility of metadata through the Governance Zones, calls to Open Metadata Security and metadata management using templates.
Open Metadata Frameworks for plug-in components
The open metadata frameworks define the interfaces implemented by components that “plug-in” to Egeria, either to integrate calls to third party technology or extend the function of Egeria. The frameworks are as follows:
- Open Connector Framework (OCF) - base framework for all types of plug-in components called connectors.
- Open Discovery Framework (ODF) - specialized connectors called discovery services that support automated metadata discovery,
- Governance Action Framework (GAF) - specialized connectors for the triage and remediation of issues found in the digital landscape.
- Audit Log Framework (ALF) - extensions for all types of connectors to enable natural language diagnostics such as exceptions and audit log messages.
Return to Status Overview
License: CC BY 4.0, Copyright Contributors to the ODPi Egeria project.