# Distributed components

Components that need to be operated by a federated participant.

# Access Node

The [DOME Access-Node](https://github.com/DOME-Marketplace/access-node) is a set of services, that can be used to access the DOME Marketplace. A registered participant can use it to act as a federated marketplace in DOME.

The Access-Nodes consists of 3 logical building blocks:

[![Building Blocks](https://github.com/DOME-Marketplace/integration-guide/raw/main/doc/img/building-blocks.png)](https://github.com/DOME-Marketplace/integration-guide/blob/main/doc/img/building-blocks.png)

The TM-Forum-API Service is a service providing a growing subset of the [TMForum API](https://www.tmforum.org/oda/open-apis/table)'s while using an [NGSI-LD](https://www.etsi.org/deliver/etsi_gs/CIM/001_099/009/01.06.01_60/gs_CIM009v010601p.pdf) context broker as persistence backend and change notificator.

<section aria-label="mermaid rendered output container" class="js-render-needs-enrichment render-needs-enrichment position-relative" data-host="https://viewscreen.githubusercontent.com" data-identity="6be2bc9d-0c03-4d91-8594-4f7fb80a6794" data-src="https://viewscreen.githubusercontent.com/markdown/mermaid?docs_host=https%3A%2F%2Fdocs.github.com" data-type="mermaid" id="bkmrk-%C2%A0">[![image.png](https://knowledgebase.dome-marketplace-dev.org/uploads/images/gallery/2024-05/scaled-1680-/Apximage.png)](https://knowledgebase.dome-marketplace-dev.org/uploads/images/gallery/2024-05/Apximage.png)

</section>TODO: Add description of blockchain connector

#### Overview and sub-components

The TM-Forum-API service is a cluster of individual services providing one specific API each, enabling the participant to only run the necessary subset for its use-case. Apart from offering CRUD operations on the managed entities, the service also enables the subscription to notifications based on given queries.

The services are stateless and support horizontal scaling, but require an external cache to avoid having inconsistent caches. Inconsistent caches can result from either changes due to calls to the API, or due to notifications for changes reported by the underlying persistence. If run in a single instance mode, a local cache is acceptable but for larger setups a [Redis](https://redis.io/) installation is recommended.

For reasons of convenience, the TM-Forum-API service can be deploying with an [Envoy API proxy](https://www.envoyproxy.io/) which provides the individual APIs via a single service, routed based on the path. Another convenient feature is a [RapiDoc](https://rapidocweb.com/) container, that can be deployed with the TM-Forum-API service that provides a Openapi based API documentation for the deployed services, with the functionality of querying the API too.

The requirement for the persistence is to be compliant to the NGSI-LD API v1.6 enabling the use of different available context brokers. The currently recommended Context-Broker for the access node is [Scorpio](https://github.com/ScorpioBroker/ScorpioBroker), mainly due to good cloud integration and overall support. The Scorpio context-broker allows a variety of adjustments to cover the operator's specific needs ( e.g. horizontal scaling utilizing [Kafka](https://kafka.apache.org/)) and uses [Postgresql](https://www.postgresql.org/) as it's persistence layer. The Postgresql is extended with [Postgis](https://postgis.net/) for supporting geospatial data.

#### Infrastructure requirements

The base memory consumption per deployed pod is listed below but is will increase with the amount of traffic, therefor should only be used as a rough estimate.

<table id="bkmrk-service-memory-%28mi%29-"><thead><tr><th>Service</th><th>Memory (Mi)</th></tr></thead><tbody><tr><td>TM-Forum-API Pod</td><td>250</td></tr><tr><td>Scorpio</td><td>400</td></tr><tr><td>Postgresql/Postgis</td><td>150</td></tr><tr><td>Redis</td><td>10</td></tr><tr><td> </td><td> </td></tr><tr><td> </td><td> </td></tr></tbody></table>

Apart from the database service, no other service will maintain a own persistence, therefor only for this service a persistent volume claim has to be dimensioned.

#### How to deploy

The recommended and endorsed way of deployment is via the provided helm charts ( optionally wrapped in ArgoCD Applications).

To deploy a setup, the [umbrella chart](https://helm.sh/docs/howto/charts_tips_and_tricks/#complex-charts-with-many-dependencies) of the access-node can be used as followed:

- create a configuration values file according to the own environment, as described [here](https://github.com/DOME-Marketplace/integration-guide?tab=readme-ov-file#how-to-configure).
- add helm chart repository to helm installation
    
    <div><div class="cm-editor ͼ1 ͼ2 ͼ4 ͼ1p"><div class="cm-scroller" tabindex="-1"><div aria-hidden="true" class="cm-gutters"></div><div aria-multiline="true" aria-readonly="true" autocapitalize="off" autocorrect="off" class="cm-content" contenteditable="true" role="textbox" spellcheck="false" translate="no"><div class="cm-line">helm repo add dome-access-node https://dome-marketplace.github.io/access-node</div><div class="cm-line">helm repo update</div></div></div><button class="cm-copy-button" type="button"><svg height="16" viewbox="0 0 24 24" width="16" xmlns="http://www.w3.org/2000/svg"></svg></button></div></div><div class="snippet-clipboard-content notranslate position-relative overflow-auto"><div class="zeroclipboard-container"><svg aria-hidden="true" class="octicon octicon-copy js-clipboard-copy-icon" data-view-component="true" height="16" version="1.1" viewbox="0 0 16 16" width="16"></svg>  
    </div></div>> ? All releases of the Access-Node reside in the helm-repository [https://dome-marketplace.github.io/access-node](https://dome-marketplace.github.io/access-node). In addition to that, all Pre-Release versions(build from the Pull Requests) are provided in the pre-repo [https://dome-marketplace.github.io/access-node/pre](https://dome-marketplace.github.io/access-node/pre). The pre-repo will be cleaned-up from time to time, in order to keep the index manageable.
- install the components using the prepared configuration
    
    <div><div class="cm-editor ͼ1 ͼ2 ͼ4 ͼ1q"><div class="cm-scroller" tabindex="-1"><div aria-multiline="true" aria-readonly="true" autocapitalize="off" autocorrect="off" class="cm-content" contenteditable="true" role="textbox" spellcheck="false" translate="no"><div class="cm-line">helm install &lt;RELEASE_NAME&gt; dome-access-node/access-node --namespace &lt;NAME_SPACE&gt; --version &lt;CHART_VERSION&gt; -f values.yaml</div></div><div aria-hidden="true" class="cm-layer cm-layer-above cm-cursorLayer">  
    </div></div><button class="cm-copy-button" type="button"><svg height="16" viewbox="0 0 24 24" width="16" xmlns="http://www.w3.org/2000/svg"></svg></button></div></div><div class="snippet-clipboard-content notranslate position-relative overflow-auto"><div class="zeroclipboard-container"><svg aria-hidden="true" class="octicon octicon-copy js-clipboard-copy-icon" data-view-component="true" height="16" version="1.1" viewbox="0 0 16 16" width="16"></svg>  
    </div></div>

#### How to configure

The chart is released with a set of [default values](https://github.com/DOME-Marketplace/access-node/blob/main/charts/access-node/values.yaml) which act as a good starting point for an adoption. These values are also documented, enhancing the understanding. Additionally, the [respective charts](https://github.com/FIWARE/helm-charts/tree/main/charts/tm-forum-api) of the components should be consulted.

<table id="bkmrk-component-chart-tm-f"><thead><tr><th>Component</th><th>Chart</th></tr></thead><tbody><tr><td>TM-Forum-API</td><td>[https://github.com/FIWARE/helm-charts/tree/main/charts/tm-forum-api](https://github.com/FIWARE/helm-charts/tree/main/charts/tm-forum-api)</td></tr><tr><td>blockchain-connector</td><td>[https://github.com/DOME-Marketplace/access-node/tree/main/charts/blockchain-connector](https://github.com/DOME-Marketplace/access-node/tree/main/charts/blockchain-connector)</td></tr><tr><td>broker-adapter</td><td>[https://github.com/DOME-Marketplace/access-node/tree/main/charts/broker-adapter](https://github.com/DOME-Marketplace/access-node/tree/main/charts/broker-adapter)</td></tr><tr><td>dlt-adapter</td><td>[https://github.com/DOME-Marketplace/access-node/tree/main/charts/dlt-adapter](https://github.com/DOME-Marketplace/access-node/tree/main/charts/dlt-adapter)</td></tr><tr><td>kafka</td><td>[https://github.com/bitnami/charts/tree/main/bitnami/kafka](https://github.com/bitnami/charts/tree/main/bitnami/kafka)</td></tr><tr><td>postgresql</td><td>[https://github.com/bitnami/charts/tree/main/bitnami/postgresql](https://github.com/bitnami/charts/tree/main/bitnami/postgresql)</td></tr><tr><td>scorpio-broker-aaio</td><td>[https://github.com/FIWARE/helm-charts/tree/main/charts/scorpio-broker-aaio](https://github.com/FIWARE/helm-charts/tree/main/charts/scorpio-broker-aaio)</td></tr><tr><td>scorpio-broker</td><td>[https://github.com/FIWARE/helm-charts/tree/main/charts/scorpio-broker](https://github.com/FIWARE/helm-charts/tree/main/charts/scorpio-broker)</td></tr></tbody></table>

TODO: Replace with charts [in](https://github.com/in2workspace/helm-charts/tree/main/charts) once they are used.

To have a starting point, the [this](https://github.com/DOME-Marketplace/integration-guide/blob/main/config/accessnode.yaml) minimal config reduces the configuration to items that are likely changed by integrators. TODO: include config for the blockchain components

#### How to validate a deployment

All components are configured with health and readiness checks to validate their own status, therefor being the base for a validation. These checks are utilized in the kubernetes checks as defined in the helm charts. TODO: Include RapiDoc Container for validation and add explanation here

#### How to operate

> - Management/admin APIs.
> - Instrumentation, metrics, logs, alerts

The underlying database service holds the persisted data and therefor requires a backup&amp;recovery mechanism when operated in a production environment. The use of managed database is strongly encouraged for safety and convenience.

The TM-Forum-API service used a json based log output by default, which can be parsed easily by log aggregators but can also be replaced if needed. The verbosity is controlled via [environment variables](https://github.com/FIWARE/helm-charts/blob/05552c4c97a21df68f14e78de80a56e3934e179d/charts/tm-forum-api/templates/deployment.yaml#L165) and can be fine tuned to the operators needs.

TODO: Prometheus Metrics TODO: Grafana Dashboard

#### How to update

Upgrade to both a different chart version and new configuration can be accomplished with the following command

<div id="bkmrk-1-helm-upgrade-%3Crele"><div class="cm-editor ͼ1 ͼ2 ͼ4 ͼ1r"><div class="cm-scroller" tabindex="-1"><div aria-hidden="true" class="cm-gutters"><div class="cm-gutter cm-lineNumbers"><div class="cm-gutterElement"></div></div></div><div aria-multiline="true" aria-readonly="true" autocapitalize="off" autocorrect="off" class="cm-content" contenteditable="true" role="textbox" spellcheck="false" translate="no"><div class="cm-line">helm upgrade &lt;RELEASE_NAME&gt; dome-access-node/access-node --namespace &lt;NAME_SPACE&gt; --version &lt;CHART_VERSION&gt; -f values.yaml</div></div><div aria-hidden="true" class="cm-layer cm-selectionLayer"></div></div><button class="cm-copy-button" type="button"><svg height="16" viewbox="0 0 24 24" width="16" xmlns="http://www.w3.org/2000/svg"></svg></button></div></div><div class="snippet-clipboard-content notranslate position-relative overflow-auto" id="bkmrk--1"><div class="zeroclipboard-container"><svg aria-hidden="true" class="octicon octicon-copy js-clipboard-copy-icon" data-view-component="true" height="16" version="1.1" viewbox="0 0 16 16" width="16"></svg>  
</div></div>#### Release process

Versioning of the main access-node helm chart is handled based on the labels used in the pull requests used to introduce changes and is enforced in the [build pipeline](https://github.com/DOME-Marketplace/access-node/tree/main/.github/workflows). The requester and reviewers must set the label according to the [SemVer 2.0.0](https://semver.org/) versioning scheme.

Versioning of the components and sub-charts is recommended to use the same scheme.

> Versioning, release notes, stability considerations

#### Troubleshooting

> To be filled once feedback from integrators comes in

##### Timeouts occur while querying TM-Forum-API

When encountering timeouts in calls to the TM-Forum-API service it is possible to mitigate the imminent issue by increasing the timeout of the client (called "ngsi") calling the NGSI-LD broker. The necessary [client](https://docs.micronaut.io/latest/guide/configurationreference.html#io.micronaut.http.client.ServiceHttpClientConfiguration) and [server](https://docs.micronaut.io/latest/guide/configurationreference.html#io.micronaut.http.server.HttpServerConfiguration) configuration can be handed in via [additional environment variables](https://github.com/FIWARE/helm-charts/blob/05552c4c97a21df68f14e78de80a56e3934e179d/charts/tm-forum-api/templates/deployment.yaml#L243-L249).

# IAM components

The [DOME IAM-Framework](https://github.com/DOME-Marketplace/iam-components) is a set of microservices, that enables users in the DOME ecosystem to authenticate into the [DOME Marketplace](https://dome-marketplace.org/). The authentication process itself is described further below in the [Authentication](https://github.com/DOME-Marketplace/integration-guide?tab=readme-ov-file#authentication) section.

#### Overview and subcomponents

The DOME IAM-Framework consists of multiple open-source components. The components are not required to be used, as long as alternatives providing the same interfaces are used.

The IAM-Framework consists of following components:

[![IAM-components](https://github.com/DOME-Marketplace/integration-guide/raw/main/doc/img/iam.png)](https://github.com/DOME-Marketplace/integration-guide/blob/main/doc/img/iam.png)

- The [Trusted Issuers List](https://github.com/fiware/trusted-issuers-list) service provides an [EBSI Trusted Issuers Registry](https://hub.ebsi.eu/apis/pilot/trusted-issuers-registry/v4) implementation to act as the Trusted-List-Service in the DSBA Trust and IAM Framework. In addition, a Trusted-Issuers-List API is provided to manage the issuers.
- [VCVerifier](https://github.com/fiware/vcverifier) provides the necessary endpoints to offer SIOP-2/OIDC4VP compliant authentication flows. It exchanges VerifiableCredentials for JWT, that can be used for authorization and authentication in down-stream components.
- [Credentials Config Service](https://github.com/fiware/credentials-config-service) manages and provides information about services and the credentials they are using. It returns the scope to be requested from the wallet per service. Furthermore, it specifies the credentials required and the issuers list endpoints to validate against, when checking access for a certain service.
- The [Keycloak-VC-Issuer](https://github.com/fiware/keycloak-vc-issuer) is plugin for Keycloak to support SIOP-2/OIDC4VP clients and issue VerifiableCredentials through the OIDC4VCI-Protocol to compliant wallets.
- [PDP](https://github.com/fiware/dsba-pdp) is an implementation of a Policy-Decision Point, evaluating Json-Web-Tokens containing VerifiableCredentials in a DSBA-compliant way. It also supports the evaluation in the context of i4Trust.
- [Keyrock](https://github.com/ging/fiware-idm) is the FIWARE component responsible for Identity Management. Within DOME IAM-Framework, currently Keyrock is being used as the iSHARE-compliant Authorization Registry (see for details: [https://dev.ishare.eu/delegation/endpoint.html](https://dev.ishare.eu/delegation/endpoint.html)), where attribute-based access policies are stored and used during the authorization process. Note, that this will be replaced by an ODRL-compliant policy registry. A description of the policies is given in the [policies](https://github.com/DOME-Marketplace/integration-guide?tab=readme-ov-file#policies) section.
- [Kong Plugins](https://github.com/fiware/kong-plugins-fiware) allow to extend the API Gateway Kong by further functionalities. Kong Gateway is a lightweight, fast, and flexible cloud-native API gateway. One of the plugins is the PEP plugin, which is especially required within the IAM-components as PEP component and interacts with the PDP mentioned above.
- [Waltid](https://github.com/walt-id/waltid-ssikit) manages Keys, DIDs and VCs. It is used by VC Issuer and VCVerifier.

#### How to deploy

The recommended way of deployment is via the provided [Helm charts](https://github.com/DOME-Marketplace/iam-components).

To deploy a setup, the [umbrella chart](https://helm.sh/docs/howto/charts_tips_and_tricks/#complex-charts-with-many-dependencies) of the iam-components can be used as followed:

- create a configuration values file according to the own environment, as described [here](https://github.com/DOME-Marketplace/integration-guide?tab=readme-ov-file#how-to-configure-1).
- add helm chart repository to helm installation
    
    <div><div class="cm-editor ͼ1 ͼ2 ͼ4 ͼ1s"><div class="cm-scroller" tabindex="-1"><div aria-multiline="true" aria-readonly="true" autocapitalize="off" autocorrect="off" class="cm-content" contenteditable="true" role="textbox" spellcheck="false" translate="no"><div class="cm-line">helm repo add dome-iam https://dome-marketplace.github.io/iam-components</div><div class="cm-line">helm repo update</div></div><div aria-hidden="true" class="cm-layer cm-layer-above cm-cursorLayer">  
    </div></div><button class="cm-copy-button" type="button"><svg height="16" viewbox="0 0 24 24" width="16" xmlns="http://www.w3.org/2000/svg"></svg></button></div></div><div class="snippet-clipboard-content notranslate position-relative overflow-auto"><div class="zeroclipboard-container"><svg aria-hidden="true" class="octicon octicon-copy js-clipboard-copy-icon" data-view-component="true" height="16" version="1.1" viewbox="0 0 16 16" width="16"></svg>  
    </div></div>> ? All releases of the IAM-components reside in the helm-repository [https://dome-marketplace.github.io/iam-components](https://dome-marketplace.github.io/iam-components). In addition to that, all Pre-Release versions(build from the Pull Requests) are provided in the pre-repo [https://dome-marketplace.github.io/iam-components/pre](https://dome-marketplace.github.io/iam-components/pre). The pre-repo will be cleaned-up from time to time, in order to keep the index manageable.
- install the components using the prepared configuration
    
    <div><div class="cm-editor ͼ1 ͼ2 ͼ4 ͼ1t"><div class="cm-scroller" tabindex="-1"><div aria-multiline="true" aria-readonly="true" autocapitalize="off" autocorrect="off" class="cm-content" contenteditable="true" role="textbox" spellcheck="false" translate="no"><div class="cm-line">helm install &lt;RELEASE_NAME&gt; dome-iam/iam-components --namespace &lt;NAME_SPACE&gt; --version &lt;CHART_VERSION&gt; -f values.yaml</div></div><div aria-hidden="true" class="cm-layer cm-layer-above cm-cursorLayer">  
    </div></div><button class="cm-copy-button" type="button"><svg height="16" viewbox="0 0 24 24" width="16" xmlns="http://www.w3.org/2000/svg"></svg></button></div></div>

#### How to configure

The chart is released with a set of documented [default values](https://github.com/DOME-Marketplace/iam-components/blob/main/charts/iam-components/values.yaml). The parameters listed below are important to set and should be updated at least:

- `rbac` and `serviceAccount`: Depending on your requirements, you might need to adapt settings for RBAC and service account
- `did`s of participants: Replace/add the DIDs of the issuer and other participants
- In the case of did:key provide correct key in [keyfile.json](https://github.com/DOME-Marketplace/iam-components/blob/main/charts/iam-components/templates/keycloak.yaml) for your issuer
- `keycloak.frontendUrl`: Externally accessible address of the keycloak (should be the same as defined in ingress/route)
- `keycloak.realm`: Adapt clients, users and roles according to your needs
- `<tir.com>`: replace everywhere with actual TIR URL
- `<dome-marketplace.org>`: replace with your own domain
- `keyrock.initData.scriptData`: Adapt the roles as in keycloak realm
- `kong.configMap`: Adapt the kong services and their routes

However, it is suggested to consult the respective charts listed below and check their documentation and configuration.

<table id="bkmrk-component-chart-post"><thead><tr><th>Component</th><th>Chart</th></tr></thead><tbody><tr><td>postgresql</td><td>[https://github.com/bitnami/charts/tree/main/bitnami/postgresql](https://github.com/bitnami/charts/tree/main/bitnami/postgresql)</td></tr><tr><td>mysql</td><td>[https://github.com/bitnami/charts/tree/main/bitnami/mysql](https://github.com/bitnami/charts/tree/main/bitnami/mysql)</td></tr><tr><td>vcwaltid</td><td>[https://github.com/i4Trust/helm-charts/tree/main/charts/vcwaltid](https://github.com/i4Trust/helm-charts/tree/main/charts/vcwaltid)</td></tr><tr><td>keycloak</td><td>[https://github.com/bitnami/charts/tree/main/bitnami/keycloak](https://github.com/bitnami/charts/tree/main/bitnami/keycloak)</td></tr><tr><td>credentials-config-service</td><td>[https://github.com/FIWARE/helm-charts/tree/main/charts/credentials-config-service](https://github.com/FIWARE/helm-charts/tree/main/charts/credentials-config-service)</td></tr><tr><td>trusted-issuers-list</td><td>[https://github.com/FIWARE/helm-charts/tree/main/charts/trusted-issuers-list](https://github.com/FIWARE/helm-charts/tree/main/charts/trusted-issuers-list)</td></tr><tr><td>vcverifier</td><td>[https://github.com/i4Trust/helm-charts/tree/main/charts/vcverifier](https://github.com/i4Trust/helm-charts/tree/main/charts/vcverifier)</td></tr><tr><td>keyrock</td><td>[https://github.com/FIWARE/helm-charts/tree/main/charts/keyrock](https://github.com/FIWARE/helm-charts/tree/main/charts/keyrock)</td></tr><tr><td>dsba-pdp</td><td>[https://github.com/FIWARE/helm-charts/tree/main/charts/dsba-pdp](https://github.com/FIWARE/helm-charts/tree/main/charts/dsba-pdp)</td></tr><tr><td>kong</td><td>[https://github.com/Kong/charts/tree/main/charts/kong](https://github.com/Kong/charts/tree/main/charts/kong)</td></tr></tbody></table>

#### How to validate a deployment

All components are configured with health and readiness checks to validate their own status, therefore being the base for a validation. These checks are utilized in the Kubernetes checks as defined in the helm charts.

#### How to operate

The underlying database service holds the persisted data and therefore requires a backup&amp;recovery mechanism when operated in a production environment. The use of managed database is strongly encouraged for safety and convenience.

#### How to update

Upgrade to both a different chart version and new configuration can be accomplished with the following command

<div id="bkmrk-helm-upgrade-%3Creleas"><div class="cm-editor ͼ1 ͼ2 ͼ4 ͼ1u"><div class="cm-scroller" tabindex="-1"><div aria-multiline="true" aria-readonly="true" autocapitalize="off" autocorrect="off" class="cm-content" contenteditable="true" role="textbox" spellcheck="false" translate="no"><div class="cm-line">helm upgrade &lt;RELEASE_NAME&gt; dome-iam/iam-components --namespace &lt;NAME_SPACE&gt; --version &lt;CHART_VERSION&gt; -f values.yaml</div></div><div aria-hidden="true" class="cm-layer cm-layer-above cm-cursorLayer">  
</div></div><button class="cm-copy-button" type="button"><svg height="16" viewbox="0 0 24 24" width="16" xmlns="http://www.w3.org/2000/svg"></svg></button></div></div><div class="snippet-clipboard-content notranslate position-relative overflow-auto" id="bkmrk--3"><div class="zeroclipboard-container"><svg aria-hidden="true" class="octicon octicon-copy js-clipboard-copy-icon" data-view-component="true" height="16" version="1.1" viewbox="0 0 16 16" width="16"></svg>  
</div></div>#### Release process

Versioning of the main iam-components helm chart is handled based on the labels used in the pull requests used to introduce changes and is enforced in the [build pipeline](https://github.com/DOME-Marketplace/iam-components/tree/main/.github/workflows). The requester and reviewers must set the label according to the [SemVer 2.0.0](https://semver.org/) versioning scheme.

Versioning of the components and sub-charts is recommended to use the same scheme.

#### Troubleshooting

> To be filled once feedback from integrators comes in