Mirrors/api-guidelines
1
0
Fork 0
mirror of https://github.com/adidas/api-guidelines.git synced 2025-10-25 15:19:19 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #90 from adiglp/release/update-guidelines

Browse Source 
Release/update guidelines
This commit is contained in:
Cesareo
2025-06-10 17:25:02 +02:00
committed by GitHub
parent 1312cd0a8d 4e58982220
commit f7e34bf1b5
15 changed files with 108 additions and 92 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
asynchronous-api-guidelines/kafka-asynchronous-guidelines/README.md
View File

@@ -1,6 +1,6 @@
# Kafka Asynchronous Guidelines
There are several technologies to implement event-driven architectures, but this section is going to focus on the predominant technology on this subject : Apache Kafka.
There are several technologies to implement event-driven architectures, but this section is going to focus on the predominant technology on this subject: Apache Kafka.
**Apache Kafka** can be considered as a Streaming Platform which relies on the several concepts:

2
asynchronous-api-guidelines/kafka-asynchronous-guidelines/a_introduction/README.md
View File

@@ -2,7 +2,7 @@
This section is specific to the definition of API specs with [AsyncAPI](https://www.asyncapi.com/) for Kafka protocol.
Also, take into account that across the section there will be multiple references to this [AsyncAPI reference specification](https://design.api.3stripes.io/apis/adidas/asyncapi-adoption-initiative/1.0.0) which is publicly available for reference. 
Also, take into account that across the section there will be multiple references to this [AsyncAPI reference specification](https://design.api.3stripes.io/apis/adidas/asyncapi-adoption-initiative/1.0.0) which is publicly available for reference.
#### Kafka to AsyncAPI concept mapping

24
asynchronous-api-guidelines/kafka-asynchronous-guidelines/a_introduction/why-asyncapi.md
View File

@@ -7,24 +7,24 @@ It supports various messaging protocols, including MQTT, Web Socket, Kafka, AMQP
The benefits of using AsyncAPI are, amongst others:
* Standardization
* AsyncAPI defines a STANDARD format (YAML or JSON) for describing asynchronous APIs.
* By defining the structure of messages, channels, and events, you can ensure that all components adhere to the same conventions.
* Using a single standard ensures consistency in the design and documentation of all your asynchronous APIs.
* This simplifies integration, maintenance, and troubleshooting across different parts of your system.
* AsyncAPI defines a STANDARD format (YAML or JSON) for describing asynchronous APIs
* By defining the structure of messages, channels, and events, you can ensure that all components adhere to the same conventions
* Using a single standard ensures consistency in the design and documentation of all your asynchronous APIs
* This simplifies integration, maintenance, and troubleshooting across different parts of your system
* Improved Developer Experience
* AsyncAPI documents the messages being exchanged, their structure, and the events triggered by them.
* It provides developers with a clear picture of how to interact with the API, what data to expect, and how to interpret responses without digging into the implementation details. 
* AsyncAPI documents the messages being exchanged, their structure, and the events triggered by them
* It provides developers with a clear picture of how to interact with the API, what data to expect, and how to interpret responses without digging into the implementation details
* Code scaffolding
* Using tools like asyncapi-generator allow to easily generate the skeleton of applications that can work with the resources described in the spec.
* This can be done in different programming languages (Python, Java, Node.js. ...), reducing significantly the development time and the coding errors.
* Design-first approach: It encourages designing the API first before writing code, leading to better planned and more reliable APIs.
* Using tools like asyncapi-generator allow to easily generate the skeleton of applications that can work with the resources described in the spec
* This can be done in different programming languages (Python, Java, Node.js. ...), reducing significantly the development time and the coding errors
* Design-first approach: It encourages designing the API first before writing code, leading to better planned and more reliable APIs
In addition to those benefits, Platform & Engineering is working hard to create a data catalogue built upon AsyncAPI that allows to have a good level of discoverability, allowing teams to be able to find exactly the data they need with regards to any data object in the company.
Questions like:
* Who is responsible for a specific data object
* Where is that data hosted
* Which kind of information is available
* Who is responsible for a specific data object?
* Where is that data hosted?
* Which kind of information is available?
Will be easy to answer once this catalogue is in place. Also, it is important to have a good discoverability and search & filtering capabilities.

2
asynchronous-api-guidelines/kafka-asynchronous-guidelines/b_asyncapi_version.md
View File

@@ -5,6 +5,6 @@ Any version of AsyncAPI **MAY** be used for spec definitions.
However, to be aligned with adidas tooling, spec versions **SHOULD** be _v2.6.0_, because to the date of this document creation (April 2024) this is the highest supported version on Swaggerhub, the current API portal to render, discover and publish specs.
```yaml
asyncapi: 2.6.0
asyncapi: '2.6.0'
...
```

2
asynchronous-api-guidelines/kafka-asynchronous-guidelines/c_internal_public_specs.md
View File

@@ -1,6 +1,6 @@
# Internal vs Public Specifications
AsyncAPI specs **MAY** be created both for public APIs or for internal APIs. 
AsyncAPI specs **MAY** be created both for public APIs or for internal APIs:
* Public APIs are those who are created to be consumed by others
* Internal APIs are only for development teams for a particular project

6
asynchronous-api-guidelines/kafka-asynchronous-guidelines/f_schema_data_evolution/README.md
View File

@@ -17,9 +17,9 @@ As a summary, the available compatibility types are listed below:
| Mode | Description |
| -------------------- | --------------------------------------------------------------------------- |
| BACKWARD | new schema versions are backward compatible with older versions |
| BACKWARD\_TRANSITIVE | backward compatibility across all schema versions, not just the latest one. |
| BACKWARD\_TRANSITIVE | backward compatibility across all schema versions, not just the latest one |
| FORWARD | new schema versions are compatible with older consumer versions |
| FORWARD\_TRANSITIVE | forward compatibility across all schema versions. |
| FORWARD\_TRANSITIVE | forward compatibility across all schema versions |
| FULL | both backward and forward compatibility with the latest schema version |
| FULL\_TRANSITIVE | both backward and forward compatibility with all schema versions |
| NONE | schema compatibility checks are disabled |
@@ -28,7 +28,7 @@ As a summary, the available compatibility types are listed below:
#### Upgrading process of clients based on compatibility
Depending on the compatibility mode, the process of upgrading producers/consumers will be different based on the compatibility mode enabled.
Depending on the compatibility mode, the process of upgrading producers/consumers will be different based on the compatibility mode enabled:
* NONE
* As there are no compatibility checks, no order will grant a smooth transition

2
asynchronous-api-guidelines/kafka-asynchronous-guidelines/f_schema_data_evolution/full-compatibility.md
View File

@@ -2,7 +2,7 @@
This is a combination of both compatibility types (backward and forward). It also has 2 variants:
* FULL - Backward and forward compatible between schemas X and X - 1.
* FULL - Backward and forward compatible between schemas X and X - 1
* FULL\_TRANSITIVE - Backward and forward compatible between schemas X and all previous ones (X - 1, X - 2, ...)
**Important** Once more, FULL\_TRANSITIVE is the default compatibility mode in adidas, it is set at cluster level and all new schemas will inherit it

2
asynchronous-api-guidelines/kafka-asynchronous-guidelines/f_self_contained_specs/e_meaningful_descriptions.md
View File

@@ -1,3 +1,3 @@
# Meaningful Descriptions
AsyncAPI specs **MUST** make use of the _description_ fields, including there meaningful information to understand the purpose of the different elements. 
AsyncAPI specs **MUST** make use of the _description_ fields, including there meaningful information to understand the purpose of the different elements.

2
asynchronous-api-guidelines/kafka-asynchronous-guidelines/g_key_value_format.md
View File

@@ -2,6 +2,6 @@
Kafka messages **MAY** include a key, which needs to be properly designed to have a good balance of data across partitions.
The message key and the payload (often called value) can be serialized independently and can have different formats. For example, the value of the message can be sent in AVRO format, while the message key can be a primitive type (string). 
The message key and the payload (often called value) can be serialized independently and can have different formats. For example, the value of the message can be sent in AVRO format, while the message key can be a primitive type (string).
Message keys **SHOULD** be kept as simple as possible and use a primitive type when possible.

4
asynchronous-api-guidelines/kafka-asynchronous-guidelines/h_asyncapi_id.md
View File

@@ -2,10 +2,10 @@
According to [AsyncAPI documentation](https://v2.asyncapi.com/docs/reference/specification/v2.6.0#A2SIdString), every AsyncAPI spec **SHOULD** use a unique identifier for the application being defined, following RFC-3986.
More concretely, ASyncAPI specs created in adidas should use the following pattern
More concretely, ASyncAPI specs created in adidas should use the following pattern:
```yaml
...
id: urn:fdp:adidas:com:namespace:asyncapi_reference_spec
id: 'urn:fdp:adidas:com:namespace:asyncapi_reference_spec'
...
```

16
asynchronous-api-guidelines/kafka-asynchronous-guidelines/i_servers.md
View File

@@ -4,20 +4,20 @@ All AsyncAPI specs **MUST** include a _servers_ section including references to
For example, in Swaggerhub it is possible to refer to those reusable definitions through:
[https://design.api.3stripes.io/domains/adidas/asyncapi\_adoption\_commons/1.0.0](https://design.api.3stripes.io/domains/adidas/asyncapi\_adoption\_commons/1.0.0)
[https://design.api.3stripes.io/domains/adidas/cluster-landscape/1.0.0](https://design.api.3stripes.io/domains/adidas/cluster-landscape/1.0.0)
more specifically, it is possible to add the right Kafka servers like this:
```yaml
...
servers:
pivotalDev:
$ref: https://design.api.3stripes.io/v1/domains/adidas/asyncapi_adoption_commons/1.0.0#/components/servers/pivotalDev
pivotalSit:
$ref: https://design.api.3stripes.io/v1/domains/adidas/asyncapi_adoption_commons/1.0.0#/components/servers/pivotalSit
pivotalPro:
$ref: https://design.api.3stripes.io/v1/domains/adidas/asyncapi_adoption_commons/1.0.0#/components/servers/pivotalPro
playground-dev:
$ref: 'https://design.api.3stripes.io/v1/domains/adidas/cluster-landscape/1.0.0#/components/servers/playground-dev'
playground-sit:
$ref: 'https://design.api.3stripes.io/v1/domains/adidas/cluster-landscape/1.0.0#/components/servers/playground-sit'
playground-pro:
$ref: 'https://design.api.3stripes.io/v1/domains/adidas/cluster-landscape/1.0.0#/components/servers/playground-pro'
...
```
**Important note** Don't forget to include '_/v1/_' in the URL of the domain
**Important note** Don't forget to include '_/v1/_' in the URL of the domain.

86
asynchronous-api-guidelines/kafka-asynchronous-guidelines/j_channels.md
View File

@@ -14,14 +14,14 @@ All AsyncAPI specs **MUST** include definitions for the channels (Kafka topics)
* External Docs
* Message details
In addition to those supported fields, it **MAY** be possible to use extension attributes (using the x- prefix) to specify specific configuration parameters and metadata. In so, the recommended attributes to use are :
In addition to those supported fields, it **MAY** be possible to use extension attributes (using the x- prefix) to specify specific configuration parameters and metadata. In so, the recommended attributes to use are:
* x-metadata
* To include additional configuration specific to your team or project
* x-configurations
* To include Kafka configuration parameters and producers/consumers
As the parameters can be different per environment, it is very convenient to add an additional level for the environment
As the parameters can be different per environment, it is very convenient to add an additional level for the environment.
As part of the publish/subscribe operations, the spec **SHOULD** specify the different Kafka clients currently consuming from the different topics for each cluster/environment. For this, the extended attributes x-producers and x-consumers will be used.
@@ -29,60 +29,60 @@ As part of the publish/subscribe operations, the spec **SHOULD** specify the dif
...
channels:
namespace.source.event.topic-name:
description: A description of the purpose of the topic and the contained information
servers: ["pivotalDev", "pivotalSit", "pivotalPro"]
description: 'A description of the purpose of the topic and the contained information'
servers: ['playground-dev', 'playground-sit', 'playground-pro']
x-metadata:
myField1: myValue1
myField2: myValue2
myField1: 'myValue1'
myField2: 'myValue2'
x-configurations:
pivotal.dev:
playground.dev:
kafka:
partitions: 12
replicas: 1
partitions: '12'
replicas: '1'
topicConfiguration:
min.insync.replicas: "1"
retention.ms: "2592000000"
pivotal.sit:
min.insync.replicas: '1'
retention.ms: '2592000000'
playground.sit:
kafka:
partitions: 12
replicas: 2
partitions: '12'
replicas: '2'
topicConfiguration:
min.insync.replicas: "1"
retention.ms: "2592000000"
min.insync.replicas: '1'
retention.ms: '2592000000'
    publish:
operationId: "producer"
summary: "Description for the operation"
description: "An extensive explanation about the operation"
operationId: 'producer'
summary: 'Description for the operation'
description: 'An extensive explanation about the operation'
security:
- producerAcl: []
tags:
- name: tagA
- name: tagB
- name: 'tagA'
- name: 'tagB'
x-producers:
pivotal.dev:
- producer1
- producer2
pivotal.sit:
- producer1
- producer2
pivotal.pro:
- producer3
- producer4
playground.dev:
- 'producer1'
- 'producer2'
playground.sit:
- 'producer1'
- 'producer2'
playground.pro:
- 'producer3'
- 'producer4'
externalDocs:
description: documentation
url: http://confluence.adidas.fdp/catalogue/myTopic
...
description: 'documentation'
url: 'http://confluence.adidas.fdp/catalogue/myTopic'
...
subscribe:
operationId: "consumer"
...
operationId: 'consumer'
...
x-consumers:
pivotal.dev:
- consumer1
- consumer2
pivotal.sit:
- consumer1
- consumer2
pivotal.pro:
- consumer3
playground.dev:
- 'consumer1'
- 'consumer2'
playground.sit:
- 'consumer1'
- 'consumer2'
playground.pro:
- 'consumer3'
...
```

30
asynchronous-api-guidelines/kafka-asynchronous-guidelines/k_schemas.md
View File

@@ -2,9 +2,9 @@
Kafka messages **SHOULD** use schemas (AVRO, JSON, Protobuf) registered in the Schema Registry to ensure compatibility between producers/consumers.
If so, always refer to the schema definitions directly in the schema registry instead of duplicating the schema definitions inline. This is to avoid double maintenance. 
If so, always refer to the schema definitions directly in the schema registry instead of duplicating the schema definitions inline. This is to avoid double maintenance.
An example directly taken from reference spec is shown below
An example directly taken from reference spec is shown below.
```yaml
...
@@ -14,18 +14,24 @@ channels:
publish:
...
message:
$ref: '#/components/messages/topic1Payload'
$ref: '#/components/messages/namespace.source.event.topic-name'
components:
...
schemas:
...
topic1SchemaValue:
schemaFormat: 'application/vnd.apache.avro;version=1.9.0'
payload:
$ref: https://sit-fdp-pivotal-schema-registry.api.3stripes.io/subjects/pea_fd_fdp.sample.test-value/versions/latest/schema
messages:
topic1Payload:
$ref: '#/components/schemas/topic1SchemaValue'
namespace.source.event.topic-name:
$ref: '#/components/schemas/namespace.source.event.topic-name'
...
schemas:
namespace.source.event.topic-name:
description: 'Schema[s] retrieved from Schema Registry'
schemaFormat: 'application/vnd.apache.avro;version=1.9.0'
payload:
$ref: 'https://dev-fdp-playground-schema-registry.api.3stripes.io/subjects/namespace.source.event.topic-name-value/versions/latest/schema'
bindings:
kafka:
key:
$ref: 'https://dev-fdp-playground-schema-registry.api.3stripes.io/subjects/namespace.source.event.topic-name-key/versions/latest/schema'
```
**Important note** The used schema is a very simple one, it is only used to illustrate how to refer to it.
**Important note** The used schema is a very simple one, it is only used to illustrate how to refer to it. In case the message doesn't use schema for the key, _bindings_ field can be omitted from the schema.

16
asynchronous-api-guidelines/kafka-asynchronous-guidelines/l_security_schemes.md
View File

@@ -1,6 +1,8 @@
# Security Schemes
Specs **MAY** use security schemas to reflect the fact that the Kafka servers use mTLS. It is something quite static at the moment so the recommendation is reuse the ones specified in the reference spec.
Specs **MAY** use security schemas to reflect the fact that the Kafka servers use mTLS or SASL. It is something quite static at the moment so the recommendation is refer to reusable definitions.
Below example includes references to other security schemas used by the Kafka servers.
```yaml
channels:
@@ -15,7 +17,15 @@ components:
securitySchemes:
...
consumerAcl:
type: X509
$ref: 'https://design.api.3stripes.io/v1/domains/adidas/cluster-landscape/1.0.0#/components/securitySchemes/consumerAcl'
producerAcl:
type: X509
$ref: 'https://design.api.3stripes.io/v1/domains/adidas/cluster-landscape/1.0.0#/components/securitySchemes/producerAcl'
mtlsV1Server:
$ref: 'https://design.api.3stripes.io/v1/domains/adidas/cluster-landscape/1.0.0#/components/securitySchemes/mtlsV1Server'
mtlsV2Server:
$ref: 'https://design.api.3stripes.io/v1/domains/adidas/cluster-landscape/1.0.0#/components/securitySchemes/mtlsV2Server'
saslV1Server:
$ref: 'https://design.api.3stripes.io/v1/domains/adidas/cluster-landscape/1.0.0#/components/securitySchemes/saslV1Server'
saslV2Server:
$ref: 'https://design.api.3stripes.io/v1/domains/adidas/cluster-landscape/1.0.0#/components/securitySchemes/saslV2Server'
```

4
asynchronous-api-guidelines/kafka-asynchronous-guidelines/m_external_docs.md
View File

@@ -5,6 +5,6 @@ The external docs **SHOULD** be used to refer to LeanIX fact-sheet associated to
```yaml
...
externalDocs:
description: LeanIX
url: https://adidas.leanix.net/adidasProduction/factsheet/Application/467ff391-876c-49ad-93bf-facafffc0178
description: 'LeanIX'
url: 'https://adidas.leanix.net/adidasProduction/factsheet/Application/467ff391-876c-49ad-93bf-facafffc0178'
```