Mirrors/api-guidelines
1
0
Fork 0
mirror of https://github.com/adidas/api-guidelines.git synced 2025-10-25 15:19:19 +00:00
Code Issues Packages Projects Releases Wiki Activity

Added asynchronous APIs guidelines

Browse Source
This commit is contained in:
Juan Manuel Pérez
2024-04-22 16:38:33 +02:00
parent 56ccedbc84
commit 153f933dc2
9 changed files with 496 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
asynchronous-api-guidelines/03_asyncapi_kafka_specs/a_introduction.md Normal file
View File

@@ -0,0 +1,60 @@
# adidas Asynchronous API guidelines
## Introduction to AsyncAPI spec definitions for Kafka protocol
This section is specific to the definition of API specs with [AsyncAPI](https://www.asyncapi.com/) for Kafka protocol.
Also, take into account that across the section there will be multiple references to this [AsyncAPI reference spec](https://design.api.3stripes.io/apis/adidas/asyncapi-adoption-initiative/1.0.0) which is publicly available for reference. 
### Basic concepts about AsyncAPI
#### Why AsyncAPI?
Event-driven architectures are becoming increasingly popular for building scalable, responsive, and efficient applications. AsyncAPI plays a crucial role in this landscape by offering a standardized way to describe asynchronous APIs, similar to how OpenAPI does for REST APIs. AsyncAPI seeks to make the development, maintenance, and testing of asynchronous APIs easier by providing a machine-readable specification.
It supports various messaging protocols, including MQTT, WebSocket, Kafka, AMQP, and more, making it versatile for different use cases. In adidas, we will use it mainly to document Kafka resources created in FDP but nothing prevents you from using it for a different purpose.
The benefits of using AsyncAPI are, amongst others:
- Standardization
- AsyncAPI defines a STANDARD format (YAML or JSON) for describing asynchronous APIs.
- By defining the structure of messages, channels, and events, you can ensure that all components adhere to the same conventions.
- Using a single standard ensures consistency in the design and documentation of all your asynchronous APIs.
- This simplifies integration, maintenance, and troubleshooting across different parts of your system.
- Improved Developer Experience
- AsyncAPI documents the messages being exchanged, their structure, and the events triggered by them.
- It provides developers with a clear picture of how to interact with the API, what data to expect, and how to interpret responses without digging into the implementation details. 
- Code scaffolding
- Using tools like asyncapi-generator allow to easily generate the skeleton of applications that can work with the resources described in the spec.
- This can be done in different programming languages (Python, Java, Node.js. ...), reducing significantly the development time and the coding errors.
- Design-first approach: It encourages designing the API first before writing code, leading to better planned and more reliable APIs.
In addition to those benefits, Platform & Engineering is working hard to create a data catalogue built upon AsyncAPI that allows to have a good level of discoverability, allowing teams to be able to find exactly the data they need with regards to any data object in the company.
Questions like:
- Who is responsible for a specific data object
- Where is that data hosted
- Which kind of information is available
Will be easy to answer once this catalogue is in place. Also, it is important to have a good discoverability and search & filtering capabilities.
#### Kafka to AsyncAPI concept mapping
|Kafka Concept|AsyncAPI Concept|
|-------------|----------------|
|broker|server|
|topic|channel|
|consumer|subscriber|
|producer|publisher|
#### First level items in AsyncAPI structure
|Element|Meaning|
|-------|-------|
|asyncapi|Specifies the AsyncAPI specification version|
|info|Provides metadata about the API such as the version, title and description|
|servers|Describes servers where the API is available|
|channels|Defines the channels through which messages are received/published|
|components|Reusable elements to be references across the spec|

249
asynchronous-api-guidelines/03_asyncapi_kafka_specs/b_guidelines.md Normal file
View File

@@ -0,0 +1,249 @@
# adidas Asynchronous API guidelines
## AsyncAPI guidelines for Kafka
### AsyncAPI version
Any version of AsyncAPI **MAY** be used for spec definitions.
However, to be aligned with adidas tooling, spec versions **SHOULD** be *v2.6.0*, because to the date of this document creation (April 2024) this is the highest supported version on Swaggerhub, the current API portal to render, discover and publish specs.
```yaml
asyncapi: 2.6.0
...
```
### Internal vs public specs
AsyncAPI specs **MAY** be created both for public APIs or for internal APIs. 
- Public APIs are those who are created to be consumed by others
- Internal APIs are only for development teams for a particular project
There are no differences with regards to the spec definition, but internal APIs **SHOULD** have restricted access limited only to the internal development team for a particular project or product.
This access control is handled through Role-Based Access Control (RBAC) implemented in Swaggerhub.
### Spec granularity
In FDP all resources are grouped by namespace.
For that reason specs **SHOULD** be created with a relation 1:1 with namespaces. In other words, every namespace will have an AsyncAPI spec including all the assets belonging to that namespace.
Different granularities **MAY** be chosen depending on the needs. 
### Meaningful descriptions
All fields included in the specs **MUST** include a proper description. 
### Self-contained specs
All AsyncAPI specs **SHOULD** include as much information as needed in order to make the spec self-contained and clearly documented
### Contact information
AsyncAPI specs **MUST** include at least one main contact under the info.contact section.
The spec only allows to include one contact there, but it **MAY** also include additional contacts using extension fields. For example:
```yaml
...
info:
...
contact:
name: "Main point of contact"
email: "team_dl@adidas.com"
x-additional-responsibles:
- person2@adidas.com
- person3@adidas.com
- person4@adidas.com
```
### AsyncAPI ID
According to [AsyncAPI documentation](https://v2.asyncapi.com/docs/reference/specification/v2.6.0#A2SIdString), every AsyncAPI spec **SHOULD** use a unique identifier for the application being defined, following RFC-3986.
More concretely, ASyncAPI specs created in adidas should use the following pattern
```yaml
...
id: urn:fdp:adidas:com:namespace:asyncapi_reference_spec
...
```
### Servers
All AsyncAPI specs **MUST** include a servers section including references to the right Kafka clusters, defined and maintained by FDP team and made available through domains in Swaggerhub.
Those definitions are handled in Swaggerhub as reusable domains publicly available:
https://design.api.3stripes.io/domains/adidas/asyncapi_adoption_commons/1.0.0
that can be referred from any spec, picking the right kafka servers as required (see example below).
```yaml
...
servers:
pivotalDev:
$ref: https://design.api.3stripes.io/v1/domains/adidas/asyncapi_adoption_commons/1.0.0#/components/servers/pivotalDev
pivotalSit:
$ref: https://design.api.3stripes.io/v1/domains/adidas/asyncapi_adoption_commons/1.0.0#/components/servers/pivotalSit
pivotalPro:
$ref: https://design.api.3stripes.io/v1/domains/adidas/asyncapi_adoption_commons/1.0.0#/components/servers/pivotalPro
...
```
**Important note** Don't forget to include '*/v1/*' in the URL of the domain
### Channels
All AsyncAPI specs **MUST** include definitions for the channels (kafka topics) including:
- Description of the topic
- Servers in which the topic is available
- This is a reference to one of the server identifiers included in the servers section
- publish/subscribe operations
- Operation ID
- Summary or short description for the operation
- Description for the operation
- Security schemes
- Tags
- External Docs
- Message details
In addition to those supported fields, it **MAY** be possible to use extension attributes (using the x- prefix) to specify specific configuration parameters and metadata. In so, the recommended attributes to use are :
- x-metadata
- To include additional configuration specific to your team or project
- x-configurations
- To include Kafka configuration parameters and producers/consumers
As the parameters can be different per environment, it is very convenient to add an additional level for the environment
As part of the publish/subscribe operations, the spec **SHOULD** specify the different kafka clients currently consuming from the different topics for each cluster/environment. For this, the extended attributes x-producers and x-consumers will be used.
```yaml
...
channels:
namespace.source.event.topic-name:
description: A description of the purpose of the topic and the contained information
servers: ["pivotalDev", "pivotalSit", "pivotalPro"]
x-metadata:
myField1: myValue1
myField2: myValue2
x-configurations:
pivotal.dev:
kafka:
partitions: 12
replicas: 1
topicConfiguration:
min.insync.replicas: "1"
retention.ms: "2592000000"
pivotal.sit:
kafka:
partitions: 12
replicas: 2
topicConfiguration:
min.insync.replicas: "1"
retention.ms: "2592000000"
    publish:
operationId: "producer"
summary: "Description for the operation"
description: "An extensive explanation about the operation"
security:
- producerAcl: []
tags:
- name: tagA
- name: tagB
x-producers:
pivotal.dev:
- producer1
- producer2
pivotal.sit:
- producer1
- producer2
pivotal.pro:
- producer3
- producer4
externalDocs:
description: documentation
url: http://confluence.adidas.fdp/catalogue/myTopic
...
subscribe:
operationId: "consumer"
...
x-consumers:
pivotal.dev:
- consumer1
- consumer2
pivotal.sit:
- consumer1
- consumer2
pivotal.pro:
- consumer3
...
```
### Schemas
Kafka messages **SHOULD** use schemas (AVRO, Json, Protobuf) registered in the Schema Registry to ensure compatibility between producers/consumers.
If so, always refer to the schema definitions directly in the schema registry instead of duplicating the schema definitions inline. This is to avoid double maintenance. 
An example directly taken from reference spec is shown below
```yaml
...
channels:
namespace.source.event.topic-name:
...
publish:
...
message:
$ref: '#/components/messages/topic1Payload'
components:
...
schemas:
...
topic1SchemaValue:
schemaFormat: 'application/vnd.apache.avro;version=1.9.0'
payload:
$ref: https://pro-fdp-pivotal-schema-registry.api.3stripes.io/subjects/sap_retail_pricing.sap_retail.master.prices-value/versions/latest/schema
messages:
topic1Payload:
$ref: '#/components/schemas/topic1SchemaValue'
```
**Important note** To have a reference to a real schema, a schema from sap_retail_pricing in pivotal.pro schema registry was used in the spec reference as an example
### Security Schemes
Specs **MAY** use security schemas to reflect the fact that the kafka servers use mTLS. It is something quite static at the moment so the recommendation is reuse the ones specified in the reference spec.
channels:
namespace.source.event.topic-name:
...
publish:
...
security:
- producerAcl: []
...
components:
securitySchemes:
...
consumerAcl:
type: X509
producerAcl:
type: X509
### External docs
The external docs **SHOULD** be used to refer to LeanIX factsheet associated to the spec.
```yaml
...
externalDocs:
description: LeanIX
url: https://adidas.leanix.net/adidasProduction/factsheet/Application/467ff391-876c-49ad-93bf-facafffc0178
```

44
asynchronous-api-guidelines/03_asyncapi_kafka_specs/c_tooling.md Normal file
View File

@@ -0,0 +1,44 @@
# adidas Asynchronous API guidelines
## AsyncAPI tools
### API Design platform
The current platform available in adidas to design, host, and render AsyncAPI specs is [Swaggerhub](https://design.api.3stripes.io/).
Every AsyncAPI spec **MUST** be hosted in Swaggerhub under the *adidas* organization.
In the future, Fast Data Platform team will provide mechanisms to auto generate your specs as part of the self-service initiative that is ongoing.
But until then, the specs will be created manually in the platform following the API-first approach if possible.
**Important note** Swaggerhub has limited capabilities with regards to discoverability, search and filtering of APIs. Other alternatives are being evaluated. Any upcoming decision impacting this will be reflected in this document in the future.
### Editors
Aside from Swaggerhub editing capabilities, other alternative editor options are available:
- AsyncAPI Studio: A web-based editor designed specifically for creating and validating AsyncAPI documents.
- Visual Studio Code: VS Code can be extended with plugins like "AsyncAPI for VS Code" to provide AsyncAPI-specific features,  for editing AsyncAPI files.
### Command Line Interface (CLI) tool
Unfortunately, Swaggerhub is not offering a Command Line Interface (CLI) tool which allows including this capability as part of CICD workflows. 
For this, there is an official AsyncAPI CLI tool which can be checked here: https://www.asyncapi.com/tools/cli. This includes a validator against the AsyncAPI spec, templated generators, version conversion, spec optimizer, bundler, etc.
For example, to validate a yaml spec file:
```
asyncapi validate --file your-asyncapi-file.yaml
```
### Generators
These tools are capable of generate a variety of outputs from any valid AsyncAPI spec, including:
- API documentation in various formats like HTML, Markdown, or OpenAPI
- Code samples in various programming languages like Python, Java, and Node.js based on your API definition. 
- Functionally complete applications
There is an official generator tool which can be checked here: https://www.asyncapi.com/docs/tools/generator.