From 2a0df79e63198f437ddb17492726f827fdd6f095 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Juan=20Manuel=20P=C3=A9rez?= Date: Thu, 9 May 2024 10:31:13 +0200 Subject: [PATCH] Content review, removing references to FDP, rephrasing some sentences, fixed some erratas. --- .../01_introduction/b_basic_concepts_edas.md | 6 ++++-- .../02_asynchronous_api_guidelines/c_immutability.md | 2 +- .../02_asynchronous_api_guidelines/e_schema_registration.md | 2 +- .../f_schema_data_evolution.md | 5 ++--- .../03_asyncapi_kafka_specs/a_introduction.md | 2 +- .../03_asyncapi_kafka_specs/d_spec_granularity.md | 2 +- .../03_asyncapi_kafka_specs/i_servers.md | 2 +- .../03_asyncapi_kafka_specs/n_tooling.md | 4 +--- 8 files changed, 12 insertions(+), 13 deletions(-) diff --git a/asynchronous-api-guidelines/01_introduction/b_basic_concepts_edas.md b/asynchronous-api-guidelines/01_introduction/b_basic_concepts_edas.md index ff15fee..9b73307 100644 --- a/asynchronous-api-guidelines/01_introduction/b_basic_concepts_edas.md +++ b/asynchronous-api-guidelines/01_introduction/b_basic_concepts_edas.md @@ -26,6 +26,8 @@ In most cases, EDAs are broker-centric, as seen in the diagram below. ![EDA overview](../../assets/eda_overview.png) +*The figure above was taken from AsyncAPI official documentation* + #### Problem statement Typically, the architectural landscape of a big company grows in complexity and as a result of that it is possible to end up with a bunch of direct connections between a myriad of different components or modules. @@ -61,12 +63,12 @@ There are several technologies to implement event-driven architectures, but this - Performance - Partitioned, replayable log (collection of messages appended sequentially to a file) - Data copied directly from disk buffer to network buffer (zero copy) without even being imported to the JVM - - Extreme throughput by using the concept of consumer group and + - Extreme throughput by using the concept of consumer group - Security - Secure encrypted connections using TLS client certificates - Multi-tenant management through quotas/acls - Client APIs on different programming languages : Go, Scala, Python, REST, JAVA, ... - - Stream processing APIs (currently Kafka Streams and ksqlDB) + - Stream processing APIs like Kafka Streams - Ecosystem of connectors to pull/push data from/to Kafka - Clean-up processes for storage optimization - Retention periods diff --git a/asynchronous-api-guidelines/02_asynchronous_api_guidelines/c_immutability.md b/asynchronous-api-guidelines/02_asynchronous_api_guidelines/c_immutability.md index 02f3220..14b8bec 100644 --- a/asynchronous-api-guidelines/02_asynchronous_api_guidelines/c_immutability.md +++ b/asynchronous-api-guidelines/02_asynchronous_api_guidelines/c_immutability.md @@ -4,6 +4,6 @@ ### Immutability -After agreement with the stakeholders the contract **MUST** be published in order to do it immutable. Changes to the API related to the data model, **MUST** be published in a schema registry. +After agreement with the stakeholders the contract **MUST** be published in order to make it immutable. Changes to the API related to the data model, **MUST** be published in a schema registry. Schema registry acts as a central location for storing and accessing the schemas of all published APIs. \ No newline at end of file diff --git a/asynchronous-api-guidelines/02_asynchronous_api_guidelines/e_schema_registration.md b/asynchronous-api-guidelines/02_asynchronous_api_guidelines/e_schema_registration.md index a75ae25..7e3da18 100644 --- a/asynchronous-api-guidelines/02_asynchronous_api_guidelines/e_schema_registration.md +++ b/asynchronous-api-guidelines/02_asynchronous_api_guidelines/e_schema_registration.md @@ -4,4 +4,4 @@ ### Automatic schema registration -Applications **MUST NOT** enable automatic registration of schemas because FDP's operational model for the Schema Registry relies on GitOps (every operation is done through GIT PRs + automated pipelines) \ No newline at end of file +Applications **MUST NOT** enable automatic registration of schemas because in adidas schemas have a separate lifecycle, intended to be independent from API contract and API implementing code. \ No newline at end of file diff --git a/asynchronous-api-guidelines/02_asynchronous_api_guidelines/f_schema_data_evolution.md b/asynchronous-api-guidelines/02_asynchronous_api_guidelines/f_schema_data_evolution.md index 8878564..6507e6d 100644 --- a/asynchronous-api-guidelines/02_asynchronous_api_guidelines/f_schema_data_evolution.md +++ b/asynchronous-api-guidelines/02_asynchronous_api_guidelines/f_schema_data_evolution.md @@ -6,7 +6,7 @@ All asynchronous APIs **SHOULD** leverage Schema Registry to ensure consistency across consumers/producers with regards to message structure and ensuring compatibility across different versions. -The default compatibility mode in Schema Registry is FULL_TRANSITIVE. This is the more restrictive compatibility mode, but others are also available. More on this on the subsection below. +The default compatibility mode **SHOULD** be FULL_TRANSITIVE, which is the default compatibility mode in adidas for Schema Registry. Check the sections below to know more about compatibility modes. #### Compatibility modes @@ -67,8 +67,7 @@ This is a combination of both compatibility types (backward and forward). It als - FULL - Backward and forward compatible between schemas X and X - 1. - FULL_TRANSITIVE - Backward and forward compatible between schemas X and all previous ones (X - 1, X - 2, ...) - -**Important** FULL_TRANSITIVE is the default compatibility mode in FDP, it is set at cluster level and all new schemas will inherit it +**Important** Once more, FULL_TRANSITIVE is the default compatibility mode in adidas, it is set at cluster level and all new schemas will inherit it This mode is preserved only if using the following operations diff --git a/asynchronous-api-guidelines/03_asyncapi_kafka_specs/a_introduction.md b/asynchronous-api-guidelines/03_asyncapi_kafka_specs/a_introduction.md index 293509d..53f37b8 100644 --- a/asynchronous-api-guidelines/03_asyncapi_kafka_specs/a_introduction.md +++ b/asynchronous-api-guidelines/03_asyncapi_kafka_specs/a_introduction.md @@ -12,7 +12,7 @@ Also, take into account that across the section there will be multiple reference Event-driven architectures are becoming increasingly popular for building scalable, responsive, and efficient applications. AsyncAPI plays a crucial role in this landscape by offering a standardized way to describe asynchronous APIs, similar to how OpenAPI does for REST APIs. AsyncAPI seeks to make the development, maintenance, and testing of asynchronous APIs easier by providing a machine-readable specification. -It supports various messaging protocols, including MQTT, WebSocket, Kafka, AMQP, and more, making it versatile for different use cases. In adidas, we will use it mainly to document Kafka resources created in FDP but nothing prevents you from using it for a different purpose. +It supports various messaging protocols, including MQTT, WebSocket, Kafka, AMQP, and more, making it versatile for different use cases. In adidas, ASyncAPI is used mainly to document Kafka resources created across the company in the scope of the Streaming Platform, but nothing prevents you from using it for a different purpose. The benefits of using AsyncAPI are, amongst others: diff --git a/asynchronous-api-guidelines/03_asyncapi_kafka_specs/d_spec_granularity.md b/asynchronous-api-guidelines/03_asyncapi_kafka_specs/d_spec_granularity.md index 8bf7253..8572953 100644 --- a/asynchronous-api-guidelines/03_asyncapi_kafka_specs/d_spec_granularity.md +++ b/asynchronous-api-guidelines/03_asyncapi_kafka_specs/d_spec_granularity.md @@ -4,7 +4,7 @@ ### Spec granularity -In Fast Data Platform (FDP) all resources are grouped by namespace. +In adidas all resources are grouped by namespace. For that reason specs **SHOULD** be created with a relation 1:1 with namespaces. In other words, every namespace will have an AsyncAPI spec including all the assets belonging to that namespace. diff --git a/asynchronous-api-guidelines/03_asyncapi_kafka_specs/i_servers.md b/asynchronous-api-guidelines/03_asyncapi_kafka_specs/i_servers.md index 6d20fb3..1920541 100644 --- a/asynchronous-api-guidelines/03_asyncapi_kafka_specs/i_servers.md +++ b/asynchronous-api-guidelines/03_asyncapi_kafka_specs/i_servers.md @@ -4,7 +4,7 @@ ### Servers -All AsyncAPI specs **MUST** include a servers section including references to the right Kafka clusters, defined and maintained by FDP team and made available through domains in Swaggerhub. +All AsyncAPI specs **MUST** include a servers section including references to the right Kafka clusters, defined and maintained globally and made available through domains in Swaggerhub. Those definitions are handled in Swaggerhub as reusable domains publicly available: diff --git a/asynchronous-api-guidelines/03_asyncapi_kafka_specs/n_tooling.md b/asynchronous-api-guidelines/03_asyncapi_kafka_specs/n_tooling.md index 0c3d701..428b462 100644 --- a/asynchronous-api-guidelines/03_asyncapi_kafka_specs/n_tooling.md +++ b/asynchronous-api-guidelines/03_asyncapi_kafka_specs/n_tooling.md @@ -8,9 +8,7 @@ The current platform available in adidas to design, host, and render AsyncAPI sp Every AsyncAPI spec **MUST** be hosted in Swaggerhub under the *adidas* organization. -In the future, Fast Data Platform team will provide mechanisms to auto generate your specs as part of the self-service initiative that is ongoing. - -But until then, the specs will be created manually in the platform following the API-first approach if possible. +In the future, some mechanisms will be provided to auto generate AsyncAPI specs automatically from the assets created in the Streaming Platform. Until then, those specs will be created manually in the platform following the API-first approach if possible. **Important note** Swaggerhub has limited capabilities with regards to discoverability, search and filtering of APIs. Other alternatives are being evaluated. Any upcoming decision impacting this will be reflected in this document in the future.