Mirrors/api-guidelines
1
0
Fork 0
mirror of https://github.com/adidas/api-guidelines.git synced 2025-10-25 15:19:19 +00:00
Code Issues Packages Projects Releases Wiki Activity

Changes related to Apiary and Mashery. Also related to LRTs cases and mentions to SwaggerHub.

Browse Source
This commit is contained in:
dediejes
2021-02-12 14:59:23 +01:00
parent d0b751e470
commit 82e9dd5bc9
21 changed files with 407 additions and 121 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
rest-api-guidelines/execution/long-running-tasks/README.md Normal file
View File

@@ -0,0 +1,10 @@
# Long Running Tasks
This section includes the recommended approaches to handling long runnint tasks (LRTs) in REST APIs.
You can identify a LRT quite easily. The main factor to consider are the metrics from latency of the endpoint. If it requiress tens of seconds even minutes we are facing a problem related to LRTs.
LRTs cannot be handled in a regular straight synchronous call. The amount of commited recources at the network, client and server levels are huge when connections are blocked for several minutes.
It is strongly recommended to follow a non-blocking solution as it is proposed in this section.

136
rest-api-guidelines/execution/long-running-tasks/callback.md Normal file
View File

@@ -0,0 +1,136 @@
# Callback
Callback or Webhooks are another way of handling long running tasks (LRTs). Callbacks are based on the subscription principle, whereas the API notifies the API Consumer in a different connection. This pattern is also applicable to the subscription to any kind of events to get notifications from your API.
The roles are:
- API Consumer / Subscriber
- API Producer / Publisher
If the chosen way is based on using callbacks, the response to such an asynchronous operation **MUST** return, in the case of success, the **202 Accepted** status code together with an `application/hal+json` representation of a new **task-tracking resource**.
> This pattern is described by [OAS v3.0.x](https://swagger.io/docs/specification/callbacks/).
## Subscription
The subscriber enrolls to specfic notifications. The subscriber resource **MUST** provide the information about the callback URL. Any data needed to require the execution of a task **MUST** be included in the request body.
The subscription is created by using the HTTP POST Request Method. It **SHOULD** be designed as follows:
1. Subscription is accepted
Return **201 Created** and representation of the current status. Content type: `application/hal+json`
The publisher resource **MUST** provide a UUID to identify the subscription.
2. Subscription is not accepted
Return **403 Forbidden** . Content type: `application/problem+json` with the problem detail information.
## Notification
The publisher resource **MUST** use callback URL provided by the subscriber. Any data with the output of the requested task **SHOULD** be sent to the subscriber in this request.
The callback request has to use the HTTP POST Request Method **SHOULD** as follows:
1. The subscriber accepts the callback. Content type: `application/hal+json`
Return **200 OK**.
2. The subscriber does not accept the callback
Return **403 Forbidden** . Content type: `application/problem+json` with the problem detail information.
## Cancel Subscription
The subscriber resource **MUST** include the UUID to identify the subscription.
It has to be used the HTTP PUT Request Method **SHOULD** as follows:
1. Subscription is accepted
Return **202 Accepted**. Content type: `application/hal+json`
2. Subscription is not accepted
Return **403 Forbidden** . Content type: `application/problem+json` with the problem detail information.
## Design Note
- The subscription pattern supports two main approaches:
- On one side, it can be **only-once**. The callback will be invoked only once by the publisher and it will be cancelled automatically after.
- On the other side, it can be **continuous**. In this case the subscription **MUST** be explicitly cancelled. Regarding the subscriber, its API is also the subject of the adidas API guidelines.
- The callback can be based on an Asynchronous/Streaming API topic. In this case the subscription is made as mentioned above but with the following differences in the workflow:
- The API Consumer does not send a callback URL in the initial request.
- The API Producer **SHOULD** provide the name of the topic and the UID of the task to correlate the input.
- It is up to the API Consumer to subscribe to the Asynchronous/Streaming API topic to receive the input from the provider. Please read the Asynchronous/Streaming API section.
### Example
1. **Settle the subscription**
```
POST /items/tasks/ HTTP/1.1
Content-Type: application/json
{
"callbackUrl": "https://myserver.com/send/callback/here"
}
...
HTTP/1.1 201 Created
Content-Type: application/hal+json
{
"_links": {
"self": { "href": "/items/tasks/4746" }
},
"message": "Your request to subscribe to the progress of the task has been accepted.",
"UUID": "4746"
}
```
2. **The Publisher sends the callback**
```
POST https://myserver.com/send/callback/here HTTP/1.1
{
"_links": {
"self": { "href": "/items/tasks/4746" }
},
"UUID": "4746",
{
<Data with the callback>
}
}
...
HTTP/1.1 200 Ok
Content-Type: application/hal+json
```
3. **Eventually the subscriber cancels the subscription**
```
PUT /feeds/tasks/1 HTTP/1.1
...
HTTP/1.1 202 Accepted
Content-Type: application/hal+json
{
"_links": {
"self": { "href": "/feeds/tasks/4746" }
},
"message": "Your subscription is cancelled."
}
```

82
rest-api-guidelines/execution/long-running-tasks/files-upload.md Normal file
View File

@@ -0,0 +1,82 @@
# Files Upload
The upload of files using a REST API endpoint is a common practice. It implies certain concerns taht have to be addressed in the design phase of the API.
The API Consumer performs a key role in this case. The MIME type in the Content-Type header of the request is an important factor for a successful operation. An operation that needs to upload binary files **SHOULD** uses a collection resource with the POST HTTP Request Method. When processing an existing resource the request message body **MUST** contain the right MIME type of the resources being processed.
## Main Issues
- Too long time periods in timeout settings, blocking open HTTP connections for too long. It makes the API less reliable and more error-prone as it is more vulnerable to network-related issues.
- Interrupted connections that can result into corrupted files and false response status to the API Consumer.
- No size limit in server can suppose an unacceptable load to the API operation in terms of resources, security and robustness as well as a huge increase in operational cost.
## Checklist in File Upload Operations
### Use the right MIME Type in the API Consumer Side
It is a common practice to use
[IANA](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types) distinguishes between two main generic types, **discrete** and **multipart**:
- Discrete types are types which represent a single file or medium, such as a single text, video, or music file.
- Multipart type represents a document that is comprised of multiple component parts, each of which may have its own individual MIME type. It can also encapsulate multiple files being sent together in one single transaction.
#### Using a Multipart Type
- multipart/form-data
- multipart/byteranges
Frameworks like Spring offfer support for multipart files sending like the [MultipartFile](https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/web/multipart/MultipartFile.html) interface.
#### Using a Discrete Type
It is recommended to upload the file alone, with no other content in the request. This approach allows to include the MIME type corresponding to the specific type of file. For instance:
- Graphical file -> image/jpeg, image/gif, image/bmp, etc.
- Data file -> text/csv,
- Text file -> text/plain
- PDF -> application/pdf
etc.
It is also recommended to compress the file to be uploaded, then using these MIM types (examples):
- gzip -> application/gzip
- zip -> application/zip
- 7z -> application/x-7z-compressed
- tar -> application/x-tar
etc.
> You can find a complete reference about the MIME types [here](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types).
### Set Maximum Size Limit
The maximum size of the whole file **MUST** be set for the given endpoint/s in the APi upstream/backend service side.
The maximum size value depends on the use case and the expected payload in upload requests.
The settings **MUST** included in the upstream/backend service as a part of the configuration.
Otherwise, the API Gateway (Kong) **COULD** be configured enabling a maximum size of the payload for specific endpoint/s.
Frameworks like Spring includes configuration settings for multipart file uploading. The operation **SHOULD** be constrained as follows:
```
spring.servlet.multipart.enabled=true # enables multipart uploads
spring.servlet.multipart.file-size-threshold=2KB # the threshold after which files are written to disk.
spring.http.multipart.max-file-size=128KB # the total file size cannot exceed the amount o.
spring.http.multipart.max-request-size=128KB # the total request size for a multipart/form-data cannot exceed 128KB.
```
### Configure Properly all the Components
Load tests should give you metrics about the average latency of the operations. Use these metrics to calcuate the best value for the timeout settings in the upstream/backend service.
The API Gateway timeout settings have to be considered for the expected timeout values, aligned with the values in the upstream/backend service. Al other components in the infrastructure **MUST** be considered for the calculation of the final metrics.
```
|API Consumer/Client Timeout| ---> |External Load Balancer| ---> |API Gateway Timeout| ---> |Internal Load Balancer| ---> |Upstream/Backend Service Tiemout|
```
The approach based on too long timeout values is not acceptable. You **MUST** follow a fast-fail approach with a expected duration of the upload. If this time is exceeded a timeout error **SHOULD** be sent to the API Consumer. The maximum size limit **SHOULD** be consistent with the timeout value.
> Please also consider the client and API Gateway Timeout settings. In this case the lack of retrieval of a response during a too long upload operation can trigger a timeout error.

108
rest-api-guidelines/execution/long-running-tasks/polling.md Normal file
View File

@@ -0,0 +1,108 @@
# Polling
If an API operation can be considered as a long running task and the API Consumer can track its progress, the response to the LRT **MUST** return, in the case of success, the **202 Accepted** status code together with an `application/hal+json` representation of a new **task-tracking resource**.
## Task Tracking Resource
The task-tracking resource **SHOULD** convey the information about the status of an asynchronous task.
Retrieval of such a resource using the HTTP GET Request Method **SHOULD** be designed as follows:
1. Task is Still Processing
Return **200 OK** and representation of the current status.
2. Task Successfully Completed
Return **303 See Other** together with [HTTP Location Header](https://tools.ietf.org/html/rfc7231#section-7.1.2) with URI or a outcome resource.
3. Task Failed
Return **200 OK** and `application/problem+json` with the problem detail information on the task has failed.
## Design Note
The polling (task-tracking) operation requires a clear adaptation on the API Consumer side:
- Polling requests frequency depend on the type of operation and specific latency of thre resource.
- The identification of the resource has to be correlated along the series of polling requests. The API Consumer has to be able to save this ID and the API Producer has to be able to identify the progress of the operation with that ID.
- A security problem can be raised if a non-authorized client retrieves the response for a different resource ID. The authorization data and tasks in progress have to be strongly correlated and controlled to avoid consistency issues.
### Example
1. **Initiate the polling task**
```
POST /feeds/tasks/ HTTP/1.1
Content-Type: application/json
...
HTTP/1.1 202 Accepted
Content-Type: application/hal+json
Retry-After: 60
{
"_links": {
"self": { "href": "/feeds/tasks/1" }
},
"message": "Your task to generate feed has been accepted. Try query for result after 60 seconds.",
"retryAfter": 60
}
```
1. **Poll the task status: In progress**
```
GET /feeds/tasks/1 HTTP/1.1
...
HTTP/1.1 200 Ok
Content-Type: application/hal+json
Retry-After: 30
{
"_links": {
"self": { "href": "/feeds/tasks/1" }
},
"message": "Your feed is being generated. Try query for result after 30 seconds.",
"retryAfter": 30
}
```
1. **Poll the task status: Finished**
```
GET /feeds/tasks/1 HTTP/1.1
...
HTTP/1.1 303 See Other
Location: /feeds/1
Content-Location: /feeds/tasks/1
Content-Type: application/hal+json
{
"_links": {
"self": { "href": "/feeds/tasks/1" },
"feed": { "href": "/feeds/1" }
},
"message": "Your feed is ready."
}
```
1. **Poll the task status: Failure**
```
GET /feeds/tasks/1 HTTP/1.1
...
HTTP/1.1 200 OK
Content-Type: application/problem+json
{
"title": "Wrong input parameters",
"detail: "Missing required input parameter XYZ.",
"status": 400
}
```