Error handling for RESTful APIs

Introduction

When designing RESTful APIs, it is essential to pay sufficient attention to application error reporting. Simply returning a “400” (the HTTP equivalent of “an error has occurred”) is clearly not sufficient. Furthermore, it is important that error reporting is handled consistently across all RESTful APIs so that API consumers are not confronted with different mechanisms, structures, and naming conventions for different APIs.

The objective of this chapter is to introduce a standardized approach to error reporting.

A Standard: RFC-9457

The Dutch government API strategy designates the RFC-7807 standard as the basis for error reporting. However, this RFC has since been replaced by RFC-9457, and the recommendation to the sector is to follow this latest version. The differences between the two are minimal.

The content type to be used for error responses according to RFC-9457 is:

application/problem+json

The RFC-9457 standard defines a ‘Problem Details Record,’ which should include at least the following attributes:

In addition to the required fields, the RFC-9457 standard defines several optional elements:

Example:

"type": https://cmf.energysector.nl/reference-data/error-catalogue[/{categorie}]/{fout-type}",
"title": "Description of the error",
"status": 401,
"detail": "More details regarding the error can be found here",
"instance": "urn:uuid:ebd2e7f0-1b27-11e8-accf-0ed5f89f718b"

General rules

When implementing a Problem Details record, it is important to remember that not everything necessarily requires a new type. For some error messages, the HTTP status code is already sufficiently descriptive, and the detail attribute can be used to provide a more specific error message. For example:

{
 "type": "https://cmf.energysector.nl/reference-data/error-catalogue/default/http-503",
 "title": "Service Unavailable",
 "status": 503,
 "detail": "Database cannot be reached"
}

In this case, the HTTP status code already conveys the core issue, while detail provides additional context. No additional attributes are required.

When defining new problem types, the following rules must be observed:

1. Avoid exposing sensitive information in error messages, whether in human-readable text or additional attributes in the response payload, that could be exploited by third parties.
2. Do not include implementation-specific or technical details such as stack traces in the response.
3. Avoid defining problem types that are already clearly described by the HTTP status code. Example: A 403 Forbidden response for a PUT operation on a resource already clearly indicates that the resource cannot be overwritten. There is no need for an additional message stating the same. As a best practice, the type catalog should include standard entries for commonly used HTTP error codes to ensure consistent reporting.
4. It is allowed to extend the standard with additional fields. The API standard makes use of this flexibility to add extra records, allowing for arbitrarily complex error messages to be constructed in a standardized way.

Error types

We should aim to define a limited number of error types. Consider the following list, where the HTTP status code itself already provides a hint:

• Authentication / Authorisation Error
• Operation Precondition Error
• Communication Error
• Data Validation Error / Semantic Error
• Syntax Error
• Object(s) Not Found / Object(s) Deleted
• Parameter Validation Error / Missing Parameter(s) / Conflicting Parameters
• State Error (e.g., record locked, conflicting update, etc.)
• Too Many Results / Response Limit Reached
• Timeout / Server Busy / Server or Application Unreachable
• Software Error / Processing Error

Error Type Catalogue

Ideally, we should maintain a catalogue of error types. This catalogue should contain the following information:

1. Error Type

An URL that provides access to a specific entry in the catalogue. The format is:
https://<provider-host>/reference-data/error-catalogue/<category>/<error-type>

in more detail:

Example:

"https://cmf.energysector.nl/reference-data/error-catalogue/default/http-503"

2. Title

A textual description of the error type, readable by end users, which can be used as the title attribute in error responses.

3. HTTP Status

The HTTP error status code (or codes) that this error type applies to. Ideally, there should be a one-to-one mapping, but some types may be relevant for multiple status codes.

4. Detail Template

A string containing placeholders for parameters, such as:

"Conflicting parameters are: {param-list}"

This template, when expanded, provides the value for the detail field.

5. Extension Schema

A JSON schema that describes any extension attributes. This must align with an extension object as described in Enterprise Architect.

Generic ProblemDetails Type

Based on the above information, it is possible to define a generic model that can serve as the standard error response object for RESTful APIs:

Standard model for problem details

The root of this model is a ProblemDetails class, which is essentially a one-to-one implementation of the model from the RFC-9457 standard. An additional field, traceID, has been added to facilitate faster and more efficient error tracing and to correlate related messages.

The traceID format is designed to be compatible with OpenTelemetry. It consists of the following components: \<version>-\<trace_id>-\<span_id>-\<trace_flags>

Example:

00-80e1afed08e019fc1110464cfa66635c-7a085853722dc6d2-01

TraceID structure:

In principle, objects of the ProblemDetails type can be used directly as error responses, as they contain the relevant information needed for many generic errors. If additional details are required, an extension in the form of an array of ApplicationProblem objects can be used to provide feedback on application-specific errors.

Extension class: ApplicationProblem

An ApplicationProblem class can be used to report a specific category of errors. Each ApplicationProblem class can, in turn, include an optional list of ApplicationMessage classes, which are intended to provide additional details about the problem.

Attributes are defined as follows:

Extension class: ApplicationMessage

Each category typically defines its own detail records, represented by objects of type ApplicationMessage, which include the following attributes:

Implicit Enumerations

The categoryCode and code attributes function as implicit enumerations (i.e., the schema does not explicitly define a fixed list of values). To still allow range validation, the enumeration type includes two additional attributes:

• codeListName = The name of the enumeration list used; this can also be a URI where the list can be retrieved.
• codeListVersion = The version of the enumeration list. This is optional if the name already includes the version (such as in a URI).

Example

Below is an example of a complete error response for a 400 Bad Request error.

The recommendation is to use type, title, and detail to provide as much generic, consistent, and reusable error information as possible (i.e., general error details), while using the ApplicationProblem extension record to return detailed information.

This approach allows for a relatively small set of error types to cover a wide variety of errors.

{
  "type": "https://cmf.energysector.nl/reference-data/error-catalogue/user-errors/parameter-validation",
  "title": "One or more parameters did not validate correctly.",
  "status": 400,
  "detail": "Conflicting parameters are: A, B, C, D",
  "instance": "urn:uuid:4017fabc-1b28-11e8-accf-0ed5f89f718b",
  "traceID": "00-80e1afed08e019fc1110464cfa66635c-7a085853722dc6d2-01",
  "applicationProblem": [
  {
    "categoryCode": "PARAMETER_VALIDATION_ERROR",
    "severityCode": "ERROR",
    "message": [
    {
      "identifier": "VAL01", 
      "code": "CONFLICT", 
      "text": "Parameters A and B cannot occur at the same time."
    },
    {
      "identifier": "VAL12",
      "code": "VALUE",
      "text": "Value of parameter C must be greater than value of parameter D."
    },
    {
      "identifier": "VAL12",
      "code": "VALUE", 
      "text": "Value of parameter B must be greater than 100"
    }]
  },
  {
    "categoryCode": "RESULT_SET_SIZE",
    "severityCode": "WARNING",
    "message": [
    {
      "identifier": "WRN03",
      "code": "RESULT_SET",
      "text": "Specified parameters may result in a very large result-set, not all results might be available."
    }]
  }]
}