Base your API error response model on a solid standard with the Problem Details RFC

If you’re trying to come up with an error response model for your API, but are finding it difficult to settle on a format that is simple, yet future-proofed; look no further!

In this article, I expound the virtues of the ‘Problem Details’ RFC standard and why I believe it’s a no-brainer to use it for your project rather than trying to reinvent the wheel.

I explain the key aspects of the standard and provide code samples to help you understand how you can implement it in your application.

Why use a standard?

Aside from helping you to avoid agonising over the exact format of your API error model, it makes sense to use a standard model for a number of other reasons.

For a start, why waste time rethinking the structure of your error response for every new API you create? Basing your error model on a solid standard will save you valuable time. It’s like making use of design patterns and one of the key reasons that we use patterns is to avoid solving the same problem multiple times.

Let’s also consider this from an API client point of view. Why should developers who are integrating with our APIs have to rewrite their error handling logic for every new API we provide? It makes much more sense to keep things consistent for the client’s sake too.

I believe it is highly valuable to relay error data to clients in a way that is both human-readable and machine-readable. By choosing this path we will make it straightforward for developers to implement error handling for our API and we will make it possible for future systems to respond to errors intelligently by working to a standard format.

Problems Details explained

Problem Details for HTTP APIs is an RFC standard. It describes an API error response format that is both machine-readable and easy to understand for humans.

To make this more concrete, let’s look at an example error response taken from the RFC.

HTTP/1.1 403 Forbidden
Content-Type: application/problem+json
Content-Language: en

{
 "type": "https://example.com/probs/out-of-credit",
 "title": "You do not have enough credit.",
 "detail": "Your current balance is 30, but that costs 50.",
 "instance": "/account/12345/msgs/abc",
 "balance": 30,
 "accounts": ["/account/12345",
              "/account/67890"]
}

As you can see, the preferred format of the response is JSON. According to the RFC, the standard parts that make up the error response model are type, title, status, detail and instance.

Let’s look at the various fields within the model to see what they are intended for. I have adapted the descriptions below which are based on the official RFC descriptions.

type

A URI reference that identifies the problem type. The specification encourages that the type provided should be human-readable using HTML. If the member is not present its value is assumed to be "about:blank".

title

A short, human-readable summary of the problem type. It SHOULD NOT change from occurrence to occurrence of the problem, except for the purposes of localisation.

status

The HTTP status code generated by the server.

detail

A human-readable explanation that is specific to this occurrence of the problem.

instance

A URI reference that identifies the specific occurrence of the problem. It may or may not yield further information if dereferenced.

The good parts

One of the things I particularly like about the standard is that it clearly describes a base error model containing the properties that you must support. However, it also provides you with scope to extend the model should you need to.

The RFC permits what are known as ‘problem-specific extensions’. These allow you to expand on the basic model with any additional members that make sense for your application. The balance and account fields from the RFC sample located at the beginning of the previous section are examples of extension members.

In regards to the standard members of the model, my preference is to take a pragmatic approach.

For example, let’s consider the type property. For most of the projects I am working on, it isn’t practical to have a webpage dedicated to each type of possible error. Given that the standard specifically states that the value is assumed to be "about:blank" if it is not present, I usually leave this member out.

The title and detail members are straightforward to implement and I always make sure that an appropriate HTTP status code is returned for every possible error.

For the instance property, the most practical way I’ve found of implementing this is to define a URN that encapsulates additional information regarding the error.

Here is an example URN for reference.

urn:companyname:api:error:protocol:badRequest:f29f57d7-e1f8-4643-b226-fa18f15e9b71

Note that the colon (:) character is used to separate the URN into its constituent parts.

In this example, the URN can be broken down as follows.

urn

The standard URN prefix.

companyname

The Namespace Identifier (Company).

api

The first part of the Namespace Specific String (i.e. the API for our ‘company’).

error

The second part of the Namespace Specific String (i.e. this URN describes an error).

protocol

The third part of the Namespace Specific String (i.e. this is a generic HTTP protocol error as opposed to a custom one).

badRequest

The fourth part of the Namespace Specific String (i.e. the HTTP status code in JSON format).

f29f57d7-e1f8-4643-b226-fa18f15e9b71

The fifth part of the Namespace Specific String (i.e. a unique GUID which we can use for cross-referencing in our logs).

Of course, the structure of this URN can be adjusted to suit your own particular needs. However, I feel that the example described above is universal in many respects. If logged, this URN string would help us to identify what kind of error we are looking at very quickly.

Codifying the standard

Now, let’s look at how to implement the Problem Details pattern in an API application.

In my code examples, I’m using C# within an ASP.NET Web API project.

First of all, let’s consider what the base definition of the Problem Details model looks like as a class in code.

/// <summary>
/// A machine-readable format for specifying errors in HTTP API responses based on https://tools.ietf.org/html/rfc7807.
/// </summary>
public class ProblemDetails
{
    #region Properties
 
    /// <summary>
    /// A URI reference [RFC3986] that identifies the problem type. This specification encourages that, when
    /// dereferenced, it provide human-readable documentation for the problem type
    /// (e.g., using HTML [W3C.REC-html5-20141028]).
    /// When this member is not present, its value is assumed to be
    /// "about:blank".
    /// </summary>
    [JsonProperty(DefaultValueHandling = DefaultValueHandling.Ignore)]
    public string Type { get; set; }
 
    /// <summary>
    /// A short, human-readable summary of the problem type.
    /// It SHOULD NOT change from occurrence to occurrence
    /// of the problem, except for purposes of localization(e.g., using proactive content negotiation;
    /// see[RFC7231], Section 3.4).
    /// </summary>
    public string Title { get; set; }
 
    /// <summary>
    /// The HTTP status code([RFC7231], Section 6) generated by the origin server for this occurrence of the problem.
    /// </summary>
    public int? Status { get; set; }
 
    /// <summary>
    /// A human-readable explanation specific to this occurrence of the problem.
    /// </summary>
    public string Detail { get; set; }
 
    /// <summary>
    /// A URI reference that identifies the specific occurrence of the problem.
    /// It may or may not yield further information if dereferenced.
    /// </summary>
    public string Instance { get; set; }
 
    #endregion
}

As you can see, the base model represents most of the members as a string. I’m using the JSON.NET library to signify that the Type property should not be serialised if a value has not been specified for it.

Note that if you’re using ASP.NET Core there is a ProblemDetails class built into the framework. The code I’m using above is based on this built-in class.

Next, let’s extend the standard ProblemDetails class with our own custom members.

// <summary>
/// A machine-readable format for specifying errors in HTTP API responses based on https://tools.ietf.org/html/rfc7807.
/// Contains 'extended members' which are allowable in accordance with the official RFC.
/// </summary>
public class ApiProblemDetails : ProblemDetails
{
    #region Properties
 
    /// <summary>
    /// The API Error Code, represented as a string value.
    /// </summary>
    public string Code { get; set; }
 
    /// <summary>
    /// The API Error Category, represented as a string value.
    /// </summary>
    public string Category { get; set; }
 
    /// <summary>
    /// A collection of model Validation Errors relating to the API request.
    /// </summary>
    [JsonProperty(DefaultValueHandling = DefaultValueHandling.Ignore)]
    public ICollection<ValidationError> ValidationErrors { get; set; }
 
    #endregion
}

In the ApiProblemDetails class above we add a Code property to represent our own custom error code. The value for this could be "resourceNotFound" as an example. You can make this a numeric code if you prefer. Personally, I’ve decided to make it a string/enum to avoid the quandary regarding the categorisation of custom error codes i.e. should you divide your error codes into client and server errors like HTTP does?

Here is an enumeration of possible ApiErrorCode values which can be extended to suit your application.

/// <summary>
/// Possible API Error Codes.
/// </summary>
public enum ApiErrorCode
{
    None,
    ResourceInvalid,
    ResourceNotFound
}

The Category property allows us to separate standard protocol (HTTP) errors from custom errors which are specific to our application e.g. "custom" or "yourProductName".

Lastly, the ValidationErrors property provides a way for us to pass a list of validation errors to the client whenever an invalid model has been supplied to POST/PUT operations etc.

The ValidationError class simply holds the Name of the invalid model property and the Description of the validation error.

/// <summary>
/// Represents a model Validation Error.
/// </summary>
public class ValidationError
{
    #region Properties
 
    /// <summary>
    /// The Name of the invalid model property.
    /// </summary>
    public string Name { get; set; }
 
    /// <summary>
    /// The Description of the model error.
    /// </summary>
    public string Description { get; set; }
 
    #endregion
}

This concludes our tour of the error model classes. We’ve taken the standard Problem Details model and extended it, making it a little more flexible with our own extension members.

For your project, the base model may be enough to meet your requirements. However, I believe that the additional members we’ve looked at complement the structure of the base model nicely. They give us a way to let our clients know when something very specific happens. This removes our sole reliance on HTTP status codes which can sometimes be too generic.

Building a framework

I’ve created a sample Web API project and made it available on GitHub. The project implements the Problem Details pattern as described in the section above and demonstrates a framework for handling application errors globally.

The key classes within the project which you should check out, are discussed below.

ApiException

To make it simpler for us to trigger API error responses I’ve created an ApiException class that encapsulates everything that we need to generate a Problem Details error response.

We can throw an ApiException at any time from our API Controllers and the error will be caught and processed by a custom global ‘Exception Handler’. An appropriate Problem Details error response will then be generated and returned to the client.

Here is an example of throwing an ApiException from an API Controller action.

// Fire an error and return a 'Not Found' status if there are no Todos.
throw new ApiException(HttpStatusCode.NotFound, ApiErrorCode.ResourceNotFound);

Note that by specifying ApiErrorCode.ResourceNotFound we provide the client with the means to differentiate between general 404 HTTP errors and cases where the resource itself cannot be found.

ApiExceptionHandler

We can handle an ApiException (or any other type of exception) globally within an ASP.NET Web API project by deriving a class from the ExceptionHandler base class.

Within the ApiExceptionHandler class, we create, log and return the Problem Details error response.

ApiValidationActionFilter

In order to handle model validation errors automatically and reduce noise in our API Controller code, I’ve implemented a custom ‘Action Filter’. The filter checks the ‘Model State’ automatically and throws an ApiException if there are any problems, with the validation errors passed along for convenience.

This is how the ValidationError collection gets populated and passed back to our clients.

ApiErrorInstanceUrn

To simplify the generation of the URN for our Instance property, I’ve created a class that encapsulates the parts which make up the URN and the logic for generating and formatting it.

Some other classes which you may find interesting within the project are as follows.

  • ApiErrorFactory
  • ApiExceptionLogger
  • HttpStatusCodeExtensions

Note that the ‘Action Filters’, ‘Handlers’ and ‘Loggers’ mentioned above are all registered globally within the Register method of the WebApiConfig class.

If you want to try the project out in your own development environment, after cloning the repo and running the application, type the following into the address bar of your web browser to see an example error response.

http://localhost:54283/api/todos?userId=4

This should yield the following JSON output.

{
  "code": "resourceNotFound",
  "category": "custom",
  "title": "Not Found",
  "status": 404,
  "detail": "The item you are looking for cannot be found.",
  "instance": "urn:jc:api:error:custom:resourceNotFound:1269d728-1170-42e1-8805-d1b9870bd7f7"
}

Note that the port number (54283) in the above URL may be different on your machine, so be sure to adjust it accordingly.

Conclusion

I really believe that the standardisation of error responses is a great thing for both API producers and consumers. It helps both parties focus on what matters, with the comfort of knowing that errors are being handled in a uniform manner.

In my opinion, the Problem Details standard offers both simplicity and flexibility and is a great basis for creating an error response model. The ability to extend the model when needed provides assurance that it can be adapted to suit future requirements.

I strongly encourage you to try adopting the Problem Details standard for your next project.


I hope you enjoyed this post! Comments are always welcome and I respond to all questions.

If you like my content and it helped you out, please check out the button below 🙂

Comments

Sridhar Krishnamurthy

Thank you. Most of us tend to backseat ‘error reporting’ and botch up the error/exception data model therein.

July 26, 2021

Jonathan Crozier

Yes, it is vital to carefully consider your API error handling format upfront. I’m glad you found the post helpful.

July 27, 2021