Ensuring your OpenAPI (Swagger) Specification is never outdated through testing with Chai or Jest

Testing your OpenAPI Specification using Chai OpenAPI response validator or Jest OpenAPI. Additional Chai/Jest matchers for asserting that HTTP responses satisfy an OpenAPI spec.

The OpenAPI Initiative, previously known as the Swagger Specification, is a machine readable specification to describe RESTful web services such as an API. It's used by many to describe what API queries an application can handle and when combined with the use of tools such as Swagger UI can make even the most complex APIs easy for developers to understand.

Using an OpenAPI Specification in your project should make it more understandable and easier for new developers to consume it. It is, however, easy for it to become outdated when changes are made to the functionality of the application, an unfortunately common issue that is present in documentation all over the web. Even the smallest detachment between actual function and documentation can cause hours of frustration for a user of your product.

In fact, when developing an application which communicates with a third-party API, incorrect OpenAPI Specifications can make it impossible to carryout critical functionality and stop a project in its tracks. I have experienced this first hand when trying to use a certain bank's API to create a money handling application while at University.

So how can we ensure that OpenAPI Specifications are up-to-date?

Short answer: With tests!

Longer answer:

The open source OpenAPIValidators project on GitHub contains two packages that extend both the Chai Assertion Library and Jest two popular JavaScript testing frameworks.

It describes itself as a plugin that will "automatically test whether your server's behaviour and documentation match". It is as simple to use as it sounds, to make an assertion you add expect(responseObject).toSatisfyApiSpec() in your tests, pretty nice.

In addition to being used to ensure documentation is updated, the testing plugins can also aid Test Driven Development practices by introducing the opportunity to design your OpenAPI Specification first and implement the API second. This means that the team implementing the API will have strict documentation to follow with the plugins ensuring that the correct responses are sent with the correct response codes. Moreover, in a time where frontend and backend development teams are not co-located, both teams can agree on a specification and not worry that one will deviate from the agreed design.

The two plugins can be found on NPM:

• chai-openapi-response-validator

• jest-openapi

How does it work?

1. You pass it your OpenAPI Specification via an object or by absolute path.
2. You add expect(responseObject).toSatisfyApiSpec() to each test where you want to validate the response.
3. It matches the request path and HTTP response code returned from the API call with your OpenAPI Specification and then asserts that the response is what is defined in the Specification.
4. If the response matches the Specification then the test passes, if it doesn't then the test will fail with a helpful error.

What does it look like to use?

Take the test below and the Specification that follows, it makes a GET request, asserts that the HTTP response code will be 200 and then, using the OpenAPIValidators package, asserts that the response from the request matches exactly the OpenAPI Specification.

it('should make a GET request and satisfy OpenAPI spec', async () => {
  // Make request (supertest used here)
  // /hello should return the code 200 with the body "hello world"
  const res = await supertest(server).get('/hello');
  // Make any assertions as normal
  expect(res.status).toEqual(200);
  // Assert that the HTTP response satisfies the OpenAPI spec
  expect(res).toSatisfyApiSpec();
});

The OpenAPI Specification used for this test:

openapi: 3.0.3
info:
  title: Open Api Validator
  version: 0.0.1
paths:
  /hello:
    get:
      responses:
        200:
          description: Response body should be a string
          content:
            text/plain:
              schema:
                type: string
                example: Hello World

When the response matches the OpenAPI Specification the test will pass as expected with no additional output:

but when the response does not match the OpenAPI Specification the test fails and provides helpful output so you can either change the implementation or update the Specification:

The screenshot above shows a failure where the actual API response has returned an object rather than the string that is documented in the OpenAPI Specification.

When the OpenAPI validator fails to match an API call to the Specification it fails with this error:

Which will prompt you to either remove the assertion from the test or update the OpenAPI Specification to contain the new route, preferably the latter.

Round up

In this blog I've introduced the OpenAPI Validators which can be used in conjunction with Jest or Chai and an OpenAPI Specification to ensure that both your API documentation is kept up to date and that APIs are not accidentally modified to return incorrect responses. Additionally, frontend and backend developers can use the testing plugins to ensure that an agreed API Specification is implemented correctly.

In the future I will release another post where I demonstrate how the jest-openapi validator can be used with an Express API.

Drop me a reaction if this post has helped or if outdated API documentation has inconvenienced you in the past!