What is Contract Testing?

Contract testing is a technique to verify integration between each applications independently in order to test what has been sends or receives matches with the document named “contract”. For applications that communicate via HTTP, these “contract” would be the HTTP request and response, and for an application that used queues, this would be the message that goes on the queue. In practice, a common way of implementing contract tests is to check that all the calls to your test return the same results as a call to the real application would.

Contract testing fits nicely anywhere where you have two services that need to communicate: such as an API client and a web front-end, one service and its downstream service. Although a single client and a single service is a common use case, contract testing really shines in an environment with many services. Having well-formed contract tests makes it easy for developers to avoid version hell. Contract testing is the killer app for micro-service development and deployment.

Why we need Contract Testing?

For example, a Web Application where the front-end is being developed by Team Diana and the API is being developed by Team Ash. The project starts with a kick-off meeting where the requirements are presented and agreed upon between the teams. Each team takes the requirements and starts creating the backlog by refining user stories.  The development starts in both teams following the user stories, integration testing is left for later sprints. As Team Diana finds additional requirements and relating to error scenarios, the API documentation is updated accordingly. Test cases are added related to the updated scenarios based on the documentation.

Already we can see a couple of flaws with this process:

  1. API document changes may not be communicated effectively.
  2. Test team creates integration test cases based on documentation and the document may not up to date.
  3. Integration environment is the first time when full integration is tested.
  4. Different API version on integration environment vs production.

Consumer-Driven Contract Testing

Consumer-driven contract testing has two sides the consumer and the provider.
Consumer:
An application that makes use of the functionality or data from another application to do its job.
For applications that use HTTP, the consumer is always the application that initiates the HTTP request (eg. the web front end), regardless of the direction of data flow.
For applications that use queues, the consumer is the application that reads the message from the queue.

Provider:
An application (often called a service) that provides functionality or data for other applications to use, often via an API.
For applications that use HTTP, the provider is the application that returns the response.
For applications that use queues, the provider (also called producer) is the application that writes the messages to the queue.

A contract between a consumer and provider is called a pact. Each pact is a collection of interactions.
For HTTP:
An expected request – describing what the consumer is expected to send to the provider
A minimal expected response – describing the parts of the response the consumer wants the provider to return.

For messages:
The minimal expected message – describing the parts of the message that the consumer wants to use.

The first step in writing a pact test is to describe this interaction.

Consumer testing

Following the diagram:

  1. Using the Pact DSL, the expected request and response are registered with the mock service.
  2. The consumer test code fires a real request to a mock provider.
  3. The mock provider compares the actual request with the expected request, and emits the expected response if the comparison is successful.
  4. The consumer test code confirms that the response was correctly understood

Pact tests are only successful if each step completes without error.

Provider verification

Following the diagram:

In provider verification, each request is sent to the provider, and the actual response it generates is compared with the minimal expected response described in the consumer test.
Provider verification passes if each request generates a response that contains at least the data described in the minimal expected response.

Contract Testing Vs Integration Testing

Integration Testing Contract Testing
API Configuration Yes No
Deployment Checks Yes No
API Versioning Yes Yes
Debug Locally No Yes
Environmental Issues Yes No
Feedback Time Slow Fast
Clearly Pinpoint Failure Many layers Very Easy

Firstly, contract testing does not replace integration testing. But it probably can replace some of your existing integration test scenarios, shift left, and provides faster feedback to your software development life-cycle.

In integration testing, you will be verifying the context in which the API lives, such as the environment architecture, the deployment process, etc.

Therefore you want to be running the core test scenarios which would confirm the configuration. Also proving whether the deployment was successful by returning a correct response.

In contract testing, you are testing the specifics of the API, which includes the edge cases related to the API structure, content, and error responses.

Next steps

In this tutorial, we learned what contract testing means and what it looks like in a micro-service infrastructure, and saw how it looks.
In addition, contract testing can reduce costs to your team by reducing feedback times related to integration issues.
Follow us you will know more about Consumer testing and PACT best practices.