This tutorial will show you how to write web service tests with the Karate framework. We will start with a short introduction about the basic features, then we will explore the domain-specific language (DSL), and learn how to apply it to verify a web service’s response and the returned JSON structures.
As integration with existing frameworks is important, we will demonstrate how to integrate well-known tools like JUnit and Maven to execute our tests and generate reports that may be used for integration-servers/services.
This tutorial will also cover other interesting aspects of this framework like configuration files, data tables, switching HTTP client implementations or more complex use cases like testing multipart file uploads.
Finally, we will show you how easy it is to set up a Continuous Integration (CI) on Semaphore for Karate tests in a few seconds.
We assume that you possess general knowledge of the HTTP protocol, the JSON notation, basic Java knowledge and experience in using build tools like Maven.
To integrate Karate in our project using Maven as a build tool, we only need to add the following two dependencies to
We also have to add the surefire-plugin to the plugin section in
Karate is a web service testing framework build on top of the popular Cucumber library. It eases the testing process allowing developers without Java knowledge to script sequences of HTTP calls and specify expectations for the web services response easy and fast, offering a custom domain-specific language.
Features of Karate
In addition to the features mention above, the Karate library offers a variety of additional features:
Many other features are listed on the GitHub project.
Implementing Karate Tests
We’re now ready to write our first tests. Tests consist of a Java class that might be recognized by the designated test framework used and a feature file that describes our interaction with the service under test.
Karate tests scripts use the
*.feature extension. The naming convention recommends putting the features next to the
*.java files. We can configure Maven to follow the recommended convention by editing
On the final account, the naming convention follows this pattern:
Karate features are written in a DSL that we’ll be covering in the following examples.
Our features are generally stored in
src/test/java/ so that feature files and Java tests are matched by their name and package structure.
All we need to do to integrate JUnit is to create a test class addressing our corresponding JUnit runner so that a minimal approach could look like the following class.
If we need to set up instrumentation for our tests, we can use JUnit‘s lifecycle mechanisms like @Before, @BeforeClass, @After, @AfterClass, test rules and so on.
Simple Hello World Example
We’re now ready to start with a simple example.
Assuming that we’ve got a simple RESTful web service running on localhost (e.g.
http://localhost:3000/quote) that returns a random quote when called in a GET request.
The response is returned using the JSON format and looks like this one:
We’re starting by writing our expectation in a new feature file named
quote.feature saved in
In this feature file, we’re defining a new feature named “Quote generator”, and setting a global setting for our base URL.
Afterward, we’re adding one scenario for fetching random quotes and assure that the response status code is 200 and that the quote in the response’s JSON structure is not null using a special marker.
The following markers are currently supported:
More detailed information about the rich syntax of Karate’s DSL can be found in the project’s wiki.
In the next step, we’re adding a Java class so that JUnit‘s test-runner executes our test.
Following the naming conventions, we’re creating the following class named
We may now run our test using Maven in the command line as follows:
Or alternatively using our IDE of choice:
Data Tables Example
Data tables allow us to add datasets in a tabular form as examples for our tests, and also use this information to fill our tests and assertions.
Again, we can run these tests using Maven in the command line like this:
Or alternatively using our IDE of choice:
To demonstrate Karate’s capabilities when dealing with file uploads and Multipart requests, we’re going to write tests for a RESTful web service for file management.
This service allows us to upload files that are stored in a temporary directory, and also to list all stored files.
Throughout its APIs, the JSON format is used for the server’s response messages.
For demonstration purposes, we have hacked together a quick and ugly implementation using Spring Boot.
File Store Service
This is our simple file store application written with Spring Boot:
This is our RESTful web service implementation:
Starting the RESTful Web Service
We may run our service now using our IDE of choice or by using Maven in the command line as follows:
Accessing the Service
We are now ready to access our RESTful web service and practice some Karate with it.
First, let’s have a short introduction of the APIs exposed and how to access them using regular tools like curl or Postman.
Storing a New File
We may upload files to the service that are stored in a temporary directory.
In the following examples, we’re uploading sample PDF files to our web service.
Uploading a file with Curl:
Uploading a file with Postman:
Listing Stored Files
The second API allows us to list stored files and receive a JSON response containing the file’s names and size (in bytes).
List stored files with Curl:
Assuming that we have uploaded two files named test.txt and test1.txt the following JSON structure is returned.
List stored files with Postman:
Karate Test for the File Service
To demonstrate how easy it is to test file-uploads and/or file-multipart requests, we’re going to write a quick test for the file server above.
In the first step, we’re using a separate configuration file to load the base URL from a file and support staging.
For this purpose, we’re adding a file named
karate-config.js to the directory
From now on, we’re referencing our base URL in our feature-files like this:
This is our feature file that describes how we’re interacting with our web service’s upload and file-listing functions.
Our test directory structure should now look similar to this one:
Running the Test
We may run our tests again using Maven in the command line or using our IDE of choice.
The Karate demo section offers a test-setup using an embedded Spring Boot instance, too as well as other interesting examples.
Choosing the HTTP Client
Since Karate 0.3.0 we may choose which HTTP client to use for running our tests, by adding the designated Maven dependencies — at the time of writing this article, Apache and Jersey implementations are supported.
For Apache, we’re adding the following dependency to our project’s
For Jersey we’re using this dependency:
Karate uses the surefire-plugin to generate standard reports in XML format that may be processed by standard mechanisms of common integration servers and providers like Semaphore.
Continuous Integration With Semaphore
Karate tests don’t run themselves, you must run them every time you make a change to your codebase. Manual tests do not scale well, less so when working as part of a team.
We can use Semaphore Continuous Integration (CI) to automate testing, build the application and even make deployments. Setting up CI takes a few minutes and it’s free, and once you have it configured, it’ll run your tests on every push.
To get the ball rolling, we’re going to clone an existing Spring Boot example that showcases a lot of the concepts we learned so far:
Add the project to Semaphore:
Improving Build Times
The started workflow should work out of the box, but with a few tweaks, we can make it run much faster. Here’s the plan:
To modify the pipeline you can use the Edit Workflow button on the top-right corner:
We’re presented with the Workflow Builder. Its main components are:
Make the following modifications to configure the cache:
To check that it’s using the cache, click on the job to see it’s logs:
You should see a message like this near the end:
The Semaphore Toolbox
We’ve used special commands in the last section. I think this is a good point to learn about the Semaphore built-in toolbox:
We’ll create a Tests block with two jobs:
Open the Workflow Editor again:
Your CI pipeline is now ready!
Next Steps: Deployment
Semaphore is not limited to building and testing our code, you extend your workflow with Continuous Deployment (CD) to automatically release new versions to the platform of your choice.
Kubernetes is one increasingly popular alternative for several reasons:
We hope that this tutorial helped you to get a basic understanding of Karate’s features and possibilities and that you now have another utility in your toolbox for testing web services of all kinds.
If you found this testing framework interesting, please feel free to share this tutorial. You’re also welcome to leave any comments or questions here!
Happy testing 🙂
This content was originally published here.