Automate Your APIs With api-flow

Hi, I’m Rob Thomas (no, not that one) (or that one). I’m a senior engineer at FormSwift, and it’s my privilege to write the first of what we hope will be many technical articles in this space.

For this first article, I’d like to describe a little project I’ve just published called “api-flow.” The project was inspired by a problem my wife (a QA Automation lead at another company) was running into. For reasons that would make sense if I felt free to go into detail, her company has a truly staggering number of sites that do much the same thing for many different audiences, but are different enough to need a certain amount of distinct infrastructure. And to test all this she’d been given a Postman collection.

So let’s talk about Postman.

We love Postman. Most everybody uses it, even though most of us never use half of what it’s grown into, which I shall generously call “highly feature-rich.” What it doesn’t do so well is run in CI, or run literally hundreds of variations of the same thing. It’s a big, heavyweight desktop app, and however much you’ve developed that collection, you’ve still got a person sitting in a chair poking a button and looking at the results. This is not a good use of people’s time at scale.

The first thing I did, of course, was to look for alternatives to Postman that could run headlessly. I’m still convinced that there has to be one out there I didn’t happen upon, but most alternatives I found were also big, unwieldy desktop apps. I’d set out expecting to decide which of several existing tools filled this niche best, and was surprised to come up empty.

Therefore I did what any sensible person wouldn’t do, and made one.

It’s early days for api-flow, with lots of potential areas of improvement, but it does a few basic things pretty well. The project runs “Flows” that are sequences of “Steps.” A Flow and its associated Steps are defined in YAML. Each Step represents a single API call. The headers, URL and body for a Step’s request are templates populated from a few sources:

1. Any template variable can be satisfied by an environment variable. If an environment variable is defined, this takes precedence over most other sources.
2. When constructing a Flow, you can provide “profiles,” other YAML files that define a base set of variables accessible to every Step in the Flow.
3. Steps define “outputs,” which map variables to JSONPath expressions. These are used to extract values from JSON response bodies. The extracted outputs are thereafter available to use in subsequent steps.
4. Very simple functions, including user-defined ones, are supported for more complex cases. A simple built-in function, for example, renders a random UUID.

Flows can also depend on other Flows, which is a simple method for DRYing your Flow definitions. You can define a Flow that creates a user or logs in, and provides a session token as an output. Multiple other test cases can then depend on this common Flow and use its output.

When a Flow is complete, the instance still retains all of its collected outputs, still attached to their respective steps. You can use this to gather results for further processing, or you can assert on them in a unit test.

I have a few use cases in mind:

• Automating day-to-day workflows: For example, as soon as I get a minute I plan to take a stab at automating some deployment tasks using the GitHub and Travis APIs.
• Accelerating standard QA Automation: Instead of walking through the UI flow to create a particular server state over and over for multiple tests, you could use api-flow to create the prerequisite state directly, and then drive the browser to test the only actual functionality you care about.
• Testing API-only stories: Rather than providing browser- or Postman-based testing instructions for stories that have no UI component, you can provide a Flow, which consumes a standard environment profile and exercises the desired behavior. The Flow can be reviewed alongside your regular code, and can be built into a complete suite of API tests that will work in all your environments with very little coding. You can provide simple instructions for manual testers to run the Flow at the command line and verify the results.

The api-flow package can be found on GitHub and PyPi. For now it supports more or less exactly the set of features I have personally needed, but I look forward to growing it according to how it gets used, so feedback is highly encouraged.