Testing API Libraries

Anymore it is nearly impossible to build something without needing to interact with one API or another. If you are anything like me, chances are you have had to write your own code to interact with one of those APIs, which usually means you also need to figuring out a way to test your code.

Unfortunately, testing API libraries isn’t always easy. More often than not, the API you are testing against won’t provide a test environment, or worse yet, they may provide a test environment that doesn’t work the same as the production environment causing your code to break when you start using the production API.

In this article we are going to explore a few techniques I have found helpful when writing API client libraries in Go. I am also going to provide some advice based on my experience integrating with various APIs.

If a good test server is available, start with integration tests

Unit tests are great at catching errors when they are setup correctly, but when integrating with an API you cannot guarantee that this will be the case. The API docs might be outdated, you might be misinterpreting the docs, or any other number of scenarios might occur where your unit test is testing based on a faulty assumption. I can’t begin to tell you how many times I have written code based on outdated or invalid docs.

When a test server is available, I recommend starting out with integration tests that make real API calls to the test server. Yes, I know that this will slow down your tests. I know that these tests will require an internet connection. Despite all of that, it will save you time in the long run because you will know for certain that your code is working with the real API.

When writing these integration tests, you will commonly need an API key of some sort. If that is the case, I suggest letting users provide it via a flag, and then using that flag to determine how to set things up or whether to skip tests.

var (
	apiKey string
)

func init() {
	flag.StringVar(&apiKey, "api_key", "", "Your secret API key for your test account")
}

func TestClient_CreateWidget(t *testing.T) {
	if apiKey == "" {
		t.Skip("skipping integration tests - api_key flag missing")
	}
	// ... run the test
}

// Alternatively, we can setup a test server for unit tests...

// Sets up a client for testing
func client(t *testing.T) *someapi.Client {
	c := someapi.Client{
		Key: apiKey,
	}
	if apiKey == "" {
		// We need a local test server
		handler := func(w http.ResponseWriter, r *http.Request) {
			// ... fill this in with a fake server of some sort
		}
		server := httptest.NewServer(http.HandlerFunc(handler))
		c.BaseURL = server.URL
		t.Cleanup(func() { server.Close() })
	}
	return &c
}

Not all API test servers are good

I added the “good” qualifier in the last section because test servers all fall on a spectrum from “utterly useless” to “amazing”.

       |-----------------------|-----------------|------------|
Utterly useless        Sometimes useful        Good        Amazing

On the utterly useless end we have test servers that return responses that differ from what production returns. Or never being available to actually interact with.

On the amazing end of the spectrum we have test servers like Stripe that give you reliable ways to simulate pretty much any behaviors you may need. You can create payment methods that will eventually have valid charges, payment methods that will eventually be rejected due to high fraud risk, and almost any other circumstance you can think of. They even provide a nice testing dashboard in their UI.

My general rule of thumb is to assume a test server is good until proven otherwise. Even if a test server doesn’t allow me to test every possible scenario, integration tests will frequently help me discover errors in my understanding of the docs that likely wouldn’t be caught with unit tests (because I would put those faulty understandings into my unit tests). I also believe it is easier to get the docs wrong than it is to return completely invalid data from a test server.

If all else fails, you might be able to create a second “live” account and test with it. When I was integrating with ConvertKit they were willing to provide me with an account for this very purpose, and while it did have limitations, it was still useful for sanity checking things.

Integration tests can be recorded and used as unit tests

When creating a Stripe API client in my Test with Go course, one technique I demonstrate there is how to record responses from integration tests to then use as part of your unit tests.

We won’t be going in as much detail as the course does here, but I do want to walk through some of main points.

The source code for this is available here: https://github.com/joncalhoun/twg/tree/master/stripe

First, your API client needs to have a customizable BaseURL so we can tell it which server to talk to depending on whether or not we are running a unit or integration test.

type Client struct {
	BaseURL    string
  // ...
}

End users shouldn’t need to set this, so you will want to use a default value if it isn’t set.

const (
	DefaultBaseURL  = "https://api.stripe.com/v1"
)

// path will be a value like "/customers"
func (c *Client) url(path string) string {
  base := c.BaseURL
  if c.BaseURL == "" {
		base = DefaultBaseURL
	}
	return fmt.Sprintf("%s%s", base, path)
}

All of this allows us to set BaseURL in unit tests. This is often done with a server derived from httptest.NewServer.

server := httptest.NewServer(handler)
defer server.Close()
c := stripe.Client{
  BaseURL: server.URL,
}

If you already have recorded responses, you can setup the http server to return then.

Recording responses is done by replacing the HTTP client used by your API client. For instance, if we started with this Client type:

type Client struct {
  APIKey string
  BaseURL string
  HttpClient interface {
    Do(*http.Request) (*http.Response, error)
  }
}

We can replace HttpClient with the following recorderClient in a test, then when the test is done we can persist the recorded responses however we want. In my specific example I stored them in testdata as a JSON file.

// whatever data your test care about
type response struct {
	StatusCode int    `json:"status_code"`
	Body       []byte `json:"body"`
}

type recorderClient struct {
	t         *testing.T
	responses []response
}

func (rc *recorderClient) Do(req *http.Request) (*http.Response, error) {
	httpClient := &http.Client{}
	res, err := httpClient.Do(req)
	if err != nil {
		rc.t.Fatalf("http request failed. err = %v", err)
	}
	defer res.Body.Close()
	body, err := ioutil.ReadAll(res.Body)
	if err != nil {
		rc.t.Fatalf("failed to read the response body. err = %v", err)
	}
	rc.responses = append(rc.responses, response{
		StatusCode: res.StatusCode,
		Body:       body,
	})
	res.Body = ioutil.NopCloser(bytes.NewReader(body))
	return res, err
}

Finally, helpers used to setup your client combined with global flags can help determine how your client should be setup for testing.

func stripeClient(t *testing.T) (*stripe.Client, func()) {
	teardown := make([]func(), 0)
	c := stripe.Client{
		Key: apiKey,
	}
	if apiKey == "" {
		count := 0
		handler := func(w http.ResponseWriter, r *http.Request) {
			resp := readResponse(t, count)
			w.WriteHeader(resp.StatusCode)
			w.Write(resp.Body)
			count++
		}
		server := httptest.NewServer(http.HandlerFunc(handler))
		c.BaseURL = server.URL
		teardown = append(teardown, server.Close)
	}
	if update {
		rc := &recorderClient{}
		c.HttpClient = rc
		teardown = append(teardown, func() {
			for i, res := range rc.responses {
				recordResponse(t, res, i)
			}
		})
	}
	return &c, func() {
		for _, fn := range teardown {
			fn()
		}
	}
}

Using this setup I find that nearly all of my integration tests can be turned into unit tests and require no code changes whatsoever. I just need to remember to run the integration tests, record the responses, and commit the testdata directory.

While the unit tests won’t be as valuable as the integration tests, they can be helpful in specific situations. Just be aware that unit tests should be viewed as helpers, not as a source of complete truth. That is what the integration tests are for.

Use convention over configuration where possible

In many cases you won’t have access to a reliable test server, so you need to setup a local server to test with. When doing this, I highly recommend using convention over configuration.

What I mean is that instead of forcing every test case to specify every request/response combination it expects, setup a test server that has a few canned responses that you can use for your tests. When writing the ConvertKit API library I opted to convert a request’s HTTP method and path into a string that was then used to determine what JSON and headers were returned.

For instance, if the following API call was made:

GET /forms/213/subscriptions

My test server would convert this into GET_forms_213_subscriptions and then would use two files in testdata to determine how to respond.

• GET_forms_213_subscriptions.json → The JSON body to return.
• GET_forms_213_subscriptions.headers.json → The headers and status code to return. When a header file was missing I just assumed a status code of 200 and no additional headers.

This made setting up any new endpoint really easy because I could just copy the sample response from the docs into a JSON file and then my local test server would use it in tests.

func TestClient_Account(t *testing.T) {
	c := client(t, "fake-secret-key")
	resp, err := c.Account()
	if err != nil {
		t.Fatalf("Account() err = %v; want %v", err, nil)
	}
	if resp.Name != "Acme Corp." {
		t.Errorf("Name = %v; want %v", resp.Name, "Acme Corp.")
	}
	if resp.PrimaryEmail != "you@example.com" {
		t.Errorf("PrimaryEmail = %v; want %v", resp.PrimaryEmail, "you@example.com")
	}
}

See https://github.com/joncalhoun/convertkit/blob/master/client_test.go#L74 for the code demonstrating how client sets up a test server along with a convertkit.Client. This also uses the new t.Cleanup from the testing package.

This works exceptionally well because it is quick and easy when you don’t need anything custom, but if you do want to test something custom you can still do so. For instance, below is a test case where I wanted to verify that one of the request options was correctly being passed to the server when making an API request.

t.Run("page option", func(t *testing.T) {
	c := clientWithHandler(t, func(w http.ResponseWriter, r *http.Request) {
		r.ParseForm()
		got := r.FormValue("page")
		if got != "2" {
			t.Errorf("Server recieved page %v; want 2", got)
		}
		testdataHandler(t, "GET_subscribers_page_2")(w, r)
	})
	resp, err := c.Subscribers(convertkit.SubscribersRequest{
		Page: 2,
	})
	// Most of these checks don't matter much. The point of this test is
	// verifying that the options made it to the server.
	if err != nil {
		t.Fatalf("Subscribers() err = %v; want %v", err, nil)
	}
	if len(resp.Subscribers) != 1 {
		t.Errorf("len(.Subscribers) = %d; want 1", len(resp.Subscribers))
	}
	if resp.Page != 2 {
		t.Errorf("Page = %d; want 2", resp.Page)
	}
})

Write unit tests for complex decoding and encoding

While I do prefer to start with integration tests for API libraries, it is also worth noting that unit tests are still useful tools.

For instance, when I was normalizing variable JSON responses from the ConvertKit API I wrote unit tests for the UnmarshalJSON logic. I didn’t necessarily have to do this, as the other tests would cover this case, but this gave me an easy way to verify that this specific code was correct. It also provides me with an easy way to add a new test case if I find out it has a bug in the future without needing to come up with a large integration or unit test that interacts with an API endpoint.