How to Parse JSON That Varies Between an Array or a Single Item With Go

Have you ever interacted with an API that returns data in different formats depending on what data you sent to the endpoint? Yeah, it is a bit of a pain… 😭

While converting my mailing list from Drip to Convert Kit I ran into this issue. ConvertKit’s V3 API endpoint for creating a new tag accepts either a single tag, or an array of tags, and the response data depends on what you send the server. More specifically, if you send it a single tag, it returns the JSON for that tag:

{
  "id": 1,
  "name": "House Stark",
  "created_at": "2016-02-28T08:07:00Z"
}

But if you send multiple tags to the API it will return an array of tags:

[
  {
    "id": 1,
    "name": "House Stark",
    "created_at": "2016-02-28T08:07:00Z"
  },{
    "id": 2,
    "name": "House Lannister",
    "created_at": "2016-02-28T08:10:00Z"
  }
]

At first I hoped this could be circumvented by always sending my tags as an array regardless of how many were being created. My hope was that the API would match the format I made the request in. That is, if I sent a request body like below, the server would also respond with an array with a single item in it.

{
  "tag": [
    { "name": "Example Tag" },
  ]
}

Yes, I am aware that I used the singular term for tag. This is what the API docs described, and is not a mistake on my part.

Unfortunately, this is not how the ConvertKit API works. Instead, it returned a single tag in this case.

I don’t believe this was an entirely awful decision, but I don’t think it was the best one either. I would have rather had one of the following:

A. An API endpoint that ONLY accepts an array of tags and ONLY returns an array of newly created tags. This handles the single tag use case easily and makes writing API clients much easier. If they absolutely wanted to support both, this could have also been done using two API endpoints.

B. A single API endpoint that returns data in a format similar to what it receives. By that I mean if a request receives an array, it should return an array irregardless of how many items are in it, and if it receives a single item it can then return a single tag.

Alas, this isn’t how the ConvertKit V3 API works, so I needed to figure out a way to handle it in my code.

I knew for certain that I only wanted a single method on my API client for creating tags. Having multiple would just make the API clunky.

I also knew that I wanted it to be as simple as possible. That is, I didn’t want people to have to construct big response types if all they really needed to pass in were a few strings representing the tags they wanted to create.

In the end I decided to accept a variadic parameter for creating tags (tags ...string), and to always return a response with a slice of newly created tags. The final code ended up looking like this:

// CreateTags will create tags using the provided values as their names.
func (c *Client) CreateTags(tags ...string) (*CreateTagsResponse, error) {
	type newTag struct {
		Name string `json:"name"`
	}
	var data struct {
		Tags []newTag `json:"tag"`
	}
	for _, tag := range tags {
		data.Tags = append(data.Tags, newTag{tag})
	}
	var ret CreateTagsResponse
	err := c.Do(http.MethodPut, fmt.Sprintf("tags"), data, &ret)
	if err != nil {
		return nil, err
	}
	return &ret, nil
}

Now there are basically two edge cases this code does not handle well:

1. When only a single tag is created - the problem this entire post is about.
2. When zero tags are passed in.

I decided to ignore (2); chances are the API will return an error and this was more or less a programmer error if it happens.

(1) could not be ignored, but first let’s look at what the CreateTagsResponse type actually looks like.

// CreateTagsResponse is the data returned from a CreateTags call.
type CreateTagsResponse struct {
	Tags []Tag
}

// Tag can be applied to subscribers to help filter and customize your mailing
// list actions. Eg you might tag a subscriber "beginner" and send them
// beginner-oriented emails, or you might tag them as interested in a paid
// course so they get information about future sales.
type Tag struct {
	ID        int       `json:"id"`
	Name      string    `json:"name"`
	CreatedAt time.Time `json:"created_at"`
}

The Tag bit is shared regardless of the response, but I needed to figure out a good way to handle parsing either a single Tag or multiple Tags. Enter the json.Unmarshaler interface.

type Unmarshaler interface {
    UnmarshalJSON([]byte) error
}

Check out json.Unmarshaler in the Go docs 👉 https://golang.org/pkg/encoding/json/#Unmarshaler

If the json.Unmarshaler interface is implemented, it will be used rather than the standard decoding logic. Sounds like a great starting point.

Now how do we determine if the JSON data is an array or a single item? Well the easiest way is to simply look at the first character and see if it is a { or a [.

// UnmarshalJSON implements json.Unmarshaler
func (ctr *CreateTagsResponse) UnmarshalJSON(b []byte) error {
	if len(b) == 0 {
		return fmt.Errorf("no bytes to unmarshal")
	}
	// See if we can guess based on the first character
	switch b[0] {
	case '{':
		return ctr.unmarshalSingle(b)
	case '[':
		return ctr.unmarshalMany(b)
  }
  // TODO: Figure out what do we do here
	return nil
}

func (ctr *CreateTagsResponse) unmarshalSingle(b []byte) error {
  // TODO: Implement this
  return nil
}

func (ctr *CreateTagsResponse) unmarshalMany(b []byte) error {
  // TODO: Implement this
  return nil
}

There are many ways to check the first character, but I opted to do the straight forward check by looking at the very first byte being unmarshaled. I then added a TODO for when this didn’t work planning to come back to it.

Next I needed to actually write the decoding logic. What is especially neat about writing your own UnmarshalJSON method is that you can still leverage the standard library’s encoding/json package in order to avoid doing too much work. You just need to define new types or variables.

func (ctr *CreateTagsResponse) unmarshalSingle(b []byte) error {
	var t Tag
	err := json.Unmarshal(b, &t)
	if err != nil {
		return err
	}
	ctr.Tags = []Tag{t}
	return nil
}

func (ctr *CreateTagsResponse) unmarshalMany(b []byte) error {
	var tags []Tag
	err := json.Unmarshal(b, &tags)
	if err != nil {
		return err
	}
	ctr.Tags = tags
	return nil
}

In both of these cases I didn’t need to create a new type because I already had the Tag type. I just needed to create either a single tag, or a slice of tags, and call json.Unmarshal. This is especially nice because it means I don’t actually have to understand how to decode JSON data. I just needed to create new types that matches the incoming JSON, leverage the standard library, and then put the decoded data where I wanted it in the CreateTagsResponse.

At this point my code works for most cases, but what happens if the first character does NOT match a curly bracket ({) or a square bracket ([)? For instance, what happens in the JSON payload is padded with spaces?

It turns out the encoding/json package seems to handle this for us already, as you can see with the example below.

// Whitespace for days
bytes := []byte(`


[
  {
          "id": 1,
          "name": "House Stark",
          "created_at": "2016-02-28T08:07:00Z"
          },{
            "id": 2,
            "name": "House Lannister",
            "created_at": "2016-02-28T08:10:00Z"
          }
      ]`)
var ctr CreateTagsResponse
err := json.Unmarshal(bytes, &ctr)
if err != nil {
  panic(err)
}
fmt.Println(ctr)

Despite seeing that my code handled whitespace, I still didn’t want to take any chances; I needed to do something if that switch statement fell through.

Ultimately, what I decided to do was just assume the data is in the “many tags” format, but to fall back to the single tag unmarshaling if that returned an error.

// UnmarshalJSON implements json.Unmarshaler
func (ctr *CreateTagsResponse) UnmarshalJSON(b []byte) error {
	if len(b) == 0 {
		return fmt.Errorf("no bytes to unmarshal")
	}
	// See if we can guess based on the first character
	switch b[0] {
	case '{':
		return ctr.unmarshalSingle(b)
	case '[':
		return ctr.unmarshalMany(b)
	}
	// This shouldn't really happen as the standard library seems to strip
	// whitespace from the bytes being passed in, but just in case let's guess at
	// multiple tags and fall back to a single one if that doesn't work.
	err := ctr.unmarshalMany(b)
	if err != nil {
		return ctr.unmarshalSingle(b)
	}
	return nil
}

Even if my code only ever arrives below the switch statement when receiving malformed JSON, I know that the standard library will return an error trying to parse it.

Hopefully this helps the next time you encounter an API that returns variable data formats.

If you want to learn more about encoding and decoding JSON in Go, you might find this article I wrote for the Gopher Academy Advent useful 👉 Advanced JSON Encoding and Decoding Techniques in Go.