Why are slices sometimes altered when passed by value in Go?

In my last blog post we talked about a few of the differences between a slice and an array. Namely, we discussed how a slice has both a capacity and a length, while an array only has a length. We also briefly covered how a slice uses an array behind the scenes as part of its data structure. If this is all news to you, I suggest you check out the article.

In this post we are going to continue that discussion and talk about how these design decisions can affect your code and introduce some unexpected bugs. Specifically, we are going to be addressing the question:

Why are slices sometimes altered when passed by value in Go?

If I am being honest, the answer to this isn’t limited to slices (more on that later), but it tends to catch people off guard most often with slices.

I believe the easiest way to illustrate the “issue” is to start with a few pop quizzes. There will be three total, and I highly recommend that you check out all three before hitting the back button. You may be surprised by the results.

Pop Quiz #1

What does the following code output?

func main() {
  var s []int
  for i := 1; i <= 3; i++ {
    s = append(s, i)
  }
  reverse(s)
  fmt.Println(s)
}

func reverse(s []int) {
  for i, j := 0, len(s) - 1; i < j; i++ {
    j = len(s) - (i + 1)
    s[i], s[j] = s[j], s[i]
  }
}

Why are changes to s being seen after the function call despite s being passed by value?

Well, as we discussed before, slices are backed by a pointer to an array. This means that even though our slice is being passed by value here, the array pointer still points to the same memory address. As a result, the slice used inside of reverse() will have a different pointer object for the array, but it will still point to the same memory address and affect the same array. This means that rearranging numbers in this array will be persisted after the function call.

Pop Quiz #2

We are going to slightly change our code by adding a single append call inside the reverse() function. How does it change our output?

func main() {
  var s []int
  for i := 1; i <= 3; i++ {
    s = append(s, i)
  }
  reverse(s)
  fmt.Println(s)
}

func reverse(s []int) {
  s = append(s, 999)
  for i, j := 0, len(s) - 1; i < j; i++ {
    j = len(s) - (i + 1)
    s[i], s[j] = s[j], s[i]
  }
}

This time when we print out s it is in the reverse order, but what happened to the 1?

When we call append a new slice is created. The new slice has new length attribute which isn’t a pointer, but it still points to the same array. Because of this, the rest of our code ends up reversing the same array that our original slice references, but the original slice doesn’t have its length altered.

Pop Quiz #3

It is time for our final quiz. We are going to make a relatively minor change - we are going to append a few extra numbers to our slice inside of the reverse() function.

func main() {
  var s []int
  for i := 1; i <= 3; i++ {
    s = append(s, i)
  }
  reverse(s)
  fmt.Println(s)
}

func reverse(s []int) {
  s = append(s, 999, 1000, 1001)
  for i, j := 0, len(s)-1; i < j; i++ {
    j = len(s) - (i + 1)
    s[i], s[j] = s[j], s[i]
  }
}

In our final quiz not only is the length not persisted, but the order of the slice isn’t affected either. Why?

As I mentioned before, when we call append a new slice is created. In the second quiz this new slice still pointed to the same array because it had enough capacity to add the new element, so the array wasn’t changed, but in this example we are adding three elements and our slice doesn’t have enough capacity. Instead, a new array is allocated and our updated slice points to it. When we finally start reversing elements in the slice it is no longer affecting our initial array, but it is instead operating on a completely different one.

Verifying what happened with the cap function

We can verify what is happening by using the cap function to check the capacity of our slice passed into reverse().

func reverse(s []int) {
  newElem := 999
  for len(s) < cap(s) {
    fmt.Println("Adding an element:", newElem, "cap:", cap(s), "len:", len(s))
    s = append(s, newElem)
    newElem++
  }
  for i, j := 0, len(s)-1; i < j; i++ {
    j = len(s) - (i + 1)
    s[i], s[j] = s[j], s[i]
  }
}

As long as we don’t go beyond the capacity of our slice, we will eventually see the changes of the reverse() function in our main() function. We still won’t see the changes to the length, but we will see the rearranging of elements in the array that backs the slice.

If we add a single append() call to s after we fill our slice to capacity we will no longer see those changes in the main() function because our reversal code ends up working with a new slice that points to an entirely different array.

Slices derived from slices or arrays are also affected

If we happen to be created new slices in our code that are derived from existing slices or arrays we can also see the same effect. For example, if you call s2 := s[:] and then pass s2 into our reverse() function it may still ultimately affect s because both s2 and s point to the same backing array. Similarly, if we append new elements to s2 that ultimately cause it to outgrow the backing array we will no longer see changes to one slice affecting the other.

This isn’t strictly a bad thing. By not copying the underlying array until it is absolutely necessary we end up having much more efficient code, but it does come at a slight mental cost of needing to account for this when writing code.

This is not limited to slices

This isn’t limited to slices. Slices are the easiest type to fall into this trap with because you may not understand their actual data structure, but any type with a pointer could be affected. This is demonstrated below.

type A struct {
  Ptr1 *B
  Ptr2 *B
  Val B
}

type B struct {
  Str string
}

func main() {
  a := A{
    Ptr1: &B{"ptr-str-1"},
    Ptr2: &B{"ptr-str-2"},
    Val: B{"val-str"},
  }
  fmt.Println(a.Ptr1)
  fmt.Println(a.Ptr2)
  fmt.Println(a.Val)
  demo(a)
  fmt.Println(a.Ptr1)
  fmt.Println(a.Ptr2)
  fmt.Println(a.Val)
}

func demo(a A) {
  // Update a value of a pointer and changes will persist
  a.Ptr1.Str = "new-ptr-str1"
  // Use an entirely new B object and changes won't persist
  a.Ptr2 = &B{"new-ptr-str-2"}
  a.Val.Str = "new-val-str"
}

Similarly to this example, a slice is defined as:

type slice struct {
  array unsafe.Pointer
  len   int
  cap   int
}

Notice how the array field is actually a pointer? This means that slices will end up acting just like any other types in Go that have a nested pointer, and they actually aren’t special at all. They just happen to be a type that very few people look at the internals of.

What does this mean for Gophers?

Ultimately, what this means is that developers need to be aware of what types of data they are passing around and how it might be affected by functions they are calling. When you pass slices to other functions or methods you should be aware that you may or may not have the elements in the original slice altered. You will never have to worry about the length or capacity changing without your knowledge, but the actual elements in the backing array may be altered.

Similarly, you should always be aware that structs with a pointer inside of them can fall into the same situation fairly easily. Any changes to data inside of a pointer will be persisted unless the pointer itself is updated to reference another object in memory first.

More than anything, I hope that this post illustrates that when you don’t fully understand why something is happening, it is worth taking the time understand it. Bugs stemming from ignorance are insanely hard to debug without knowing why they are happening, but with a little research it suddenly becomes much clearer what is happening, how to fix the issue, and just as importantly, how to test for the issue in the future.