Blog

Back to Blog

Jamming with the GYB utility

Posted on 8 Jun 2016 Written by Sam Burnstone

You may have spent some time browsing the Swift standard library source code and come across some funky file names such as CollectionAlgorithms.swift.gyb and wondered what on earth that .gyb file extension meant.

GYB is simply an acronym used by the Swift team. It stands for 'Generate Your Boilerplate'. The role of the tool is to prevent code duplication by allowing simple Python snippets to be scattered within the Swift source files. When the tool is run the python is interpreted and a fully-functioning Swift file is spat out.

Although it's just an informal tool written in Python, it's pretty powerful and can be used with lots of different file types.

A simple example

To download the tool, we'll just download the file straight from the GitHub repository:

curl -O https://raw.githubusercontent.com/apple/swift/master/utils/gyb.py

This will download the file and pop it in our current directory.

To familiarize ourselves with the tool, we'll just create a simple text file that repeats a name. First, we'll create a new file:

touch test.txt.gyb

This follows the naming convention used in the Swift repo, where the .gyb extension signifies this file requires the gyb tool to be run and the output will be of the extension type preceding .gyb (in this case a textfile).

Now within test.txt.gyb we'll add a simple script that contains some code the gyb tool can interpret:

%{
def say_hello_to(person_name):
  return 'Hello, ' + person_name
}%
% for i in range(int(iterations)):
${say_hello_to(name)}. Nice to meet you.
% end

The lines the gyb tool will interpret are delineated using % signs. Code blocks can be wrapped within %{}% which makes the file a little easier to read by removing the need to prepend each line with %. If we want to output the value of variable or the result of a function, we wrap ${<function()_or_variable} around them.

So what does this file do? Excitingly, we just iterate a certain number of times over a chunk of text that also includes a call to a python method.

You might wonder where we're defining the variables iterations and name. These are passed to the gyb tool using the -D argument.

Now we've done that, let's say we want to say hello to 'Sam' three times. The command to do this is:

python ./gyb.py -D iterations=3  -D name=Sam test.gyb.txt

This results in the following terminal output:

// ###sourceLocation(file: "~/Documents/gyb/test.txt.gyb", line: 6)
Hello, Sam. Nice to meet you.
// ###sourceLocation(file: "~/gyb/test.txt.gyb", line: 6)
Hello, Sam. Nice to meet you.
// ###sourceLocation(file: "~/gyb/test.txt.gyb", line: 6)
Hello, Sam. Nice to meet you.

Ok, so we've nearly got what we're after, but what's that '###sourceLocation' malarkey? This is just a way to determine what line of the original .gyb file resulted in the output. This is the default output, but we can turn this off by passing an empty string to the --line-directive argument.

Another argument that comes in very handy is -o which allows us to define the name of the output file.

python ./gyb.py -D iterations=5  -D name=Sam --line-directive= -o test.txt test.txt.gyb

This results in 'Hello, Sam. Nice to meet you.', repeated 5 times, being piped into a file named test.txt.

How is this used in the Swift standard library?

So you may well be thinking, how is this useful?

It's used in a number of files in the standard library. Perhaps the most obvious use case is with tuples. If you're not familiar with them then I'd recommend taking a look at the Swift user guide. In essence, they allow us to group multiple values together that are related.

The interesting thing is, although there doesn't seem to be a limit on the number of elements a tuple can contain, operator overloads for !=, ==, <, <=, >, >= are only defined for those that contain between 2 and 6 elements.

For example, this results in an error:

let longTuple = (0, 1, 2, 3, 4, 5, 6)
longTuple == longTuple // error: binary operator '==' cannot be applied to two '(Int, Int, Int, Int, Int, Int, Int)' operands

(Admittedly, if your tuple contains more than 6 items, you should probably question whether you should be using a struct or class instead.)

If you were to navigate to the standard library bundled with Xcode, you'll see a definition for all the aforementioned operators for tuples ranging from 2 to 6 elements long. That's duplicating the exact same method definitions 5 times, bar the index of the item in the tuple being compared. Call in the gyb!!

For simplicity, let's just focus on the definition for ==. Within Tuple-Equality.swift.gyb we'll grab the relevant gyb code for the method from Tuple.swift:

% for arity in range(2,7):
%   typeParams = [chr(ord("A") + i) for i in range(arity)]
%   tupleT = "({})".format(",".join(typeParams))

%   equatableTypeParams = ", ".join(["{} : Equatable".format(c) for c in typeParams])

/// Returns `true` iff each component of `lhs` is equal to the corresponding
/// component of `rhs`.
public func == <${equatableTypeParams}>(lhs: ${tupleT}, rhs: ${tupleT}) -> Bool {
  guard lhs.0 == rhs.0 else { return false }
  /*tail*/ return (
    ${", ".join("lhs.{}".format(i) for i in range(1, arity))}
  ) == (
    ${", ".join("rhs.{}".format(i) for i in range(1, arity))}
  )
}
%end

This looks a little scary at first, but all we're doing is defining a range of [2, 3, 4, 5, 6] and then for each value in the range, creating a letter to represent a value in the tuple.

For example, in the case where 'arity' equals 3:

  • typeParams = [A, B, C]
  • tupleT = (A, B, C)
  • equatableTypeParams = A : Equatable, B : Equatable, C : Equatable
  • The contents of the return block: (lhs.1, lhs.2) == (rhs.1, rhs.2)

The contents of these variables are then inserted into the method using the ${} syntax we saw in our simple example earlier in the post.

The resulting output is a method which compares the first elements in the tuples with n elements and then delegates the responsibility for checking the rest of the items to the equality method with n-1 elements.

If we run this through the gyb tool, we should see the output for all 5 variants of the method.

python ./gyb.py  --line-directive= -o Tuple-Equality.swift Tuple-Equality.swift.gyb

Inspecting the contents of Tuple-Equality.swift we should see the following:

/// Returns `true` iff each component of `lhs` is equal to the corresponding
/// component of `rhs`.
public func == <A : Equatable, B : Equatable>(lhs: (A,B), rhs: (A,B)) -> Bool {
  guard lhs.0 == rhs.0 else { return false }
  /*tail*/ return (
    lhs.1
  ) == (
    rhs.1
  )
}


/// Returns `true` iff each component of `lhs` is equal to the corresponding
/// component of `rhs`.
public func == <A : Equatable, B : Equatable, C : Equatable>(lhs: (A,B,C), rhs: (A,B,C)) -> Bool {
  guard lhs.0 == rhs.0 else { return false }
  /*tail*/ return (
    lhs.1, lhs.2
  ) == (
    rhs.1, rhs.2
  )
}


/// Returns `true` iff each component of `lhs` is equal to the corresponding
/// component of `rhs`.
public func == <A : Equatable, B : Equatable, C : Equatable, D : Equatable>(lhs: (A,B,C,D), rhs: (A,B,C,D)) -> Bool {
  guard lhs.0 == rhs.0 else { return false }
  /*tail*/ return (
    lhs.1, lhs.2, lhs.3
  ) == (
    rhs.1, rhs.2, rhs.3
  )
}

// e.t.c

One more thing

So we've seen that the gyb tool can save us from having to repeat the same method, you may think we can do a similar thing for each of the <, <=, >, >= operators.

In fact, the tool is used to define just a single method for all four of them:

// Continued from earlier with variables such as 'arity', 'typeParams', e.t.c
%   comparableTypeParams = ", ".join(["{} : Comparable".format(c) for c in typeParams])
%   for op in ["<", ">"]:
%     for opeq in ["", "="]:
public func ${op}${opeq} <${comparableTypeParams}>(lhs: ${tupleT}, rhs: ${tupleT}) -> Bool {
  if lhs.0 != rhs.0 { return lhs.0 ${op}${opeq} rhs.0 }
  /*tail*/ return (
    ${", ".join("lhs.{}".format(i) for i in range(1, arity))}
  ) ${op}${opeq} (
    ${", ".join("rhs.{}".format(i) for i in range(1, arity))}
  )
}
%     end
%   end
% end // End of file

This is very similar to what we saw earlier within the == implementation, just now we check the result of the first elements with the current comparison operator (e.g. <=) if the elements are not equal to each other. Additionally, rather than simply constraining the types to conform to the Equatable protocol, we now require them to conform to the Comparable protocol.

The nested for loops mean we can generate all four combinations very easily.

Conclusion

It's unlikely you'll need to use anything like this tool within your own projects, but it's interesting to see how Swift avoids duplication.

It's also interesting that the tool has been written in Python rather than Swift, although this is presumably so it can be written once and the Swift team don't have to worry about the underlying language changing, as is the case of Swift in its current state. I wonder if in the future, with a more stable version of Swift, we'll see tools like this being written in it rather than an older, more established language like Python.

Back to Blog