Tech Blog Python-style "Generators" in Go 1.23
Proxati is written in Golang, and we are big fans of the language and community. The language is generally very stable, new features are seldom added, so it’s a big deal when they are added.
The upcoming Go 1.23 release will introduce a powerful new feature: range-over function iterators. This enhancement brings a Python-like generators to Go, making it easier to iterate over custom data structures and sequences in a more flexible and idiomatic manner. This topic was recently discussed on HN.
What Are Range-over Function Iterators?
Range-over function iterators allow you to iterate over values produced by a function, much like Python's generators. This feature leverages the new Seq and Seq2 types, which define iterator functions that yield values on demand.
Read more about it here: https://tip.golang.org/wiki/RangefuncExperiment
package slices
func Backward[E any](s []E) func(func(int, E) bool) {
return func(yield func(int, E) bool) {
for i := len(s)-1; i >= 0; i-- {
if !yield(i, s[i]) {
return
}
}
}
}
s := []string{"hello", "world"}
for i, x := range slices.Backward(s) {
// will output:
// 0 world
// 1 hello
fmt.Println(i, x)
}
This code snippet defines a Backward function that iterates over a slice in reverse order. Using for i, x := range slices.Backward(s), you can easily loop through the values without manually managing the loop logic.
Why Does This Matter?
This feature enhances readability and expressiveness, and also makes Go more accessible to Python developers by adding a familiar design pattern.
Improved Readability: Simplify iteration logic makes the loop body cleaner and easier to understand.
Greater Flexibility: Data structures in libraries can now attach iterators, so you don’t need guess how to best implement iteration.
Lazy Evaluation: Values are produced on-the-fly, which can reduce memory demands and GC pressure by avoiding storage of large intermediate data structures.
How is iter.Pull Used?
Function iterators are normally “push” iterators where a value generator “pushes” values at a function generated from the body of the loop. In some cases, it is more convenient to “pull” values from the generator, and iter.Pullaccomplishes that. For example, the Zip function combines two range functions into a single range function that provides respective pairs from the two inputs:
// Zipped holds values from an iteration of a Seq returned by [Zip].
type Zipped[T1, T2 any] struct {
V1 T1
OK1 bool
V2 T2
OK2 bool
}
// Zip returns a new Seq that yields the values of seq1 and seq2 simultaneously.
func Zip[T1, T2 any](seq1 iter.Seq[T1], seq2 iter.Seq[T2]) iter.Seq[Zipped[T1, T2]] {
return func(yield func(Zipped[T1, T2]) bool) {
p1, stop := iter.Pull(seq1)
defer stop()
p2, stop := iter.Pull(seq2)
defer stop()
for {
var val Zipped[T1, T2]
val.V1, val.OK1 = p1()
val.V2, val.OK2 = p2()
if (!val.OK1 && !val.OK2) || !yield(val) {
return
}
}
}
}
What Will Idiomatic APIs with Range Functions Look Like?
The exact forms of idiomatic APIs using range functions are still being discussed, but we can imagine a container like a binary tree implementing an All method that returns an iterator:
func (t *Tree[V]) All() iter.Seq[V]
A List might also provide a backward iterator:
func (l *List[V]) All() iter.Seq[V]
func (l *List[V]) Backward() iter.Seq[V]
Performance and Readability
According to the release notes, the compiler has been designed to optimize these iterators so the overhead of the function calls is minimized, making range-over function iterators as performant as hand-written loops. Benchmarks will tell the real story once the feature is finalized.
Comments