a blog on programming languages and other stuff…

Iterators in Visual Basic

Actually, in this case I don’t thing the above warning is strong enough. This is a super speculative post, because I believe the chance of it appearing in the next version of the language is not extremely high, not because it’s not a worthy feature but because it’s more than a little work and we’ve got a lot of other very worthy features we’re considering. However, since it’s something that’s valuable and something we keep getting requests for, we have decided to at least generally sketch out what iterators would look like in Visual Basic, with the idea that when the time comes that we have the resources to do them we can jump on it. Consider yourself double warned.

To start with, an iterator is just a structure that returns a sequence of values. Basically, when the IEnumerable(Of T) interface method GetEnumerator() returns an instance of the interface IEnumerator(Of T), that instance is an iterator. You can iterate through the collection by calling MoveNext on the enumerator and looking at the Current property until you’ve run out of values. Or you can use the For Each statement, which will do the iteration for you.

It’s possible to construct iterators in Visual Basic today–all you have to do is implement IEnumerable(Of T) and then implement IEnumerable(Of T). But the problem is that doing this involves a whole lot of boilerplate code that is the same in almost all iterators. On top of that, if an iterator involves a lot of conditions or branching, doing state management can be quite tricky. In an ideal world, what you’d like to be able to do is simply state the values that should be iterated over and have the compiler generate all the boilerplate code and state management for you. C# does this through the yield statement, which can be used in a method that returns an iterator to automatically generate the iterator:

In this example, the FromTo method returns an iterator (typed as IEnumerable(Of Integer)) which is automatically generated by the compiler. What we’re considering in Visual Basic is essentially the same feature, but with a slightly different design. Instead of tying the iterator to a method, as in C#, we’re considering extending the multi-line lambda syntax that I talked about in the previous post to make “anonymous iterators.” The above example in VB would look something like:

Module Module1
Function FromTo(ByVal low As Integer, ByVal high As Integer) As IEnumerable(Of Integer)
Return IteratorIf low <= high Then
Return low
Return Each FromTo(low + 1, high)
End If
End IteratorEnd Function
Sub Main()
For Each i In FromTo(1, 5)
Console.WriteLine(i)
Next
End Sub
End Module

There are a couple of differences to notice here:

An anonymous iterator is just an expression, so it can be used as the return value of a function but could also appear as the target of a For Each, used in a LINQ query, stored into a field, etc. (This is particularly useful when a function produces an iterator and you want to do some validation. In the C# model, the validation code is not run until MoveNext is first called on the iterator, not when the iterator is actually produced.)

By default the type of the iterator is IEnumerable(Of T), where T is inferred from all the types returned from the iterator using the same algorithm we use for multi-line lambdas. You could also explicitly state the iterator type using an As clause, and we would allow IEnumerator(Of T) as the iterator type just like C# does.

We don’t require the yield keyword since we’ve already marked the block as being special by using a new contextual keyword Iterator. (There would also be an Exit Iterator statement, not shown.)

Our plan would be to add an Each operator that “unwraps” a collection and returns each of its values one by one. This is generally useful but also addresses some performance issues with nested iterators which you can read more about in Wes Dyer’s post on iterators. (It might also later allow things like unwrapping collections in collection initializers, like {1, 2, 3, Each a, 5, 6, 7}, where a is an array of integers or something…)

We’re interested in people’s feedback on this proposed design and any other comments you might have.

Excellent, very happy to see that iterators may become part of the language. I’ve had to write some recently, and they’re currently no fun in VB.

I like the use of lambdas (of the multi-line variety), and like most of VB, it seems a bit more logical than the C# version.

I have 2 additional minor comments:

1. The Return Iterator block does not seem consistent with other blocks, in that your End Iterator closes the "Iterator", not "Return Iterator". I would love to see a different form of "Return" in block form, maybe with a completely different keyword. For example, using the "Yield" term, you could replace "Return Iterator" with "Yield", and replace "End Iterator" with "End Yield".

(I like the word yield, in that it is an action word, unlike iterator, which requires that a "verb" be placed before it. Or maybe it could be Iterate/End Iterate?)

2. Is there a different syntax that would not require the two Returns within the If block? Many will not see how the second Return will ever get executed. How about something like:

First a minor typo. You wrote "all you have to do is implement IEnumerable(Of T) and then implement IEnumerable(Of T). " you probably meant to write IEnumerator(Of T) for the last IEnumerable.

Second, you said:

"(This is particularly useful when a function produces an iterator and you want to do some validation. In the C# model, the validation code is not run until MoveNext is first called on the iterator, not when the iterator is actually produced.) "

I’m not sure I follow what you mean there. At first it read like iterators in VB would not have delayed evaluation. But I am guessing what you meant was validation outside fo the iterator block, e.g:

Function GetIterator(switch AS Boolean) As IEnumerable(Of Fruit)

Dim var as IEnumerable(Of Fruit)

If switch THen

var = Iterator

……

End Iterator

Else

var = me.innerlist

End If

Return var

End Function

Third, having Iterator a protected keyword could be a breaking change. Any thoughts on addressing that ?

(other than the foxpro BEGIN Iterator … End Iterator of course <g>) Maybe:

#Iterator

#End Iterator

Thanks. This looks good. Can we have it as an update to VS 2008 as well ? 🙂