…On the Wings of the Software Wind

Main menu

Post navigation

Innovation: Array quirks and Generators

Speaking a little bit about innovation, we had a great discussion about Class Signatures also here in this blog as well as in a very hot thread in the .language forum, and finally I’ve submitted also a QC Report – vote for it btw.

But today I would ask for your opinion about another feature request, which, in fact, will fill a gap (or rather an oddity?) in the Delphi compiler. But the main point is how to implement it…

Hmmm… (again) …but what if my list will be generated upon request, based on a function (using a hidden enumerator)? I need many times to write something clear and concise things like this:

var
a : array of double;
begin
a := (for i in [1..4]); //gives (1,2,3,4)
a := (for i in [1..4] do i/2); //gives (0.5, 1, 1.5, 2)
a := (for i in [1..6] do i/2 if i mod 2 = 0); //gives (1,2,3)
a := (for i in [1..10] step 2); //gives (1,3,5,7,9)
end;

As an aside, yes, sons, perhaps is time to implement the Step clause in For. Just a humble hint. We are in 2009, you know.

Of course this should support anonymous lists:

a:=(for i in [(for k in [1..3] do k*2)] do i+1); //gives [3,5,7]
//which will allow us to pass as an argument like in...
procedure MyFunc((for i in [1..10)); //Aggregate functions like Sum, Avg etc. are fist candidates here.

Also, as you can comprehend ( 😉 ) this allows expressing infinite lists because the n-th element is generated on demand using the hidden enumerator inside.

And before you bite, yes, here are two different requests: one talks about a static assignment to an array in the program body and one talks about a dynamic and and concise way to express a generator – a (perhaps infinite) list implemented using an enumerator.

I like the idea of being able to assign to an array variable in code, but I can understand a) why you can’t and b) why it is likely to be impractical in practice.

When assigning to a variable in code you are removed from the declaration of the array itself so you don’t know how many elements are actually required (in the case of fixed dimension arrays). It’s one of those things that looks convenient when you sit down with a simple example but I think would be impractical in actual use.

Dynamic arrays are a different matter and could be accommodated more practically, but that then raises the issue of treating different types of arrays very differently. Then again, dynamic arrays are already very different from static arrays – it’s only that their syntax is so similar that it’s easy to forget the important distinctions that exist between them.

Think at the analogy between fixed-size strings (ie. string[200]) and the variable length strings. Then your points still apply? (Ie. a fixed-size string should be initialized only at the declaration place), assigning strings is impractical for big sizes etc.
I do think that nowadays any feature can be mis-used. It is just to see what is the advantage which brings, and if the advantage is greater than the headache then perhaps we must implement it. Also, a sign that this is useful is that other languages already have it: Python (link in post), Haskell and of course .NET’s Linq is in fact a form of it (ie. a way of generating dynamic lists based on a condition).

Oh, and the whole dynamically sizing array is great for 2 or 3 items. Hit a few hundred and performance starts to seriously take a hit. Interesting idea, simple example – hideous result in scale. If you were to write a routine, you would want to have some clue how large the result was.

So ya, interesting ideas – however all are already easily solvable already, some better solved with code as it allows you more control over the end result and how you get there.

“The biggest problem is that the ( symbol has no special meaning in the context of an assignment “
This is a grammar / parser way of behaving not a compiler issue. And it can be changed. Anyway this isn’t the gist of the issue.
Do you think that writing an entire function (like you did) for each case is “nice and simple” compared with (in your case) a:=(for i in [1..10]) ? Also in your function ForIn it doesn’t even show the rule. One has to look to the actual code to see what’s happening inside. So you must write a sea of functions called GetImpareNumbers, GetLogNumbers, GetSomeStrings etc. etc. to cover everything? And how will you nest these functions? All the IT crowd is going today to the lambdas and functional programming and should we stay to plain old procedure based style? Sorry but I don’t think so.“Oh, and the whole dynamically sizing array is great for 2 or 3 items. Hit a few hundred and performance starts to seriously take a hit.”
1. The proposed syntax improves this. Actually a for cycle (your solution) is way slower than my syntax. Do a benchmark of 10000ths assignments compared with a for cycle on the same structure to see.
2. So you propose to not use (so do not assign values to) arrays, strings and TStringLists bigger than 2 or 3 items because of “performance issues”?
3. A routine is necessarily scanning the entire array from Start to Finish as you demonstrated in your example. However a generator (because it uses internally an Enumerator) it will work in the worst case (if techniques like caching/memoization, direct access aren’t used) till the requested ‘n’ element. So it is faster than your approach. And it can be considerably faster.
4. Having a generator with an enumerator inside can also have the advantage to reduce a lot the memory used. It can use different amounts of memory (eg. the cached elements – if applicable etc.) whereas an array initialized in a For cycle needs to ‘have room’ for all elements.

Yep. This is the idea. And this can be enhanced to have a more clearer syntax. But the main point is the other one: Having a clear syntax to generate the a list (array etc.). But for these we need lambdas.

I remember Barry Kelly talking about the problems to implement lambdas:
The parser may need to know some types before they are available the way Delphi works today…
May be with the new way they are working in, the problem can be solved.

Even if it cant be solved, y the compiler can cast an expression of type T1 to a function reference of type T2 inserting the cast from T1 to T2 in the code, the next could work:

Yep, something like this. Well even if we can improve your syntax, the biggest problem is the lambdas as you observed. Using now an anonymous function will do the work but it is way too much to write. A proper syntax should hide all these gory details.

I didn’t believe it worked in Win32…
I was thinking it was some sort of .NET extension like Array[,] …
but this way of array-constructor call works even in Delphi 2006…
The Create is red underlined, but it works.

If you look at the code, it’s a lot of code…
the SetLength and the n assignments are inlined,
but it works.

I liked your suggestions (both). The dynamic array assignment form is something would be implemented a long time ago. Btw, the anywhere-variable-assinment is something I’d like to see in OP for too long. After all, why can’t we do:

var
x: integer = 100;

in local variables too? It’s semantically the same of:

var
x: integer;
…
begin
x := 100;

Isn’t it? Who knows Delphi/OP needs a pre-compiler processor…

About the list constuctor, imho, is such a f* great idea. It’s time for OP understand the needs of modern programmers and loose a little the fear of innovating features and mainly syntax. There are several ways of extend the language without loose its identity. The Object Pascal was a big step in this way, but, for some reason, people get stucked in the stairs watching pythons and rubies going up, fearing break the Turbo-Pascal-chains.

Let’s keep getting such good ideas like these comming up and, doing that, our old good boy Pascal won’t die of “feature starvation”.

Your English is quite fine – it shouldn’t stop you to give feedback 🙂“It’s time for OP understand the needs of modern programmers and loose a little the fear of innovating features and mainly syntax. There are several ways of extend the language without loose its identity.”
Sure – your native language didn’t borrow new words from other languages, or build new words in order to express the new realities from today? Yes, (I think so). OTOH, did your native language lose its identity? No, I don’t think so.

Look at Ruby/Python: Everything is an object! So simple and so powerful [..and maybe so slow :)]. Both support easy-to-read Lambda function, Delphi’s lambda implementation seems ugly [begin..end, you know…]

I think [everything-is-an-object] + [nice-looking-lambda] = [cool-syntax]. If Delphi support such things – any your suggestion would be possible.

Example:

A := (for i in [0..20] yield i * i)

A is an array, which overrides “:=” operator, so if it gets lambda function as parameter, it iterates over it and fills itself with returned values. [let’s say (…) is simplified lambda notation]

Another question: Is it [I’m talking about dynamic/functional programming] The Delphi Way?