Walter and I discussed today and decided to fix this long-standing issue:
import std.algorithm;
void main() {
const arr = [1, 2, 3];
reduce!"a*b"(arr);
}
The problem here is that D's built-in arrays are at the same time containers and their own ranges. When the array is constant the matter becomes confusing because the range itself must be non-constant such that it can iterate the constant array.
Put simply, the type of the array in this case should be const(int[]) and the type of its range should be const(int)[].
This problem does not commonly occur with ranges because people do expect that a constant range can't be used for iteration, and an elaborate container will emit the proper type of range even when the container itself is constant.
We decided to fix this issue by automatically shedding the top-level const when passing an array or a pointer by value into a function.
The rule is simple and does not complicate lookup rules, type deduction, or template specialization. It is a semantic rewrite as follows.
Consider x an array of type qualifier(T[]). In any function call in which it is established that x is passed by value (i.e. no "ref" with the parameter), the call:
fun(..., x, ...)
is lowered into:
fun(..., cast(qualifier(T)[]) x, ...)
after which the usual language rules apply. Similarly, if x has type qualifier(T*), AND if x is passed by value into a function, the call:
fun(..., x, ...)
is lowered into:
fun(..., cast(qualifier(T)*) x, ...)
after which, again, the usual rules apply. Note that fun does not need to be a template and generally must meet no special conditions aside from taking x by value. If fun takes specifically a const(T[]), the call will go through no problem because const(T)[] is implicitly convertible to const(T[]).
This allows template functions to accept arrays by value yet modify their own private copy of the array's bounds, which is reasonable and expected.
This rule solves a host of issues related to applying range algorithms to arrays. I hope we will have this rule in action in dmd 2.057.
Andrei

On Saturday, December 10, 2011 15:47:13 Andrei Alexandrescu wrote:
> Walter and I discussed today and decided to fix this long-standing issue:
[snip]
This looks like a solid decision, and I was actually about to suggest it when thinking about the issues with const ranges today. It's the only sensible solution that I can think of. I was wondering if there might be any negative side effects to it, but I haven't been able to think of any.
It would be nice to do the same thing with static arrays, but that would make it harder to pass static arrays to templated functions, and it actually becomes quite dangerous, since it then becomes very easy to mistakenly return references to that static array when the scope is exited. So, I think that the issue of having to slice static arrays should probably remain, but unlike dynamic arrays, static arrays _do_ own their memory, so it's quite reasonably to have to slice them IMHO.
The other remaining issue is how to handle tail-const in the general case. Ideally, there would be a way to return a tail-const range from a const range. The only way I can think of enabling that at the moment is to make it so that if opSlice is defined for a type, and it returns the same type (save for how const or immutable it is), then it could be treated the same way as arrays would be as far as template instantiations go (that is, the newly suggested way). But I don't know quite what side-effects that has. Still, I do think that we should find a way to define tail-constness for ranges which are user-defined types. It's not as critical as with arrays, but given the transivity of const, it's still important, since it can be very easy to end up in a situation where it becomes somewhat problematic to get a tail-const range when you're using const much.
In any case, I applaud this decision. It should definitely help dealing with arrays as ranges easier. The fact that const and immutable don't work very well at all with ranges at this point is problematic given how const and immutable are supposed to be major features of the language, and this is definitely a step in the right direction.
- Jonathan M Davis

Le 10/12/2011 22:47, Andrei Alexandrescu a écrit :
> Walter and I discussed today and decided to fix this long-standing issue:
>> import std.algorithm;
> void main() {
> const arr = [1, 2, 3];
> reduce!"a*b"(arr);
> }
>> The problem here is that D's built-in arrays are at the same time
> containers and their own ranges. When the array is constant the matter
> becomes confusing because the range itself must be non-constant such
> that it can iterate the constant array.
>> Put simply, the type of the array in this case should be const(int[])
> and the type of its range should be const(int)[].
>> This problem does not commonly occur with ranges because people do
> expect that a constant range can't be used for iteration, and an
> elaborate container will emit the proper type of range even when the
> container itself is constant.
>> We decided to fix this issue by automatically shedding the top-level
> const when passing an array or a pointer by value into a function.
>> The rule is simple and does not complicate lookup rules, type deduction,
> or template specialization. It is a semantic rewrite as follows.
>> Consider x an array of type qualifier(T[]). In any function call in
> which it is established that x is passed by value (i.e. no "ref" with
> the parameter), the call:
>> fun(..., x, ...)
>> is lowered into:
>> fun(..., cast(qualifier(T)[]) x, ...)
>> after which the usual language rules apply. Similarly, if x has type
> qualifier(T*), AND if x is passed by value into a function, the call:
>> fun(..., x, ...)
>> is lowered into:
>> fun(..., cast(qualifier(T)*) x, ...)
>> after which, again, the usual rules apply. Note that fun does not need
> to be a template and generally must meet no special conditions aside
> from taking x by value. If fun takes specifically a const(T[]), the call
> will go through no problem because const(T)[] is implicitly convertible
> to const(T[]).
>> This allows template functions to accept arrays by value yet modify
> their own private copy of the array's bounds, which is reasonable and
> expected.
>> This rule solves a host of issues related to applying range algorithms
> to arrays. I hope we will have this rule in action in dmd 2.057.
>>> Andrei
I love to see that. This is going the right way.
You deserve your wikipedia page (semi private joke inside).

Treating whole constant arrays as ranges by automatically shedding the
top-level const is good.
But realizing it by language semantic change is definitely bad.It
breaks IFTI rule, and adding special case will make difficult to learn
language.
Instead of language change, we can add specializations that receive
non-ranges and convert them to ranges by removing top-level const.
I believe that it is Phobos issue and is never the issue of language.
Kenji
2011/12/11 Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>:> Walter and I discussed today and decided to fix this long-standing issue:
>> import std.algorithm;
> void main() {
> const arr = [1, 2, 3];
> reduce!"a*b"(arr);
> }
>> The problem here is that D's built-in arrays are at the same time containers and their own ranges. When the array is constant the matter becomes confusing because the range itself must be non-constant such that it can iterate the constant array.
>> Put simply, the type of the array in this case should be const(int[]) and
> the type of its range should be const(int)[].
>> This problem does not commonly occur with ranges because people do expect that a constant range can't be used for iteration, and an elaborate container will emit the proper type of range even when the container itself is constant.
>> We decided to fix this issue by automatically shedding the top-level const when passing an array or a pointer by value into a function.
>> The rule is simple and does not complicate lookup rules, type deduction, or template specialization. It is a semantic rewrite as follows.
>> Consider x an array of type qualifier(T[]). In any function call in which it
> is established that x is passed by value (i.e. no "ref" with the parameter),
> the call:
>> fun(..., x, ...)
>> is lowered into:
>> fun(..., cast(qualifier(T)[]) x, ...)
>> after which the usual language rules apply. Similarly, if x has type qualifier(T*), AND if x is passed by value into a function, the call:
>> fun(..., x, ...)
>> is lowered into:
>> fun(..., cast(qualifier(T)*) x, ...)
>> after which, again, the usual rules apply. Note that fun does not need to be
> a template and generally must meet no special conditions aside from taking x
> by value. If fun takes specifically a const(T[]), the call will go through
> no problem because const(T)[] is implicitly convertible to const(T[]).
>> This allows template functions to accept arrays by value yet modify their own private copy of the array's bounds, which is reasonable and expected.
>> This rule solves a host of issues related to applying range algorithms to arrays. I hope we will have this rule in action in dmd 2.057.
>>> Andrei

This is just about removing the constness for whatever the part of a type that is passed by value. This is not really a special rule and make sense.
Le 10/12/2011 23:31, kenji hara a écrit :
> Treating whole constant arrays as ranges by automatically shedding the
> top-level const is good.
> But realizing it by language semantic change is definitely bad.It
> breaks IFTI rule, and adding special case will make difficult to learn
> language.
>> Instead of language change, we can add specializations that receive
> non-ranges and convert them to ranges by removing top-level const.
> I believe that it is Phobos issue and is never the issue of language.
>> Kenji
>> 2011/12/11 Andrei Alexandrescu<SeeWebsiteForEmail@erdani.org>:>> Walter and I discussed today and decided to fix this long-standing issue:
>>>> import std.algorithm;
>> void main() {
>> const arr = [1, 2, 3];
>> reduce!"a*b"(arr);
>> }
>>>> The problem here is that D's built-in arrays are at the same time containers
>> and their own ranges. When the array is constant the matter becomes
>> confusing because the range itself must be non-constant such that it can
>> iterate the constant array.
>>>> Put simply, the type of the array in this case should be const(int[]) and
>> the type of its range should be const(int)[].
>>>> This problem does not commonly occur with ranges because people do expect
>> that a constant range can't be used for iteration, and an elaborate
>> container will emit the proper type of range even when the container itself
>> is constant.
>>>> We decided to fix this issue by automatically shedding the top-level const
>> when passing an array or a pointer by value into a function.
>>>> The rule is simple and does not complicate lookup rules, type deduction, or
>> template specialization. It is a semantic rewrite as follows.
>>>> Consider x an array of type qualifier(T[]). In any function call in which it
>> is established that x is passed by value (i.e. no "ref" with the parameter),
>> the call:
>>>> fun(..., x, ...)
>>>> is lowered into:
>>>> fun(..., cast(qualifier(T)[]) x, ...)
>>>> after which the usual language rules apply. Similarly, if x has type
>> qualifier(T*), AND if x is passed by value into a function, the call:
>>>> fun(..., x, ...)
>>>> is lowered into:
>>>> fun(..., cast(qualifier(T)*) x, ...)
>>>> after which, again, the usual rules apply. Note that fun does not need to be
>> a template and generally must meet no special conditions aside from taking x
>> by value. If fun takes specifically a const(T[]), the call will go through
>> no problem because const(T)[] is implicitly convertible to const(T[]).
>>>> This allows template functions to accept arrays by value yet modify their
>> own private copy of the array's bounds, which is reasonable and expected.
>>>> This rule solves a host of issues related to applying range algorithms to
>> arrays. I hope we will have this rule in action in dmd 2.057.
>>>>>> Andrei

On Sunday, December 11, 2011 07:31:28 kenji hara wrote:
> Treating whole constant arrays as ranges by automatically shedding the
> top-level const is good.
> But realizing it by language semantic change is definitely bad.It
> breaks IFTI rule, and adding special case will make difficult to learn
> language.
> > Instead of language change, we can add specializations that receive
> non-ranges and convert them to ranges by removing top-level const.
> I believe that it is Phobos issue and is never the issue of language.
Making the fix in Phobos is a pain in that you have to duplicate functions all over the place (though they'd just be wrappers, so you wouldn't have to duplicate their implementations). And everyone who defines their own range- based functions have to do the same thing.
The question is whether we actually lose anything by making it so that so that IFTI won't give you a fully immutable or const array. And I don't see anything that you'd lose or how it could break any code. Sure, it then becomes possible to alter the array within the function where it wasn't before, but if it couldn't before, making it possible now won't break anything, since the function would have to be changed to even try. And for new functions, if they _really_ want the array to be const or immutable, they can assign it to a new array locally. I don't see how any expressiveness is really lost here or that anything would be broken.
So, while this _does_ change the language's semantics, it seems to me that it just changes them for the better, not worse.
Making this sort of change for static arrays _would_ be a problem, but I don't see how it's an issue for dynamic arrays.
- Jonathan M Davis

On 12/10/11 4:31 PM, kenji hara wrote:
> Treating whole constant arrays as ranges by automatically shedding the
> top-level const is good.
> But realizing it by language semantic change is definitely bad.It
> breaks IFTI rule, and adding special case will make difficult to learn
> language.
There is no breakage of IFTI, just a reduction of what IFTI sees.
> Instead of language change, we can add specializations that receive
> non-ranges and convert them to ranges by removing top-level const.
> I believe that it is Phobos issue and is never the issue of language.
The problem here is scale. We're looking at an absolutely massive code duplication for every single function. I don't think this will ever work; if we do it in Phobos it will make Phobos and by extension the language more difficult to understand AND more bloated for the sake of needless consistency.
The language change is legitimate when you think of it this way - it gives template functions back the natural right to change their local state. This right was already there for non-template functions.
Andrei

On 12/10/11 4:31 PM, kenji hara wrote:
> Treating whole constant arrays as ranges by automatically shedding the
> top-level const is good.
> But realizing it by language semantic change is definitely bad.It
> breaks IFTI rule, and adding special case will make difficult to learn
> language.
>> Instead of language change, we can add specializations that receive
> non-ranges and convert them to ranges by removing top-level const.
> I believe that it is Phobos issue and is never the issue of language.
I should add there is precedent. C++ also removes top-level const when passing objects by value to templates. Deducing top-level const with pass-by-value is inherently nonsensical.
Andrei

kenji hara:
> It breaks IFTI rule,
What do you mean?
> and adding special case will make difficult to learn language.
I think that change proposed by Andrei A. (that I like) doesn't add a special case to D2. Currently (2.057beta) this works:
void main() {
immutable(int[]) a = [1, 2];
immutable(int)[] b = a;
}
Calling a function that accepts a immutable(int)[] with a immutable(int[]) means creating a local slice, it's similar. So I think Andrei A. removes a special case.
Bye,
bearophile