gives error messages that I think shouldn't be there. It seems I can work around them with

ParallelEvaluate[
Module[{slot=1},
Slot@@{slot}
]
]

and

ParallelEvaluate[
Module[{slot=1,s=Slot},
s[slot]
]
]

but I still think this is a bug in the parallel framework. Am I missing something obvious? It seems the numeric check happens with the local Module placeholder values passed to the parallelism framework in a syntactic way, not in a semantic way. But that's wrong, I should be able to pass Slot[anything]

It happens on OS X as well (8 & 9) and even if you replace Module with With
–
SzabolcsJan 13 '13 at 19:24

These things happen when the expression is put in Function: Function@Module[{s = 1}, Slot[s]]. Perhaps constructing a function is part of the parallel distribution/retrieval process.
–
SzabolcsJan 13 '13 at 19:40

So to fix this first we need to find out where is the argument passed to ParallelEvaluate wrapped by Function. Fortunately the parallel tools are defined in a plain .m (not .mx) file so we can read the source code. The relevant part turns out to be in $InstallationDirectory/AddOns/Applications/Parallel/Parallel.m, in the definition of Send:

Send[kernels:{___kernel}, expr_] := Send[#, expr]& /@ kernels

The part causing the problem is Send[#, expr]&. Looks innocent enough, doesn't it? If we change this to Function[k, Send[k, expr]] then the message goes away.

Warning: I do not know if the form of Function with named formal parameters can cause any trouble (e.g. name collisions) in certain edge cases so be careful (or ask Leonid) ... Update: Here's an example of what may go wrong with named formal parameters.

Perhaps you can report this problem to WRI and let us know what they said.

Now that we know what the problem is caused by, we can manually construct simple problem cases. For example,

Generally, passing any Slot not wrapped in a Functionwithout formal arguments will cause problems (messages and unexpected results). I would consider this a bug, so I think it's a good idea to report it.

I would not classify this as a bug. The behaviour of a Slot expression that is not directly contained within a Function expression is not defined by the documentation, and is unreliable in practice. Consider the following two functions:

f[x_] := x + 100
g[x_] := x + # &[100]

They appear to be essentially equivalent, but it just so happens that the implementation of the second function involves a pure function. However, they return different results when passed a "dangling" slot reference (i.e. a slot reference that is not directly contained within a pure function):

f[#]
(* 100 + #1 *)
g[#]
(* 200 *)

The second result might surprise us, especially if the documentation for g said "adds 100 to its argument". Consider what happens if we call g with the Module expression from the question:

g[Module[{slot = 1}, Slot[slot]]]
(* 200 *)

200 is perhaps not exactly what we expected, but at least it is consistent with g[#].

On the other hand, let's try it with a new function h which is the same as g but holds its argument:

We get the same warning message as in the question. This occurs because this expression ultimately resolves to:

Module[{slot = 1}, Slot[slot]] &

... which is manifestly incorrect since arguments to Slot in a function body must be non-negative integers -- not symbols.

ParallelEvaluate happens to be implemented somewhat like h. It holds the argument, and its implementation happens to use a pure function.

The moral of this story is that the behaviour of a dangling slot reference is essentially undefined. Its behaviour depends upon the exact implementation of the functions to which it is passed. If we are unaware of those exact implementation details, then we should avoid the use of dangling slot references since we can never be sure if that slot expression might not accidentally find its way into an internal Function expression.

To work around this difficulty, we can use a temporary symbol in place of Slot and then substitute it out after the evaluations are complete:

I agree in the sense that this is not a bug in Function or Slot, but this behavior should not be manifested in the Parallel` package. Actually I would argue that your g and h have a precedence bug; g should be defined as e.g. g[x_] := x + (# &)[100], whereupon it gives the same result as f. The documentation implicitly countenances (in the Applications section) building expressions containing Slot without a containing Function--their example of Function[Evaluate[f @@ Array[Slot, 5]]] is logically identical to e.g. With[{slots = f @@ Array[Slot, 5]}, Function[slots]].
–
Oleksandr R.Jan 13 '13 at 20:49

1

@OleksandrR It is not always a simple matter to bulletproof a pure function body against injected slot references. What about the Send function exhibited in @Szabolcs answer? Named pure function arguments don't help either -- they have problems of their own. Even if such bulletproofing were simple, it is not common practice. I'm not arguing that this behaviour is desirable, but I am saying that we shouldn't expect well-defined behaviour from an ill-defined language construct.
–
WReachJan 13 '13 at 21:21

2

The problem is that there are no clear hints in the docs that there is anything wrong with using naked slots. As Oleksandr points out there is actually an example of programmatically constructing a pure function, which could be perceived as an encouragement to do such things. (I believe @Andreas must have been doing something similar: constructing code programmatically. Though it's true that combining this with parallel evaluation is a little unusual.) Also consider that ParallelEvaluate usually takes code as its argument, unlike most other functions. I wouldn't say that this problem was ...
–
SzabolcsJan 14 '13 at 1:49

2

@Szabolcs. True the documentation has an example of generated slots, but the example conforms to my advice to only use it when the exact context is known (f has no definitions). I totally agree to report it -- I reported the issue in my blog. Sounds like a great possible issue for the docs. However, I anticipate the same response: avoid the situation. ...
–
WReachJan 14 '13 at 2:28

2

@WReach +1, agree with all your points. By the way, I wrote a tiny framework to resolve such nasty variable collisions / leaks as in your example in the blog. It lives here. When I call runWithRenamings[ ClearAll[f, g]; f[x_] := g[Function[a, x]]; g[fn_] := Module[{h}, h[a_] := fn[a]; h[0]]; f[999]], I get the correct result. Of course this does not eliminate the problem, but it might be useful for debugging, and also one can run some code in a "safer mode", when wrapped in runWithRenamings.
–
Leonid ShifrinJan 14 '13 at 10:14

The problem with this is that it needs to thread only over those lists appearing in the first argument. Personally, I'd rather do the threading explicitly than rely on the vagaries of Listable.
–
Oleksandr R.Jan 14 '13 at 13:22

Just like Power[{1, 2, 3}, 2]. So what is vague here? Listable behavior is not going to change I guess.
–
Rolf MertigJan 14 '13 at 14:21

1

You're probably right that it won't change. But what if someone wants to ParallelEvaluate a list that happens to be the same length as the list of kernels?
–
Oleksandr R.Jan 14 '13 at 14:29

It's a nice idea to rely on Listable for a solution. Hopefully this could be robust (unlike a Function with named arguments).
–
SzabolcsJan 14 '13 at 15:32

The only answer offering a simple good solution in the first lines, deserves more votes, +1!
–
RojoJan 15 '13 at 3:54

Mathematica is a registered trademark of Wolfram Research, Inc. While the mark is used herein with the limited permission of Wolfram Research, Stack Exchange and this site disclaim all affiliation therewith.