While there are some cases where a For loop might be reasonable, it's a general mantra – one I subscribe to myself – that "if you are using a For loop in Mathematica, you are probably doing it wrong". But For and Do loops are familiar to people who already know other programming languages. It is common for new Mathematica users to rely on loop constructs, only to become frustrated with performance or code complexity.

My question is: are there some general rules of thumb that would help new users select the appropriate alternative to a procedural For or Do loop, and are there other useful Mathematica constructs, aside from RandomVariate/Nest/Fold/Inner/Outer/Tuples that they should know about in order to avoid those loops?

This question came from our site for professional and enthusiast programmers.

How about operations like arithmetic on lists, matrixes, & tensors or Dot and DotProduct? "For" my part (pun intended), I stopped thinking in loops and started thinking functionally. It would help if one could tie someones "loop" arm to their body, like they do with boxers who need to switch their dominant hand. Ultimately, I think one has to give themselves over to the functional language. Maybe one can translate one language to another but poetry in English doesn't have the same meaning translated into Italian nor Italian to English. When one can dream in the language they'll have it.
–
JagraJul 5 '12 at 12:53

There are cases where I find for loops more readable. For example R = S (*list of elemetns*); Do[ R[[i]] = f[S[[i]], G[[i]]]; , {i, Length@S}]; vs R = f[#[[1]], #[[2]]] & /@ Transpose[{S, G}] The second one is more concise, but G[[i]] conveys the intent better then #[[1]].
–
AjasjaJul 5 '12 at 12:54

4

@Ajasja, even more concise and readable is R=Thread@f[S,G]
–
Simon WoodsJul 5 '12 at 13:28

3

Just one other thing one may want to add as a caveat: the preferred choice of construct can change radically when the goal is to use it with Compile.
–
JensJul 5 '12 at 15:12

1

@Jens yes, that seems to be ignored most of the time. readability is also dependent on the person: I have some code to transform a density matrix that looks like this: MapThread[Join,{rtmp,Rest /@ Flatten[Conjugate@rtmp, {{2}, {1}}]}]. Some may read this immediately, but I need a commented-out iterative version to be able to re-read it later (this one's faster though).
–
aclJul 5 '12 at 15:34

Do and Scan are (typically) appropriate for operations that do not accumulate a list of results, while Map and Table (a variant of Do) are (typically) used for ones that do. The third method uses Map (short form: /@) and accumulates a useless list of results; for this reason it is likely to be less memory efficient than the first two.

Case #2

One might ask why use Table here at all. Indeed, this simple example can be written:

func /@ list

which is preferred. However, there are more complex cases where Table is far more elegant than the alternatives.

Average Timing for 10^6 real numbers using Sin for func:

Case #3

This could be improved in the manner of case #2, but there is a far better method available: Fold. Any time you want to iterate through a list, using the result of the previous "loop" along the way, look to Fold, efficient in both syntax and computation.

Fold[# + Mod[#2, #] &, 9, lst]

Average Timing for 10^6 integers (big savings!):

Case #4

There is nothing intrinsically wrong with this. However, all of the operations within the pure function have the Attribute Listable. Therefore, this function can directly accept a list without using Map, and this will be considerably more efficient.

(3 - #)/(7 * #) & @ list

Average Timing for a list of 10^6 real numbers:

Case #5

This is similar to case #4, but a little more complicated. Once again, each subfunction in f is Listable but this is not being leveraged. One can use Apply to pass the sub-lists in a as arguments to f:

f @@ a

Suppose that not all of the functions are Listable. I create a dummy addition function g that only accepts integers, not lists. I then include this in f, and try to Apply it again:

But the result is incorrect. Once could go back to MapThread, but a better way, when possible, is to make g handle lists, which will usually be faster on large sets. Here are two ways of doing that. Give g the Listable attribute, and Mathematica will automatically thread over lists:

@Mr. I use #2 all the time .... but the version in the question. Habits are difficult to change (said the monk)
–
belisariusApr 30 '11 at 5:34

@Mr.Wizard I think it'd be nice to give examples of the Do and Map functions you use in Case #3 and #4, respectively. Lets people test out the difference in speed themselves!
–
Vincent TjengMar 31 '13 at 3:46

@Vincent Those were accidentally left out of the question for a time, but I restored them shortly after your comment. (Thanks.) Now almost a year later do you have any other suggestions for improving this Q&A?
–
Mr.Wizard♦Jan 4 '14 at 21:43

There are many alternative ways to approach various programming problems that do not use loops and are more efficient (and concise) in Mathematica. Most of them execute faster, but even where they do not, they are faster to type: development time matters, too!

Here are some rules of thumb for easier programming and iterating on lists.

1. Most arithmetic operations and many other functions are Listable, meaning that they operate element by element without having to set up a loop or operate explicitly on each element

Addition, multiplication and other standard operations work element by element when they are given conformable vectors, matrices, tensors or generalised lists:

Listable functions include standard arithmetic and trigonometric functions, Log, Exp, Mod, Abs and a huge range of exponential, Bessel-related and other special functions. The following code returns the whole list of Listable functions:

In Mathematica, one can use RandomVariate and its cousins directly, or perhaps a Table command. This generalizes to other more complex definitions inside the Table function, and one can use the fact that many arithmetic operations are Listable to avoid looping. (Yes, I know this could be written as RandomReal[{2, 3}, 100], but I wanted a simple example of vectorized arithmetic operations.)

3. If the i-th element of your list depends on the previous elements, you can construct the list using Nest, Fold, FixedPoint and their variants NestList, FoldList and FixedPointList

Another common use of For loops is when output i depends on output i-1. In Mathematica, this is a canonical use of Nest and Fold, and their NestList and FoldList counterparts that also return the intermediate results. For example, here is the For loop way to create autoregressive noise:

Just a sampling of things, some of which would require some extensive procedural solutions.

"For" my part (pun intended), I stopped thinking in loops and started thinking functionally. More so, I began to think more like a mathematician than a programer. The better I do this the more efficiently I program in Mathematica.

It would help if one could tie someone's "loop" or "procedural" arm to their body, like they do with boxers who need to switch their dominant hand.

Ultimately, I think one has to give themselves over to the functional language. Maybe one can translate one language to another but poetry in English doesn't have the same meaning translated into Italian nor Italian to English. When one can dream in the language they'll have it.

Maybe this doesn't count in this specific discussion because it demonstrates a specific type of solution available for use in Mathematica, which essentially hides how it gets the solution from the user. On the other side, I'd argue that the functional paradigm in Mathematica has made these kinds of solutions and functionality more readily available. The Wikipedia entry cited above shows how procedural approaches attack the problem. One could replace every procedural construct in that code with a functional one and produce your own functional Cholesky Decomposition pretty readily. But Mathematica has already done that for us.

Jagra, I think you should define sample list, matrix1, tensor1 objects so that this code can be evaluated. I know it's just an illustration, but to someone unfamiliar with what these do, it's not really a useful illustration without that.
–
Mr.Wizard♦Jul 5 '12 at 14:01

Jagra, I'm sorry that I am criticizing your answer a second time: it seems to me that the second half of your answer is not really appropriate here because any language can have library functions and this says little about the semantics of the language itself or good coding practice. Thanks for updating the first half however, as I think that showing that native arithmetic operations are vector, matrix, and tensor aware is very appropriate.
–
Mr.Wizard♦Jul 6 '12 at 6:46

Mathematica is a registered trademark of Wolfram Research, Inc. While the mark is used herein with the limited permission of Wolfram Research, Stack Exchange and this site disclaim all affiliation therewith.