5 Answers
5

A more efficient use of Select

If the likely bound of the problem is easily stored in memory it is practical to generate a Range, which is fast, and then Select from that. Since the Range will be unpacked by Select you must consider the unpacked size. For example:

ByteCount @ Developer`FromPackedArray @ Range@1*^7

240 000 032

This is a reasonable starting size in many cases.

Finding five values take a quarter of a second:

Select[Range@1*^7, PrimeQ[#^6 + 1091] &, 5] // Length // Timing

{0.249, 5}

Fining all values with a seed <= 10^7:

Select[Range@1*^7, PrimeQ[#^6 + 1091] &] // Length // Timing

{19.797, 3338}

You can see that this scales pretty well, without excessive overhead in the case that values are found quickly, yet for your search 3338 results can be found before the limit is reached.

(The seeds found with this method need to be converted to primes with #^6 + 1091 &.)

A method for larger problems

If this will not work, either because you have no idea what the likely upper bound is, or it is too high to hold a Range in memory, then it will be most efficient to operate in blocks, due to Mathematica's vector optimizations.

First, there is a more efficient way to build the list of candidates:

Range[100]^6 + 1091

This takes full advantage of the vectorized operations available.

Pick a large enough block size that the average element processing time is relatively low, but not so large as to process more elements than are likely needed. I will pick a block size of 100,000 and I will try to find 5,000 solutions:

The results are stored in a linked list rather than using Sow. This has the advantage of letting you open a sub-session and examine the results up to that point. For example, in a separate cell enter:

Flatten @ result // Short

And use F7 to show the results, then resume calculation.

The Monitor lets you see how many blocks have been processed.

Timings

I made the claim that these methods are efficient. Let me give some comparative timings to support my position. WReach's lazy lists code which Rojo used in his answer is a wonderful approach. It is not however, as written, fast. My methods are by comparison clunky but they are also more practical.

My Select method shows the overhead of generating and unpacking the Range but it soon catches up and exceeds the lazy lists method in performance. Remember also the human overhead of writing this if it is not going to be used many times. The Select method is very simple and direct.

Now, for my second method there is a tuning parameter: the block size. It could be argued that changing this parameter mid-test is not fair play so I will use a fixed block size of 1000.

+1 for the useful tutorial! May I ask why you are using nested lists in result= {result, #} instead of AppendTo? Is it to keep track of how far you've gone by looking at the depth of result?
–
gpapMar 18 '13 at 12:37

@gpap Linked lists are faster than AppendTo, yet provide the option of viewing intermediate results (unlike Sow). See the bottom of this page for a reference. Faster is Sow/Reap. The ultimate solution is Internal`Bag and friends, which is slightly faster than Sow/Reap and yet makes it possible to view results, but I didn't think that level of optimization was needed here.
–
Mr.Wizard♦Mar 18 '13 at 12:46

thanks for that - favorited a couple for further reading.
–
gpapMar 18 '13 at 12:59

Splitting it into blocks like that is clever, and, I think, very readable, so I am selecting this as the answer.
–
nickjamesukMar 18 '13 at 17:11

@nick Thank you, I hope this method serves you well. If you are handling a very large problem you'll want to save results as you go, like this.
–
Mr.Wizard♦Mar 19 '13 at 0:54

Ah, I missed that. Also, just a reminder, please add extra space around ~infix~ operators, especially when they are not built-in functions. This greatly improves the legibility of the code.
–
Mr.Wizard♦Mar 18 '13 at 13:21

You can also break out of inbuilt functions, though in that case I suggest using Do instead of Table (no use of building a table if it will be aborted before completion), and sowing found solutions on the fly with Sow & Reap:

Thanks István! I was looking for a more "Mathematica" way though. Using procedural approaches tends to slow things down a lot ie. This takes 3 times as long as the initial solution. Mathematica clearly has faster inbuilt code, it's just a case of getting it to stop.
–
nickjamesukMar 18 '13 at 11:19

1

Well, your problem specifically ask for a procedural break when condition is met. If you check timings, you will see that it depends on your range of scope (10000), and thus While is indeed faster.
–
István ZacharMar 18 '13 at 11:21

Yep. You're right. Is there a way to get the inbuit algorithms to stop, though?
–
nickjamesukMar 18 '13 at 11:22

@nickjamesuk since you have a PrimeQ in the loop, it's very unlikely that an extra, slow addition will be more expensive than that. Using Do or While or whatever should not make a difference, speed-wise.
–
aclMar 18 '13 at 11:23

I like the Sow and Reap thing, I have always wanted to work out how they are used. Thanks so much for all your work.
–
nickjamesukMar 18 '13 at 11:30

The pure function prime is easier to read if you change the middle argument of NestWhile i.e.

NestWhile[# + 1 &, 2, ! PrimeQ@((# + 1)^6 + 1091) &]

looks for the number (after 2) for which $ \text{number}^6 + 1091 $ is prime. So, turning this into a pure function, you can then apply it repeatedly for every number it spits out using NestList.

For some reason NestList produces its zeroth evaluation (the number 1) and I want to get rid of that from the list using Drop[ListofNumbersINeed,1] which in Infix notation is ListofNumbersINeed~Drop~1. (I learnt that from @MrWizard and it sometimes makes code easier to understand - not the case here:) ).

I then apply the pure function ((# + 1)^6 + 1091) & to the list to get the actual primes. There is bound to be a better way to do the whole thing in one go or with much more obscure notation or in a hundredth of the speed - give it a couple of hours/days and you'll see what I mean.

I think my Mathematica knowledge is not yet at the stage where I can parse that. What does the ~Drop~ do at the end?
–
nickjamesukMar 18 '13 at 11:33

I edited to make it a little clearer - NestWhile, NestList, Function and Infix are the relevant pages.
–
gpapMar 18 '13 at 11:54

Ah, so the ~Drop~ is Infix notation. Thanks very much for the clarification.
–
nickjamesukMar 18 '13 at 14:06

@nickjamesuk I'm largely the one to blame for the proliferation of that syntax around here. It apparently trips up a lot of people upon first seeing it, but after extensive use I find that it is often easier to read than the nested-bracket nightmares that can be the alternative. If you use a string of infix, as shown in Rojo's answer, it is very important IMHO to add space around each operator to distinguish these from arguments. Once this is done however code reads left to right in a logical fashion, e.g. a ~p~ b ~q~ c ~r~ d, which is equivalent to r[q[p[a, b], c], d].
–
Mr.Wizard♦Mar 18 '13 at 14:35

That doesn make sense. I'll be looking out for ways to utilise it.
–
nickjamesukMar 18 '13 at 17:06

The "vanishing function" ##&[] is used to silently remove any steps that don't yield a solution, and the All parameter passes all found solutions to the length check function, which stops the loop when n solutions are found.

Mathematica is a registered trademark of Wolfram Research, Inc. While the mark is used herein with the limited permission of Wolfram Research, Stack Exchange and this site disclaim all affiliation therewith.