8 Answers
8

You'd want to check edge cases. How big a prime number is your method supposed to be able to handle? This will depend on what representation (type) you used. If you're only interested in small (really relative term when used in number theory) primes, you're probably using int or long. Test a handful of the biggest primes you can in the representation you've chosen. Make sure you check some non-prime numbers too. (These are much easier to verify independently.)

Naturally, you'll also want to test a few small numbers (primes and non-primes) and a few in the middle of the range. A handful of each should be plenty. Also make sure you throw an exception (or return an error code, whichever is your preference) for numbers that are out of range of your valid inputs.

why would you test edge cases first? What about all these numerous and useless checks for null and other stuff making together 80% of your test code, and then it is time to go home before you even wrote the "real test"? He wants the functino to work for "normal" numbers. How would you test edges?
–
badbadboyNov 20 '08 at 20:11

Let's say function works for 125 and crashes for 12234235345. Would you spend time making sure it throws ArgumentException instead of crashing? However, I do agree - it is nice to test edge cases if you are done with all the other stuff and feel it is useful.
–
badbadboyNov 20 '08 at 20:13

2

@badbadboy The reason for testing the edge cases is very often that's where the bugs lie - and the purpose of testing is to expose the bugs.
–
ChrisNNov 20 '08 at 20:15

Of course you test all that other stuff too. The question seemed to be specifically about what data to test.
–
Bill the LizardNov 20 '08 at 20:20

@ChrisN: Yes, this we all kinda know. Could you make an example of useful edge case testing here?
–
badbadboyNov 20 '08 at 20:21

What do you mean by this? test the base/zero case, the maximum case, and at least one median/middle
–
badbadboyNov 20 '08 at 20:19

@badbadboy: suppose your function takes an input value X with range 0..N; the base/zero case would be X=0, the maximum would be X=N, and the median would be X=N/2. I would also test -1 and N+1 to make sure they failed as expected
–
Steven A. LoweNov 20 '08 at 20:21

"Beware of bugs. I have proven the above algorithm correct, but have not tested it yet."

Some people don't understand the above quote (paraphrase?), but it makes perfect sense when you think about it. Tests will never prove an algorithm correct, they only help to indicate whether you've coded it right. Write tests for mistakes you expect might appear and for boundary conditions to achieve good code coverage. Don't just be picking values out of the blue to see if they work, because that might lead to lots of tests which all test exactly the same thing.

For your example, just hand-select a few primes and non-primes to test specific conditions in the implementation.

Ask yourself. What EXACTLY do I want to test. And test the most important. Test to make sure it basically does what you are expecting it to do in the expected cases.

Testing all those nulls and edge-cases - I - don't think is real, too time consuming and someone needs to maintain that later!
And...your test code should be simple enough so that you do not need to test your test code!

If you want to check that your function correctly applied the algorithm and works in general - probably will be enough some primes.

If you want prove that the method for finding primes is CORRECT - 100000 primes will not be enough :)
But you don't want to test the latter (probably)...

Only you know what you want to test! :)

PS I thnik using loops in unit tests is not always wrong but I would think twice before doing that. Test code should be VERY simple. What if something goes wrong and there is a bug in your test???? :)
However, you should try to avoid test code duplication as regular code duplication. Someone has to maintain test code.

I didn't vote you down, but I'd guess that people disagreed with what you said about it not being necessary to test edge cases. Those are exactly the cases that need the most testing. Other than that, I thought you had good points, esp. about bugs in the test.
–
ClaytonNov 20 '08 at 21:17

@Clayton - Ok, but I don't understand this "edge" cases...I mean...I know, this is what you read in your first unit testing book, but what is the "edge case" for this example? Can anyone tell me? Making sure that 2^32 works? Or that -3 doesn't? Okay, now tell me, what is 2^32?
–
badbadboyNov 20 '08 at 21:24

@Clayton - What do I know about 2^32 in my DOMAIN? If I know this is specific important case for me - I'll test it, but why calling it an edge test? If your program crashes because of bad user input - that's error in your GUI handling, not in your domain class. However, I agree that
–
badbadboyNov 20 '08 at 21:26

if you display an error message about bad user input - then it is important to throw the right exception...but in general it is more important to test how the program WORKS and not how it DOES NOT work...I think. Maybe I am missing point here...can ANYONE give an example of good edge case here?
–
badbadboyNov 20 '08 at 21:28

This isn't worth 2 downvotes. I agree - testing all edge cases may or may not be necessary (i.e., don't test a proven algorithm, just test the implementation). I vote up.
–
CybisNov 21 '08 at 1:04

Is the implementation of this code likely to be changed in the future? (if so, test more to support the future change)

Is the public contract of this code likely to change in the future? (if so, test less - to reduce the amount of throw-away test code)

How is the code coverage, are all branches visited?

Even if the code doesn't branch, are boundary considerations tested?

Do the tests run quickly?

Edit: Hmm, so to advise in your specific scenario. Since you started writing unit tests yesterday, you might not have the experience to decide among all these factors. Let me help you:

This code is probably not too important (no one dies, no one goes to war, no one is sued), so a smattering of tests will be fine.

The implementation probably won't change (prime number techniques are well known), so we don't need tests to support this. If the implementation does change, it's probably due to an observed failing value. That can be added as a new test at the time of change.

The public contract of this won't change.

Get 100% code coverage on this. There's no reason to write code that a test doesn't visit in this circumstance. You should be able to do this with a small number of tests.

If you care what the code does when zero is called, test that.

The small number of tests should run quickly. This will allow them to be run frequently (both by developers and by automation).

I would test 1, 2, 3, 21, 23, a "large" prime (5 digits), a "large" non-prime and 0 if you care what this does with 0.

Seriously though, for this kind of function you're probably using an established and proven algorithm. The main thing you need to do is verify that your code correctly implements the algorithm.

The other thing is to make sure you understand the limits of your number representation, whatever that is. At the very least, this will put an upper limit on the size the number you can test. E.g., if you use a 32-bit unsigned int, you're never going to be able to test values larger than 4G. Possibly your limit will be lower than that, depending on the details of your implementation.

Just as an example of something that could go wrong with an implementation:
A simple algorithm for testing primes is to try dividing the candidate by all known primes up to the square root of the candidate. The square root function will not necessarily give an exact result, so to be safe you should go a bit past that. How far past would depend on specifically how the square root function is implemented and how much it could be off.

Another note on testing: In addition to testing known primes to see if your function correctly identifies them as prime, also test known composite numbers to make sure you're not getting "false positives." To make sure you get that square root function thing right, pick some composite numbers that have a prime factor as close as possible to their square root.

Also, consider how you're going to "generate" your list of primes for testing. Can you trust that list to be correct? How were those numbers tested, and by whom?

You might consider coding two functions and testing them against each other. One could be a simple but slow algorithm that you can be more sure of having coded correctly, and the other a faster but more complex one that you really want to use in your app, but is more likely to have a coding mistake.

I'm guessing you didn't subscribe to test first, where you write the test before you write the code?
Not to worry, I'm sure you are not alone.
As has been said above, test the edges, great place to start. Also must test bad cases, if you only test what you know works, you can be sure that a bad case will happen in production at the worst time.