I understand Big-O notation, but I don't know how to calculate it for many functions. In particular, I've been trying to figure out the computational complexity of the naive version of the Fibonacci sequence:

See the matrix form section here: en.wikipedia.org/wiki/Fibonacci_number . by doing this matrix ^ n (in a clever way) you can compute Fib(n) in O(lg n). The trick is in doing the power function. Theres a very good lecture on iTunesU about this exact problem and how to solve in O(lg n). The course is intro to algorithms from MIT lecture 3 (its absolutley free so check it out if you're interested)
–
AlyFeb 11 '11 at 16:50

9 Answers
9

You model the time function to calculate Fib(n) as sum of time to calculate Fib(n-1) plus the time to calculate Fib(n-2) plus the time to add them together (O(1)).

T(n<=1) = O(1)

T(n) = T(n-1) + T(n-2) + O(1)

You solve this recurrence relation (using generating functions, for instance) and you'll end up with the answer.

Alternatively, you can draw the recursion tree, which will have depth n and intuitively figure out that this function is asymptotically O(2n). You can then prove your conjecture by induction.

Base: n = 1 is obvious

Assume T(n-1) = O(2n-1), therefore

T(n) = T(n-1) + T(n-2) + O(1)which is equal to

T(n) = O(2n-1) + O(2n-2) + O(1) = O(2n)

However, as noted in a comment, this is not the tight bound. An interesting fact about this function is that the T(n) is asymptotically the same as the value of Fib(n) since both are defined as

f(n) = f(n-1) + f(n-2).

The leaves of the recursion tree will always return 1. The value of Fib(n) is sum of all values returned by the leaves in the recursion tree which is equal to the count of leaves. Since each leaf will take O(1) to compute, T(n) is equal to Fib(n) x O(1). Consequently, the tight bound for this function is the Fibonacci sequence itself (~θ(1.6n)). You can find out this tight bound by using generating functions as I'd mentioned above.

@Captain Segfault: Yeah. I clarified the answer. You'd get the tight bound using GF method as I had written above.
–
Mehrdad AfshariDec 11 '08 at 21:36

Itake your StackOverflowException as a joke. The exponential time is perceivable quite easily with rather small values for n.
–
David Rodríguez - dribeasDec 11 '08 at 23:08

@MehrdadAfshari can you explain why u take T(n-1) = O(2^n-1). T(n-1) should be (n^2), because Fibonacci have calls T(n-1)+T(n-2) so after summing all the cost (2*2*2....) should be 2^n.
–
DevJul 17 '14 at 6:22

There's a very nice discussion of this specific problem over at MIT. On page 5, they make the point that, if you assume that an addition takes one computational unit, the time required to compute Fib(N) is very closely related to the result of Fib(N).

As a result, you can skip directly to the very close approximation of the Fibonacci series:

Fib(N) = (1/sqrt(5)) * 1.618^(N+1) (approximately)

and say, therefore, that the worst case performance of the naive algorithm is

It is bounded on the lower end by 2^(n/2) and on the upper end by 2^n (as noted in other comments). And an interesting fact of that recursive implementation is that it has a tight asymptotic bound of Fib(n) itself. These facts can be summarized:

The proof answers are good, but I always have to do a few iterations by hand to really convince myself. So I drew out a small calling tree on my whiteboard, and started counting the nodes. I split my counts out into total nodes, leaf nodes, and interior nodes. Here's what I got:

What immediately leaps out is that the number of leaf nodes is fib(n). What took a few more iterations to notice is that the number of interior nodes is fib(n) - 1. Therefore the total number of nodes is 2 * fib(n) - 1.

Since you drop the coefficients when classifying computational complexity, the final answer is θ(fib(n)).

I agree with pgaur and rickerbh, recursive-fibonacci's complexity is O(2^n).

I came to the same conclusion by a rather simplistic but I believe still valid reasoning.

First, it's all about figuring out how many times recursive fibonacci function ( F() from now on ) gets called when calculating the Nth fibonacci number. If it gets called once per number in the sequence 0 to n, then we have O(n), if it gets called n times for each number, then we get O(n*n), or O(n^2), and so on.

So, when F() is called for a number n, the number of times F() is called for a given number between 0 and n-1 grows as we approach 0.

As a first impression, it seems to me that if we put it in a visual way, drawing a unit per time F() is called for a given number, wet get a sort of pyramid shape (that is, if we center units horizontally). Something like this:

It is also nice if you could explain your answer, rather than only posting a link.
–
MagnilexJan 11 at 8:36

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes.
–
JeepedJan 11 at 9:14

Well, according to me to it is O(2^n) as in this function only recursion is taking the considerable time (divide and conquer). We see that, the above function will continue in a tree until the leaves are approaches when we reach to the level F(n-(n-1)) i.e. F(1). So, here when we jot down the time complexity encountered at each depth of tree, the summation series is:

That doesn't explain how to calculate big O notation for the function.
–
Matt EllenApr 28 '10 at 20:43

@Matt Ellen: It's simple. This function is O(n), since the number of iterations of the while loop is equal to n, and there is no other loop or recursive call. (Proving 6*n + 4 is O(n) is trivial if you know the definition of the O notation.)
–
Eduardo LeónApr 29 '10 at 2:52

I agree that this can be a much better way of implementing it, but the notion of Big-O comments on the algorithm used for a problem. For instance, the expected runtime of bogosort (en.wikipedia.org/wiki/Bogosort) is O(2^n), but there are other ways to sort a list more efficiently (for instance selection sort). I think Juliet is asking about the order of the algorithm she provided, which she calls the "naive version."
–
userSep 30 '10 at 21:05

@starghost: Agreed. Since her algorithm just adds 1s, the order of her algorithm is O(Fib(n)) itself.
–
Eduardo LeónOct 1 '10 at 3:06