Python Generators

In the previous article, you learned about Iterators in Python and in this article, you will learn about Python generators, how they are created for iterations and what is the use of generators in Python.

Python Generators: Introduction

Python generators are like normal functions which have yield statements instead of a return statement. Although functions and generators are both semantically and syntactically different.

Before jumping into creating Python generators, let’s see how a generator is different from a normal function.

Difference between Python Generators and Python Functions

Syntactically different

Though the structure is almost same for both Python generators and functions, both are syntactically different. Functions have return statement whereas generators have yield statements.

Python Function Syntax

def func_name():
#statements
return something

Python Generator Syntax

def generator_name():
#statements
yield something

Function execution paused and saved between multiple calls

In normal functions, whenever a function is called it executes the statements and the result is returned. When the function is called again from the same program, the function starts execution from the beginning.

In generator functions, whenever the function is called it executes the statements and yields the result and the function is paused in the same state.

When the function is called again, the function will resume from the same state as it was in the previous call.

Local variables and their current state stored

In generator functions, the current state of the local variables is stored in between function calls which make it possible for pausing the function execution and resuming from the same state while called again.

Multiple yield statements in the same function

In normal functions, there is only one return statement. We can only return a single value from a normal function or we have to return list or tuples for returning multiple values.

On the contrary, a Python generator function can have multiple yield statements which make it easy to return multiple values.
Here is an example to yield multiple fruit names from a single generator function.

def fruits():
yield ("Apple")
yield ("Mango")
yield ("Orange")

Memory efficiency

Normal functions require more memory as they operate on the whole sequence and produce all results at once, whereas in generator only one value is produced at a time. Hence, generator functions are more memory efficient.

How to create Python generators?

Creating Python generators is as simple as creating a function with yield statement instead of the return statement. Any function with yield statement instead of the return statement is termed as Python generator.

Python generators are the simpler alternative solution of iterators because while creating our own generators we don’t need to implement class necessarily and define __init__() and __next__() functions.

Example of Python generator

Let’s create a generator to iterate between food items.

def food_items():
yield ("Pizza")
yield ("Desert")
yield ("Nuggets")

Now that we have created generator function, let’s run this code in the interpreter.

As you can see, the interpreter returns obj as a generator object. A normal function would have returned ‘Pizza’. Now let’s go further and iterate through each item. Both iterators and generators have common function __next__() for this.

>>> next(obj)
Pizza
>>> next(obj)
Desert
>>> next(obj)
Nuggets

Note: When we access item using __next__(), it is normal to return the first item as it happens in normal functions as well, but when we again use __next__() function anywhere in the program, it will return the next object because generator functions stores the last state of function and resumes from there.

Since it produces one result at a time, it requires less memory than the normal functions.

Python Generators and For Loop

In the last article on Python iterators, we learned the underlying mechanism of for loop and how it actually works behind the scene.

In iterators, we implemented a class and used objects to iterate using for loop. But in Python generators, we can use for loop directly.

Recursive Generators in Python

To this point, you know about creating Python generators.

Now you must be wondering if there is any way to use recursion in Python generators?

Well, obviously there is. Prior to the release of Python 3.3 we had to use loops inside a function to achieve recursion. But in version 3.3 Python allowed using yield from statement making it easy to use recursion.

Here is an example to display odd numbers using recursion in Python generators.