Python Generators

In the previous article, you learned about Iterators in Python and in this article, you will learn about Python generators, how they are created for iterations and what is the use of generators in Python.

Python Generators: Introduction

Generator in Python is a simple way of creating an iterator.

Python generators are like normal functions which have yield statements instead of a return statement. Although functions and generators are both semantically and syntactically different.

Before jumping into creating Python generators, let’s see how a generator is different from a normal function.

Difference between Python Generators and Python Functions

• Syntactically different
  

  ---

  Though the structure is almost same for both Python generators and functions, both are syntactically different. Functions have return statement whereas generators have yield statements.

  Python Function Syntax

  def func_name():
     #statements
     return something
  

  Python Generator Syntax

  def generator_name():
     #statements
     yield something

• Function execution paused and saved between multiple calls
  

  ---

  In normal functions, whenever a function is called it executes the statements and the result is returned. When the function is called again from the same program, the function starts execution from the beginning.

  In generator functions, whenever the function is called it executes the statements and yields the result and the function is paused in the same state.

  When the function is called again, the function will resume from the same state as it was in the previous call.

• Local variables and their current state stored
  

  ---

  In generator functions, the current state of the local variables is stored in between function calls which make it possible for pausing the function execution and resuming from the same state while called again.

• Multiple yield statements in the same function
  

  ---

  In normal functions, there is only one return statement. We can only return a single value from a normal function or we have to return list or tuples for returning multiple values.

  On the contrary, a Python generator function can have multiple yield statements which make it easy to return multiple values.
  Here is an example to yield multiple fruit names from a single generator function.

  def fruits():
     yield ("Apple") 
     yield ("Mango")
     yield ("Orange")

• Memory efficiency
  

  ---

  Normal functions require more memory as they operate on the whole sequence and produce all results at once, whereas in generator only one value is produced at a time. Hence, generator functions are more memory efficient.

How to create Python generators?

Creating Python generators is as simple as creating a function with yield statement instead of the return statement. Any function with yield statement instead of the return statement is termed as Python generator.

Python generators are the simpler alternative solution of iterators because while creating our own generators we don’t need to implement class necessarily and define __init__() and __next__() functions.

Example of Python generator

Let’s create a generator to iterate between food items.

def food_items():
   yield ("Pizza")
   yield ("Desert")
   yield ("Nuggets")

Now that we have created generator function, let’s run this code in the interpreter.

>>> obj = food_items()
>>> obj
<generator object food_items at 0x03A69180>

[adsense1]

As you can see, the interpreter returns obj as a generator object. A normal function would have returned ‘Pizza’. Now let’s go further and iterate through each item. Both iterators and generators have common function __next__() for this.

>>> next(obj)
Pizza
>>> next(obj)
Desert
>>> next(obj)
Nuggets

Note: When we access item using __next__(), it is normal to return the first item as it happens in normal functions as well, but when we again use __next__() function anywhere in the program, it will return the next object because generator functions stores the last state of function and resumes from there.

Since it produces one result at a time, it requires less memory than the normal functions.

Python Generators and For Loop

In the last article on Python iterators, we learned the underlying mechanism of for loop and how it actually works behind the scene.

In iterators, we implemented a class and used objects to iterate using for loop. But in Python generators, we can use for loop directly.

Here is how we can implement for loop directly in Python.

def food_items():
   yield ("Pizza")
   yield ("Desert")
   yield ("Nuggets")

#using for loop
for items in food_items():
  print (items)

Output

This will generate following output.

Pizza
Desert
Nuggets

Recursive Generators in Python

To this point, you know about creating Python generators.

Now you must be wondering if there is any way to use recursion in Python generators?

Well, obviously there is. Prior to the release of Python 3.3 we had to use loops inside a function to achieve recursion. But in version 3.3 Python allowed using yield from statement making it easy to use recursion.

Here is an example to display odd numbers using recursion in Python generators.

Example of Recursive Generators in Python

#using recursion in generator function
def oddnum(start):
   yield start
   yield from oddnum(start+2)

#using for loop to print odd numbers till 10 from 1
for nums in oddnum(1):
   if nums<20:
     print (nums)
   else:
     break

Output

1
3
5
7
9

PythonGenerator Expressions

Generator expression is the memory efficient generalization of list comprehensions with optimized performance.

With generators, we can evaluate the elements on demand. Though they don’t share the full power of generators, simple generators can be created on a fly using generator expressions.

Here is an example of generator expression to build a simple generator.

exp = (x ** 3 for x in range(5)) 
for i in exp:
 print(i)

As you can see, we can create a simple generator in a line to display first five perfect cube numbers including zero.

They are same like list comprehensions except the fact parenthesis is used in generator expression instead of square brackets.

Output

0
1
8
27
64

Generator expression produce one result at a time, making it memory efficient whereas list comprehensions generate a whole list.