Chapter 4 · Objects and robust code
Iterators and generators
Step 1 of 20 · Concept
for asks for the next item
By the end of this lesson, you can write functions that make values one at a time, even without end.
names = ["Ada", "Alan"]
for name in names:
print(name)
- A
forloop callsiter(names)once. It gives an iterator: an object that hands out items one at a time. - Each round calls
next(it)for the next item. - When no item is left,
nextraisesStopIteration, andforstops quietly.
Step 2 of 20 · Quiz
What does the third next do?
The list has two names. What happens at the third next(it)?
names = ["Ada", "Alan"]
it = iter(names)
print(next(it))
print(next(it))
print(next(it))
Step 3 of 20 · Demo
See it run
Run it. Two names print, then the traceback's last line is StopIteration.
Step 4 of 20 · Concept
An iterator runs once
squares = (n * n for n in [1, 2, 3])
print(list(squares))
print(list(squares))
This prints [1, 4, 9], then [].
- Lesson 13's loop without square brackets, inside
sum(...), is a generator expression. In round brackets of its own,(n * n for n in [1, 2, 3]), you can name it. - A generator expression is an iterator. Once it has handed out every item, it is empty.
- A list is not an iterator: each loop gets a new iterator from it, so a list loops again.
zip(...)andenumerate(...)give iterators too. Loop over one once, or make a list of it.
Step 5 of 20 · Quiz
Predict both lists
Type exactly what this program prints.
nums = [1, 2, 3]
doubled = (n * 2 for n in nums)
print(list(doubled))
print(list(doubled))
print(list(nums), list(nums))
Step 6 of 20 · Demo
See it run
Run it and compare with your answer.
Step 7 of 20 · Concept
yield makes a generator
def countdown(n):
while n > 0:
yield n
n -= 1
gen = countdown(2)
print(next(gen))
- A function with
yieldis a generator function. Calling it runs none of its lines yet. It returns a generator, which is an iterator. - Each
next(gen)runs the body up to ayield, which hands out a value and pauses there. - When the body ends,
nextraisesStopIteration, so aforloop over it stops.
Step 8 of 20 · Trace
A frame that pauses
A call's frame is usually gone when it returns (lesson 7). Press Next and watch the generator's frame.
def countdown(n):
while n > 0:
yield n
n -= 1
gen = countdown(1)
print(next(gen))
print(list(gen))- Line 6 made a generator. Its frame holds
n, 1, but no line of it has run. Line 7 is next. next(gen)put the frame on the stack.n > 0, so line 3,yield n, is next.yieldhanded out 1, which line 7 printed. The frame is paused at line 3, keepingn. Line 8 is next.- Line 8's
list(gen)resumed the frame where it paused. Line 4 is next. - Line 4 made
n0. Line 2 asksn > 0again. - It is not, so the loop ends and the generator is finished: its frame is gone. Back on line 8,
listgot no item. - Line 8 printed
[]. The program is finished.
Step 9 of 20 · Quiz
Where is n kept?
Line 7 has printed 1, and line 8 is next. Where is n kept?
def countdown(n):
while n > 0:
yield n
n -= 1
gen = countdown(1)
print(next(gen))
print(list(gen))Step 10 of 20 · Demo
A generator in a for loop
Run it. The for loop calls next for you, and stops at StopIteration.
Step 11 of 20 · Exercise
Your turn: evens up to n
Make evens(n) a generator that yields the even numbers from 0 up to n, n included. The program must print:
[0, 2, 4, 6, 8, 10]
12
countdown is the model: a loop in the function, and yield for each value to hand out. No return is needed.
Loop over range(n + 1), so that n is included. Yield each number whose remainder after dividing by 2 is 0.
if i % 2 == 0:
Step 12 of 20 · Concept
Lazy: values only when asked
def naturals():
n = 1
while True:
yield n
n += 1
- A generator is lazy: it makes a value only when
nextasks. So it can be endless, asnaturalsis. - Stop the loop over it with
break, or take the first few withitertools.islice(naturals(), 5). range(10 ** 12)is lazy too: it makes no list of a million million numbers. But it is not an iterator, so it can be looped again.
Step 13 of 20 · Demo
Endless, but only ten
Run it. fib() never ends, and islice takes ten values. range counts its numbers without making them.
Step 14 of 20 · Exercise
Your turn: the first squares over 50
- Write
squares(), an endless generator of 1, 4, 9, 16, and so on. - Loop over it. Print each square over 50, and stop after five.
The program must print:
64
81
100
121
144
naturals in Lazy: values only when asked is the shape of squares. A loop over it never ends by itself, so
count what you print.
Start a counter at 0 before the loop. When a square is over 50, print it and add 1 to the counter. At 5, break.
yield n * n
Step 15 of 20 · Concept
__iter__ makes your class loopable
class Playlist:
def __init__(self, name):
self.name = name
self.titles = []
def add_title(self, title):
self.titles.append(title)
def __iter__(self):
for title in self.titles:
yield title
for song in p:callsiter(p), which calls the special methodp.__iter__()(lesson 16).- A generator is the easiest
__iter__: it yields each item. - Each loop calls
__iter__again and gets a new generator, so the playlist loops again.
This Playlist keeps titles, not lesson 16's Song objects.
Step 16 of 20 · Quiz
Predict the loop
Type exactly what this program prints.
class Playlist:
def __init__(self, name):
self.name = name
self.titles = []
def add_title(self, title):
self.titles.append(title)
def __iter__(self):
for title in self.titles:
yield title
p = Playlist("Road trip")
p.add_title("Blue")
p.add_title("Gold")
for song in p:
print("Playing", song)
print(list(p))Step 17 of 20 · Demo
See it run
Run it and compare with your answer.
Step 18 of 20 · Check yourself
Check yourself: lines that matter
No shape this time.
- Write a generator
useful(lines). It yields each line without the spaces around it. - It skips blank lines and lines that start with
#. - Print the useful lines numbered from 1, with
enumerate.
The program must print:
1. eggs
2. milk
3. bread
The generator has one loop over the lines and one yield. The numbering is not its job: enumerate around the
generator does it.
Step 19 of 20 · Challenge · optional
Challenge: yield from
Optional.
walk should yield every number of a nested list, however deep. The program must print:
[1, 2, 3, 4, 5]
An item that is a list must be walked too: recursion, as lesson 14's nested sum did, with isinstance(x, list).
walk(x) returns a generator, not its values. yield from a generator hands out each value it yields, one by
one.
yield from walk(x)
Step 20 of 20 · Recap
Recap
forcallsiter()once andnext()each round, untilStopIteration.- An iterator runs once; a list or a
rangecan be looped again. yieldmakes a generator. Its frame pauses betweennext()calls, keeping its names.- Generators are lazy, so one can be endless: stop it with
breakoritertools.islice. __iter__makes your class work in aforloop.
Next: lesson 19 writes records, choices and decisions with less code.
Review
4 questions from this lesson, in a new order each time. Answer from memory: 3 of 4 right on the first try passes. On a day after you finish the lesson, a pass earns 15 XP.
The review needs JavaScript.
names = ["Ada", "Alan"]
it = iter(names)
print(next(it))
print(next(it))
print(next(it))
nums = [1, 2, 3]
doubled = (n * 2 for n in nums)
print(list(doubled))
print(list(doubled))
print(list(nums), list(nums))
def countdown(n):
while n > 0:
yield n
n -= 1
for n in countdown(3):
print(n)
print("Liftoff")
def evens(n):
# yield 0, 2, 4, ... up to n
return []
print(list(evens(10)))
print(sum(evens(6)))
import itertools
def fib():
a, b = 0, 1
while True:
yield a
a, b = b, a + b
print(list(itertools.islice(fib(), 10)))
print(len(range(10 ** 12)))
r = range(3)
print(list(r), list(r))
# 1. squares(): yield 1, 4, 9, ... for ever
# 2. print the first five squares over 50
class Playlist:
def __init__(self, name):
self.name = name
self.titles = []
def add_title(self, title):
self.titles.append(title)
def __iter__(self):
for title in self.titles:
yield title
p = Playlist("Road trip")
p.add_title("Blue")
p.add_title("Gold")
for song in p:
print("Playing", song)
print(list(p))
lines = ["# shopping", "eggs", "",
" milk ", "# done", "bread"]
# 1. useful(lines): the lines that matter
# 2. print them numbered from 1
def walk(items):
for x in items:
yield x
print(list(walk([1, [2, 3], [4, [5]]])))