I want to iterate a Python generator many times

The Python generator is useful, but once you use it in a loop in a for statement, and then try to use it in a for statement again, it looks like a carappo.

def i(n):
    yield n + 1
    yield n + 2


g = i(10)
print('first time:')
for n in g:
    print(n)

print('second time:')
for n in g:
    print(n)

The output is

first time:
11
12
second time:

It will be. No elements are iterated in the second for loop.

In situations where you want 11 and 12 to come back in the second and subsequent for loops, the following techniques worked.

class ReiteratableWrapper(object):
    def __init__(self, f):
        self._f = f

    def __iter__(self):
        return self._f()

def i(n):
    yield n + 1
    yield n + 2

import functools
f = functools.partial(i, 10)

g2 = ReiteratableWrapper(f)
for n in g2:
    print(n)

for n in g2:
    print(n)

Commentary

The ReiteratableWrapper class takes one generator function. Since it is called without giving an argument internally, if you want to make a generator call with an argument, create a function without arguments by using the functools.partial function etc. as in the example.

The expression evaluated in the context after the in of the for statement is called the __iter__ method. Therefore, an instance of ReiteratableWrapper will automatically create a new generator every time.

However, each time the __iter__ method is evaluated, the generator function is evaluated and executed, so if there are side effects in the generator function call such as the number of times the generator function can be executed is limited, it cannot be used over and over again. I think there is something.

I think the technique introduced this time is effective for content that can be repeated, such as reading from a file in order to save memory, issuing a query to the DB each time and iterating in order.

Reference link

Recommended Posts

I want to iterate a Python generator many times
I want to build a Python environment
I want to create a window in Python
I want to make a game with Python
I want to write to a file with Python
I want a mox generator
I want a mox generator (2)
I want to embed a variable in a Python string
I want to easily implement a timeout in python
I want to generate a UUID quickly (memorandum) ~ Python ~
I want to write in Python! (2) Let's write a test
I want to randomly sample a file in Python
I want to work with a robot in python.
[Python] I want to make a nested list a tuple
I want to run a quantum computer with Python
I want to debug with Python
[Python] I want to get a common set between numpy
I want to start a lot of processes from python
I want to send a message from Python to LINE Bot
I want to make input () a nice complement in python
I want to use a wildcard that I want to shell with Python remove
I want to print in a comprehension
I want to use jar from python
I want to do a full text search with elasticsearch + python
I want to analyze logs with Python
I want to play with aws with python
[Introduction] I want to make a Mastodon Bot with Python! 【Beginners】
Python: I tried to make a flat / flat_map just right with a generator
I made a password generator to teach Python3 to children (bonus) * Completely remade
[Python memo] I want to get a 2-digit hexadecimal number from a decimal number
I want to convert a table converted to PDF in Python back to CSV
I want to color a part of an Excel string in Python
Python: I want to measure the processing time of a function neatly
I want to make a web application using React and Python flask
I want to do a monkey patch only partially safely in Python
I want to make matplotlib a dark theme
I want to do Dunnett's test in Python
I want to easily create a Noise Model
I want to use MATLAB feval with python
I want to INSERT a DataFrame into MSSQL
I want to memoize including Python keyword arguments
I want to email from Gmail using Python.
[Python] I want to manage 7DaysToDie from Discord! 1/3
I don't want to take a coding test
I want to merge nested dicts in Python
I want to use Temporary Directory with Python2
I want to use ceres solver from python
#Unresolved I want to compile gobject-introspection with Python3
I want to create a plug-in type implementation
I want to solve APG4b with Python (Chapter 2)
I want to sell Mercari by scraping python
[Python] I want to manage 7DaysToDie from Discord! 2/3
I want to easily find a delicious restaurant
I want to make C ++ code from Python code!
I want to display the progress in Python!
I want to upload a Django app to heroku
I want to create a nice Python development environment for my new Mac
I just want to add scipy, but it's a messy note. Ubuntu, Python 3.
I want to create a priority queue that can be updated in Python (2.7)
I tried to explain what a Python generator is for as easily as possible.
I want to exe and distribute a program that resizes images Python3 + pyinstaller