Generator implements the interface of iterator. Iterator pulls items from collections. So iterator can not be create from an infinite series but generator could create an infinite series of items.

Write a iterator class

The main purpose for generator and iterator are to lazily load a very large collection that can’t fit into memory.

When se call functions like ‘list’, it automatically calls iter() function. So if the object didn’t implement __iter__ method, error will be raised.

A standard way to implement an interator is:

class Some:
    def __iter__(self):
        return SomeIterator()
class SomeIterator:
    def __iter__(self):
        return self
    def __next__(self):
        '''iteration detail. include StopIteration exceptions'''

yield

In the above code, ‘Some’ is a iterable class. ‘SomeIterator’ is a iterator class. We shouldn’t mix the two and we should, as a good practice, seperate them like the example so that the internal state of the interable is refreshed every time we called the iter function.

The above example however is not pythonic. Generator should be used.

class SomeGenerator:
    def __iter__(self):
        for each in something:
            yield each

This is much shorter and pythonic. The generator function does not raise StopIteration error when exhausted. When ‘yield’ is used, the function will return a generator object. When body of this function completes, the generator object raises the error.

The body of the a generator function will pause at ‘yield’ and resumes in the next call or iteration.

For a class to be lazy. It should use generator throughout the implementation, i.e. from init to all other methods. Otherwise this effort is in vain.

Like list comprehension, generator expression is embraced by ‘()’.
List comprehension will eagarly run the function.
Generator expression otherwise will lazily run the function. Generators are single-iterator objects. Instantized same generators point to the same memory location.
some generators function in standart libary: Iteratools, map, os.walk, etc.

yield from

caller calls the delegent generator. delegent generator contains the subgenerator, which contains the yield from statement.

coroutine

When we see python i = yield x, it is coroutine.

def gen():
    for i in range(10):
        x = yield i

x.send(10)

we use ‘send’ method to send in data and the generator will iterate to the next state.
the coroutine suspends exactly at yield keyword.

there are three methods for coroutine:

  1. throw. Throw an exception to the generator. such as ‘DemoException’. If exception is handled, go to next yield. If not, raise error.
  2. send. send value into the generator
  3. close. close the generator.

Asyncio I/O

Since python 3.4 coroutine has drew much attention to Guido. It’s been evolved into something totally new. I didn’t understand why ‘Fluent Python’ book mentioned this is a huge topic. But after digging a litter deeper I understand why. I won’t records any details here. There are already a whole bunch of great tutorials, Linkes are here: This tutorial explains the differences between multiprocessing, multithreading and asycio IO with focus on asyncio io. It also has a project that I could practice later. The other tutorial explains the whole story of asyncio IO. And this.

Concisely spearking, Asyncio I/O uses sytax async/await combo to define asyncio functions. And in the main function, which is also async, create a event loop by async module that manages the program in a asynchronous way. This would be extremely useful to deal with programming faced to websites or other communication with ‘slow’ devices.