python的迭代器和生成器

迭代是数据处理的基础，迭代可以理解为是一种惰性求值。在python里迭代器和生成器是一回事，使用的是yield关键字。

在语言内部，迭代器有下面的使用用途：

--for循环

--构建和扩展集合

--逐行遍历文本文件

--字典推导，列表推导，集合推导

--元组拆包

--调用函数时，使用*拆包

首先来看看iter函数：

我们来看看一个简单分词的类：

class Sentence():

def __init__(self, text):

self.word = text.split(' ')

def __getitem__(self, index):

return self.word[index]

s = Sentence('i\'m the king')

s

Out[3]: <__main__.Sentence at 0x1c8dba68588>

#在这里你可以重载__repr__使其返回你想要的字符串

for i in s:

print(i)

i'm

the

king

list(s)

Out[5]: ["i'm", 'the', 'king']

实际上是，这个调用了iter()函数，这个函数的作用呢？

--检查对象是否实现了__iter__方法，实现了就调用它，获取一个迭代器

--没有实现的话，但实现了__getitem__方法，python会创建一个迭代器，尝试按顺序获取元素

--都没有的话，会抛出"TypeError"，如下：

TypeError: 'Sentence' object is not iterable

可迭代对象和迭代器：

container.__iter__()

Return an iterator object. The object is required to support the iterator protocol described below. If a container supports different types of iteration,

additional methods can be provided to specifically request iterators for those iteration types.

从这个。可以看出python是从可迭代器对象中获取迭代器

从types的源码中，也有这么一段注释：

# Iterators in Python aren't a matter of type but of protocol. A large

# and changing number of builtin types implement *some* flavor of

# iterator. Don't check the type! Use hasattr to check for both

# "__iter__" and "__next__" attributes instead.

这个告诉我们最好使用isinstance(x, abc.Iterator)来检查是不是迭代器

迭代器对象：

This is implemented using two distinct methods(也就是__iter__,__next__); these are used to allow user-defined classes to support iteration. Sequences always support the iteration methods.

Once an iterator’s __next__() method raises StopIteration, it must continue to do so on subsequent calls. Implementations that do not obey this property are deemed broken.

要注意可迭代对象和迭代器的区别，可迭代对象有__iter__方法，每次都会实例化一个新的迭代器。迭代器要实现__next__方法，返回单个元素，__iter__返回迭代器本身。

生成器：

我们做一个简单的修改：

class Sentence():

def __init__(self, text):

self.word = text.split(' ')

def __iter__(self, index):

for i in self.word:

yield i

这里的迭代器，改成了生成器，因为在python里，只要函数的定义体有yield关键字，这就是生成器函数。

具体来看看行为：

def gen():

yield 1

yield 2

定义一个函数，看看它的行为：

gen()

Out[12]: <generator object gen at 0x000001C8DBAD36D0>

next(gen())

Out[13]: 1

g = gen()

next(g)

Out[16]: 1

next(g)

Out[17]: 2

next(g)

Traceback (most recent call last):

File "<ipython-input-18-5f315c5de15b>", line 1, in <module>

next(g)

StopIteration

通过next方法获取下一个元素，当获取结束后，再调用next会抛出StopIteration。

看看官方文档怎么描述的？

That generator then controls the execution of the generator function. The execution starts when one of the generator’s methods is called. At that time, the execution proceeds to the first yield expression, where it is suspended again, returning the value of expression_list to the generator’s caller. By suspended, we mean that all local state is retained, including the current bindings of local variables, the instruction pointer, the internal evaluation stack, and the state of any exception handling. When the execution is resumed by calling one of the generator’s methods, the function can proceed exactly as if the yield expression were just another external call. The value of the yield expression after resuming depends on the method which resumed the execution. If __next__() is used (typically via either a for or the next() builtin) then the result is None. Otherwise, if send() is used, then the result will be the value passed in to that method.

这就是所谓的惰性实现：尽可能的延后取值

在itertools,https://docs.python.org/3.4/library/itertools.html文档中可以看到很多相关元素。

那生成器和迭代器的关系：

--接口，python的迭代器协议定义了__next__，__iter__方法，而生成器实现了这两个方法。

--实现方式，生成器可以使用含有yield关键字的函数或者生成器表达式，属于GeneratorType类型，所有生成器都实现了迭代器。

--概念：迭代器用于遍历集合，从中产出元素。生成器可能无需遍历就能生成值，即使依附了集合，还可能会派生自元素的其他值。