Python – Iterators

Post contents:
– What is an iterator?
– Examples
– Pros/Cons
– Cases

What is an iterator?

https://docs.python.org/2/glossary.html#term-iterator defines an iterator as

An object representing a stream of data. Repeated calls to the iterator’s next() method return successive items in the stream. When no more data are available a StopIteration exception is raised instead. At this point, the iterator object is exhausted and any further calls to its next() method just raise StopIteration again. Iterators are required to have an __iter__() method that returns the iterator object itself so every iterator is also iterable and may be used in most places where other iterables are accepted. One notable exception is code which attempts multiple iteration passes. A container object (such as a list) produces a fresh new iterator each time you pass it to the iter() function or use it in a for loop. Attempting this with an iterator will just return the same exhausted iterator object used in the previous iteration pass, making it appear like an empty container.

More information can be found in Iterator Types.

Additionally…
An iterator, as defined by https://docs.python.org/2/howto/functional.html,

is an object representing a stream of data; this object returns the data one element at a time.

It also defines that

A Python iterator must support a method called next() that takes no arguments and always returns the next element of the stream. If there are no more elements in the stream, next() must raise the StopIteration exception. Iterators don’t have to be finite, though; it’s perfectly reasonable to write an iterator that produces an infinite stream of data.

The built-in iter() function takes an arbitrary object and tries to return an iterator that will return the object’s contents or elements, raising TypeError if the object doesn’t support iteration. Several of Python’s built-in data types support iteration, the most common being lists and dictionaries. An object is called an iterable object if you can get an iterator for it.

Supporters of iter()…

  • list()
  • tuple()
  • dict()
  • list comprehension
  • functions such as: max, min, sumIMPORTANT NOTE:
    There are many data types, functions, and objects that support iter(); however remember that a good portion of these supporters have finite iterators limitations.

Examples:

– iteration interface manually (from python site):

>>> L = [1,2,3]
>>> it = iter(L)
>>> print it

>>> it.next()
1
>>> it.next()
2
>>> it.next()
3
>>> it.next()
Traceback (most recent call last):
File “”, line 1, in ?
StopIteration
>>>


– Python expects iterable objects in several different contexts, the most important being the for statement. In the statement for X in Y, Y must be an iterator or some object for which iter() can create an iterator. These two statements are equivalent:

for i in iter(obj):
    print i

for i in obj:
    print i


– Iterators can be materialized as lists or tuples by using the list() or tuple() constructor functions:

>>> L = [1,2,3]
>>> iterator = iter(L)
>>> t = tuple(iterator)
>>> t
(1, 2, 3)


– The dict() constructor can accept an iterator that returns a finite stream of (key, value) tuples:

>>> L = [('Italy', 'Rome'), ('France', 'Paris'), ('US', 'Washington DC')]
>>> dict(iter(L))
{'Italy': 'Rome', 'US': 'Washington DC', 'France': 'Paris'}


– Files also support iteration by calling the readline() method until there are no more lines in the file. This means you can read each line of a file like this:

for line in file:
    # do something for each line
...


– Sets can take their contents from an iterable and let you iterate over the set’s elements:

S = set((2, 3, 5, 7, 11, 13))
for i in S:
    print i


– List comprehensions and generator expressions (short form: “listcomps” and “genexps”) are a concise notation for such operations, borrowed from the functional programming language Haskell (http://www.haskell.org/). You can strip all the whitespace from a stream of strings with the following code:

line_list = [' line 1\n', 'line 2 \n', ...]

# Generator expression — returns iterator
stripped_iter = (line.strip() for line in line_list)

# List comprehension — returns list
stripped_list = [line.strip() for line in line_list]

# You can select only certain elements by adding an “if” condition:
stripped_list = [line.strip() for line in line_list if line != “”]

Pros / Cons

Pros:

  • allow for iterating over, and processing of, data that would otherwise not be manageable because of resource limitations. Each iteration utilizes resources for just that instance
  • iter() can be used with many other supporting resources to make life much easier

Cons:

  • infinite iterators can not use the above mentioned supporting resources out of the box

Cases

Basic, but complete iter() example (with 2x/3x difference):

it = iter(sequence)
while True:
    try:
        value = it.next() # in Python 2.x
        value = next(it) # in Python 3.x
    except StopIteration:
        break
    it = iter(it)
    print(value)

.
.
.

Advertisements

One comment

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s