The vanishing pattern: from iterators to generators in Python

Luciano Ramalho (Python.pro.br)
Python
Location: D136
Average rating: ****.
(4.33, 12 ratings)

Draft slides available for review at: Speakerdeck

The core of the talk is refactoring a simple iterable class from the classic Iterator design pattern (as implemented in the GoF book) to compatible but less verbose implementations using generators. This provides a meaningful context to understand the value of generators. Along the way the behavior of the iter function, the Sequence protocol and the Iterable interface are presented.

Iteration provides a theme to explore core ideas in the design of Python: the data model, special methods, classic protocols, the new Collections ABC, and why it makes sense that len, reversed and iter are not methods, but built-in functions. Each of these ideas is briefly discussed in the talk, to provide insight and encourage deeper exploration of the language by the attendees.

Talk Outline (40’)

Topics will be presented roughly in the order shown below. Section times are given for 30’-45’ versions.

Motivation (3’)

  • iterating over a file, a Django QuerySet and a simple user-defined class
  • iterating to build: list, dict and set comprehensions
  • syntatic support: tuple unpacking and exploding function arguments

Implementation (10’)

  • what is a Python iterable really: the role of the iter function
  • special methods and the Python object model
  • the legacy sequence protocol
  • the Iterable interface, as defined in the Collections ABC
  • implementing the Iterator design pattern: the way of the GoF

Generators (15’)

  • motivating example: enumerate, the “missing piece” of the for loop
  • the generator concept: more general than an iterator
  • building a generator function
  • the behaviour of the simplest generator function: one yield, no loop
  • refactoring the Iterator design pattern with a generator method
  • genexps: generator expressions
  • refactoring the Iterator design pattern by returning a genexp
  • conclusion: generators preserve an iteration context

Library support (4’)

  • built-in iterable types
  • built-in functions that consume or produce iterables
  • the itertools functions and recipes

Case study (3’)

  • using generator functions to decouple reading and writing logic in a database conversion utility

Q&A (5’)

Photo of Luciano Ramalho

Luciano Ramalho

Python.pro.br

Luciano Ramalho was a Web developer before the Netscape IPO in 1995, and switched from Perl to Java to Python in 1998. Since then he worked on some of the largest news portals in Brazil using Python, and taught Python web development in the Brazilian media, banking and government sectors. His speaking credentials include OSCON 2002, and 15 talks over the years at PythonBrasil (the Brazilian PyCon) and FISL (the largest FLOSS conference in the Southern Hemisphere). In 2012 he was a keynote speaker at the RuPy Strongly Dynamic Conference in São José dos Campos, Brazil, and had 2 talks accepted for PyCon USA (to be presented March, 2013). Ramalho is a member of the Python Software Foundation and co-founder of Garoa Hacker Clube, the first hackerspace in Brazil. He is the owner of Oficinas Turing, a training company.

Sponsors

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities at the conference, contact Sharon Cordesse at (707) 827-7065 or scordesse@oreilly.com.

Contact Us

View a complete list of OSCON contacts