Functional Programming with Transducers in Python

Transducers – a portmanteau of ‘transform reducers’ – are a new functional programming concept, which were first introduced into the Clojure programming language. Although transducers are actually pretty straightforward in retrospect, wrapping your brain around them can be challenging.

Transducers allow us to fuse multiple data processing operations together so they can be applied in a single pass over a data series – similar to how Python generator functions allow us to process iterable series. However, unlike generator functions they are completely decoupled from the data series abstraction and can be used on eager sequences, lazy iterables, coroutine-based push events and events such as those modelled by Reactive Extensions for Python (RxPy).

In this workshop, we introduce transducers by implementing them from scratch in Python 3. We’ll start with the familiar staples of functional programming, map(), filter() and reduce(), and derive transducers from first principles. We’ll work towards a set of general tools which works with eager collections, lazy ‘pull’ sequences, and coroutine ‘push’ event streams and RxPy. Along the way we’ll cover stateful transducers and transducer composition, demonstrating that transducers are both more general, and more fundamental, than the functional programming tools baked into Python and many other languages.

There are no prerequisites for this workshop as we stick to core Python 3 language features although an enthusiasm for a functional programming style will be sure to help attendees get the most out of this session.