Testing with two failure seeking missiles: fuzzing and property based testing

Testing with purely random data on it’s own doesn’t get you very far. But two approaches that have been around for a while have new libraries that help you generate random input, that homes in on failing testcases.

First Hypothesis, a Python implementation and update of the Haskell library QuickCheck. Known as property based testing, you specify a property of your code that must hold, and Hypothesis does its best to find a counterexample. It then shrinks this to find the minimal input that contradicts your property.

Second, American fuzzy lop (AFL), is a young fuzzing library that’s already achieved an impressive trophy case of bug discoveries. Using instrumentation and genetic algorithms, it generates test input that carefully searches out as many code paths as it can find, seeking greater functional coverage and ultimately locating crashes and hangs that no other method has found. I’ll be showing how with Python-AFL we can apply this tool to our Python code.