How to break Python

Published: November 28, 2016. Filed under: Django, Python.

Edited December 2018: added sections for Python 3.7 and upcoming 3.8, updated Python 3.6 section since Python 3.6 has been released, and updated Python 3.3 section since 3.3 has reached end-of-life.

Don’t worry, this isn’t another piece about Python 3. I’m fully in favor of Python 3, and on record as to why. And if you’re still not convinced, I suggest this thoroughly comprehensive article on the topic, which goes over not just the bits people get angry about but also the frankly massive amount of cool stuff that only works in Python 3, and that you’re missing out on if you still only use Python 2.

No, this is about how you as a developer can break Python, and break it thoroughly, whenever you need to.

Why break Python?

If you maintain applications, libraries or other code written in Python, and you ever intend to distribute any of that code to third parties, you need to make some decisions. You need to decide how to license your code, you need to decide how to distribute it (if you’re open source, the Python Package Index is the place to do it), etc.

But you also need to — and this is an important and oft-overlooked step — decide which versions of Python you’ll support. And here I don’t just mean Python 2 vs. Python 3. If you support Python 2, which specific Python 2 releases do you want to support? On Python 3, which specific Python 3 releases?

Supporting lots of versions of Python can significantly increase your workload in writing and maintaining code. Python improves with each release of the language, but — aside from some 2-to-3 compatibility features present in Python 2.6/2.7, and some security stuff that got backported to 2.7, those improvements don’t migrate backward into older versions of the language. Which means that every version of Python you support adds another set of restrictions on what you can write, and can leave you having to invent workarounds for convenient features that unfortunately don’t exist in a version you’re still supporting. This is a recipe for burning out as a maintainer, so you really need to pick a set of Python versions and stick to that in order to put some limits on how much work you’ll have to do.

Also, once you’ve decided on a set of versions to support, you really want to be sure about it. Many people nowadays are good about having some kind of continuous integration running against a matrix of supported versions, but that only handles half the problem: you know your code works on the versions you support, but are you sure it doesn’t work on unsupported Python versions?

This is more important than it sounds at first. But one of the nightmare scenarios as a maintainer is to suddenly have someone who’s very angry because they didn’t change anything in their setup and suddenly it stopped working. And that’s exactly what happens with code that accidentally “works” — or at least mostly works — on an unsupported Python version. Everything is fine until one day the fateful code path gets reached for the first time, or you push a new release that turns your declaration of incompatibility into actual incompatibility.

So if at all possible, you should take steps to ensure that when you declare a set of supported Python versions, you’re actually enforcing them.

Which versions should you support?

This is entirely up to you, and will depend on how much work you feel like doing and how badly you want features that only exist in newer versions of Python. My own personal policy, though, is:

There are plenty of other sensible options available. You could support only the latest stable 2.x and latest stable 3.x, or only the latest stable 3.x, or only supported 3.x releases, and so on. The important thing is not the specific set of versions you decide on, but that you decide and take action on that decision.

You should also make sure — for anything which will go on the Python Package Index or a similar service — to use trove classifiers in your setup.py to let people know which versions you’re supporting. The full list of trove classifiers is quite large, and lets you also indicate topical categories and things like framework version compatibility, but at the very least you should use the Python-version classifiers so people know what to expect (you can also browse PyPI by classifiers, in order to find, say, game-related code which supports Python 3.5 and is known to work on macOS, or any other arbitrary set of classifiers you care to combine).

How not to break Python

There is a very easy brute-force way to enforce use of specific versions of Python: you can just check the version someone’s running, and crash if it’s not one you want to support. For example, if you wanted to only allow Python 3.3 and later, you could do something like:

import sys

if sys.version_info < (3, 3):
    sys.exit("You must use Python 3.3, or newer.")

This works because sys.version_info is a tuple, and Python supports ordered comparisons on tuples (and on versions of Python which include namedtuple in the collections library, sys.version_info is a namedtuple with useful names on its fields, letting you inspect it semantically instead of having to memorize the indices of the different version components).

But it’s not exactly elegant, and it litters a bunch of these brute-force version checks throughout your code. If you’re OK with that, then it does of course get the job done.

Personally, I prefer an approach which involves simply writing natural Pythonic code to do whatever it is my library or application does, but in a way that also ensures compatibility with only the specific set of Python versions I’ve chosen to support. Which isn’t terribly hard once you know a few things, and those things are what the rest of this article will cover.

This will not, however, just be a list of what was new in each version of Python. Instead, the focus will be on generally-useful things which were added to Python and which achieve one of the following (in decreasing order of how preferable they are to use):

  1. Causing an import-time SyntaxError. This is the ideal, because it ensures absolutely that your code will never work on an unsupported version of Python, with no chance of accidentally being able to sort-of work for long enough to mislead someone.
  2. Causing an import-time ImportError, NameError or AttributeError. This is almost as good as a syntax error, but not quite: these exceptions will get the job done, just without clearly communicating that your code is written for a different version of Python.
  3. Causing an exception fairly quickly and predictably, but not automatically on import. This is the least desirable option, because it runs the risk of someone thinking it’s a bug in your code instead of a deliberate way of breaking compatibility with an unsupported version.

Now let’s get started.

Python 2.6

As I mentioned above, I support any version of Python that still has upstream support, which means that for now I still support Python 2 even if I don’t recommend or use it personally. The only 2.x release still receiving upstream support is Python 2.7, but thanks to long-term third-party support contracts there are people still running 2.6. You really should try to stop supporting or using Python 2.6 as soon as possible, but if you must support it — and I highly recommend you make sure you’re paid well for doing so — here’s how to ensure your code works on 2.6 but not on 2.5:

Note: do not attempt to use the print/print() distinction to break Python 2.5 support. Python 2.6 introduced the from __future__ import print_function flag for emulating Python 3’s behavior, but print followed by parentheses is syntactically valid in earlier versions of Python. It just reads as “print this tuple”.

Python 2.7

This is ideally the minimum Python version anyone should support now, which means you’ll want to break Python 2.6 compatibility. For that:

Python 3

If you only want to support Python 3, and cut off Python 2 altogether, it’s somewhat tricky precisely because the compatibility tools for supporting Python 2 and 3 in a single codebase were so good. This makes it hard to trigger a generic “this doesn’t work on Python 2” SyntaxError, since Python 3 held off on adding new syntax for a few releases in order to give people a chance to start their porting and shake out issues in the early 3.x releases. Even worse, several syntactic changes got backported for compatibility into 2.7, taking away the option to use them.

The easiest and most obvious thing you can do, syntactically, is to use function annotations. These were never backported into a 2.x release, and are generally useful on a Python 3 codebase. Unfortunately, they’re most useful when combined with the typing type-hint library which was first shipped in Python 3.5, so unless you’re 3.5+ you’ll need to install the generic backported version for earlier Python versions.

If you can’t or don’t want to use annotations, you can:

Python 3.3

Python 3.3 was the oldest currently-supported Python 3.x release when this article was originally written, but reached end-of-life in 2017. If you still need to support Python 3.3 but not older 3.x releases:

Python 3.4

This one’s tricky, because Python 3.4 did not add any new syntax to the language. So there’s no way to get the ideal situation of code that works on 3.4 and is a SyntaxError in 3.3. However, you can still force some errors in 3.4:

Python 3.5

Python 3.5 has new syntax we can take advantage of:

Note: Python 3.5 added the typing module to the standard library, but as mentioned above it’s available as a separate download from PyPI for earlier Python versions. Which means importing from typing is not guaranteed to break compatibility with older Pythons; someone might have installed it separately, or might go and install it separately to make your code work.

Python 3.6

Python 3.6 made multiple additions to Python’s syntax:

Python 3.7

As of the last edit to this article, Python 3.7 was the most recent released version of Python. It provided a new way to break things immediately:

Python 3.7 also made async and await into reserved keywords, but that one goes the other way round: rather than stopping new code from running on an old Python, it stops old code from running on a new Python (using async or await as identifiers is a SyntaxError in Python 3.7)

The new built-in breakpoint() function is a NameError in Python 3.6, but since it’s a debugging tool you probably don’t want to leave it in production code.

Three new modules were added to the standard library, providing opportunities to immediately cause ImportError in older Python versions: contextvars, dataclasses, and importlib.resources. Of these, dataclasses is the one most likely to be generally useful.

Finally, it’s possible to write code using the typing module which works in Python 3.7 but raises TypeError at import time in older versions. Previously, the generics in typing had the metaclass typing.GenericMeta, which would in turn cause TypeError (due to metaclass conflict) whenever a class was a subclass of a generic and any parent that had a different metaclass. PEP 560 resolved this problem for Python 3.7.

Python 3.8

As of the last edit to this article, Python 3.8 was in development, but already contained some changes which could be used to break older Pythons:

And that’s it… for now

I’ve already implemented some of the above in a piece of code I’m shipping (webcolors 1.7 uses u prefixes to break Python 3.0-3.2, and a dict comprehension to break 2.6), and plan to do the same with my other projects over the course of their future release cycles. And hopefully the above list will help someone else manage the same in their projects, and perhaps prevent some maintenance headaches and bad days for developers and users.