How to break Python

Published: November 28, 2016. Filed under: Django, Python.

Don’t worry, this isn’t another piece about Python 3. I’m fully in favor of Python 3, and on record as to why. And if you’re still not convinced, I suggest this thoroughly comprehensive article on the topic, which goes over not just the bits people get angry about but also the frankly massive amount of cool stuff that only works in Python 3, and that you’re missing out on if you still only use Python 2.

No, this is about how you as a developer can break Python, and break it thoroughly, whenever you need to.

Why break Python?

If you maintain applications, libraries or other code written in Python, and you ever intend to distribute any of that code to third parties, you need to make some decisions. You need to decide how to license your code, you need to decide how to distribute it (if you’re open source, the Python Package Index is the place to do it), etc.

But you also need to — and this is an important and oft-overlooked step — decide which versions of Python you’ll support. And here I don’t just mean Python 2 vs. Python 3. If you support Python 2, which specific Python 2 releases do you want to support? On Python 3, which specific Python 3 releases?

Supporting lots of versions of Python can significantly increase your workload in writing and maintaining code. Python improves with each release of the language, but — aside from some 2-to-3 compatibility features present in Python 2.6/2.7, and some security stuff that got backported to 2.7, those improvements don’t migrate backward into older versions of the language. Which means that every version of Python you support adds another set of restrictions on what you can write, and can leave you having to invent workarounds for convenient features that unfortunately don’t exist in a version you’re still supporting. This is a recipe for burning out as a maintainer, so you really need to pick a set of Python versions and stick to that in order to put some limits on how much work you’ll have to do.

Also, once you’ve decided on a set of versions to support, you really want to be sure about it. Many people nowadays are good about having some kind of continuous integration running against a matrix of supported versions, but that only handles half the problem: you know your code works on the versions you support, but are you sure it doesn’t work on unsupported Python versions?

This is more important than it sounds at first. But one of the nightmare scenarios as a maintainer is to suddenly have someone who’s very angry because they didn’t change anything in their setup and suddenly it stopped working. And that’s exactly what happens with code that accidentally “works” — or at least mostly works — on an unsupported Python version. Everything is fine until one day the fateful code path gets reached for the first time, or you push a new release that turns your declaration of incompatibility into actual incompatibility.

So if at all possible, you should take steps to ensure that when you declare a set of supported Python versions, you’re actually enforcing them.

Which versions should you support?

This is entirely up to you, and will depend on how much work you feel like doing and how badly you want features that only exist in newer versions of Python. My own personal policy, though, is:

There are plenty of other sensible options available. You could support only the latest stable 2.x and latest stable 3.x, or only the latest stable 3.x, or only supported 3.x releases, and so on. The important thing is not the specific set of versions you decide on, but that you decide and take action on that decision.

You should also make sure — for anything which will go on the Python Package Index or a similar service — to use trove classifiers in your setup.py to let people know which versions you’re supporting. The full list of trove classifiers is quite large, and lets you also indicate topical categories and things like framework version compatibility, but at the very least you should use the Python-version classifiers so people know what to expect (you can also browse PyPI by classifiers, in order to find, say, game-related code which supports Python 3.5 and is known to work on macOS, or any other arbitrary set of classifiers you care to combine).

How not to break Python

There is a very easy brute-force way to enforce use of specific versions of Python: you can just check the version someone’s running, and crash if it’s not one you want to support. For example, if you wanted to only allow Python 3.3 and later, you could do something like:

import sys

if sys.version_info < (3, 3):
    sys.exit("You must use Python 3.3, or newer.")

This works because sys.version_info is a tuple, and Python supports ordered comparisons on tuples (and on versions of Python which include namedtuple in the collections library, sys.version_info is a namedtuple with useful names on its fields, letting you inspect it semantically instead of having to memorize the indices of the different version components).

But it’s not exactly elegant, and it litters a bunch of these brute-force version checks throughout your code. If you’re OK with that, then it does of course get the job done.

Personally, I prefer an approach which involves simply writing natural Pythonic code to do whatever it is my library or application does, but in a way that also ensures compatibility with only the specific set of Python versions I’ve chosen to support. Which isn’t terribly hard once you know a few things, and those things are what the rest of this article will cover.

This will not, however, just be a list of what was new in each version of Python. Instead, the focus will be on generally-useful things which were added to Python and which achieve one of the following (in decreasing order of how preferable they are to use):

  1. Causing an import-time SyntaxError. This is the ideal, because it ensures absolutely that your code will never work on an unsupported version of Python, with no chance of accidentally being able to sort-of work for long enough to mislead someone.
  2. Causing an import-time ImportError, NameError or AttributeError. This is almost as good as a syntax error, but not quite: these exceptions will get the job done, just without clearly communicating that your code is written for a different version of Python.
  3. Causing an exception fairly quickly and predictably, but not automatically on import. This is the least desirable option, because it runs the risk of someone thinking it’s a bug in your code instead of a deliberate way of breaking compatibility with an unsupported version.

Now let’s get started.

Python 2.6

As I mentioned above, I support any version of Python that still has upstream support, which means that for now I still support Python 2 even if I don’t recommend or use it personally. The only 2.x release still receiving upstream support is Python 2.7, but thanks to long-term third-party support contracts there are people still running 2.6. You really should try to stop supporting or using Python 2.6 as soon as possible, but if you must support it — and I highly recommend you make sure you’re paid well for doing so — here’s how to ensure your code works on 2.6 but not on 2.5:

Note: do not attempt to use the print/print() distinction to break Python 2.5 support. Python 2.6 introduced the from __future__ import print_function flag for emulating Python 3’s behavior, but print followed by parentheses is syntactically valid in earlier versions of Python. It just reads as “print this tuple”.

Python 2.7

This is ideally the minimum Python version anyone should support now, which means you’ll want to break Python 2.6 compatibility. For that:

Python 3

If you only want to support Python 3, and cut off Python 2 altogether, it’s somewhat tricky precisely because the compatibility tools for supporting Python 2 and 3 in a single codebase were so good. This makes it hard to trigger a generic “this doesn’t work on Python 2” SyntaxError, since Python 3 held off on adding new syntax for a few releases in order to give people a chance to start their porting and shake out issues in the early 3.x releases. Even worse, several syntactic changes got backported for compatibility into 2.7, taking away the option to use them.

The easiest and most obvious thing you can do, syntactically, is to use function annotations. These were never backported into a 2.x release, and are generally useful on a Python 3 codebase. Unfortunately, they’re most useful when combined with the typing type-hint library which was first shipped in Python 3.5, so unless you’re 3.5+ you’ll need to install the generic backported version for earlier Python versions.

If you can’t or don’t want to use annotations, you can:

Python 3.3

Python 3.3 is the oldest currently-supported Python 3.x release, and so is a good target for a minimum version in Python 3 land. This means you’ll want to break earlier 3.x releases. Two very easy ways to do this are:

Python 3.4

This one’s tricky, because Python 3 did not add any new syntax to the language. So there’s no way to get the ideal situation of code that works on 3.4 and is a SyntaxError in 3.3. However, you can still force some errors in 3.4:

Python 3.5

Python 3.5 is the latest current 3.x release, though 3.6 is on its way. It also has new syntax we can take advantage of:

Note: Python 3.5 added the typing module to the standard library, but as mentioned above it’s available as a separate download from PyPI for earlier Python versions. Which means importing from typing is not guaranteed to break compatibility with older Pythons; someone might have installed it separately, or might go and install it separately to make your code work.

Python 3.6

Python 3.6 isn’t out yet, but it’s in beta and pretty close to its final form. This lets us anticipate some things:

And that’s it… for now

Python 3.7 is lurking in the future, but so far nothing’s locked in for it that could be used to forcibly break compatibility with 3.6 or other earlier versions. So that’s where I’ll leave off.

I’ve already implemented some of the above in a piece of code I’m shipping (webcolors 1.7 uses u prefixes to break Python 3.0-3.2, and a dict comprehension to break 2.6), and plan to do the same with my other projects over the course of their next release cycles. And hopefully the above list will help someone else manage the same in their projects, and perhaps prevent some maintenance headaches and bad days for developers and users.