I’m curious about the transition in our ecosystem from Python 2 to Python 3. On stage at our monthly PyDataLondon meetups I’m known to badger folk to take the step and upgrade to reduce the support burden on developers. The transition gathers pace but it still feels slow. I’ve noted my recommendations for moving to Python 3+ at the end. See also the reddit discussion.
I’m wondering – when will we see the point where open source projects say “We support Python 3.x for free but if you want bugs fixed for Python 2.7, you’ll have to pay“? I’m not saying “if”, but “when”. There’s already one example below and others will presumably follow.
In the last couple of years a slew of larger projects have dropped or are dropping support for Python 2.6 – numpy (some discussion), pandas, scipy, matplotlib, NLTK, astropy, dask, ipython, django, numba, twisted, scrapy. Good – Python 2.6 was deprecated when 2.7 was released in 2010 (that’s 6 years ago!).
The position of the matplotlib and django teams is clearly “Python 2.7 and Python 3.4+”. Django states that Python 2.7 will be supported until the 2020 sunset date:
“As a final heads up, Django 1.11 is likely to be the last version to support Python 2.7 as it will be supported until the end of Python 2 upstream support in 2020. We’ve adopted a Python version support policy…”
We can expect the larger projects to support legacy userbases with a mix of Python 2.7 and 3.4+ for 3.5 years (at least until 2020). After this we should expect projects to start to drop 2.7 support, some (hopefully) more aggressively than others.
What about smaller projects? Several have outright dropped Python 2.7 support already – errbot (2016 is the last Python 2.7-supported year), nikola, python-thumbnails – or never supported it – wordfreq, featherweight. Which others have I missed? Update – JupyterHub (cheers Thomas) too. IPython 6.0 in 2007 will be Python 3.4+ only (IPython 5.0 is the last supported Python 2.7 release). mitmproxy has just switched to a Python-3-only branch (Sept 2016).
More interestingly David MacIver (of Hypothesis) stated a while back that he’d support Python 2.7 for free but Python 2.6 would be a paid support option. He’s also tagged (regardless of version) a bunch of bugs that can be fixed for a fee. Viewflow is another – Python 3.4 is free for non-commercial use but a commercial license or support for Python 2.7 requires a fee. Asking for money to support old, PITA or difficult options seems mightily sensible. I guess we’ll see this first for tools that have a good industrial userbase who’d be used to paying for support (like Viewflow).
Aaron Meurer (lead dev on SymPy) has taken the position that library leaders should pledge for a switch to Python 3.x only by 2020. The pledge shows that scikit-bio is about to go Python 3-only and that IPython 6.x+ will be Python 3 only (from 2017). Increasingly we’ll see new libraries adding the shiny features for their Python 3 branch only.
What next? I imagine most new smaller projects will be Python 3.4+ (probably 3.5+ only soon), they’ll have no legacy userbase to support. They could widen their potential userbase by supporting Python 2.7 but this window only exists for 3 years and those users will have to upgrade anyhow. So why bother going backwards?
Once users notice that cooler new toys are Python 3.4+ only they’ll want to upgrade (e.g. NetworKit is Python 3.3+ only for high volume graph network analysis). They’ll only hold back if they’re supporting legacy internal systems (which will be the case for an awful lot of people). We’ll see this more as we get closer to 2020. What about after 2020?
I guess many companies will be slow to jump to Python 3 (it’d take a lot of effort for no practical improvement), so I’d imagine separate groups will start to support Python 2.7 libraries as forks. Hopefully the main library developers will drop support fairly quickly, to stop open source (cost-free) developers having a tax on their time supporting both platforms.
Separate evidence – Drupal 6 adopted a commercial-support-for-old-versions policy (thanks @chx). It is also worth noting that Ubuntu 16.04 LTS ships without Python 2. Microsoft and Brett Cannon have discussed the benefits of moving to Python 3+ recently.
My recommendations (coming from a Senior Industrial Data Scientist with 15+ years commercial experience and 10+ years using Python):
- Where feasible – all new projects must use Python 3.5+ (e.g. Proof of Concepts, Research, new isolated systems) – this is surprisingly easy
- If Python 2.7 compatibility is required – write all new code in a Python 3.5+ compatible way (1, 2, 3, 4), make extensive tests for the later inevitable migration (you already have good test-coverage, right?)
- Accept that support for Python 2.7 gets turned off in 3.5 years and that all Python 2.7 code written now will likely have to be migrated later (this is a business cost that you can estimate now)
- Accept that as we get closer to 2020 more programmers (both new and experienced) will be using Python 3.5+, so support for Python 2.7-based libraries will inevitably decline (there’s a big business risk here)
Graham and I did a lightning talk on jumping to Python 3 six months back, there’s a lot of new features in Python 3.4+ that will make your life easier (and make your code safer, so you burn less time hunting for problems). Jake also discussed the general problem for scientists back in 2013, it’ll be lovely when we get past this (now-very-boring) discussion.
Ian is a Chief Interim Data Scientist via his Mor Consulting. Sign-up for Data Science tutorials in London and to hear about his data science thoughts and jobs. He lives in London, is walked by his high energy Springer Spaniel and is a consumer of fine coffees.
36 Comments