Is working with Python Environments a barrier to Python use?

Vanilla Installs
easy!
What is everyone complaining about?
I only need one python ever!

Oh I got app 1 installing dependencies over app 2 because I’m installing everything to the system python and I have to run as admin or sudo pip install everything!

Virtual envs
python -m venv venv
Pycharm’s venv creator
venv wrapper
activating with activate.bat and dot sourcing
looking at PATH to figure out why nothing works
adding PYTHONPATH for things that for some reason can’t be in the venv

Oh, but I will never need another python.
Yeah right, python 2 and 3 everywhere, 2 often unavoidable.
Pypy can solve perf problems
You probably don’t want to migrate to 3.n to 3.n+1 because libraries take a while to catch up so you’re switching back and forth
Oh, you’re going to stick with 3.1 forever? Pip just installed the latest version of a lib that doesn’t support 3.1, do you know how to pin your dependencies?
Oh you found a bug that is fixed, but now you need to move away from 3.1, etc.
So now you need pyenv, pywin and or tox. Tox does cross python testing, pyenv/pywin do interpreter switching

Ah, my versions are pinned, I have no more problems
Dependency A depends on B 2.1 and Dependency C depends on B 2.3. Pip out of box provides no clues why everything is broken
Thanks okay, I’ll use pipenv. Now I can get a graph to see where the transitive problems are and get warnings when they happen. Also, I can separate out development dependencies so that the dependency graph for a production release is simpler.

No more problems, yes sir.
“Can’t find FORTRAN compiler” – Yes, some common data science library have FORTAN dependencies.
Oh, easy, I just won’t do datascience, just Postgres. “Can’t find C++ compiler. Can’t find Python.h. Can’t find libfoo.so”…. Oh, I guess I need to install tool chains. Or learn to install wheels. Or realize that if you have a lot of native dependencies, maybe you should use conda.

No more problems? Everyone on the team did the exact same thing, identically. All easy now?
Well actually they didn’t, all of the above are slightly different for each OS.
But lets imagine everyone did have the same workstation
The build environment runs in docker, often in a linux distro with most dependencies.This is so reliable, why didn’t we just use docker all the time? It is very, very slow if you’re writing to it all the time.
The production environment runs in docker often a small distribution like alpine, except many ordinary packages lack the corresponding apk to do installs
VMs and Vagrant are often too slow.
And finally, you generally need a bunch of bash scripts to make this run in production. To get them to work on windows, you’ll need gitbash, or mingw64, or cygwin, all of which behave differently
But what about Ubuntu on Windows (WSL?) – Until WSL2 is release, performance is really bad for anything involving IO, but some people can put up with it. Also not supported by free Pycharm, AFAIK.

Posted in Uncategorized