I rarely disagree with Drew, but this right here is the problem. I'm a DBA/sysadmin, and I can tell you with certainty that this is a horrific idea. You don't manage python dependencies at the...
I manage my Python packages in the only way which I think is sane: installing them from my Linux distribution’s package manager.
I rarely disagree with Drew, but this right here is the problem. I'm a DBA/sysadmin, and I can tell you with certainty that this is a horrific idea. You don't manage python dependencies at the system level for the same reason you don't use one git repo for every programming project you ever work on.
Unless you can get every single python program that was ever written to agree to the same set of dependencies and perpetually keep them up to date, you can't use system python for everything. Tons of libraries and other programs have breaking changes between versions and it takes time to upgrade them all.
Virtual environments at the project level are the answer. Most of those other tools are merely frontends to manage them (I find poetry the best at the moment), which I don't really have a problem with. If you want the best system python, it's just the raw runtime , venv, and pip. Everything else you install with pip install --user and keep the muck out of the system itself.
If you don't want to do that as a system packager...use a venv build system, leverage cython to create a statically linked python program so you don't need to load and install a dependency chain.
This may be true but, loads of Perl and PHP stuffs "Just Work" out of the box with the system provided packages - rarely does one need to reach directly for CPAN or PECL to get things to work. So...
This may be true but, loads of Perl and PHP stuffs "Just Work" out of the box with the system provided packages - rarely does one need to reach directly for CPAN or PECL to get things to work. So maybe the expectation that a system provided language (an LSB one at that) should not require special care and feeding is reasonable? Maybe it's not the Python team screwing over the users but the devs of the specific Python apps/packages that don't play nice with the distros?
One of Python’s killer features is its cross-platform nature. PyPI/pip works well universally, whereas the package management ecosystem on Linux is fractured to hell and back. Library devs would...
One of Python’s killer features is its cross-platform nature. PyPI/pip works well universally, whereas the package management ecosystem on Linux is fractured to hell and back. Library devs would have to spend a lot of time spinning their wheels for very little ROI to support linux package managers as an alternative to pip; with an ecosystem as big as Python’s, that’s not going to happen - not when there’s a standardized, accepted system already in place.
Vord speaks truth: pip + venv “just works” for python.
I've not doven too deep into perl, but tons of python stuff works ootb too. I'm not even really opposed to having system-level python stuff like pyodbc or requests available in the repos, but...
I've not doven too deep into perl, but tons of python stuff works ootb too. I'm not even really opposed to having system-level python stuff like pyodbc or requests available in the repos, but those things need to be standalone and only depending on the standard library. You shouldn't really leverage them outside of system management for the reasons I describe though.
Pretty much all languages with a centralized package management suffer these problems. Just the ones that compile to binaries don't need to have the build system cruft moved to the client's machine too.
This is incidentally why I generally prefer shell scripts over either for system stuff. Instead of libraries you have programs, thus abstracting the majority of version dependance issues.
Personally, I prefer pipenv, but yes, system Python has screwed me over way too many times. Just getting a package to be installed system-wide can become a huge PITA (e.g. it was installed...
Personally, I prefer pipenv, but yes, system Python has screwed me over way too many times. Just getting a package to be installed system-wide can become a huge PITA (e.g. it was installed root-local by the distro). Writing for your Python packages in your distro package manager will always screw you over whenever you try running it on another system.
I use OpenSUSE, which is a fantastic distro, but their naming scheme for package management is different from almost all other major linux distros. So compiling programs of all types which depend...
I use OpenSUSE, which is a fantastic distro, but their naming scheme for package management is different from almost all other major linux distros.
So compiling programs of all types which depend on system stuff and have scripts to try to fill dependances mostly only factor Debian/Redhat derivatives, and I'll often have to rework bits of things to get them working. C++ stuff certainly isn't immune either, and having to track down dozens of dependancies that are otherwise expected to be installed at the system level, often at different paths, are a pita.
There are benefits to using a mono repo. I don't currently, but I'm not sure why other than github etc making it easy to setup, distribute and manage code in new repos. I guess tracking issues...
You don't manage python dependencies at the system level for the same reason you don't use one git repo for every programming project you ever work on.
There are benefits to using a mono repo. I don't currently, but I'm not sure why other than github etc making it easy to setup, distribute and manage code in new repos. I guess tracking issues would be a hassle on GitHub if all my projects were in the same repo... then again it would be easier to track cross project issues. Not sure if/how it would work with golang as there a dependency was usually pointing to some GitHub repo... Would be fun to try though 🙂
I rarely disagree with Drew, but this right here is the problem. I'm a DBA/sysadmin, and I can tell you with certainty that this is a horrific idea. You don't manage python dependencies at the system level for the same reason you don't use one git repo for every programming project you ever work on.
Unless you can get every single python program that was ever written to agree to the same set of dependencies and perpetually keep them up to date, you can't use system python for everything. Tons of libraries and other programs have breaking changes between versions and it takes time to upgrade them all.
Virtual environments at the project level are the answer. Most of those other tools are merely frontends to manage them (I find poetry the best at the moment), which I don't really have a problem with. If you want the best system python, it's just the raw runtime , venv, and pip. Everything else you install with
pip install --userand keep the muck out of the system itself.If you don't want to do that as a system packager...use a venv build system, leverage
cythonto create a statically linked python program so you don't need to load and install a dependency chain.This may be true but, loads of Perl and PHP stuffs "Just Work" out of the box with the system provided packages - rarely does one need to reach directly for CPAN or PECL to get things to work. So maybe the expectation that a system provided language (an LSB one at that) should not require special care and feeding is reasonable? Maybe it's not the Python team screwing over the users but the devs of the specific Python apps/packages that don't play nice with the distros?
One of Python’s killer features is its cross-platform nature. PyPI/pip works well universally, whereas the package management ecosystem on Linux is fractured to hell and back. Library devs would have to spend a lot of time spinning their wheels for very little ROI to support linux package managers as an alternative to pip; with an ecosystem as big as Python’s, that’s not going to happen - not when there’s a standardized, accepted system already in place.
Vord speaks truth: pip + venv “just works” for python.
I've not doven too deep into perl, but tons of python stuff works ootb too. I'm not even really opposed to having system-level python stuff like pyodbc or requests available in the repos, but those things need to be standalone and only depending on the standard library. You shouldn't really leverage them outside of system management for the reasons I describe though.
Pretty much all languages with a centralized package management suffer these problems. Just the ones that compile to binaries don't need to have the build system cruft moved to the client's machine too.
This is incidentally why I generally prefer shell scripts over either for system stuff. Instead of libraries you have programs, thus abstracting the majority of version dependance issues.
Personally, I prefer pipenv, but yes, system Python has screwed me over way too many times. Just getting a package to be installed system-wide can become a huge PITA (e.g. it was installed root-local by the distro). Writing for your Python packages in your distro package manager will always screw you over whenever you try running it on another system.
I use OpenSUSE, which is a fantastic distro, but their naming scheme for package management is different from almost all other major linux distros.
So compiling programs of all types which depend on system stuff and have scripts to try to fill dependances mostly only factor Debian/Redhat derivatives, and I'll often have to rework bits of things to get them working. C++ stuff certainly isn't immune either, and having to track down dozens of dependancies that are otherwise expected to be installed at the system level, often at different paths, are a pita.
Agree totally.
pip installmethod has served me best over the years and I've rarely found the need to install a distro's python package.There are benefits to using a mono repo. I don't currently, but I'm not sure why other than github etc making it easy to setup, distribute and manage code in new repos. I guess tracking issues would be a hassle on GitHub if all my projects were in the same repo... then again it would be easier to track cross project issues. Not sure if/how it would work with golang as there a dependency was usually pointing to some GitHub repo... Would be fun to try though 🙂