Arch Linux's Python Protects Me From Myself

Lesson learned: Don't use python

I've been using virtual environments but I didn't know about pipx. I literally ended up making my own script for that but I guess I won't be needing that anymore :D

My problem with this virtual environment is... It breaks. It has never done anything else than break my applications. I can't use the libraries I installed when writing a script, because they are in pipx's venv. The shell I was using could not find its addons, because they where in different venv's. It was a mess, and in the end I just made an alias for pip to be pip --break-system-packages, and I never got a conflict with apt. (Exept when I upgraded ubuntu, but that's another story.)

tfw nixos

Unless I so out of date, I wish virtual environments would look at what the system have and if it is what it needs, pull it in. If not, install something else. The nightmare of added space if virtual environments all having a lot of the same files.

It should only give you this error if you are running pip as root. Otherwisd it should default to --user which solves the problem completely. This is stupid. They want to fix one problem and they create another while doing so!

Nix(OS) bindmounts everything readonly, and only nix-daemon can access the nix store for write access, your entire system is just a bunch of symlinks to the nix store, every single app on the OS is hardcoded to a specific package version by its hash, its a little steep of a learning curve, but it its unyielding pedantic versioning prevents stuff like system wide installs from tools like pip from happening, and because your system is just symlinks, you can revert to an old configuration pinned to a single consistent package hierarchy.

You can then pin an app or dev environment to a specific version of nixpkgs (you can use it on any OS you like), this is like venv or cargo/yarn lock files but for the entire package hierarchy down to systemd and libc. You can even specify patches and it'll recompile that package in a clean sandbox, its like having first class aur support, no third party tools to download, nix already knows exactly how that package was built, you just describe what you want changed, like your own git repo.

Want a stripped down Linux kernel, debugging flags, or custom patch? Just specify them and it'll apply those changes to the latest kernel every time its updated on nixpkgs.

I rarely needed to use pip, but thanks for letting me know. Just installed pipx to have it there if this ever pops up.

TBF: the only place where you can install python (other language packages) system-wide with PIP without having to worry about breaking package managed dependencies is /usr/local/ because linux fhs mandates that this place be not touched by distros package managers.

This is what's profesionally known as a "Second System Problem" & it's to be avoided at all costs!
I have never understood why some software environments & distros in the past didn't do that at the very first step when they got deployed as a system-wide stable. Programming environments & languages should absolutely have coordinated with distros/package environments from the get go - not DECADES after it the language became a stable standard.

thank you for bringing attention to this!

The inability to process errors is what I call "red text blindness".

It seems to me that a lot of people simply can not process red text or errors in general, and I don't think dyslexia is common enough to explain the trouble.

Getting an unexpected error can literally feel pretty awful. I guess many people's brains just shortcut and look for help immediately.

One of my thoughts: /usr/local/ is a thing. Why doesn't pip use it?

I honestly hate this because it doesn't actually help solve the problem. I remember I had to install some obscure package, Judging by my history I'm guessing pipx but that could be something entirely different, and it said to install it with python-pipx, (assuming, I obviously don't have the terminal output but that's the format) but that just flat out wasn't a package in the repos. Granted that could be argued to be an issue with python naming conventions (tk-kill me by smothering me with a PILlow) but it's still annoying to have the error telling you to do something that's just flat out not a thing you can do. I don't remember how I actually got past it, maybe I just used different software or something, in any case I now despise this error.

edit : oh hey pipx, I'm guessing that's why it was most recent in my history then, I probably used a venv for it, neat.

great news. I sometimes screw up and by mistake install some dev packages outside of intended venv. The The message is maybe going overboard and simple "you are about to install it system wide. Are you sure you want to do that?"

This one I stumbled on the last several days for the first time -- I wanted to run a python thingy on my laptop that I acquired and set up for arch several months ago. I was glad for the information provided but I wondered how it was I had never seen that before on other arch boxes I have and do more code banging on... Now I know.

I was really surprised there was no mention of Docker containers in the entire video. Because (for me, at least) that is where most things will break due to PEP668.

For two reasons:
1) in most cases, containers will (and should) run only a single app, so you really don't care enough about potential conflicts to use venvs
2) using venvs in container is (a bit of) a pain. It can be done, but not by simply calling "activate"

I can see many workflows (including, admittedly, some of mine :) ) that will break when this gets widely implemented. Such as in Ubuntu 24.04...

Yes, I know, obligatory xkcd "workflow" reference here :)

bruh stop peepin my search history

Wait, do you install python packages using pip as root?

I only install pip packages as a normal user, they end up in ~/.local/lib/

Start using NIX OS

Isn't there is pip install --user that does the same thing that pipx, but without virtual environments. Just instead of system it installs everything locally to user. Same with cabal (for Haskell) and I suspect for many other languages with package managers.

Pip is not a package manager. Pip installs packages. It does not resolve any conflict.

Nix solves this problem entirely

Problems like these remind me why things like NixOS exist and how i need to stop procrastinating and try it

and I just passed the parameter to ignore the error and then spend half an hour fixing cuda

I wanted to install something with pip once, got this error, read it, didn't understand it, and proceeded to not use python.

I love Python as a language, but the way it's now considered normal that the most lauded packages can't be installed simultaniously because they apparently all depend on another version of this one function only available in this specific version of one other package that is in no other version ... what happened to programming to an interface?

this is kinda weird for arch, I use arch as a developer because I like bleeding edge stuff, and also wasn't arch supposed to be a "do your own thing" distro?

Whats up with using doas instead of sudo?

Good thing. Alternatively banish Python completely. Whatever devs may like about it (IMHO all languages since C++ are just the same with a new name...) - it's a pure 100% PITA for the user all the time! Think I haven't seen a single python application run on my system ever without many wasted hours of fiddling around and contaminating my system with dozens of modules, python versions, dependeny issues, all this confusing venv crap and libraries or just important application like i.e. system-config-users simply no longer being able. Did not know about pipx, yet. Maybe this will make it more bearable.

Users don't perceive error messages as "The computer is telling you what went wrong and what to do about it". Users perceive it as "the damn machine randomly decided to not do what I want"

why do they not make pip work like pipx ?

ahh venv; every project is 5 GB now 🙃 And people complain about node_modules

Venvs were such a hassle to me I was just writing Dockerfiles for my stuff, I didn't know pipx existed. The more you know lol

My only concern about this change is who is gonna maintain that gazillion of python packages for every package manager. Unless they automatized it somehow, this feels like duplicated work.

once pip wrote to your system, you can never delete that pollution ... love it ...

Soo what I'm hearing si that installing packages on linux now involves pulling a Wirtual?

I dont know how to code (@LifeOfBoris and his coding lessons are my maximum knowledge, add onto that minecraft commands...) so this... KINDA SOUNDS WORRYING

oh wow pipx for the win

This has bit me so many times. Glad someone is sorting it.

I never saw this error because I was already using venv to avoid pip conflicting with the system package manager.

Huh. Maybe ive been using pipx this whole time. All of my pip installed stuff has been in a folder in .local, not just the symlinks

Nice, just yesterday I tried to install python with django-htmx to learn a bit of py... Great timing to learn about pipx.

So this is how our ancestors did things before nix was invented? Really interesting stuff!

I'd really like pipx to have the feature to rename the virtual environments it creates, sometimes i want to create a group of python packages with inject but it has the name pf the first installed.

What bothers me is that it doesn't tell you if or which package is causing the conflict, by running:

grep -f <(pip list --path /usr/lib/python3.11/site-packages/ | cut -d' ' -f1) \
<(pip list --path ~/.local/lib/python3.11/site-packages/ | cut -d' ' -f1)

You can see if a package has been installed on the system or locally.

I usually have things like i3ipc, yt-dlp (dev), gallery-dl (dev) installed locally with pip install --user <package_name>
since my scripts use them, so i might rename that file that you shouldn't : )

Its a great addition but it kind of broke my already installed packages on arch

pipx also doesn't work always, like I was trying to install pygame. What works for me was poetry.