I ran into a strange problem yesterday when I wanted to run the Python program I use to print a nice version of our weekly menu plans – the system complained that a package was missing, even though I’d been able to run the program just a few weeks ago and I certainly hadn’t uninstalled any packages since then.
I installed the package so I could run the program and called it a night. But I was bothered.
Today, I looked at the problem again. I knew I’d installed the most recent version of Python, Python 3.11, installed a bunch of packages (including the one that was missing last night) and made it the default version, but suddenly I was getting Python 3.9. Python 3.11 was available, complete with packages, but I had to explicitly specify that version or I’d get Python 3.9.
It took a bit of research before I figured out what was happening. When you open up a terminal window on a Unix system (or otherwise start a shell), the system runs several scripts before you get control. One of the things that gets done in those script is setting the path that the system uses to find a program if you don’t specify its exact location (so you can say python3 instead of /opt/homebrew/bin/python3, for example). In a fit of cleanliness and optimization, I’d moved the line that adds programs (including my preferred version of Python) which were installed by Homebrew to the path from .zshrc to .zshenv, and that was the cause of my woes. I fixed the problem by moving the line to yet another initialization script, .zprofile and all was well again.
In case anyone else is confused about the various initialization files that are called when a zsh shell is started, here’s a brief summary of what runs when, at least on Mac OS (thanks to harrymc for his clear explanation, which I’m simplifying further for Future Me). I’m omitting files that don’t seem to exist by default in Mac OS.
| File | Typical Purpose | Sourced when… |
|---|---|---|
.zshenv |
Set environment variables | always |
/etc/profile |
This is a SYSTEM-PROVIDED file, typically used to set the PATH. On Mac OS, calls /usr/libexec/path_helper which prepends its result to any PATH that’s already been set. |
when logging in (or starting a new terminal) |
.zprofile |
Runs any commands appropriate for a login shell, including PATH modifications | when logging in (or starting a new terminal) |
.zshrc |
Runs any commands to set up an interactive environment | when starting an interactive shell |
.zlogin |
Final commands to set up a login shell | when logging in (or starting a new terminal) |
Putting the Homebrew copy of Python into the PATH in .zshenv would have been OK if there hadn’t been a copy of Python in one of the system directories that /etc/profile put in the front of the PATH; after /etc/profile made its modifications, the Homebrew copy was hidden by the system copy.
So Future Me, please note: don’t set up the PATH in .zshenv. It’s OK to set up other environment variables, but not that one!