Processes and the environment
Tell processes apart, read what they inherited, and use exit status as evidence.
A process is a running program plus its context
When the shell starts a program, the kernel creates a process. It has a numeric PID, a parent PID, a user and group identity, a working directory, a set of open files (including the three standard streams), and an environment. Two copies of the same program are two processes with different PIDs and possibly very different contexts.
That context is what you investigate. ps -o pid,ppid,user,etime,args -p <pid> shows identity and age. ls -l /proc/<pid>/cwd shows the working directory, and /proc/<pid>/environ shows the environment it started with. ps -ef --forest draws the parent-child tree, which shows whether a process was started by a service manager, by cron, or by someone's shell.
Never act on a name fragment
pgrep waybill or ps aux | grep waybill is a search, not an identification. It can match a deploy helper, an editor with the file open, or the grep itself. Before signalling or reporting a process, confirm it by at least two independent facts. Useful ones are the full command line, the owner, the parent, and the listening socket. ss -ltnp maps listening ports to PIDs, which is usually the decisive fact for a service. The Linux Administration track builds signals on this. Sending a signal to the wrong PID is the classic way to turn a slow incident into an outage.
Inheritance flows one way
A child process receives a copy of its parent's environment at start. Afterwards they are independent. Three operational consequences follow:
- Your shell is not the service's shell. Variables you exported, a
PATHyour profile set, or a directory youcd'd into do not exist for a cron job or a service. A command that "works by hand" but fails when scheduled is almost always missing context its author's shell provided. - Children cannot change parents. A script that
cds or exports affects only itself.source script.shruns it in the current shell when that is what you want. - Startup files matter. Login shells, interactive shells, and non-interactive shells read different files. Something set in
.bashrcmay be absent in a script.
env prints the environment, and env -i PATH=/usr/bin:/bin ./job runs a command with an almost empty one. That is a quick way to reproduce how a scheduled job sees the world.
PATH chooses the program
When you type a bare name, the shell checks aliases, functions, and built-ins, then searches each directory in PATH from left to right and runs the first match. type -a name lists every candidate. A stale copy earlier in PATH, or a missing directory in a service's minimal PATH, explains many "wrong version" and "command not found" reports. Scripts that run unattended should call programs by absolute path or set PATH explicitly.
Exit status is the first piece of evidence
Every process ends with a status. Zero means success, and the meaning of anything else is defined by the program. $? holds the last status. && runs the next command only on success and || only on failure. In scripts, set -e stops at the first failure and set -o pipefail makes a pipeline fail if any stage fails, not only the last. Read the EXIT STATUS section of a tool's manual before relying on its codes. grep's "1 means no match" has fooled many alerting scripts.
Key terms
- PID
- Process ID, a number the kernel assigns. It is the only unambiguous handle on a running process.
- Parent process
- The process that started this one. Children inherit its environment, working directory, and open files.
- Environment variable
- A
NAME=valuepair copied into each child process at start. Changes in a child never flow back to the parent. - Exit status
- A number from 0 to 255 a process returns. 0 means success. Anything else is a program-defined failure.
Read further
- The Linux Command Line, Chapter "Processes" (Free PDF (CC BY-NC-ND))
Howps,top, and job control show processes, and what the process states mean. Read the section on signals but leave the details for the Linux Administration track. - The Linux Command Line, Chapter "The Environment" (Free PDF (CC BY-NC-ND))
The difference between shell variables and environment variables, whatexportdoes, and which startup files a login shell and a non-login shell read. - How Linux Works, 3rd edition, Ch. 8, "A Closer Look at Processes and Resource Utilization" (Purchase)
The first sections on tracking processes withpsandlsof. Note how open files and the working directory belong to a process.