Core Code
Core Code Modern Dev Handbook
LINUX-PRODUCTION Contents
Linux Fundamentals for Production
Filesystem Layout (FHS) in Practice Users, Groups, and Service Accounts Permissions & ACLs (chmod/chown/setuid) Sudoers Safe Defaults Packages: apt/dnf, Repos, and Pinning Shell & Environment Hygiene Editing Configs Safely Time, Timezones, and NTP System Introspection Toolkit
Processes & Signals
Linux Process Model (PID, PPID, Fork/Exec) ps/top/htop: Finding What Hurts Signals in Practice (TERM/KILL/HUP) nice/renice and Scheduling Basics ulimits and Resource Limits Background Jobs (nohup, tmux, systemd-run) /proc and What It Tells You cgroups Basics (CPU/Memory Isolation) Zombies, Orphans, and Stuck Processes
systemd & Services
systemd Unit Types Overview Service Unit Best Practices Dependencies & Ordering (After/Wants/Requires) Restart Policies and Backoff Environment Files and Secrets Handling journald Basics (Reading, Filtering, Retention) Logging Practices with systemd Timers vs Cron Debugging systemd Services
Networking Basics for Production
ip, routes, and interfaces DNS Debugging (dig, resolvectl, hosts) Ports and Sockets with ss curl for HTTP Debugging TCP Basics (Handshake, States, Retries) Firewall Overview (nftables/iptables/ufw) NAT and Port Forwarding Concepts TLS and Certificates Basics Network Troubleshooting Playbook
Storage & Filesystems
Disk Layout and Partitioning (lsblk/fdisk) Filesystems (ext4/xfs) Basics Mounts and /etc/fstab Safety Disk Usage: df vs du (and Why They Differ) Inodes and “Disk Full” Mysteries Swap Strategy in Production Log Rotation (logrotate) Basics Temporary and Runtime Storage Hygiene Backup Storage Considerations
Security Basics
SSH Hardening in Production Users and Least Privilege in Production Firewall Minimum Rules for Production Linux Patching Strategy in Production Managing Secrets on Linux Hosts Linux Audit and Basic Logging in Production Fail2ban and Rate Limiting in Production SELinux and AppArmor Overview in Production Linux Security Baseline Checklist for Production
Observability on Linux
Linux Log Sources Map in Production journalctl Power Usage in Production Log Structure and Correlation in Production Metrics Hooks Overview in Production Health Checks and Readiness in Production Tracing Mindset for Linux and Production Systems Core Dumps Basics in Production Debug Packages and Symbols in Production Linux Observability Checklist for Production
Performance & Debugging
Load Average Explained for Production Debugging CPU Profiling Basics in Linux Production Memory Pressure Basics in Linux Production IO Bottlenecks and iostat in Production lsof and Open Files in Production Debugging Practical strace Usage in Production Sockets Debugging Tools in Linux Production dmesg and Kernel Signals for Production Debugging Production Debugging Workflow for Linux Systems
Deployment Patterns
Environment and Configuration Management in Production Release Directory Pattern for Safe Linux Deployments Zero-Downtime Restarts in Production Running Migrations Safely in Production Secrets Deploy Patterns in Production Backups and Restore Drills in Production Rollback Strategies for Production Deployments Maintenance Windows in Production Operations Production Deployment Checklist for Linux Systems
Production Checklists
Writing Effective Runbooks for Production Systems Failure Modes Mapping for Linux Production Systems Incident Response Basics for Linux Production Break-Glass Access in Production Capacity Planning Basics for Linux Production Systems Monitoring and Alerting Baseline for Linux Production Production Readiness Review for Linux Systems Configuration Drift in Linux Production Systems Go-Live Checklist

Zombies, Orphans, and Stuck Processes

Recognize zombie/orphan patterns and resolve stuck shutdowns without panic.
On this page

Why Zombies and Orphans Matter in Production

Zombie and orphan processes are not everyday concerns — until they are. When process trees behave unexpectedly, production systems become unstable, unpredictable, or impossible to debug. Understanding these states helps you diagnose supervision and shutdown failures.

What Is a Zombie Process?

A zombie process is a process that has finished execution, but its parent has not yet collected its exit status. The kernel keeps a small entry in the process table.

Zombies:

  • Do NOT consume CPU
  • Do NOT consume memory
  • DO consume process table entries

How Zombies Appear

When a child exits, the parent must call wait() to collect its exit code. If the parent fails to do so, the child remains as zombie. This usually indicates a bug in the parent process.

How to Detect Zombies

ps aux | awk '$8 ~ /Z/ {print}'

Or: 

ps -eo pid,ppid,stat,comm | grep Z

If you see many zombies under one parent, that parent is mismanaging children.

What Is an Orphan Process?

An orphan process is one whose parent has exited. The kernel reassigns it to PID 1 (systemd). This is normal in some cases, but can indicate broken supervision. 

ps -eo pid,ppid,comm | awk '$2==1 {print}'

Production Scenario: Worker Crash

Common pattern:

  • Master process spawns workers
  • Master crashes
  • Workers become orphaned
  • Systemd restarts master
  • Now you have old orphaned workers + new workers

This can cause port conflicts or duplicated background jobs.

Zombie Accumulation Risk

If zombies accumulate:

  • Process table may fill
  • New forks may fail
  • You may see: “Resource temporarily unavailable”

Fix is not killing zombies — you must fix or restart the parent.

How to Identify the Parent Causing Zombies

ps -eo pid,ppid,stat,comm | grep Z

Take the PPID and inspect that process: 

ps -fp <ppid>

If it is stuck or misbehaving, restart the service cleanly via systemd.

Stuck Processes (D-State)

A process in D (uninterruptible sleep) is waiting on I/O. It cannot be killed even with SIGKILL. 

ps -eo pid,stat,comm | grep D

If many processes are in D state:

  • Suspect disk problems
  • Suspect NFS issues
  • Investigate storage before restarting services

Restart Strategy

If zombie/orphan issues exist:

  • Restart via systemctl
  • Do NOT manually kill individual children randomly
  • Confirm process tree is clean after restart
systemctl restart myapp
ps -ef --forest | grep myapp

Common Production Mistakes

  • Killing zombie processes (you cannot)
  • Ignoring parent process bugs
  • Restarting random PIDs instead of supervised unit
  • Ignoring D-state processes and blaming CPU

Mental Model

Processes live in trees. Supervisors manage trees. Zombies mean broken parenting. Orphans mean broken supervision. D-state means I/O pain. Production debugging is about understanding which of these is happening.

Production Checklist

  • Check for Z state during strange process growth
  • Inspect PPID to find misbehaving parent
  • Restart via systemd, not raw kill
  • Investigate D-state before forcing kills
  • Validate process tree after recovery