Core Code
Core Code Modern Dev Handbook
LINUX-PRODUCTION Contents
Linux Fundamentals for Production
Filesystem Layout (FHS) in Practice Users, Groups, and Service Accounts Permissions & ACLs (chmod/chown/setuid) Sudoers Safe Defaults Packages: apt/dnf, Repos, and Pinning Shell & Environment Hygiene Editing Configs Safely Time, Timezones, and NTP System Introspection Toolkit
Processes & Signals
Linux Process Model (PID, PPID, Fork/Exec) ps/top/htop: Finding What Hurts Signals in Practice (TERM/KILL/HUP) nice/renice and Scheduling Basics ulimits and Resource Limits Background Jobs (nohup, tmux, systemd-run) /proc and What It Tells You cgroups Basics (CPU/Memory Isolation) Zombies, Orphans, and Stuck Processes
systemd & Services
systemd Unit Types Overview Service Unit Best Practices Dependencies & Ordering (After/Wants/Requires) Restart Policies and Backoff Environment Files and Secrets Handling journald Basics (Reading, Filtering, Retention) Logging Practices with systemd Timers vs Cron Debugging systemd Services
Networking Basics for Production
ip, routes, and interfaces DNS Debugging (dig, resolvectl, hosts) Ports and Sockets with ss curl for HTTP Debugging TCP Basics (Handshake, States, Retries) Firewall Overview (nftables/iptables/ufw) NAT and Port Forwarding Concepts TLS and Certificates Basics Network Troubleshooting Playbook
Storage & Filesystems
Disk Layout and Partitioning (lsblk/fdisk) Filesystems (ext4/xfs) Basics Mounts and /etc/fstab Safety Disk Usage: df vs du (and Why They Differ) Inodes and “Disk Full” Mysteries Swap Strategy in Production Log Rotation (logrotate) Basics Temporary and Runtime Storage Hygiene Backup Storage Considerations
Security Basics
SSH Hardening in Production Users and Least Privilege in Production Firewall Minimum Rules for Production Linux Patching Strategy in Production Managing Secrets on Linux Hosts Linux Audit and Basic Logging in Production Fail2ban and Rate Limiting in Production SELinux and AppArmor Overview in Production Linux Security Baseline Checklist for Production
Observability on Linux
Linux Log Sources Map in Production journalctl Power Usage in Production Log Structure and Correlation in Production Metrics Hooks Overview in Production Health Checks and Readiness in Production Tracing Mindset for Linux and Production Systems Core Dumps Basics in Production Debug Packages and Symbols in Production Linux Observability Checklist for Production
Performance & Debugging
Load Average Explained for Production Debugging CPU Profiling Basics in Linux Production Memory Pressure Basics in Linux Production IO Bottlenecks and iostat in Production lsof and Open Files in Production Debugging Practical strace Usage in Production Sockets Debugging Tools in Linux Production dmesg and Kernel Signals for Production Debugging Production Debugging Workflow for Linux Systems
Deployment Patterns
Environment and Configuration Management in Production Release Directory Pattern for Safe Linux Deployments Zero-Downtime Restarts in Production Running Migrations Safely in Production Secrets Deploy Patterns in Production Backups and Restore Drills in Production Rollback Strategies for Production Deployments Maintenance Windows in Production Operations Production Deployment Checklist for Linux Systems
Production Checklists
Writing Effective Runbooks for Production Systems Failure Modes Mapping for Linux Production Systems Incident Response Basics for Linux Production Break-Glass Access in Production Capacity Planning Basics for Linux Production Systems Monitoring and Alerting Baseline for Linux Production Production Readiness Review for Linux Systems Configuration Drift in Linux Production Systems Go-Live Checklist

ps/top/htop: Finding What Hurts

Quickly identify hot processes, CPU steal, memory pressure, and runaway tasks.
On this page

Why ps/top/htop Are Production Tools (Not Just Commands)

In production, you do not debug by guessing. You debug by observing. ps/top/htop are the fastest way to answer:

  • Which process is hurting the system?
  • Is it CPU, memory, I/O, or just “looks busy”?
  • Is the problem one process, many processes, or the host itself?

Start With a Snapshot: ps

ps is your “frozen frame”. It is ideal for quick, scriptable checks.

Top CPU Consumers

ps aux --sort=-%cpu | head -n 15

Top Memory Consumers

ps aux --sort=-%mem | head -n 15

Production hint: %MEM is not enough by itself. Also watch RSS (resident memory), because VIRT can be huge without real usage.

Understand VIRT vs RES vs SHR

  • VIRT: virtual address space (can be large, not always a problem)
  • RES (RSS): real RAM used (this hurts)
  • SHR: shared memory (libraries shared between processes)

If RES grows steadily, suspect memory leak or cache growth. If VIRT is huge but RES stable, it may be normal (mmap, shared libs, etc.).

Live View: top

top is your “moving picture”. Use it when load is changing and you need trends. Start top: 

top

Critical top Metrics

  • load average: 1m / 5m / 15m pressure trend
  • %Cpu: user vs system vs iowait
  • Mem: used vs available
  • Swap: activity is usually a bad sign in production

CPU Steal (Virtual Machines)

If you run on a VM, watch for “steal” time. High steal means the hypervisor is starving your VM of CPU, even if your app is fine. In top, look for st (steal).

Symptom:

  • App is slow
  • CPU seems high or inconsistent
  • Nothing obvious in process list

If steal is high, the fix may be infrastructure-level (bigger instance / less noisy neighbors), not code.

I/O Wait: The Silent Killer

If %Cpu shows high wa (iowait), the machine is waiting on disk. This is NOT a “CPU problem”. Restarting apps won’t fix slow storage.

Quick signal:

  • High load average
  • Many processes in D state
  • wa is high

Readability Upgrade: htop

htop is often easier for production work because:

  • Tree view (parent/child)
  • Interactive sorting
  • Per-core CPU view
  • Kill/renice actions with less typing

Install: 

sudo apt install htop
sudo dnf install htop

Threads vs Processes Confusion

Some applications use many threads. CPU usage can look confusing. In top you can enable thread view: 

top -H

Production use case:

  • Java apps
  • Some Node native modules
  • Databases

When ps/top Are Not Enough: pidstat

If you need per-process CPU, memory, and I/O over time, use pidstat (from sysstat). 

pidstat -u 1
pidstat -r 1
pidstat -d 1

This helps answer: is the process consistently heavy, or only spiking?

Common Production Mistakes

  • Confusing high load with high CPU (load can be I/O wait)
  • Chasing VIRT instead of RES
  • Ignoring steal time on VMs
  • Killing processes without checking why they are busy
  • Restarting apps when the real bottleneck is disk/network

Mental Model

Always classify the pain before acting:

  • CPU-bound: high user/system CPU, low iowait
  • I/O-bound: high iowait, D-state processes, high load
  • Memory pressure: low available memory, swap activity, OOM events
  • Infrastructure starvation: high steal, noisy neighbor signals

Once you classify correctly, your next tool choice becomes obvious.

Production Checklist

  • Use ps for snapshots, top/htop for trends
  • Interpret load together with CPU (especially iowait)
  • Track RES for real memory usage
  • Watch steal time on VMs
  • Use pidstat when you need time-series behavior
Signals in Practice (TERM/KILL/HUP) →