Core Code
Core Code Modern Dev Handbook
LINUX-PRODUCTION Contents
Linux Fundamentals for Production
Filesystem Layout (FHS) in Practice Users, Groups, and Service Accounts Permissions & ACLs (chmod/chown/setuid) Sudoers Safe Defaults Packages: apt/dnf, Repos, and Pinning Shell & Environment Hygiene Editing Configs Safely Time, Timezones, and NTP System Introspection Toolkit
Processes & Signals
Linux Process Model (PID, PPID, Fork/Exec) ps/top/htop: Finding What Hurts Signals in Practice (TERM/KILL/HUP) nice/renice and Scheduling Basics ulimits and Resource Limits Background Jobs (nohup, tmux, systemd-run) /proc and What It Tells You cgroups Basics (CPU/Memory Isolation) Zombies, Orphans, and Stuck Processes
systemd & Services
systemd Unit Types Overview Service Unit Best Practices Dependencies & Ordering (After/Wants/Requires) Restart Policies and Backoff Environment Files and Secrets Handling journald Basics (Reading, Filtering, Retention) Logging Practices with systemd Timers vs Cron Debugging systemd Services
Networking Basics for Production
ip, routes, and interfaces DNS Debugging (dig, resolvectl, hosts) Ports and Sockets with ss curl for HTTP Debugging TCP Basics (Handshake, States, Retries) Firewall Overview (nftables/iptables/ufw) NAT and Port Forwarding Concepts TLS and Certificates Basics Network Troubleshooting Playbook
Storage & Filesystems
Disk Layout and Partitioning (lsblk/fdisk) Filesystems (ext4/xfs) Basics Mounts and /etc/fstab Safety Disk Usage: df vs du (and Why They Differ) Inodes and “Disk Full” Mysteries Swap Strategy in Production Log Rotation (logrotate) Basics Temporary and Runtime Storage Hygiene Backup Storage Considerations
Security Basics
SSH Hardening in Production Users and Least Privilege in Production Firewall Minimum Rules for Production Linux Patching Strategy in Production Managing Secrets on Linux Hosts Linux Audit and Basic Logging in Production Fail2ban and Rate Limiting in Production SELinux and AppArmor Overview in Production Linux Security Baseline Checklist for Production
Observability on Linux
Linux Log Sources Map in Production journalctl Power Usage in Production Log Structure and Correlation in Production Metrics Hooks Overview in Production Health Checks and Readiness in Production Tracing Mindset for Linux and Production Systems Core Dumps Basics in Production Debug Packages and Symbols in Production Linux Observability Checklist for Production
Performance & Debugging
Load Average Explained for Production Debugging CPU Profiling Basics in Linux Production Memory Pressure Basics in Linux Production IO Bottlenecks and iostat in Production lsof and Open Files in Production Debugging Practical strace Usage in Production Sockets Debugging Tools in Linux Production dmesg and Kernel Signals for Production Debugging Production Debugging Workflow for Linux Systems
Deployment Patterns
Environment and Configuration Management in Production Release Directory Pattern for Safe Linux Deployments Zero-Downtime Restarts in Production Running Migrations Safely in Production Secrets Deploy Patterns in Production Backups and Restore Drills in Production Rollback Strategies for Production Deployments Maintenance Windows in Production Operations Production Deployment Checklist for Linux Systems
Production Checklists
Writing Effective Runbooks for Production Systems Failure Modes Mapping for Linux Production Systems Incident Response Basics for Linux Production Break-Glass Access in Production Capacity Planning Basics for Linux Production Systems Monitoring and Alerting Baseline for Linux Production Production Readiness Review for Linux Systems Configuration Drift in Linux Production Systems Go-Live Checklist

Managing Secrets on Linux Hosts

Secure secrets on production Linux hosts: eliminate plaintext exposure, control file permissions, prevent environment leaks, audit access, and reduce blast radius from host compromise.
On this page

Why Secrets on Host Are a High-Risk Area

API keys, database passwords, cloud credentials, TLS private keys — once a host is compromised, attackers look for secrets first. Poor secret handling turns a single host breach into full infrastructure takeover.

Symptom

  • Plaintext passwords inside config files
  • .env files readable by all users
  • Credentials committed to backup archives
  • Environment variables leaking through process inspection
  • Application running as root exposing private keys

Root Cause

  • No centralized secret management
  • Weak file permission discipline
  • Service accounts over-privileged
  • Secrets stored in home directories
  • No separation between runtime user and admin user

Investigation

Find world-readable sensitive files:

sudo find / -type f -perm -004 -name "*.env" 2>/dev/null

Check permissions of critical directories:

ls -ld /etc /var/www /home

Inspect application config permissions:

ls -l /var/www/app/.env

Check process environment variables (current user):

cat /proc/$(pgrep -f app_process_name)/environ | tr "" "
"

List private keys on system:

sudo find / -type f -name "id_rsa" 2>/dev/null

Mitigation

1) Restrict File Permissions

Secrets must not be world-readable.

chmod 600 /var/www/app/.env
chown appuser:appuser /var/www/app/.env

Ensure .ssh directory is restricted:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

2) Run Applications as Dedicated Users

Never run application processes as root.

ps aux | grep app_process_name

If running as root, fix service configuration (systemd example):

User=appuser
Group=appuser

3) Avoid Storing Secrets in User Home Directories

Use restricted system paths such as:

  • /etc/appname/
  • /run/secrets/

4) Protect TLS Private Keys

ls -l /etc/ssl/private

Keys should be owned by root and readable only by required services.

5) Reduce Environment Variable Exposure

Environment variables are visible to processes with sufficient privileges.

Avoid exporting secrets globally in:

  • /etc/profile
  • ~/.bashrc
  • system-wide environment files

Hardening Strategy

  • Principle of minimum exposure: only the running service should read its secrets
  • Separate runtime identity: isolate service accounts
  • Rotate credentials regularly
  • Use external secret managers (cloud-native or vault systems)
  • Audit file permissions regularly
  • Prevent backups from including plaintext secrets unintentionally

Common Escalation Pattern

  • Attacker gains low-privilege shell
  • Searches for readable .env files
  • Extracts database password
  • Connects to database and dumps user data
  • Uses API keys to access cloud infrastructure

Secret hygiene determines whether compromise stays local or spreads.

Verification Checklist

sudo find /etc /var/www -type f -perm -004 2>/dev/null
ps aux
ls -l /etc/ssl/private
  • No world-readable secret files
  • No services running as root unnecessarily
  • Secret files owned by correct service account
  • No plaintext secrets in home directories

Why This Matters in Real Infrastructure

Secrets are the bridge between one machine and the rest of your infrastructure. A single exposed credential often leads to database compromise, cloud account takeover, or lateral movement. Production systems assume compromise can happen — secret management ensures the damage stops at one boundary.

Linux Audit and Basic Logging in Production →