Core Code
Core Code Modern Dev Handbook
LINUX-PRODUCTION Contents
Linux Fundamentals for Production
Filesystem Layout (FHS) in Practice Users, Groups, and Service Accounts Permissions & ACLs (chmod/chown/setuid) Sudoers Safe Defaults Packages: apt/dnf, Repos, and Pinning Shell & Environment Hygiene Editing Configs Safely Time, Timezones, and NTP System Introspection Toolkit
Processes & Signals
Linux Process Model (PID, PPID, Fork/Exec) ps/top/htop: Finding What Hurts Signals in Practice (TERM/KILL/HUP) nice/renice and Scheduling Basics ulimits and Resource Limits Background Jobs (nohup, tmux, systemd-run) /proc and What It Tells You cgroups Basics (CPU/Memory Isolation) Zombies, Orphans, and Stuck Processes
systemd & Services
systemd Unit Types Overview Service Unit Best Practices Dependencies & Ordering (After/Wants/Requires) Restart Policies and Backoff Environment Files and Secrets Handling journald Basics (Reading, Filtering, Retention) Logging Practices with systemd Timers vs Cron Debugging systemd Services
Networking Basics for Production
ip, routes, and interfaces DNS Debugging (dig, resolvectl, hosts) Ports and Sockets with ss curl for HTTP Debugging TCP Basics (Handshake, States, Retries) Firewall Overview (nftables/iptables/ufw) NAT and Port Forwarding Concepts TLS and Certificates Basics Network Troubleshooting Playbook
Storage & Filesystems
Disk Layout and Partitioning (lsblk/fdisk) Filesystems (ext4/xfs) Basics Mounts and /etc/fstab Safety Disk Usage: df vs du (and Why They Differ) Inodes and “Disk Full” Mysteries Swap Strategy in Production Log Rotation (logrotate) Basics Temporary and Runtime Storage Hygiene Backup Storage Considerations
Security Basics
SSH Hardening in Production Users and Least Privilege in Production Firewall Minimum Rules for Production Linux Patching Strategy in Production Managing Secrets on Linux Hosts Linux Audit and Basic Logging in Production Fail2ban and Rate Limiting in Production SELinux and AppArmor Overview in Production Linux Security Baseline Checklist for Production
Observability on Linux
Linux Log Sources Map in Production journalctl Power Usage in Production Log Structure and Correlation in Production Metrics Hooks Overview in Production Health Checks and Readiness in Production Tracing Mindset for Linux and Production Systems Core Dumps Basics in Production Debug Packages and Symbols in Production Linux Observability Checklist for Production
Performance & Debugging
Load Average Explained for Production Debugging CPU Profiling Basics in Linux Production Memory Pressure Basics in Linux Production IO Bottlenecks and iostat in Production lsof and Open Files in Production Debugging Practical strace Usage in Production Sockets Debugging Tools in Linux Production dmesg and Kernel Signals for Production Debugging Production Debugging Workflow for Linux Systems
Deployment Patterns
Environment and Configuration Management in Production Release Directory Pattern for Safe Linux Deployments Zero-Downtime Restarts in Production Running Migrations Safely in Production Secrets Deploy Patterns in Production Backups and Restore Drills in Production Rollback Strategies for Production Deployments Maintenance Windows in Production Operations Production Deployment Checklist for Linux Systems
Production Checklists
Writing Effective Runbooks for Production Systems Failure Modes Mapping for Linux Production Systems Incident Response Basics for Linux Production Break-Glass Access in Production Capacity Planning Basics for Linux Production Systems Monitoring and Alerting Baseline for Linux Production Production Readiness Review for Linux Systems Configuration Drift in Linux Production Systems Go-Live Checklist

Time, Timezones, and NTP

Keep time correct with chrony/systemd-timesyncd; avoid timezone surprises.
On this page

Why Time Is a Production Dependency

Time is not just a display setting. It is a dependency. If server clocks drift, production systems fail in strange ways: TLS breaks, logs become useless, distributed systems disagree, and debugging turns into guesswork.

Common Symptoms of Bad Time

  • TLS/SSL errors (“certificate not yet valid”, “expired”)
  • Authentication failures (JWT, OAuth, session timeouts)
  • Replication lag and out-of-order events
  • Logs that appear out of sequence
  • Cache TTL behaving incorrectly

UTC vs Local Timezone

Production best practice: use UTC on servers unless you have a very strong reason not to. UTC avoids daylight saving problems and makes correlation across systems easier. Local timezone is fine for user-facing apps, but infrastructure should prefer UTC.

Check Current Time Settings

timedatectl
date

Look for:

  • Time zone
  • NTP service status
  • System clock synchronized: yes/no

NTP: How Servers Stay Correct

NTP keeps system time aligned with trusted sources. Most distros use one of:

  • systemd-timesyncd (simple, built-in)
  • chrony (more robust, preferred for many production setups)

systemd-timesyncd Basics

systemctl status systemd-timesyncd
timedatectl show-timesync

Configure servers: 

/etc/systemd/timesyncd.conf

chrony Basics

Chrony is often preferred for production because it handles unstable networks better. 

systemctl status chronyd
chronyc tracking
chronyc sources

Do Not Manually Set Time in Production

Manually changing the clock can break running systems (databases, caches, auth). If time is wrong, fix synchronization, not just the displayed value. Only use manual changes in emergency situations with a clear plan.

Time and TLS Certificates

Certificates rely on time validity windows. If your server clock is behind or ahead, clients can reject valid certificates. Always check time before assuming “certificate problem”.

Time in Logs and Incident Debugging

During incidents, logs are your truth source. If time is inconsistent:

  • Cross-service correlation becomes unreliable
  • Alert timelines become misleading
  • Root cause analysis becomes slower

Production Hard Rules

  • Enable NTP and verify sync
  • Prefer UTC on servers
  • Monitor clock sync status
  • Investigate time drift as a first-class incident cause

Quick Verification Checklist

timedatectl
journalctl -u systemd-timesyncd -n 50 --no-pager
chronyc tracking

Mental Model

Time is part of system correctness. If distributed systems are “not agreeing”, time is one of the first things to verify. Production engineering treats time sync like networking: invisible until it breaks everything.

Production Checklist

  • NTP enabled and synchronized
  • Timezone decision documented (prefer UTC)
  • Alerting on time sync failures
  • Time checked during TLS/auth incidents
System Introspection Toolkit →