Docker Debugging — exec, logs, inspect, and Why Your Container Won't Start

Your container exits immediately with code 1. Or it starts but the application is unreachable. Or it runs for an hour and then OOMs. Docker gives you powerful debugging tools, but most people only know docker logs. Here is the full toolkit for diagnosing container problems.

Container Lifecycle States

Before debugging, understand where your container is in its lifecycle:

docker ps -a --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"

Status	Meaning	What to Check
Created	Container exists but never started	Start command, entrypoint
Up 5 minutes	Running	Logs, connectivity
Up 5 minutes (healthy)	Running, healthcheck passing	Everything looks good
Up 5 minutes (unhealthy)	Running, healthcheck failing	Health endpoint, resource limits
Exited (0)	Process completed successfully	Expected? Check if CMD is correct
Exited (1)	Process crashed with error	Logs, environment variables
Exited (137)	Killed (OOM or docker stop)	Memory limits, docker events
Exited (139)	Segfault	Binary compatibility, base image

Exit code 137 deserves special attention. It means the process received SIGKILL — usually because it exceeded its memory limit. Exit code 143 is SIGTERM, which is a graceful shutdown from docker stop.

docker logs: Your First Stop

Always start with logs. Most container problems reveal themselves immediately in the output.

# View all logs
docker logs myapp

# Follow logs in real-time (like tail -f)
docker logs -f myapp

# Last 50 lines only
docker logs --tail 50 myapp

# Logs with timestamps
docker logs -t myapp

# Logs since a specific time
docker logs --since "2025-04-28T10:00:00" myapp

# Logs from the last 5 minutes
docker logs --since 5m myapp

# Combine: follow, timestamps, last 20 lines
docker logs -f -t --tail 20 myapp

If the container exited, docker logs still works on stopped containers. This is critical — the logs persist until the container is removed with docker rm.

docker exec: Get Inside a Running Container

When logs are not enough, get a shell inside the container and poke around.

# Interactive shell
docker exec -it myapp /bin/sh
# or if bash is available:
docker exec -it myapp /bin/bash

# Run a specific command without entering the shell
docker exec myapp cat /app/config.json

# Run as a specific user
docker exec -u root myapp whoami

# Check environment variables inside the container
docker exec myapp env

# Check network connectivity from inside
docker exec myapp ping -c 2 database
docker exec myapp wget -qO- http://localhost:3000/health

# Check running processes
docker exec myapp ps aux

# Check disk usage
docker exec myapp df -h

What if the container has no shell? (distroless images, scratch-based images)

# Use a debug container that shares the PID namespace
docker run -it --rm --pid=container:myapp --network=container:myapp \
  nicolaka/netshoot /bin/bash

# Now you can inspect the target container's network and processes

docker inspect: The Configuration Bible

docker inspect dumps every piece of configuration — mounts, network settings, environment variables, restart policies, and more.

# Full JSON dump
docker inspect myapp

# Get specific fields with Go templates
# Container IP address
docker inspect myapp --format '{{.NetworkSettings.IPAddress}}'

# Mounted volumes
docker inspect myapp --format '{{json .Mounts}}' | python3 -m json.tool

# Environment variables
docker inspect myapp --format '{{json .Config.Env}}' | python3 -m json.tool

# Restart count and last exit code
docker inspect myapp --format 'Restarts: {{.RestartCount}}, ExitCode: {{.State.ExitCode}}'

# Health check status
docker inspect myapp --format '{{json .State.Health}}' | python3 -m json.tool

# Port bindings
docker inspect myapp --format '{{json .NetworkSettings.Ports}}' | python3 -m json.tool

docker events: Watch Everything in Real-Time

docker events is like an audit log for all Docker operations. Invaluable when you need to see the sequence of events leading up to a failure.

# Watch all events in real-time
docker events

# Filter events for a specific container
docker events --filter container=myapp

# Filter by event type
docker events --filter type=container --filter event=die

# Events in the last hour
docker events --since 1h

# Output as JSON for parsing
docker events --format '{{json .}}' --filter container=myapp

When a container dies and restarts repeatedly, docker events shows you the exact timeline — when it started, when it died, the exit code, and when it restarted.

Common Startup Failures and Fixes

Wrong CMD or ENTRYPOINT

# Symptom: Container exits immediately with code 0 or 127
docker logs myapp
# /bin/sh: mycommand: not found

# Debug: Check what CMD is set to
docker inspect myapp --format '{{json .Config.Cmd}}'
docker inspect myapp --format '{{json .Config.Entrypoint}}'

# Fix: Override CMD to test
docker run -it --entrypoint /bin/sh myapp:latest
# Now you can look around and find the right executable path

Missing Environment Variables

# Symptom: Application error, "undefined", connection refused to database
docker logs myapp
# Error: DATABASE_URL is not set

# Debug: Check what env vars are set
docker inspect myapp --format '{{json .Config.Env}}' | python3 -m json.tool

# Fix: Run with the missing variable
docker run -e DATABASE_URL=postgresql://postgres:secret@db:5432/myapp myapp:latest

Port Conflicts

# Symptom: "Bind for 0.0.0.0:3000 failed: port is already allocated"
# Debug: Find what is using the port
docker ps --format "table {{.Names}}\t{{.Ports}}" | grep 3000

# On the host
# Linux/Mac:
lsof -i :3000
# or
ss -tlnp | grep 3000

# Fix: Use a different host port
docker run -p 3001:3000 myapp:latest

Permission Denied

# Symptom: EACCES, Permission denied
docker logs myapp
# Error: EACCES: permission denied, open '/app/data/config.json'

# Debug: Check what user the process runs as
docker exec myapp whoami
docker exec myapp id
docker exec myapp ls -la /app/data/

# Fix: Correct ownership or run as the right user
docker run -u root myapp:latest chown -R appuser:appuser /app/data

Debugging Crashed Containers

The container exited and you cannot exec into it. But you can still extract information.

# Copy files out of a stopped container
docker cp myapp:/app/logs/error.log ./error.log
docker cp myapp:/app/config.json ./config-dump.json

# Create a new image from the stopped container's filesystem
docker commit myapp myapp-debug:latest

# Now run the debug image with a shell
docker run -it --entrypoint /bin/sh myapp-debug:latest
# Inspect the filesystem at the moment of crash

The docker commit trick is powerful — it snapshots the container's filesystem including any files created at runtime, temp files, core dumps, and log files that were not on a mounted volume.

docker stats: Resource Monitoring

Is your container running out of memory? Using too much CPU? docker stats gives you a live view.

# Live resource usage for all containers
docker stats

# Specific containers
docker stats myapp db redis

# One-shot (no streaming) for scripting
docker stats --no-stream --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.MemPerc}}\t{{.NetIO}}"

Metric	Warning Sign	Action
MEM USAGE near limit	Container will be OOM-killed	Increase --memory limit or optimize app
CPU at 100% constantly	Application bottleneck	Profile app, add CPU limit
NET I/O very high	Possible network loop	Check application logic
BLOCK I/O very high	Disk thrashing	Check logging volume, add tmpfs

# Set resource limits to prevent runaway containers
docker run -d --name api \
  --memory=512m \
  --cpus=1.0 \
  --restart=on-failure:5 \
  myapp:latest

Troubleshooting Flowchart

When a container will not start or keeps crashing, follow this sequence:

# Step 1: Check the current state
docker ps -a | grep myapp

# Step 2: Read the logs
docker logs --tail 100 myapp

# Step 3: Inspect the configuration
docker inspect myapp --format '{{json .State}}' | python3 -m json.tool

# Step 4: Check events timeline
docker events --filter container=myapp --since 1h

# Step 5: If running, check resources
docker stats --no-stream myapp

# Step 6: If running, exec in and investigate
docker exec -it myapp /bin/sh

# Step 7: If crashed, extract the filesystem
docker cp myapp:/app/logs/ ./debug-logs/

# Step 8: If all else fails, override entrypoint
docker run -it --entrypoint /bin/sh myapp:latest

Wrapping Up

Docker debugging is a skill that separates beginners from professionals. Start with docker logs, move to docker inspect for configuration issues, use docker exec to investigate live containers, and keep docker events running when you need to catch intermittent failures. For crashed containers, docker cp and docker commit let you perform a post-mortem without losing evidence.

With this post, we have covered the core Docker fundamentals — from containers and images to volumes, networking, Compose, Dockerfiles, registries, and debugging. The next step is to take these skills into orchestration territory with Kubernetes, where you will manage fleets of containers across clusters of machines.