Docker ps images exec logs Explained

Quick take: Docker command-line interface provides powerful tools for managing containerized applications.

Docker ps, images, exec, and logs: Complete Command Reference Guide

Docker command-line interface provides powerful tools for managing containerized applications. Among the most frequently used commands are docker ps, docker images, docker exec, and docker logs. These commands form the backbone of container lifecycle management and troubleshooting in production environments. This article provides an exhaustive exploration of each command, their options, practical applications, and real-world scenarios you will encounter as a DevOps engineer.

Understanding Docker Container Management Fundamentals

Before diving into specific commands, it is essential to understand the Docker architecture and container lifecycle. Docker containers are lightweight, standalone, executable packages containing everything needed to run an application. Unlike virtual machines, containers share the host OS kernel while maintaining isolation through namespaces and control groups. When you deploy containers, you need visibility into their status, resource usage, and behavior. This is where the core Docker commands become indispensable for daily operations.

The container lifecycle follows a predictable path: creation, running, paused, stopped, and removed states. Each state transition requires specific commands and produces specific outcomes. Understanding this lifecycle is crucial before mastering the individual commands covered in this article. DevOps teams must be able to query container status at any point, inspect running processes, retrieve logs for debugging, and execute maintenance tasks within containers without stopping them.

Docker ps: Listing and Managing Containers

Basic Docker ps Syntax and Output

The docker ps command is your primary tool for querying container status. In its simplest form, it lists all currently running containers with essential information about each. The command displays container ID, image name, command being executed, when it was created, its current status, exposed ports, and assigned name.

docker ps

This basic invocation shows only running containers. The output typically looks like this:

CONTAINER ID   IMAGE           COMMAND                  CREATED        STATUS        PORTS                   NAMES
a1b2c3d4e5f6   nginx:latest    "nginx -g 'daemon of"   2 hours ago     Up 2 hours     0.0.0.0:80->80/tcp     web-server
f6e5d4c3b2a1   postgres:13     "docker-entrypoint.s"   5 hours ago     Up 5 hours     5432/tcp                db-primary

Essential Docker ps Flags and Options

Docker ps supports numerous flags that modify its behavior and output format. The most critical flag is --all or -a, which displays both running and stopped containers. This is essential for understanding the full history of containers on your system, especially when troubleshooting failed deployments or investigating why containers are not starting.

docker ps -a

When you run this command, you see containers in various states including stopped, exited, and created states. This reveals containers that may have crashed or been intentionally stopped. The status column shows exactly what happened, with exit codes for stopped containers indicating the reason for termination.

The --quiet or -q flag returns only container IDs without any additional information. This is particularly useful when scripting, as container IDs can be piped to other commands for bulk operations.

docker ps -q

You can use this to stop all running containers at once:

docker stop $(docker ps -q)

The --format flag allows custom output formatting using Go template syntax. This enables you to extract exactly the information you need for monitoring, logging, or integration with other tools. For example, to display only container names and their IP addresses:

docker ps --format "table {{.Names}}\t{{.ID}}\t{{.Status}}"

Advanced filtering with --filter enables you to query containers based on specific criteria like label, status, image name, or exit code. This is invaluable in complex environments with numerous containers.

docker ps -a --filter "status=exited"

This command shows only containers that have exited, helping you identify problematic deployments quickly.

The --size flag displays the size of each container on disk, which helps identify containers consuming excessive storage. This is important for capacity planning and troubleshooting disk space issues.

docker ps --size

Advanced Filtering Scenarios

In production environments with hundreds of containers, basic listing is insufficient. Docker supports filtering by multiple criteria simultaneously. To find all containers running a specific image that exited within the last day:

docker ps -a --filter "ancestor=myapp:latest" --filter "status=exited"

Label-based filtering is particularly powerful for container orchestration. If you have labeled containers with environment tags, you can list all production containers:

docker ps --filter "label=environment=production"

You can combine formatting with filtering to create monitoring dashboards or generate reports. For example, listing all containers with their resource limits:

docker ps --format "table {{.Names}}\t{{.Image}}\t{{.Status}}" --filter "status=running"

Docker images: Managing Container Images

Understanding Docker Image Structure

Before using docker images effectively, understand that Docker images are layered filesystems. Each instruction in a Dockerfile creates a new layer, and images are composed of multiple read-only layers stacked together. The docker images command displays information about these image layers and their metadata.

Basic Docker images Command

The docker images command lists all images stored on your system, showing repository, tag, image ID, creation time, and size. This is essential for understanding what images are available for deployment.

docker images

Output appears as follows:

REPOSITORY      TAG       IMAGE ID       CREATED        SIZE
nginx           latest    a8efb6b4f8b5   2 weeks ago     142MB
postgres        13        e4f2e3c1d4a5   3 weeks ago     314MB
python          3.9       f6c5b4a3d2e1   1 month ago     886MB
myapp           v1.0      a1b2c3d4e5f6   5 days ago      256MB

Filtering and Formatting Docker Images

The --filter flag for docker images works similarly to docker ps. You can filter images by dangling status, which identifies images not tagged and not used by any container. These are ideal candidates for cleanup to free disk space.

docker images --filter "dangling=true"

Dangling images accumulate quickly during development and CI/CD pipelines. To remove all dangling images safely:

docker image prune

You can filter images by reference, showing only images matching a specific pattern:

docker images --filter "reference=nginx*"

Custom formatting allows you to extract specific information needed for your workflow:

docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}"

Managing Multiple Image Versions

In production environments, you often have multiple versions of the same application image. Understanding image relationships is crucial. The --digests flag shows the SHA256 digest of each image, which uniquely identifies image content independent of tags.

docker images --digests

This is important because two images with the same name and tag might have different digests if rebuilds occurred. Digests ensure you are deploying exactly the version you intended, regardless of tag changes.

To see only images with specific tag patterns, combine grep with docker images output:

docker images | grep -E "(latest|prod|staging)"

Understanding Image Size and Disk Usage

Image size directly impacts deployment speed, storage costs, and container startup time. Large images are particularly problematic in microservices architectures where you might have dozens of containers. The docker images output shows the virtual size, which is the amount of disk space used by the image.

To understand actual disk usage including shared layers:

docker system df

This command provides a summary of Docker's disk usage including images, containers, and volumes. It shows how much space could be freed by removing unused resources.

For development teams, optimizing image size is critical. Large base images impact both build time and deployment speed. Consider using minimal base images like alpine for non-critical components.

Docker exec: Executing Commands in Running Containers

Fundamentals of Docker exec

The docker exec command allows you to run commands inside a running container without stopping it or establishing a new session from scratch. This is invaluable for debugging, maintenance, and administration tasks. Unlike docker run which creates a new container, exec operates within an existing container's namespace and filesystem.

Basic syntax requires the container identifier (name or ID) and the command to execute:

docker exec container_name command

Interactive Execution and Shell Access

To debug container issues, you typically need an interactive shell session. Use the -i flag for interactive mode and -t for a pseudo-terminal. Combined as -it, this gives you a functioning shell prompt inside the container.

docker exec -it web-server /bin/bash

This command drops you into a bash shell inside the web-server container with full interactive capabilities. You can execute commands just as if you were inside the container directly. This is your primary troubleshooting mechanism for understanding container state, checking logs within the container, inspecting mounted volumes, or verifying configuration files.

For containers with only sh available (common in alpine-based images), use sh instead of bash:

docker exec -it app-container /bin/sh

Common Troubleshooting Tasks with Docker exec

When a containerized application is misbehaving, docker exec allows you to investigate directly. For example, to check if a network service is responding correctly:

docker exec web-server curl http://localhost:8080/health

To verify environment variables inside a container match your expectations:

docker exec app-container env | grep IMPORTANT_VAR

To check if required files exist at expected paths:

docker exec database-container ls -la /var/lib/postgresql/data

For database containers, you can execute database-specific commands directly:

docker exec db-primary psql -U postgres -c "SELECT version();"

This query verifies PostgreSQL is running and returns the version without entering an interactive session.

Running Scripts and Complex Commands

Docker exec can execute complex commands by using sh -c to invoke a shell interpreter:

docker exec web-server sh -c "ps aux | grep nginx"

This approach allows piping, redirection, and conditional execution within the container. You can chain multiple commands together:

docker exec app-container sh -c "cd /app && npm test && npm build"

For copying files from container to host or host to container during runtime, docker exec with redirection works effectively:

docker exec app-container cat /app/config.json > local-config.json

User Specification and Permissions

By default, docker exec runs commands as root inside the container. For security-sensitive operations, specify a different user with the --user flag:

docker exec --user appuser web-server whoami

This is particularly important when containers run with specific unprivileged users for security purposes. Running as the correct user ensures proper permissions and reproduces production conditions.

To determine what user the container normally runs as, check the Dockerfile or inspect the container configuration.

Working Directory Specification

The --workdir flag sets the working directory for the command execution:

docker exec --workdir /app container-name ls -la

This is useful when you need to execute commands relative to a specific directory without explicitly changing directories first.

Advanced Execution Scenarios

For containers running as root but needing to preserve the original user, use the --user flag to maintain consistency with production deployments:

docker exec --user 1000:1000 container-name id

Running privileged commands within containers requires understanding the container's capabilities. By default, docker exec inherits the container's privilege level. If you need specific privileges, check the container configuration before attempting operations.

For long-running background operations, you can detach from output:

docker exec -d container-name long-running-process &

The -d flag runs the command detached from your terminal, useful for asynchronous operations.

Docker logs: Analyzing Container Output

Understanding Container Logging Architecture

Docker captures container output (stdout and stderr) and stores it in logs accessible through the docker logs command. This is the primary mechanism for understanding container behavior, diagnosing problems, and monitoring application health. Different logging drivers can be configured to send logs to various backends, but the docker logs command accesses the locally stored logs.

Basic Docker logs Usage

The simplest docker logs invocation displays all captured output from container startup:

docker logs container_name

Output shows all stdout and stderr from the container, displayed in chronological order. For containers that have been running for extended periods, this can be substantial output.

Following Live Log Output

During troubleshooting and development, you need to watch logs as they occur. The --follow or -f flag mirrors tail -f functionality, displaying new log entries as they are generated:

docker logs -f web-server

Press Ctrl+C to exit the follow mode. This is essential when you are debugging live issues, testing code changes, or monitoring deployment progress in real-time.

Limiting Log Output

For containers running for days or weeks, logs become very large. The --tail flag limits output to the most recent N lines:

docker logs --tail 100 web-server

This shows only the last 100 lines, which is usually sufficient for diagnosing recent issues without overwhelming output.

Combining tail with follow shows recent logs plus new entries as they arrive:

docker logs --tail 50 -f app-container

Timestamp-Based Log Filtering

The --timestamps or -t flag adds timestamps to each log line, essential for understanding when events occurred:

docker logs -t web-server

Timestamps help correlate container events with external system events and identify timing-related issues.

Filter logs to specific time windows using --since and --until flags with duration or timestamp format:

docker logs --since 2024-01-15T10:00:00 --until 2024-01-15T11:00:00 app-server

You can also use relative durations:

docker logs --since 30m web-server

This shows only logs from the last 30 minutes, useful for examining recent activity without scrolling through days of logs.

Combining Docker logs with System Monitoring

In production environments, individual container logs must be collected and aggregated. Most teams use centralized logging solutions like ELK stack, Splunk, or cloud-native solutions. However, understanding local logs remains essential for troubleshooting.

To search logs for specific error patterns:

docker logs web-server | grep -i error

To count error occurrences:

docker logs app-container 2>&1 | grep -c "ERROR"

Extracting specific log entries for incident investigation:

docker logs db-server | grep "connection refused"

Analyzing Exit Status and Error Logs

When containers exit, their logs remain accessible until the container is removed. This is crucial for post-mortem debugging of failed containers:

docker logs failed-container

Combined with docker ps -a to find stopped containers, you can reconstruct what happened before failure.

To check the exit code of a stopped container:

docker inspect --format='{{.State.ExitCode}}' container_name

Exit code 0 indicates successful completion, while non-zero codes indicate errors. Common codes include 1 (general error), 126 (permission denied), 127 (command not found), and 130 (terminated by signal).

Advanced Log Analysis Patterns

For containers generating extensive logs, filter for specific patterns. To find database connection issues:

docker logs db-container | grep -i "connection\|timeout\|refused"

To identify performance issues, search for timeout or latency messages:

docker logs app-server | grep -i "slow\|timeout\|exceeded"

Combine multiple containers into one view for correlation analysis:

docker logs web-server | head -200 && echo "---" && docker logs db-server | head -200

Exporting Logs for Analysis

Save container logs to a file for offline analysis or sharing with other team members:

docker logs container-name > container-logs.txt 2>&1

For compressed storage of large logs:

docker logs container-name 2>&1 | gzip > container-logs.txt.gz

This approach preserves logs even after container removal, essential for incident investigations and compliance requirements.

Combining Commands for Real-World Workflows

Comprehensive Container Status Checks

In production environments, understanding overall system health requires combining multiple commands. A comprehensive health check script might look like:

#!/bin/bash

echo "=== Running Containers ==="
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Image}}"

echo -e "\n=== Stopped Containers ==="
docker ps -a --filter "status=exited" --format "table {{.Names}}\t{{.ExitCode}}\t{{.Image}}"

echo -e "\n=== Image Summary ==="
docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}"

echo -e "\n=== System Resource Usage ==="
docker system df

echo -e "\n=== Recent Container Activity ==="
for container in $(docker ps -q); do
    echo "Container: $(docker ps --filter id=$container --format '{{.Names}}')"
    docker logs --tail 5 $container | sed 's/^/  /'
done

Automated Troubleshooting Workflow

When containers fail unexpectedly, an automated workflow helps diagnose issues consistently:

#!/bin/bash

CONTAINER=$1

echo "Troubleshooting container: $CONTAINER"
echo ""

echo "1. Container Status:"
docker ps -a --filter name=$CONTAINER

echo -e "\n2. Container Logs (last 50 lines):"
docker logs --tail 50 $CONTAINER

echo -e "\n3. Container Configuration:"
docker inspect $CONTAINER | grep -A 5 "Env"

echo -e "\n4. Container Network:"
docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}' $CONTAINER

echo -e "\n5. Recent Process Activity:"
docker exec -it $CONTAINER ps aux 2>/dev/null || echo "Container not running"

Monitoring Container Lifecycle Events

Understanding when containers start, stop, and fail is essential for operations. Combining docker ps with timestamps helps track lifecycle events:

#!/bin/bash

echo "Container Lifecycle Events:"
docker ps -a --format "table {{.Names}}\t{{.Status}}\t{{.CreatedAt}}"

Practical DevOps Scenario: Production Issue Investigation

Consider a scenario where a production web service is not responding correctly. The investigation workflow combines multiple commands:

First, verify the container is running:

docker ps | grep web-service

If found, check recent logs for error patterns:

docker logs --tail 200 web-service | grep -i error

If errors reference database connections, verify the database container:

docker ps | grep database

Check database logs for connection errors:

docker logs database | grep -i "connection\|accept" | tail -20

Execute a connectivity test from the web service to the database:

docker exec web-service nc -zv database 5432

Verify environment variables are correctly set in the web service:

docker exec web-service env | grep DATABASE

This systematic approach, combining ps, logs, and exec, provides comprehensive visibility into the issue.

Best Practices and Performance Considerations

Log Management and Rotation

Docker's default logging driver stores logs locally, which can consume significant disk space for long-running containers. Configure log rotation to prevent disk exhaustion:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

Add this configuration to /etc/docker/daemon.json and restart the Docker daemon. This limits individual log files to 10MB with a maximum of 3 rotated files, preventing runaway disk usage.

Efficient Container Listing with Filters

In large deployments with hundreds of containers, unfiltered docker ps commands become slow and produce overwhelming output. Always use filters to narrow results to relevant containers:

docker ps --filter "status=running" --filter "label=app=myapp"

This approach scales much better than processing all containers.

Strategic Use of Image Tags

Properly tagging images improves trackability and simplifies management. Use consistent tagging strategies combining version numbers, build timestamps, and environment identifiers:

docker tag myapp:latest myapp:1.2.3-prod-20240115

This makes it easy to identify exactly which version is deployed and when it was built.

Container Naming Conventions

Descriptive container names make docker ps output immediately useful without requiring additional lookups. Use naming patterns that encode function and environment:

docker run --name web-api-prod-01 -d nginx

Better than generic names like container1 or app, this name immediately conveys purpose and environment.

Conclusion

Mastering docker ps, docker images, docker exec, and docker logs is fundamental to effective container management and DevOps operations. These commands provide the visibility and control necessary for deploying, monitoring, and troubleshooting containerized applications at scale. The docker ps command enables container lifecycle visibility, docker images manages your image repository, docker exec allows direct interaction with running containers for debugging and administration, and docker logs provides application and system visibility for troubleshooting and monitoring. Combining these commands with appropriate filtering, formatting, and automation creates a powerful toolkit for production Docker environments. This article is part of the Docker Complete Course on learnwithirfan.com, which provides comprehensive training for DevOps engineers seeking to master containerization and orchestration technologies.

Final Thoughts

Docker ps images exec logs Explained is worth reviewing with a practical lens: understand the risk or opportunity, map it to your environment, and take clear next steps instead of reacting to headlines.