Lesson 15 of 16Advanced20 min

Container Security

By the end of this lesson

Run as a non-root user and reduce what the image contains.

Unless an image says otherwise, the process inside a container runs as root. That is the default, and it is the starting point for this lesson rather than a footnote at the end of it.

Root inside a container is not the same as root on the host — namespaces and the default capability set hold it back, so it is a reduced form of root. It is still more than an application needs. A web API that reads configuration and talks to a database has no reason to be able to install packages, write anywhere in the filesystem, or change network settings. When a dependency turns out to have a flaw, the difference between root and an unprivileged account is the difference between what the attacker can attempt and what they can achieve.

Everything here is defensive. The aim is to reduce what an image contains and what a container is permitted to do, so that a problem somewhere in your dependency tree has less to work with. Each measure costs something, and the cost is named each time.

Dockerfile — a non-root account on an Alpine-based runtime

Dockerfile

FROM mcr.microsoft.com/dotnet/aspnet:9.0-alpine AS final
WORKDIR /app

# Alpine uses BusyBox tools, so the user-creation commands differ from Debian.
# -S makes a system account; -H skips the home directory; -D sets no password.
RUN addgroup -S -g 64198 appuser \
 && adduser -S -u 64198 -G appuser -H -D appuser

COPY --from=build --chown=appuser:appuser /app/publish .

ENV ASPNETCORE_HTTP_PORTS=8080
EXPOSE 8080

USER appuser
ENTRYPOINT ["dotnet", "Anvi.Employees.Api.dll"]

The base image is the Alpine variant of the ASP.NET Core runtime. Alpine is a small Linux distribution, so the image is smaller and contains fewer packages that could need patching.
addgroup and adduser here are BusyBox versions with short flags, not the Debian commands from the previous lesson. Copying a Debian RUN line into an Alpine image fails with an unhelpful usage message, so the base image decides the syntax.
A fixed UID and GID of 64198 matches the convention the official .NET images use for their own app account. Any unused number above 1000 works; fixing it means volume permissions stay predictable.
-H means no home directory and -D means no password, so the account cannot be logged into. An application account does not need either.
USER sits after the COPY for the reason the previous lesson gave: the copy needs write permission on /app. Everything after this line, including the container's process, runs unprivileged.
The .NET images from version 8 onward already contain a non-root account called app and expose its UID as the build argument APP_UID, so USER $APP_UID is a shorter alternative. Creating the account yourself is shown here because it works on any base image, including ones that provide nothing.

Base image choice is the other half of reducing what is in the image. Three broad options, in order of how much they contain:

A full distribution base (Debian, Ubuntu): A shell, a package manager, and a long list of libraries. Familiar, easy to debug, and the largest surface to keep patched. This is the default for most official runtime images.
Alpine: A much smaller distribution with BusyBox tools and the musl C library rather than glibc. Typically a fraction of the size. The musl difference occasionally matters: a native dependency compiled for glibc may not load, and behaviour around DNS and locales can differ, so test rather than assume.
Chiselled or distroless: Ubuntu chiselled images and Google's distroless images contain the runtime and its libraries and almost nothing else — no shell, no package manager, no utilities. Smallest, and least to patch. Microsoft publishes chiselled variants of the .NET runtime images.

Trade-off

Minimal images are harder to debug, and that is a real cost

docker exec -it container sh is how most people investigate a misbehaving container. On a chiselled or distroless image there is no shell, so that command fails and the usual next steps — check a file, run a DNS lookup, list processes — are unavailable from inside.

The alternatives work, and they need to be in place before you need them. Inspect from outside with docker logs, docker inspect and docker stats. Use docker debug or an ephemeral debug container attached to the same namespaces, where your platform supports it. Publish a diagnostics endpoint in the application itself, which is under your control and works everywhere.

A reasonable middle path for a team new to this: run Alpine or the standard runtime image in development, where you want a shell, and a chiselled image in production, where you want the smallest surface. The images then differ between environments, which is itself a trade-off — you are testing against something slightly different from what you ship. Pick deliberately and write down which way you went.

Watch out

A secret in any layer stays in the image forever

Layers are additive. Each instruction in a Dockerfile records a change on top of the layers below it, and nothing is ever removed from the layers below. Deleting a file in a later instruction writes a deletion marker that hides it from the assembled filesystem. The bytes are still in the earlier layer, and anyone who pulls the image can read them.

So a Dockerfile that copies a credentials file, uses it, and deletes it has published that credential. So has one that clones a private repository with a token in the URL, or passes an API key as a build argument — build arguments are recorded in the image metadata and docker history prints them.

What to do instead: use BuildKit's build secrets, where a value is mounted only for the duration of one RUN instruction and is not written into any layer; or fetch the credential at run time from a secret store or a mounted file, as the Environment Variables lesson described. Treat a credential that has been in an image as compromised and rotate it — deleting the image from your own registry does not recall the copies already pulled.

Reducing what a running container is allowed to do

Shell

# Read-only root filesystem, with writable space only where it is needed
docker run -d --name employees-api -p 8080:8080 \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=64m \
  --cap-drop ALL \
  --security-opt no-new-privileges \
  anvi/employees-api:1.5.0

# Confirm the account and the capability set the process ended up with
docker exec employees-api id
docker inspect -f '{{.HostConfig.CapDrop}} {{.HostConfig.ReadonlyRootfs}}' employees-api

# Scan an image before you push it, and again periodically after
docker scout cves anvi/employees-api:1.5.0

# See what a base image tag resolves to today, and when it was built
docker pull mcr.microsoft.com/dotnet/aspnet:9.0-alpine
docker image inspect -f '{{.Created}} {{.Id}}' mcr.microsoft.com/dotnet/aspnet:9.0-alpine

--read-only mounts the container's filesystem read-only. An attacker who reaches code execution cannot drop a script or a binary anywhere it will persist. Applications that expect to write somewhere will fail, which is why the next flag exists.
--tmpfs /tmp gives one writable directory backed by memory, discarded when the container stops. noexec means nothing in it can be run, nosuid ignores set-user-id bits, and size caps it so a runaway write cannot exhaust host memory. Add one of these for each path your application genuinely writes to.
--cap-drop ALL removes the Linux capabilities the container would otherwise keep. Capabilities are the individual privileges root normally holds, split up — binding a low port, changing file ownership, loading kernel modules. A published .NET API on port 8080 needs none of them. Add back only what breaks, with --cap-add, and write down why.
--security-opt no-new-privileges stops a process inside the container from gaining privileges through a set-user-id binary. There is no legitimate reason for an application container to do that.
docker scout cves compares the packages in your image against published vulnerability data. Trivy and Grype do the same job and run well in a pipeline. All of them report what is known today, which is why scanning once at build time is not enough.
The last two commands make the point about patching: a tag like 9.0-alpine is a moving pointer, and the image behind it is rebuilt when its packages are patched. Your image does not change until you rebuild it, so a rebuild-and-redeploy schedule is part of the security work, not an optional extra.

A practical order to work through, cheapest and highest value first:

Add a non-root USER to every image you build, and check it with docker exec ... id rather than assuming
Ship a runtime base image, never an SDK, so compilers and package managers are not in production
Pin base images to a specific minor version and rebuild on a schedule so patches actually reach you
Scan images in the build pipeline and fail the build on severe, fixable findings — fixable matters, or the gate becomes noise everyone ignores
Drop capabilities and mount the root filesystem read-only, adding back only what the application demonstrably needs
Keep secrets out of layers and out of build arguments; read them at run time
Move to a smaller base image last, once you have a way to debug without a shell

Common mistake

Common mistakes

Assuming an image runs unprivileged because it is someone else's. Plenty of published images default to root. docker exec ... id answers the question in a second.

Adding USER at the top of the Dockerfile, so every later instruction lacks permission and the build fails in a way that looks unrelated.

Running with --privileged to make a permission error go away. That flag removes most of the isolation, including the capability restrictions, and turns a small problem into a large one. Find the specific capability instead.

Deleting a secret in a later RUN and believing it is gone. It is in the earlier layer, and docker history will show the instruction that put it there.

Passing a token as a build argument. Build arguments are stored in image metadata; they are configuration, not secrets.

Scanning once, at the point the image was built, and never again. Vulnerabilities are published after your build. An image that was clean in March is not clean in September because nothing changed.

Treating a non-root container as the boundary that matters. It is one layer among several — the application still needs input validation, authorisation on every request, and a database account with only the permissions it uses.

Summary

Containers run as root unless the image says otherwise, and an application rarely needs that
Create a fixed non-root account, place USER after the copies, and verify with docker exec ... id
Smaller base images mean less to patch and fewer tools for an attacker, at the cost of harder debugging
Layers are additive, so a secret added in any layer ships in the image even if a later layer deletes it
Drop capabilities, mount the root filesystem read-only, and rebuild on a schedule so base image patches arrive

Practice

Attempt each one before opening the solution. Getting it wrong first is how the idea sticks.

Try it yourself

Find out what your container is actually allowed to do

Start your employees API with no security flags and run docker exec employees-api id. Note the user. Then try to create a file at the filesystem root from inside the container.

Rebuild with a non-root USER, run again with --read-only, --tmpfs /tmp and --cap-drop ALL, and repeat both checks. Read the application logs for anything that broke.

Show solution

The first run reports uid=0(root) and the file creation succeeds. The second reports your application account, and the write fails with a read-only filesystem error. That gap is what the lesson is about: the same application, the same image content, a much smaller set of things it can do.

Reading the logs after the second run is the part that matters in practice. Applications write to places you did not think about — a temporary file during a file upload, a data-protection key ring, a diagnostic dump directory. Each one either gets its own tmpfs mount or gets reconfigured to write somewhere shared and persistent.

There is also a lesson in the order. Tightening everything at once and then seeing a failure tells you something broke but not which flag did it. Change one thing, confirm the application still works, then change the next.

Shell

docker run -d --name api-loose -p 8080:8080 anvi/employees-api:1.5.0
docker exec api-loose id
docker exec api-loose sh -c 'touch /proof && ls -l /proof'
docker rm -f api-loose

docker run -d --name api-tight -p 8080:8080 \
  --read-only --tmpfs /tmp:rw,noexec,nosuid,size=64m --cap-drop ALL \
  anvi/employees-api:1.5.0
docker exec api-tight id
docker exec api-tight sh -c 'touch /proof' || echo "blocked, as intended"
docker logs api-tight
docker rm -f api-tight

Think about it

A pipeline builds the employees API image, scans it, finds nothing, and deploys. The image is untouched for eight months because the API needed no changes. The scanner is still green in the pipeline, because the pipeline has not run.

What is the actual state of that image, and what would you change about the process?

Show solution

The image contains whatever its base layers contained eight months ago. Vulnerabilities published since then are present and unreported, because nothing has looked at the image since the day it was built. A green result from a run in February says nothing about September.

Two process changes address it. Scan the images that are deployed, on a schedule, rather than only at build time — the registry or your platform can do this and report against what is actually running. And rebuild on a schedule, so base image patches reach production without waiting for a feature. A weekly or monthly rebuild-and-deploy of unchanged code feels strange the first time and is the only way patches arrive.

The honest trade-off: rebuilding unchanged code means redeploying something that was working, and every deployment carries a small risk. That risk is manageable with a health check and a rollback path, both of which you want anyway. Leaving an unpatched image running for eight months is the larger exposure.

Knowledge check

Nothing is recorded and there is no score. The explanation appears either way.

A Dockerfile copies a deployment key, uses it during the build, and deletes it with a later RUN instruction. Who can read that key?

Nobody — the file was deleted before the image was finishedOnly someone with root access to the host running the containerAnyone who can pull the image, because the earlier layer still contains itOnly the account named in the USER instruction

What is the main cost of moving to a chiselled or distroless base image?

The application runs measurably slowerThere is no shell, so debugging from inside the container no longer worksEnvironment variables are no longer availableImages can no longer be scanned for vulnerabilities

Saved in this browser only.