Container debugging made easy: zsh, tools, packages on any image

It's 12 AM. My Docker container isn't working. Time to debug it with docker exec:

I'm using a distroless image to save space and minimize vulnerable dependencies, but... it's nearly impossible to debug.

Okay, let's rebuild the image temporarily with debian:sid-slim for debugging:

...forget it.

The problem

With container images constantly being stripped to the bare minimum, it's a miracle to even find a /bin/sh, let alone curl.

There isn't really a good solution:

• Adding packages to the image bloats the image size and introduces potentially vulnerable dependencies.
• Keeping the package manager works but requires running apt update and reinstalling tools every time you restart the container.
• You can't install anything in distroless containers.
• Read-only containers can't be modified at all.
• Even after installing commands, a default /bin/sh is missing bells and whistles.

Introducing: Debug Shell

We've built OrbStack Debug Shell to solve these papercuts. It provides a nice zsh setup with auto-complete, syntax highlighting, colorful output, editors (vim, nano), common tools (htop, curl, jq, less, etc.), and access to over 80,000 packages. All of this works with any container—distroless, read-only, you name it.

In addition to everything Debug Shell provides, you can install anything from NixOS' expansive collection of over 80,000 packages. Installed packages persist across container restarts, image rebuilds, and even across different containers.

Debug Shell automatically asks you to install commands the first time you try to run them:

Since it's is an "overlay" on top of the container, it doesn't modify the actual container or image at all, and works on read-only containers. All the extra tools work exactly as if they were running directly in your container.

How it works

Debug Shell works by injecting a debugging environment using:

• NixOS for a large package collection, and flexibility with filesystem paths
• Linux namespaces to create a secondary view of the container
• Other Linux magic to glue everything together

In particular, mount namespaces are what Docker and runc use to give each container its own image and view of the filesystem. But unlike chroot(2), you can copy an existing mount namespace into a new one. Debug Shell uses this to copy a container's namespace, creating a new view where we can inject things without them showing up in the original mount namespace or filesystem:

The future of container debugging

OrbStack is a Docker Desktop replacement that makes it fast and easy to work with container and Linux environments. This means that Debug Shell makes local development easier, but stay tuned for more 👀

Debug Shell comes with a nice zsh setup but we'll also be working on customizability, such as support for fish, to make it feel like your usual environment.

Try it now

Along with Debug Shell, we've got an assortment of bug fixes, performance improvements, and small features and UI tweaks in the OrbStack 1.5 release.

To use Debug Shell in OrbStack, simply select a container or click the "Debug" button in the sidebar. You can also use it from the command line: orb debug <container name or ID>

Check the documentation for more details.

Follow @OrbStack on Twitter and join the Discord community to stay up to date with OrbStack news 👀