We fixed container memory usage on macOS

Earlier this year we introduced a new file system that makes containers run 2–10x faster in OrbStack when a lot of file I/O is involved, reaching up to 95% of native file system performance.

We've been hard at work since then, digging into the biggest issue with Docker containers and virtual machines on macOS: memory usage. Today we're excited to announce that OrbStack now has dynamic memory management, so that it only uses as much memory as your containers need. Any extra memory is automatically released and made available to other apps, improving performance of the entire system and preventing slowdowns. No more high memory usage for seemingly no reason.

Before and after

One of the use cases that benefits most from dynamic memory is heavy containers on a low-memory machine, so we tried running a Compose project with 25 NATS servers on a 16 GB M1 MacBook Air. You can see memory usage and pressure dropping as soon as the containers are stopped:

Memory pressure is a measure of how memory usage is affecting performance on the system. When pressure is low, the system has enough memory to meet the needs of all running apps. When pressure is high, macOS must resort to compressing memory and moving less-recently-used memory contents to disk (swapping) in order to make room for new memory allocations. Compression and swap are pretty fast on Apple Silicon thanks to hardware-accelerated compression and fast SSDs, but it's still orders of magnitude slower than keeping everything in RAM.

Why does it use so much memory?

OrbStack, and all other container runtimes on macOS, run a Linux virtual machine to host containers. This is a given because macOS doesn't have the APIs necessary for creating containers, and it also makes the "build once, run anywhere" idea of containers possible by ensuring binary compatibility.

Modern virtualization can be very efficient, but it typically still involves running a full operating system in the virtual machine — in this case, Linux. A virtual machine tricks Linux into thinking that it's running on a real computer rather than as part of an app, allowing it to run just as well as it would on a physical machine. To do this, it provides components and devices usually seen in a physical computer, such as a CPU, memory, disk, network card, and more.

The memory part is where things get tricky. The virtual machine's RAM is exposed as a large block of addresses where data can be stored, like a giant array:

const virtualMemory = new Uint8Array(4 * 1024 * 1024 * 1024); // 4 GiB

The VM is free to use this memory however it wants by reading to, and writing from, addresses in the array. We can minimize memory usage by lazily allocating parts of the array as they get used. However, there's no concept of "used" or "free" memory; we only know that Linux has written data to various parts of the array. Maybe it wrote to the first 512 MiB earlier, but now only the first 128 MiB is actually in use. The rest is just sitting there, but we can't discard it because we don't know whether it contains important data that Linux might need later.

On real hardware, this doesn't matter because the machine has a certain amount of RAM that is always available. If it's not being used, no one else is going to use it, and there's no extra cost to keeping all memory contents around. But we're running Linux on a system with a lot of other apps competing for memory, so we can't just keep everything around.

Dynamic memory management

In OrbStack v1.7.0, we've introduced a new memory management system that builds on top of the "giant array" concept to track which parts of the virtual RAM are actually in use, and which parts are likely no longer needed and can be released if other apps need more memory. This makes our memory usage much more efficient and cooperative, leading to better performance for all apps running on the system.

The biggest improvements will, of course, mainly be seen on system with less memory. Even on high-memory machines, however, releasing memory can make the difference between macOS being forced to compress data, and being able to keep everything in RAM. It also allows macOS to keep more files cached so that it doesn't have to read them from disk again, which can be a big performance win for all apps on the system, not only for OrbStack.

Try it out

OrbStack replaces Docker Desktop with a fast and easy way to work with containers, Linux, and Kubernetes. Dynamic memory is now available to all users in OrbStack v1.7.0, along with many more improvements.

Over the next few weeks, we'll be tweaking and improving other aspects of memory management to make OrbStack even better. Stay tuned for more updates!

Follow @OrbStack on Twitter and join the Discord community to stay up to date with OrbStack news!