Minikube on M1 pro without Docker desktop

Down the M1 rabbit hole.

This year I started my own company: 0to1. To get up and running quickly I decided to stick with technology that I knew before, hardware I have used for the past 8 years without breaking down has been Apple MacBook Pro’s. The new Macbook M1 Pro had raving reviews, super performance, and long battery life.

Knowing that the processor architecture changed from Intel (x86) to ARM (M1), after a year things should certainly have been ironed out… right?

After checking I found out that some tools didn’t work, like VirtualBox but there are often others that do work without issues.

After installing Rancher desktop on the m1 I ran into an issue that /usr/local/bin was not created and ownership was not set to my user, this was easy to resolve by the solution below.


Rancher Desktop was working fine, but I needed an etcd node integrated with Kubernetes for a little programming project I have going on. Looking back I should have just used Rancher Desktop to run an etcd cluster locally, but I remembered Minikube had etcd installed automatically.

I wanted to see if Minikube on the M1 could be easily set up. So decided to try out Minikube.

Minikube it is!

brew install minikube

The brew install was easy, but starting minikube gave an error that Virtualbox was not found, this is the default driver that is used by minikube. As I mentioned above, VirtualBox isn’t supported on the M1 and likely never will. So I needed another driver.

😄 minikube v1.24.0 on Darwin 12.1 (arm64)

✨ Automatically selected the virtualbox driver

❌ Exiting due to DRV_UNSUPPORTED_OS: The driver ‘virtualbox’ is not supported on darwin/arm64

A list of all the available supported drivers for Apple can be found here:

Docker Desktop recently changed its policy to a paid plan for certain users, this gave me a new challenge:

Can I run Docker without docker-desktop on the new M1?

All the solutions online gave the Intel processor solution: Install minikube. This quickly became a chicken or egg situation. Using Docker as the driver without using docker-desktop is not an option at the moment, the only solution mentioned time and time again seems to be the docker-desktop on the M1 Pro.

Surely there are other drivers that I could use?

Other drivers

Since Rancher desktop is using Qemu on the M1 Pro, I surely could use it… Right? Since KVM uses Qemu I decided to give the kvm2 driver a try:

Rogiers-MBP:0to1 rogierdikkes$ minikube start –driver=kvm2

😄 minikube v1.24.0 on Darwin 12.1 (arm64)

❌ Exiting due to DRV_UNSUPPORTED_OS: The driver ‘kvm2’ is not supported on darwin/arm64

No support in Minikube, so I tried hyperkit:

brew install hyperkit

Error: hyperkit: no bottle available!

You can try to install from source with:

brew install –build-from-source hyperkit

Slowly options started running out, building from source? This is while the package is easy to install on the Intel MacBooks, so I didn’t want to go this route. Time to give good ol’ Redhat a try.

Podman to the rescue?

With podman the installation went very smoothly, unfortunately, I didn’t make the finish line with podman. With Podman I ran into bugs on the M1. I decided to write down what I did in case the bug I encountered gets fixed.

brew install podman

After installing Podman you have to init the machine. Minikube requires 2 CPUs, so provide the podman VM with 2 CPUs at init.

podman machine init --cpus 2

Downloading VM image: fedora-coreos-35.20220103.2.0-qemu.aarch64.qcow2.xz: done

Extracting compressed file

Then start the machine:

podman machine start

INFO[0000] waiting for clients…

INFO[0000] listening tcp://

INFO[0000] new connection from to /var/folders/s1/cythsgh50hd9tl7_38qknb2r0000gn/T/podman/qemu_podman-machine-default.sock

Waiting for VM …

Machine “podman-machine-default” started successfully

As you can see podman downloads a qcow2 file to use with Qemu. A similar approach Rancher Desktop takes. After this is completed you can start minikube with the podman driver:

minikube start --driver=podman

😄 minikube v1.24.0 on Darwin 12.1 (arm64)

✨ Using the podman (experimental) driver based on user configuration

👍 Starting control plane node minikube in cluster minikube

🚜 Pulling base image …

E0107 13:11:31.539966 90940 cache.go:201] Error downloading kic artifacts: not yet implemented, see issue #8426

🔥 Creating podman container (CPUs=2, Memory=1953MB) …| ERRO[0123] accept tcp [::]:38413: use of closed network connection

ERRO[0123] accept tcp [::]:41795: use of closed network connection

ERRO[0123] accept tcp [::]:36729: use of closed network connection

ERRO[0123] accept tcp [::]:44515: use of closed network connection

ERRO[0123] accept tcp [::]:35925: use of closed network connection

✋ Stopping node “minikube” …

🔥 Deleting “minikube” in podman …

🤦 StartHost failed, but will try again: creating host: create: creating: create kic node: create container: podman run -d -t –privileged –device /dev/fuse –security-opt seccomp=unconfined –tmpfs /tmp –tmpfs /run -v /lib/modules:/lib/modules:ro –hostname minikube –name minikube –label –label –label –label –network minikube –ip –volume minikube:/var:exec –memory-swap=1953mb –memory=1953mb –cpus=2 -e container=podman –expose 8443 –publish= –publish= –publish= –publish= –publish= exit status 126



Error: the requested cgroup controller `cpu` is not available: OCI runtime error

🔥 Creating podman container (CPUs=2, Memory=1953MB) …

😿 Failed to start podman container. Running “minikube delete” may fix it: creating host: create: creating: setting up container node: creating volume for minikube container: podman volume create minikube –label –label exit status 125



Error: volume with name minikube already exists: volume already exists

❌ Exiting due to GUEST_PROVISION: Failed to start host: creating host: create: creating: setting up container node: creating volume for minikube container: podman volume create minikube –label –label exit status 125



Error: volume with name minikube already exists: volume already exists

What seems to be happening is that the group CPU is unavailable in the first deployment. Currently, there is an issue open for this, so podman was not the way forward.

The issue:

next! VFTOOL

Since podman did not work and the Minikube setup on the M1 was broken I had the hope that the binaries for the Ubuntu setup would work. Again I just should have used Rancher Desktop, but now I was curious if there was any way to get Minikube working on the M1. The idea was to get Ubuntu running on the M1, since VirtualBox was no longer an option, my approach was to use vftool to run a virtual machine. Inside that virtual machine, you run minikube. I had the hope that the Ubuntu binaries would not have issues and Minikube would just get installed.

First, let’s create a folder so we can isolate our mess and allows us to start setting up vftool to build a virtual machine.

mkdir -p ~/VM/Ubuntu
cd ~/VM/
git clone
cd ./vftool
file ./build/vftool
cp ./build/vftool /usr/local/bin/

Above copy command to /usr/local/bin is optional, it saved me a couple of seconds to find the binary when you want to use vftool after the build. The file the command should give the following output:

Mach-O 64-bit executable arm64

After this is complete you can move onward to setting up the VM vftool will be using. First, download the iso for Ubuntu. We also need 7zip to unpack the ISO to get access to the kernel and initrd. We need to gunzip the kernel and copy the initrd to our VM folder.

cd ~/VM/Ubuntu
curl --silent -L -o ~/VM/Ubuntu/focal.iso
brew install 7zip
7zz x -tiso -y ~/VM/Ubuntu/focal.iso -o~/VM/Ubuntu/focal-unpacked
cp ~/VM/Ubuntu/focal-unpacked/casper/initrd ~/VM/Ubuntu/
cp ~/VM/Ubuntu/focal-unpacked/casper/vmlinuz ~/VM/Ubuntu/vmlinuz.gz
gunzip ~/VM/Ubuntu/vmlinuz.gz

After this is done you have the 2 files you need, ISO and the focal-unpacked folder is no longer necessary. The next step is to create a block device to use for Ubuntu, I have tried to use the ISO downloaded however vftool kept giving the error that it was an invalid storage device. So we create an empty block device to use as the disk for the virtual machine:

dd if=/dev/zero of=~/VM/Ubuntu/focal-server-cloudimg-arm64.img seek=60000000 obs=1024 count=0

This creates an empty raw device for you to use with the size of 60G, the reason is that minikube uses around 33G for its installation alone. We will be using this volume for the docker images and minikube images etc. If we don’t use this image we will run out of disk space when starting minikube. Since this virtual machine is entirely in memory we need to add a lot of memory to make minikube run without getting OOM. Now let’s start the virtual machine!

vftool -k ~/VM/Ubuntu/vmlinuz -i ~/VM/Ubuntu/initrd -d ~/Ubuntu/VM/focal-server-cloudimg-arm64.img -m 8192 -p 2 -a "console=tty0 console=hvc0 root=/dev/vda1 ds=nocloud"

The following message returns:

2022-01-07 18:11:43.308 vftool[1565:1811117] vftool (v0.3 10/12/2020) starting

2022-01-07 18:11:43.309 vftool[1565:1811117] +++ kernel at /Users/rogierdikkes/VM/Ubuntu/vmlinuz, initrd at /Users/rogierdikkes/VM/Ubuntu/initrd, cmdline ‘console=tty0 console=hvc0 root=/dev/vda1 ds=nocloud’, 2 cpus, 4096MB memory

2022-01-07 18:11:43.310 vftool[1565:1811117] +++ fd 3 connected to /dev/ttys007

2022-01-07 18:11:43.310 vftool[1565:1811117] +++ Waiting for connection to: /dev/ttys007

2022-01-07 18:11:47.523 vftool[1565:1811117] +++ Attaching disc /Users/rogierdikkes/VM/Ubuntu/focal-server-cloudimg-arm64.img

2022-01-07 18:11:47.523 vftool[1565:1811117] +++ Configuration validated.

2022-01-07 18:11:47.523 vftool[1565:1811117] +++ canStart = 1, vm state 0

2022-01-07 18:11:47.602 vftool[1565:1811140] +++ VM started

With screen we will now connect to this virtual machine, you need to search for the tty which you can attach to. In my case it was /dev/ttys007

screen /dev/ttys007

This should open a session with Ubuntu. Now you can finish the Ubuntu install on the virtual machine. While installing you can keep everything default, except for 1 field: URL.

The URL the field is incorrect in the installation, the defaults point towards images that do not exist:

Replace these 2 URLs with the ISO we downloaded earlier:

After this, you see the download and you get the login prompt. You can use the username ubuntu without a password.

Now this is done we need to mount the raw disk to /var/lib/docker so all the images end up here:

sudo mkdir -p /var/lib/docker
sudo mkfs.ext4 /dev/vda
sudo mount /dev/vda /var/lib/docker

Once this is done we will create a symlink to the /var/lib/docker directory to place the minikube files at the same location as the docker files. It’s a bit messy, but I kept getting out of disk space errors and was done with this…

sudo ln -s /var/lib/docker /var/lib/minikube

After this is done let’s install the Docker engine on Ubuntu, you can use this guide.

For the completion guide I list the steps here in a oneliner:

sudo apt update -y && \
sudo apt install -y ca-certificates curl gnupg lsb-release && \
curl -fsSL | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg && \
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null && \
sudo apt update -y && \
sudo apt install -y docker-ce docker-ce-cli conntrack

OverlayFS cannot mount on top of the liveCD, this will give the following error:

docker: Error response from daemon: error creating aufs mount to /var/lib/docker/aufs/mnt/8017e1251b9eab3500fcd0b677b96c9e5f9430dd5249354b120feb0019aa09e8-init: mount target=/var/lib/docker/aufs/mnt/8017e1251b9eab3500fcd0b677b96c9e5f9430dd5249354b120feb0019aa09e8-init data=br:/var/lib/docker/aufs/diff/8017e1251b9eab3500fcd0b677b96c9e5f9430dd5249354b120feb0019aa09e8-init=rw:/var/lib/docker/aufs/diff/077e60cb2a82a857e71c73b6ec7cd8c667d0f1b15fdc1f5a228eac2e8f852872=ro+wh,dio,xino=/dev/shm/aufs.xino: invalid argument.

In dmesg you can see the following error:

[ 681.897119] overlayfs: missing ’lowerdir’

You can solve this by using a heredoc to configure the vfs storage-driver in the Docker daemon.json file, this has some negative side effects but it was the only storage driver I got working since aufs was deprecated and overlayfs doesn’t work in the liveCD.

sudo bash -c "tee <<EOF > /etc/docker/daemon.json
    \"storage-driver\": \"vfs\"

Because we just used the vfs storage-driver we cannot run the minikube command anymore with driver type none, minikube will fail in its preflight check:

error execution phase preflight: [preflight] Some fatal errors occurred:

    \[ERROR SystemVerification\]: unsupported graph driver: vfs

After adding the storage driver vfs you need to restart the docker process:

sudo systemctl restart docker

Check if the hello-world runs:

sudo docker run hello-world

After this is completed correctly you can download minikube:

curl -Lo minikube
chmod +x ./minikube
sudo cp ./minikube /usr/local/bin

Adjust the following parameter:

sudo sysctl fs.protected_regular=0

Set a password, else you cannot add yourself to the docker group:


After setting the password add the ubuntu user to the docker group:

sudo usermod -aG docker $USER && newgrp docker

After this is complete you can run the minikube command, running it with sudo will make it fail:

minikube start --driver=docker

Since minikube starts just like your OS in memory this is the moment where OOM events happen. Now let’s check if minikube works:

minikube kubectl -- get pods -A


kube-system coredns-78fcd69978-dt8cw 1/1 Running 0 12s

kube-system etcd-minikube 1/1 Running 0 32s

kube-system kube-apiserver-minikube 1/1 Running 0 28s

kube-system kube-controller-manager-minikube 1/1 Running 0 25s

kube-system kube-proxy-qc7sm 1/1 Running 0 12s

kube-system kube-scheduler-minikube 1/1 Running 0 28s

kube-system storage-provisioner 1/1 Running 0 20s


Halfway through the vftool the setup I started to realize that connecting from my MacBook to the Minikube setup would take more and more time. I started to think about alternatives. I had the Rancher Desktop running before on Intel and had the idea that it was an alternative. There is another post on how to set up Rancher Desktop. Having the ability to correct directly from my M1 terminal and my vscode setup was not possible in the vftool setup above.

The vftool setup is not 100% stable, I had dozens of kernel panics during the setup and trying to figure out how to set up minikube inside vftool. I even had kernel oops during the creation of the symlink.