I’m trying to noodle out how to make this basic workflow happen in Docker Desktop for Mac, with Kaniko running inside of a Kubernetes pod:
- Pull a base image from the local Docker context
- Build an image on top of that with other resources
- Publish the new image back to Docker to the local context
with the goal of starting a new pod in Kubernetes that references the image that was just built, and having Kubernetes launch the pod with that new image.
We’re already kind of doing this in a full-blown Kubernetes cluster where Kubelet is configured to pull from an external registry via an alias of cluster.local
so instead of pulling from / publishing to Docker we’re talking to that registry. The Dockerfile
would resemble:
ARG REGISTRY_HOST=external.registry.fqdn
FROM ${REGISTRY_HOST}/path/to/my-base-image:1.0.0
COPY ...
RUN ...
...etc...
and we publish to external.registry.fqdn/path/to/my-new-image:2.0.0
. But the image name spec for the pod would be cluster.local/path/to/my-new-image:2.0.0
. So in Docker Desktop we’d need to be able to configure the build argument REGISTRY_HOST so that it points to the Docker Desktop context (I think).
This is primarily for being able to test the process locally, instead of having to push helm charts, etc. up to a cluster to test changes. Any suggestions on how one might go about doing something like this, perhaps with a local registry outside of Kubernetes and tweaks to kubelet to have it work like the full cluster configuration we’re using would be greatly appreciated.
If someone has managed to do something like this using colima
and kubernetes
, I will gladly take a look at that solution as well.
Is there a foundation to begin this work or has someone solved this already?