Streamlining Kubernetes Project Management with folder-based Contexts
The Problem
We often find ourselves working on multiple Kubernetes projects, which requires frequent switching between
Kubernetes contexts. To be honest, there have been instances in the past where changes were mistakenly applied to the wrong context—fortunately, only on a local testing Kubernetes cluster. This experience led us to consider ways to securely manage different Kubernetes-based projects locally using
kubectl
from our development machines.
Approach
Developing and managing multiple projects in parallel necessitates proper separation; we achieve this by ensuring each project is as self-contained within its folder as possible. Thus, all configurations, dependencies, and even tooling should be confined to that project's folder. Modern tooling often supports this approach, exemplified by npm's use of the
package.json
and
node_modules
folder.
We aim to adopt a similar strategy for our Kubernetes contexts that is both easy to maintain and straightforward to understand.
Setup
So, how do we set up our projects to make this happen? The core tool of our approach is a small but powerful open-source tool called " direnv ". It is not specifically developed for Kubernetes but for switching the shell environment based on project folders.
With direnv, we can create a
.envrc
file where we can specify environment variables that are only set when we
change directory (
cd
) into the project folder containing the
.envrc
file. If we leave the project folder with
cd ..
, the previous shell environment is restored. So, that’s already cool, but how can we use this for our Kubernetes context? Easy. We set our
KUBECONFIG
environment variable in the
.envrc
file like so:
export KUBECONFIG=$(pwd)/.kube/config
How does that help? Normally, we have the configuration for
kubectl
in the user's home directory at
~/.
kube/config
. We can specify multiple Kubernetes contexts in that file, but they need to be switched manually and
are then globally scoped — not on a shell basis like we want.
However, by overriding the
KUBECONFIG
environment variable, we can instruct
kubectl
and tools like Helm to use a different configuration file. That's where
.envrc
becomes useful: it directs the
KUBECONFIG
environment variable to a local
kubectl
config stored in our project folder, instead of our home directory. For instance, if our project folder is
myproj
, the kubeconfig file should be placed in
myproj/.kube/config
. Of course, you can change this location, but for the sake of maintaining an easy-to-understand mental model, we use a similar location to that in the user's home directory.
After that, everything should work as expected if you have set up direnv properly according to the documentation. Don’t forget the second step after installing direnv — configuring your shell. (That’s what I initially forgot and wondered why it doesn’t work.)
Advantages
In contrast to the default behavior of working with
kubectl
, which uses a global Kubernetes context, having a folder-scoped Kubernetes context presents several advantages:
Folder-Scoped Context
With the Kubernetes context being scoped by the folder, there’s no risk of accidentally changing configurations for a different cluster while working within the project folder. This specificity enhances safety and reduces errors.
Multiple Kubernetes Contexts in Parallel
The folder-based approach facilitates working on multiple projects across multiple terminals without needing to worry about whether the correct Kubernetes context is set in each terminal. This seamless operation is possible with the setup described, eliminating the need for manual intervention.
Quickly Switch Between Kubernetes Projects and Contexts
Switching between different Kubernetes projects and their corresponding contexts is as straightforward as changing directories in your terminal. This method is super fast and significantly less error-prone compared to manually changing the context.
Demo
Based on two example projects
myproj
and
anotherproj
both setup with this approach, this is how it appears when changing a Kubernetes project/context:
The source code for this demo can be found in this repository .
Additional Tools
In addition to the core setup, we leverage several other tools to streamlines our Kubernetes workflow for our development machines:
- kube-ps1 (Zsh Plugin): Displays the current Kubernetes context and namespace directly in the shell prompt, keeping you constantly informed of the context you're working in without needing to run additional commands.
-
kubectl
(Oh My Zsh Plugin): Enhances the shell experience by adding autocomplete functionality for
kubectl
, extending even to the Kubernetes resource ID level. This feature significantly speeds up command entry and reduces errors. -
kubectx
: From this toolset, I specifically utilize the
kubens
command for swiftly switching between namespaces. It's a simple yet powerful addition that facilitates quick context adjustments. -
ctlptl
: While not directly related to the shell experience,
ctlptl
is an indispensable tool I find essential for spinning up local development Kubernetes clusters declaratively with just a single command. Developed by the talented team at Tilt , it simplifies the management of local clusters, making it easier to test and develop applications in a Kubernetes environment. If you haven’t checked out Tilt yet, I highly recommend it. It's a superb tool for local Kubernetes development with super-fast live reloading.
If you're interested in leveling up your web development skills, you might want to consider our web development masterclass.