The KubeCon / CloudNativeCon EU 2024 in Paris has just wrapped up, and it was nothing short of amazing. Alongside the main event, we saw some standout specific conferences like ArgoCon, BackstageCon, and Cilium + eBPF Day, just to drop a few names. With over 300 talks across the main conference tracks, there’s a lot to revisit or catch up on, and you can do just that on YouTube . More than 12,000 people joined us live at the Paris Expo Porte de Versailles, making it a truly vibrant gathering.

“Why consider attending KubeCon when building a SaaS or WebApp?” you might wonder. The division between development and operations has been fading for years, and the smaller your team, the more crucial it becomes for developers to grasp the operations (ops) side of things. Yet, even if your company is large enough to have dedicated dev and ops teams, it’s time to explore DevOps. This approach helps optimize your workflows and tools and restructure your teams to save time and money while increasing agility. The old “throw it over the wall” mentality between dev and ops ought to be left behind.

In this blog post, I don’t intend to dive into why containerization, Kubernetes, and the cloud-native ecosystem are critical for your SaaS or cloud application. Instead, I’d like to discuss the current state of play and where the future might be heading in the cloud-native landscape. My insights are based on my experience, talks from KubeCon, as well as conversations with various projects and teams.

Status Quo - Kubernetes for Everyone

After 10 years of Kubernetes and a widespread adoption across various industries and companies of all sizes, it's clear that it's here to stay. Originally built for large systems (1000+ nodes, 10k+ pods/containers), Kubernetes required complex system management, which initially seemed like overkill for smaller systems or startups. However, its versatility and scalability have proven invaluable for organizations regardless of their size.

I think we're at a juncture where Kubernetes distributions have become straightforward to set up and maintain, taking the sting out of their inherent complexities. The ecosystem of tools surrounding Kubernetes has also matured significantly, now offering a robust and satisfying developer experience.

For those of you building a SaaS or web application, regardless of whether you're starting with a microservice architecture, beginning your journey on a Kubernetes system can facilitate smooth scaling without necessitating significant changes to your processes or toolchain as your team and company expand. A few years back, I wouldn't have given this recommendation to smaller teams or solo entrepreneurs. However, today, the benefits of adopting Kubernetes, even over straightforward Docker or Docker Compose setups, are too significant to ignore.

You can kick things off with a single-node cluster. It's not high-availability, but it'll still get you zero-downtime deployments, load balancing, rollbacks, as well as config and secret management—and all of this without much more effort than running Docker Compose. The beauty is, you can apply the same principles to a multi-node, high-availability cluster later on. I'm not claiming Kubernetes suits every use case—I mean, you probably don't need it for hosting a private WordPress blog (though you could, and you'd learn heaps from it).

To be honest, setting up a small cluster isn't rocket science these days, but you still need to learn how to work with Kubernetes. Given its powerful features—like self-healing, load balancing, high availability, and zero-downtime deployments—you first need to understand the challenges these features address before you can fully appreciate the possibilities Kubernetes offers.

The real challenge with Kubernetes and the entire cloud-native ecosystem lies in deciding which capabilities and tools are genuinely beneficial for the current state of your team, company, or product.

The Future - LLM/AI, WebAssembly, Security

The status quo after 10 years of Kubernetes is already quite impressive, but KubeCon also shed light on where Kubernetes and the Cloud Native Ecosystem are headed next. Here are a few key takeaways from the conference:

AI and LLMs

As you might have guessed, AI and LLMs (Large Language Models) featured in the KubeCon schedule. However, audience feedback during a keynote by Priyanka Sharma, Executive Director of CNCF, on this question about "how many organizations are prototyping AI-enabled features right now" indicates that AI hasn't yet become a mainstay among DevOps and ops teams in the Kubernetes ecosystem. This doesn't surprise me, as many companies are still - even after 10 years of Kubernetes - in the midst or just beginning their migration to containers and Kubernetes. The feeling that everyone at KubeCon has been using Kubernetes in their organisations for years is misleading and doesn't reflect reality. Moreover, regardless of their Kubernetes migration status, LLMs are not yet a standard part of most companies' internal infrastructure but are more often found within R&D teams as proof of concepts (POCs).

But I also believe that companies aiming to host, use, or train their own LLMs, instead of leveraging APIs from external LLM providers like OpenAI, will eventually find Kubernetes indispensable for providing the necessary infrastructure. This infrastructure will need to handle the demands of scaling, dynamically managing computing power, and supporting long-running training tasks for their LLM use cases.

The CNCF has significantly emphasized AI/LLMs, dedicating its own track at KubeCon. While Kubernetes is a solid and proven choice for non-AI workloads, from simple web apps to hosting services like Google Docs on a global scale, we are still at an early stage in developing tooling and support for the specific requirements of training and hosting LLMs efficiently.

Besides mastering the technical requisites, DevOps teams need to start collaborating closely with Data Scientists to grasp the specific needs and workflows involved in developing, training, and hosting LLMs. This collaboration will ensure the infrastructure and processes are primed to deliver an optimal “Data-Science-Experience.” This approach advocates for agile working methodologies and efficient hosting of LLM workloads. Kubernetes is on its way to laying down the technical groundwork for generative AI, but the realization that DevOps and Ops play a critical role in the efficient development and hosting of LLMs hasn’t fully taken hold yet.

On the subject of technical groundwork, the DRA API (Dynamic Resource Allocation) - which is currently in alpha phase - represents a significant step forward. It caters to the demands of generative AI by ensuring sufficient GPU power while maintaining overall system dynamics.

WebAssembly

I'm a huge fan of WebAssembly, and not just for processing audio in web apps through the browser but also for its applications on the server and in "serverless" environments.

To give you a quick primer on WebAssembly: it's incredibly versatile but, at its core, serves as a compilation target for various programming languages. These can then be executed by WebAssembly runtimes, which are available in many different programming languages. Essentially, this means you can run one program inside another, offering a highly configurable and secure execution environment with clear methods for communication between the host program and the WebAssembly program. Plus, it's blazing fast, with virtually no overhead.

Sounds familiar? Running something inside something else? That's essentially what Docker and Kubernetes do as well. A container runs a program isolated from the host system. If we compile that program inside the container to WebAssembly, we can then execute that program in a WebAssembly runtime, eliminating the need for a container. Essentially, we could replace the container runtime in Docker or Kubernetes with a WebAssembly runtime.

What's the win here? TL;DR: Much less overhead compared to a container runtime. The detailed version: Particularly in serverless architectures, where "cold start" times are a bottleneck—because users don’t want to wait several seconds for a web application to respond—the duration it takes for a serverless function to be processed is crucial. Traditionally, a container hosting the serverless function (program) would spin up to handle an HTTP request from the end user. Just the container startup time could take a few hundred milliseconds; with the program inside processing the request, it might take 1-2 seconds to get a response, which is impractical for live user interaction. With WebAssembly (WASM) as the runtime, we can reduce startup time to as little as 1 millisecond(!), making it about 1000 times faster than container-based serverless systems. This opens up entirely new possibilities and areas of application.

The tooling around WebAssembly improves daily, and with SpinKube , we finally have a solution that significantly simplifies the implementation of WebAssembly in a Kubernetes cluster. And the great news is that SpinKube was submitted to the CNCF Sandbox during KubeCon.

Software Supply Chain Security

This is a highly relevant topic, with attacks constantly on the rise, forcing companies to respond. Although its implementation is becoming increasingly urgent, the tooling and workflows involved are complex, especially considering the need to secure every step from software development to deployment. There were many insightful talks on this subject, and we're seeing a growing landscape of tools designed to address these challenges. The setup and implementation of these tools are highly specific to the DevOps workflows and tools currently in use, prompting the development of general guidelines that are continuously being refined. For instance, we have the SLSA framework and the Software Supply Chain Best Practices (v1) by the CNCF TAG Security. A new version of the Software Supply Chain Best Practices (v2) is already underway.

Conclusion

It was a fantastic conference, highly recommended for anyone looking to grasp the big picture of running modern cloud applications and SaaS systems.

If you're navigating the complex world of Kubernetes and seeking the right tooling and training to maximize efficiency, feel free to get in touch with us.

Get in touch