KubeCon + CloudNative North America 2022

CustomResourceDefinitions are prolific in Kubernetes. With so many new projects being prototyped, developed and released into the ecosystem, it's essential to ensure you're designing your APIs in a scalable, well tested way. Projects like Kubebuilder and the Operator Framework provide plentiful tooling to keep you on 'the straight and narrow', but some fundamental issues still trip up even the most experienced Kubernetes engineers. In this talk, we'll go over strategies & tooling that can help ensure you are successful when authoring and evolving your APIs, including:

* How to ensure your APIs follow best practices/standards (and linting OpenAPI schemas!)
* Fuzz & integration testing API version conversion functions to ensure platform stability in your clusters
* Automatically validating your OpenAPI schemas are up to date for your type definitions
* Best practices in 2022 for writing validations & mutations
* Ensuring your OpenAPI schema changes are backwards compatible within an API version

James has helped bring various API groups to stability across projects in the ecosystem and both he and Andrea have experience with unique problems that are found when projects reach a certain scale & maturity, having helped take CRDs through a full alpha to beta to GA deprecation cycle.

kube-scheduler is a key component of kubernetes. It has evolved with many new features over the years. In order to better use and manage kubernetes to meet practical needs of today’s increasingly diverse workloads in large production clusters with complex configurations, it’s essential to understand how kube-scheduler works, what features are available, and how to properly configure and manage them. This tutorial will review the basics of kube-scheduler, including kube-scheduler basic features, scheduling framework, scheduler plugins and profiles, how to configure kube-scheduler with different parameters, plugins and multiple profiles, and how to test and evaluate kube-scheduler on a local machine and in a cluster. The tutorial will present the latest scheduling features and how to use them. Advanced topics such as bin-packing, scheduling scalability, batch support, and how to extend the default kube-scheduler with custom scheduler plugins will be introduced too. The tutorial includes demos and hands-on sessions and is suited for kubernetes administers, users and developers, from beginning to advanced levels. The audience is expected to come prepared with a personal laptop with kubernetes, e.g., Minikube, installed.

Cloud-native community has long desired the ability to resize pods in-place because pod restarts are expensive for long-running applications and disruptive to services. To resize pods, we commonly rely on Vertical Pod Autoscaler (VPA) to observe usage, recommend, and reactively enact resource updates.

An alternative approach is to resize pods based on deterministic events. For example, you can capture events with eBPF to detect when a CPU-intensive command is going to be executed, and proactively resize the pod CPU accordingly. In this talk, Pablo will show an interesting use case where remote development environments run inside pods. These pods need minimal resources when a developer is writing code, but need significantly higher CPU & memory when a developer issues a “build” command or runs a battery of tests. In-place resize is mandatory in this scenario, otherwise, the development experience would be broken on every pod restart.

Vinay will then talk about the current in-place pod resize feature design, which is soon landing as alpha in Kubernetes. He will go over the CRI changes, discuss the design rationale & trade-offs. He will then lay out the next steps and discuss what the community can do to help to drive this feature to a rock-solid GA over the next year.

Three years ago ING Wholesale Banking Advanced Analytics team set up an ambitious goal to gather in one place a curated portfolio of internal data sources together with a large scale compute platform. At its core the idea of allowing internal projects to get access to a rich toolset of open source and industry standards frameworks and preprocessed data to validate business ideas in the secure exploration environment. Extensive growth with over 300 internal projects so far and more than 2000 internal users proofs advanced analytics i.e. ML, AI, NLP capabilities should become easily consumable not only by specialized, dedicated teams, but make them close to subject matter experts. In this session we would like to shed more light on how a specialized cloud native Kubernetes scheduler (Volcano) enables us to deliver multi-tenant large scale processing capabilities. The optimal resource usage with stability of core services are key for our cloud native platform. To enable dynamic allocation and hdrf (hierarchical dominant resource fairness) we have created an extension to Apache Spark binaries. This allows users to use Volcano with Spark interactive mode in a Jupyter notebook. Additionally we have created interfaces to visualize all the scheduling metrics like the yarn ui.

With the motto of "making cluster lifecycle management easy", Cluster API adoption has been increasing each year since its inception. Its pluggable architecture via providers is vendor agnostic and makes its adoption across various infrastructure environments (public clouds, on-premises) easy and gives its users the flexibility of using a variety of infrastructures with the same set of tools and workflows. Writing a provider follows a de-facto pattern (there are >20 providers created following this pattern). The talk “Building Your Own Cluster API Provider the easy way” at KubeCon EU 2022 was well received with 600+ registering (and more than that attending), and many indicating they are at various levels in their provider journey. While the talk touched upon high-level constructs of writing a provider, there is a lot of functionality to be covered in order to make the provider usable or to reach the first release milestone. This tutorial aims to be an in-depth hands-on exercise where the audience can get to a fully CAPI conformant provider starting from scratch.

Everything can be done from your laptop. Where possible, please come to the session with the prerequisites installed.

"Tales from the Kubernetes controller factory floor" If you have used the Kubernetes controller-tools and kubebuilder projects to create a custom Kubernetes controller, you already know the immense power of these tools. However, a significant amount of work remains to implement a controller once kubebuilder has produced API types and basic controller scaffolding. What if you had to build dozens of controllers managing thousands of resources? You'd need a factory to produce full controller implementations from API model schemas. Amine and Jay happen to work in such a Kubernetes controller factory! In this talk, they will give you a tour of the Kubernetes controller factory, showing you how to use the API machinery and what dangers linger on the factory floor. They will teach you how to be the most productive worker in the whole factory by building on top of controller-tools and kubebuilder functionality. You will be introduced to open source tools and strategies that make Kubernetes controller factory life safe and enjoyable!

The days of needing custom scripts and hard-won knowledge to set up and configure Windows Server hosts are over. The new Windows HostProcess Container feature in Kubernetes has begun to close a major feature gap between Windows and Linux containers. We now have a way to carry out key tasks such as running kube-proxy and CNIs (Container Network Interface) as containers in the cluster. Beyond the basics, HostProcess containers also open the door to more effective logging, monitoring and debugging of the Windows environment.   In this session, we will cover the basics of using HostProcess containers and see how they differ from other Windows containers. After gaining an understanding of these differences, we will explore three examples covering the common use cases for HostProcess containers: CNIs, debugging, and on-demand monitoring. The attendees will see concrete examples of HostProcess Containers, novel approaches to debugging, and gain inspiration for new ways of interacting with Windows enabled clusters.

Kubernetes controllers are responsible for making the current state of your cluster continue to become closer to your desired state. Have you ever wondered how these built-in controllers work? Or have you ever wanted to write your own controller to manage Custom Resources? In this 90 minute tutorial, we'll walk you through building your own controller using controller runtime, the set of common libraries on which core controllers are built. We'll use Kubebuilder, a framework for building APIs using custom resource definitions (CRDs). We'll also explain lesser-documented best practices and conventions for writing controllers that the community has developed through trial and error learning, through projects such as Flux and Cluster API. Attendees will gain an understanding of what Kubernetes conditions are, how to set and respond to them, and why they matter. We’ll review common pitfalls and additional helper libraries to make writing these easier, more reliable, and enjoyable!

Kubernetes adopters have realized the benefits of declarative APIs and the ability to leverage modern deployment practices such as GitOps for safe and repeatable application delivery. These teams naturally wish to apply these same processes and tools to manage infrastructure deployments. The Crossplane project extends Kubernetes to enable the provisioning of cloud infrastructure. Combined with Argo CD, they become a powerful infrastructure management dashboard. This talk covers the benefits of using Kubernetes as a control plane of your cloud infrastructure over Terraform and CloudFormation, such as native RBAC and seamless integration with other cloud-native tools. See how Akuity uses Argo CD and Crossplane to manage its production AWS infrastructure. Learn to leverage advanced Argo CD features (health checks, resource actions, extensions) to get the most out of your Crossplane installation. Implement best practices recommended directly from the project maintainers Upbound and Akuity.

In Kubernetes 1.23 we integrated the CEL expression language into open source Kubernetes, making it possible to support the vast majority of CRD validation use cases without a webhook. This includes multi-field validation rules, immutability checks and more.  And this is just the beginning, we plan to extend admission control to support CEL expressions as well, which will make it possible to replace far more of those operationally troublesome webhooks with a much simpler alternative. We're convinced this leads to a better development experience for anyone extending Kubernetes. And more importantly, it makes cluster operations simpler and safer.  Learn about this future of Kubernetes extensibility from a contributor who has been involved in Kubernetes extensibility for over 5 years, including the projects to bring CRDs and Webhooks to GA, and who has been involved in improving the stability of Kubnernetes control planes in GKE for years.  In this talk I'll introduce CEL and how we've integrated it into Kuberentes and answer questions including: What can you do with CEL in Kubernetes today? What future features are planned? Can there really a future where webhooks are the exception instead of the norm?

CustomResourceDefinitions are driving the extension ecosystem around Kubernetes. This talk is about the search for the next step, a successor for CRDs in a post-operator world where service providers use CRDs as first-class API for the services they are building and offering to tenants. CRDs as we know them are installed in customer clusters, usually together with operators or controllers. With that they are under control of the users: - users can tweak the CRDs. - users are the ones updating and controlling the operators with all the complexity and pitfalls updating operators and APIs can have. This situation is not a good fit for today's problems, and it's mostly an artifact of how CRDs and their life-cycle were conceived years ago as a tool to add in-cluster concepts. This talk is about lifting CRDs up to be a first-class verhicle for APIs provided and consumed by different parties, without the operator-glue, in different clusters, standardized, securely and federated.