9 CNCF tools for automation and configuration

Provisioning is an increasingly important element in the cloud-native stack. As software delivery times accelerate, there is a growing need to automate and quickly configure IT environments. Applying standard configurations on a cluster of nodes can greatly facilitate deployment and maintenance. Fortunately, many tools now exist to help operators quickly configure IT resources.

Below, we continue our exploration of useful tools from the Cloud Native Computing Foundation (CNCF). This group of tools focuses on automation and configuration concepts. Many CNCF tools in this area are native to Kubernetes, which automates the extension of the cloud-native architecture for edge computing, bare metal, and AI/ML processing. As of 2022, there are nine CNCF project incubation and sandbox projects that fall into this category.


Kubernetes native state-of-the-art computing framework (project under CNCF)

Website | GitHub

Edge computing is becoming more commonplace as businesses seek to reduce the costly egress of data to the Internet. Organizations may also prefer to manage IT at the edge for security reasons. KubeEdge, which became a CNCF incubator project in 2020, helps extend the cloud-native capabilities operators have come to expect at the edge. The framework can be used to help build a cloud computing ecosystem at the edge, managing unique constraints such as network reliability and resource limitations on edge nodes. Using KubeEdge, you can deploy ML/AI applications at the edge or scale highly distributed edge architectures.


A Kubernetes resource interface for the edge

Website | GitHub

Some operators may want to run Kubernetes on edge nodes. However, at the edge of a network, you can support many devices that are too small to run Kubernetes on their own. These devices often have intermittent availability and use unique communication protocols. For example, ONVIF is a standard used by many IP cameras.

The Akri open source project is designed to help better discover and manage small edge devices, also known as leaf devices. Akri is built on the native Kubernetes Device Plugins framework. According to the documentation, Akri excels at “handling the dynamic appearance and disappearance of leaf devices”. As of this writing, Akri is a sandbox project with the CNCF.


Cloud SDK for Kubernetes

Website | GitHub

One of the problems with Kubernetes is its complexity, which can lead to a steep learning curve. So being able to configure Kubernetes in the languages ​​you know best is one way to lower that barrier to entry.

CDK for Kubernetes (cdk8s) is a cloud-based development toolkit for Kubernetes that lets you use the same language to build and configure your application.

Using cdk8s, you can define applications in TypeScript, JavaScript, Python, Java, and Go. It then produces YAML, which can be applied to define Kubernetes applications for any cluster. This reduces the need to write a bunch of YAML templates and copy-paste; what some call YAML engineering. The tool was developed by AWS developers and later made open source for anyone to use. It is now a sandbox project within the CNCF.

Cloud custodian

Apply standard rules and cost optimizations for the cloud

Website | GitHub

Cloud Custodian is a robust yet simple toolkit for enforcing standard policies across your cloud infrastructure. It is a domain-specific language (DSL) YAML for defining policies that take management actions on cloud resources. The tool supports the three major cloud service providers: AWS, GCP, and Azure.

Using Cloud Custodian, you can replace ad hoc configurations with standard rules for things like security policies, access control, cloud cost optimization, and more. The documentation has many examples of policies that can be applied to your environment. For example, this policy will find any service running at 60% and raise the threshold by 30%:

  - name: account-service-limits
    resource: account
      - type: service-limit
        threshold: 60
      - type: request-limit-increase
        percent-increase: 30

The Cloud Custodian application itself is written in Python and can be run on most operating systems. To give it a try, you can read the getting started guides for AWS, Azure, and GCP.


A utility to easily run deep learning models on Kubernetes

Website | GitHub

KubeDL is another open source CNCF project that can be used to more easily configure and run your machine learning workloads using Kubernetes. KubeDL supports popular deep learning frameworks including TensorFlow, PyTorch, XGBoost, Mars, and MPI. These can all be run from a single controller.

For example, here is an example of setting up a training job with Tensorflow:

apiVersion: training.kubedl.io/v1alpha1
kind: "TFJob"
  name: "mnist"
  namespace: kubedl
  cleanPodPolicy: None
      replicas: 1
      restartPolicy: Never
            - name: tensorflow
              image: kubedl/tf-mnist-with-summaries:1.0
                - "python"
                - "/var/tf_mnist/mnist_with_summaries.py"
                - "--log_dir=/train/logs"
                - "--learning_rate=0.01"
                - "--batch_size=150"
                - mountPath: "/train"
                  name: "training"
                  cpu: 2048m
                  memory: 2Gi
                  cpu: 1024m
                  memory: 1Gi
            - name: "training"
                path: /tmp/data
                type: DirectoryOrCreate

Using KubeDL, you can manage models, track model versions, and automatically tune features to optimize running machine learning workloads in K8s. It also provides a way to store metadata for your projects, advanced scheduling features, the ability to sync files when launching the container, and other features. At the time of writing, KubeDL is a CNCF sandbox project.

Provisioning Bare Metal Hosts for Kubernetes

Website | GitHub

Metal3.io is a tool for provisioning Kubernetes on bare metal hosts. It offers a Kubernetes API to manage bare metal provisioning details; the provisioning stack itself runs on Kubernetes. Metal3.io uses the concept of a BareMetalHost to define the desired state of the host, bare metal health states, and provisioning details such as settings related to deploying an image.

Below is an example snippet from the documentation. Written in YAML, this is a partial example of running a cluster of a BareMetalHost resource.

apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
  creationTimestamp: "2019-09-20T06:33:35Z"
  - baremetalhost.metal3.io
  generation: 2
  name: bmo-controlplane-0
  namespace: bmo-project
  resourceVersion: "22642"
  selfLink: /apis/metal3.io/v1alpha1/namespaces/bmo-project/baremetalhosts/bmo-controlplane-0
  uid: 92b2f77a-db70-11e9-9db1-525400764849
    address: ipmi:
    credentialsName: bmo-controlplane-0-bmc-secret
  bootMACAddress: 98:03:9b:61:80:48
    apiVersion: machine.openshift.io/v1beta1
    kind: Machine
    name: bmo-controlplane-0
    namespace: bmo-project
  externallyProvisioned: true
  hardwareProfile: default

New features include pivoting as part of the CI workflow, which allows objects to be moved between clusters. As of this writing, Metal3.io is a sandbox project within the CNCF.


Extend the K8s to the Edge

Website | GitHub

OpenYurt is another tool to consider if you’re looking to bring cloud-native infrastructure like Kubernetes to the edge. It is a framework extensible enough to bring cloud-native features such as elasticity, high availability, logging, and DevOps to edge environments.

For example, OpenYurt provides self-healing capabilities. So if a node connection is disconnected, it can sync automatically once the connection is restored. It provides these and other features for orchestrating edge services and managing endpoint devices.

Many companies have used OpenYurt to extend the native Kubernetes experience to edge environments in logistics, transportation, IoT, CDN, retail, and manufacturing spaces. As of this writing, OpenYurt is a sandbox project within the CNCF.


Container Management for Edge Computing

Website | GitHub

SuperEdge is another framework for extending Kubernetes to edge environments. Its core feature set includes components like edge-health, which runs on endpoints to detect their health status. There is also lite-apiservera lightweight version of the Kubernetes API server that provides caching and authentication functionality.

SuperEdge also uses a network tunnel to proxy requests between cloud and edge. By using these proxies, the project presents itself as a non-intrusive tool for configuring edge devices. SuperEdge was created by Tencent cloud and is now a CNCF sandbox project.

Tinker Bell

A workflow engine for bare metal provisioning

Website | GitHub

Another utility designed to make bare metal provisioning easier is Tinkerbell, the open source bare metal provisioning engine powered by Equinix. It includes five key microservices: a network server, a metadata service, an operating system installation environment, and a workflow engine. The workflow engine, called Tink, is the main provisioning engine that communicates using gRPC and offers a CLI that developers can work with.

Tinkerbell is generic enough to work with any operating system and provides declarative APIs to control automation programmatically. And since Tinkerbell is backed by Equinix Metal, you can pretty much guarantee that the project will be actively maintained well into the future. Tinkerbell is a CNCF sandbox project. For more information, you can consult the documentation here.

Final Thoughts

As you’ll notice, many of these tools can be run on Kubernetes, allowing you to manage your infrastructure the same way you manage your applications. Since we’ve already seen so much investment in cloud setup, much of the new development in this area is in setup for alternative scenarios, like bare metal, edge, and IoT.

The Cloud Native Computing Foundation (CNCF) has become a hub of great tools to support operations across the entire cloud native stack. And apart from configuration, there are many packages for service proxies, persistent storage, scheduling, CI/CD, etc.

Comments are closed.