Building a Kubernetes CRD

Kubernetes CustomResourceDefinitions (CRDs) are a powerful mechanism for exposing custom objects as native Kubernetes primitives. This article will provide an overview of CRDs and demonstrate how to build a CRD that reveals the dependency details of a deployed microservice.

Introduction

Kubernetes CRDs

In the realm of Kubernetes, a multitude of resources are readily accessible and manageable through standard Kubernetes commands. Yet, when the need arises to extend Kubernetes capabilities for custom objects, Custom Resource Definitions (CRDs) emerge as a pivotal solution.

Kubernetes Custom Resource Definitions (CRDs) transform an object into a first-class citizen within the Kubernetes ecosystem. This means you can harness the full potential of standard Kubernetes tools to interact with and oversee these custom objects, just as you would with the built-in Kubernetes resources.

Build a Kubernetes CRD for a CI/CD pipeline

To witness Kubernetes in action, let’s embark on a journey to create our own Custom Resource Definition (CRD). In this endeavor, we’ll craft a CRD that serves as a repository for critical data about your microservice’s external service dependencies. These dependencies span a wide spectrum, including data storage components like databases and in-memory caches, as well as connections to vital cloud services, such as secrets managers like Key Vault in Microsoft Azure.

Our custom resource will be an object containing information about our external dependencies. We will capture the following information for our service:

Dependency nameDependency URLDependency type (should be one of datastore, cloudservice, or microservice)

To do this, we need to create a custom resource definition and apply it to our Kubernetes cluster using the kubectl apply command. Then, we’ll create an instance of the custom resource and upload it using the kubectl apply command.

Steps to Perform

To complete this walkthrough, you’ll need access to a Kubernetes environment. We are using a local environment powered by minikube.

Create a CustomResourceDefinition

CustomResourceDefinitions are YAML-formatted files. Here’s an example definition for our external dependencies resource structure. Save this as a file named dependencies.yaml. 

apiVersion: "apiextensions.k8s.io/v1"
kind: CustomResourceDefinition
metadata:
  name: dependencies.example.com
spec:
  group: example.com
  scope: Namespaced
  names:
    plural: dependencies
    singular: dependency
    kind: Dependency
    shortNames:
      - dep
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            dependency:
              type: object
              properties:
                name:
                  type: string
                  minimum: 3
                url:
                  type: string
                  minimum: 10
                type:
                  type: string
                  minimum: 3

apiVersion specifies the version for the CRD extension itself. There are older versions (for example, v1beta1) that uses slightly different YAML formats. Additionally, not all versions may be supported in your deployment of Kubernetes. So make sure the version and syntax specification align. In this example, we’re using CRD version v1.

The metadata section scopes our CRD for API purposes. Kubernetes will automatically create API endpoints that enable basic CRUD (Create, Read, Update, Delete) operations on our objects. The name property will become part of the path of our newly created APIs, which we’ll be able to access at api/dependencies.example.com/v1/.

Next comes the specification for our CRD, defined under spec. Before defining our object, we include some meta-information:

group must match the latter part of the namespace we chose under name. scope determines whether our CRD is scoped to the cluster or scoped to our namespace. names controls how we’ll refer to our objects in API calls and using the Kubernetes CLI.

We can also define a shortName to help save on typing. 

After this, we specify our CRD under versions. We can publish multiple versions so that we don’t break backward compatibility with existing tools or CI/CD scripts. The served flag means this version is enabled. 

storage –  must be used for the most recent version to designate it’s the version format used for storage in etcd. Kubernetes will convert requests for previous versions to the appropriate version spec on the fly.

Then, under schema, we define our custom resource using OpenAPI definition syntax. You’ll recognise this format if you’ve created OpenAPI specifications or used Swagger. 

Now, we will discuss how you can use additional OpenAPI definition features to strengthen type safety and runtime parameter checking.

Create a CustomResourceDefinition

After creating our CustomResourceDefinition, we can upload it to our Kubernetes cluster using the kubectl apply command:

kubectl apply -f dependencies.yaml

If your CRD is syntactically correct, Kubernetes will create it immediately. The CRD is simply a template for a custom resource. To store resources, you must create them in a separate YAML file and upload them to your cluster. The file below creates a dependency object that conforms to our custom resource definition. Save this to a file named dependency-cr.yaml.

apiVersion: "example.com/v1"
kind: Dependency
metadata:
  name: api-dependency
dependency:
  name: storageapi
  url: https://example.com/storageapi/v1/
  type: microservice

Key considerations:

• The apiVersion field is our spec group combined with the version of our custom resource definition.

• name will appear in URL paths, so Kubernetes enforces the use of RFC 1123 syntax. This means you must use only lowercase alphanumeric characters (a-z, 0-9) or the ‘-’ and ‘.’ characters.

• dependency and all of its fields must correspond to the OpenAPI definition we published earlier. 

  
  

  Save this to a file named dependency-cr.yaml and upload it to Kubernetes: 

kubectl apply -f dependency-cr.yaml

Operate on a custom resource with kubectl

Once published, you can use kubectl commands to manage your dependency objects. To list the object you just created, run the below command: 

kubectl get dependencies

Amnics-MacBook-Pro: tmp amnic$ kubectl get dependencies
NAME                AGE
api-dependency       9s

For dependencies, we can use the shortName we defined earlier:

kubectl get dep

You can use other basic kubectl commands to manage your custom resource. To view the full details of the resource, run the below command:

kubectl get dep api-dependency -o yaml

To delete it, run the below command: 

kubectl delete dep api-dependency

Apply data constraints

We can enhance our CRD by incorporating further restrictions on the values that can be assigned to specific fields. For instance, as previously mentioned, we will only permit three options for the type field: datastore, cloudservice, and microservice.

Add the below snippet to our file dependencies.yaml:

enum:
 - datastore
 - cloudservice
 - microservice

After adding the above snippet to our file, it should look like:

type: string
minimum: 3
enum:
 - datastore
 - cloudservice
 - microservice

You can force update your existing CRD like so: 

kubectl replace -f dependencies.yaml

Now, try and publish a custom dependency resource with an unsupported value for type. Create another file, dependency-cr-1.yaml, with the below content: 

apiVersion: "example.com/v1"
kind: Dependency
metadata:
  name: api-dependency-1
dependency:
  name: storageapi
  url: https://example.com/storageapi/v1/
  type: macroservice

Now, let’s try to apply the newly made changes:

kubectl apply -f dependency-cr-1.yaml

This command will result in an error indicating that macroservice is not a supported type.

Conclusion

In this blog, we demonstrated how to set up and utilize custom resources in Kubernetes. In our next post, we will delve deeper by constructing a custom Kubernetes operator.