Unlocking Full Potential of AWS with Karpenter-2

After the Introduction to Karpenter Part 1 of the Blog, it’s time to dive deeper into its key features and the benefits it offers.

Key Features & Benefits of Karpenter

EKS best practices recommend using Karpenter over Autoscaling Groups, Managed Node Groups, and Kubernetes Cluster Autoscaler in most cases. Here’s why Karpenter has the edge:

Advantages of Karpenter

1. Consolidation: A noteworthy attribute of Karpenter is its competence in consolidation, an automated process that optimizes resource utilization and efficiency. This is achieved by fine-tuning the cluster size, purging underutilized nodes, and consolidating pods to harness available resources optimally. Consequently, this contributes to additional cost savings and an enhanced overall efficiency.

   In the Custom Resource Definition (CRD) for Karpenter’s Provisioner, workload consolidation can be enabled. The lifecycle activities of the cluster’s Karpenter-controlled nodes are the responsibility of the provisioners. They enable teams to specify capacity restrictions as well as node behaviour (such as expiration and consolidation).

   You can refer to more resource here.

   Note: Karpenter enables rapid node startup and swift response to dynamic resource requests. This capability allows for seamless handling of workload provisioning, ensuring efficient utilisation of resources in a timely manner.

   
   

2. Scalability: Karpenter provides more sophisticated scaling ability to recognise particular workload requirements. That is used to divide resources. This indicates that it can grow in accordance with actual usage.

Downscaling with Karpenter:

Karpenter provides downscaling control that is more precise and granular. According to the demands of the job, it enables you to establish guidelines and regulations for when and how to cut back. Costs can be decreased and underutilization prevented as a result.

1. Efficiency: Karpenter can help you save money on cloud costs by allowing you to pay only for the resources used by your Kubernetes workloads. You can utilize spot instances with On-Demand fallbacks to achieve cost savings.

   Instance Types for the Karpenter – you can refer more here in order to get the instances that is contributing to the possible node scaling.

   
   

2. Cost-effectiveness: Karpenter actively minimizes cluster costs by determining when it can remove nodes because other cluster nodes can handle their workload. It also identifies opportunities to replace nodes with more cost-effective alternatives based on changes in the workload.

   Karpenter considers your pod disruption budgets while scaling down nodes and evicting pods.

   
   

3. Efficient GPU Node Provisioning : Karpenter’s auto scaling efficiently allocates GPU instances based on workload needs, optimizing resource utilization by saving costs. Flexible scheduling allows effective allocation of GPU resources for multiple applications within the Kubernetes cluster.

You can leverage custom resources at aws.amazon.com/neuron for use cases that require accelerated EC2 instances..

Component diagram for the GPU Instance Allocation

Updating and Upgrading the Karpenter Nodes:

Efficient way to update and Upgrade Cloud AMI Images: Karpenter updates and rotates nodes with the latest updates and patches to the AMI, which will allow us to run effective compute resources based on workload requirements and upgrade instances with a lesser amount of technical Depth and downtime.

Updating and Upgrading the AMI Images in the Nodes from Different Instances

Karpenter’s Limitations:

While Karpenter offers various advantages, it is important to consider its limitations, which include:

1. Inability to optimise Cloud Spend based on existing commitments:

   Karpenter currently does not have the ability to optimise spending by taking into account existing commitments such as savings plans or reserved instances. As a result, there is a possibility of spending more if users are not aware of their pre-existing commitments.

   Note: Users must deploy Karpenter’s pod in a controlled node group. However, they can now run it on Fargate Instances with some modifications.

   
   

2. Can’t dynamically adjust spot prices(Missing Feature):

   Karpenter’s spot pricing strategy remains static and doesn’t dynamically adapt to changes in compute utilisation. As a result, over time, this can lead to less-than-optimal cost outcomes since the pricing may not align with the current workload demands.

   You can also refer to this GitHub issues: https://github.com/aws/karpenter/issues/3044

   
   

3. Complexity in configuration:

   Configuring Karpenter can prove complex and demands technical expertise. Users must prepare for this intricacy when implementing Karpenter. While it offers immense power, it may necessitate greater expertise and resources compared to alternative scaling solutions.

   Note: Karpenter can plan workloads according to pre-determined standards/configurations. Resource requirements, availability regions, and price can all be criterion. In order to reduce expenses and increase efficiency, workload scheduling can be optimised. However, to get the schedule perfect, extra configuration and fine-tuning may be required.

   
   

4. Spot Instance Terminations at a short notice:

   EC2 spot instances typically give only a 2-minute warning before terminating, which might not be sufficient for certain workloads to handle termination and rescheduling optimally.

   In contrast, Karpenter takes a different approach by relying on AWS Node Termination Handler (NTH) to gracefully cordone and drain spot nodes in case of disruptions, avoiding the limitations of the two-minute Spot Interruption Termination Notice (ITN).

Always Avoid using Custom Launch Template while Adopting Karpenter

It is not Advisable to use Custom Launch Templates, as they can hinder multi-architecture flexibility, automatic node upgrades, and precise securityGroup detection. Furthermore, these templates can introduce confusion by replicating certain fields while disregarding others, including crucial aspects like subnets and instance types. Optimize your cluster’s potential by aligning with Karpenter’s recommendations for seamless, efficient provisioning.

You can usually skip using launch templates by using your own user data and/or directly starting custom AMIs in the AWS node template. You can find more details on how to do this in the Node Templates section.

Efficient Performance with Karpenter Provisioning

Karpenter exhibits a distinct advantage over the Kubernetes Cluster Autoscaler in provisioner configurations, showcasing superior performance. This superiority stems from its direct management of node groups, eliminating the dependence on Auto Scaling Groups (ASG) and resulting in swift pod binding and enhanced overall performance.

Through the utilization of Karpenter’s adaptable provisioner, enterprises can effectively allocate both Spot and On-Demand instances, dynamically adapting to the demands of their workloads. Moreover, Karpenter extends support to arm64 (AWS Graviton) worker nodes, which gives optimal performance at a lower cost.

In the next blog we can learn more about how to deploy applications in Action.

References to Karpenter and some useful resources:

1. Karpenter consolidation in Kubernetes

2. Optimizing your Kubernetes compute costs with Karpenter consolidation

3. K8’s Device Plugin