Blog

Committing to AWS compute capacity through Savings Plans or Reserved Instances can save 40-70% versus On-Demand pricing, but choosing the wrong commitment type or size can lock you into wasted spend. Dynamic EKS workloads that scale with traffic and use Spot instances require a fundamentally different commitment strategy than traditional VM-based infrastructure.

Key Takeaways

• Compute Savings Plans offer more flexibility than Reserved Instances for dynamic EKS fleets
• Savings Plans apply to Fargate and Lambda; Reserved Instances do not
• Spot instances don’t count toward Savings Plan or RI commitments—commit only for baseline On-Demand capacity
• Start with conservative commitments at 50-60% of baseline and increase gradually based on usage data
• Combine Savings Plans for steady-state workloads with Spot for burst capacity to maximize savings

Understanding Reserved Instances vs Savings Plans

Reserved Instances (RIs)

Reserved Instances are capacity reservations for specific instance types in specific regions. You commit to paying for a particular instance (e.g., m5.large in us-east-1) for 1 or 3 years, and AWS gives you a discount (typically 30-60% for 1-year, up to 70% for 3-year commitments).

How they work:

• Purchase an RI for m5.large, us-east-1, Linux
• AWS billing automatically applies the RI discount to matching instances
• If you run 10× m5.large but only bought 5 RIs, 5 instances get the discount and 5 pay On-Demand
• RIs are region-specific but can float across AZs (regional RIs)

Key constraints:

• Instance type specific (m5.large RI doesn’t apply to m5.xlarge)
• Cannot change instance family (m5 to c5 requires selling in RI marketplace)
• Does not apply to Fargate or Lambda
• Risk of waste if workload patterns change

Compute Savings Plans

Savings Plans are commitments to spend a certain dollar amount per hour (e.g., $10/hour) on compute, regardless of instance type, region, or even service. AWS applies discounts automatically to any eligible usage up to your commitment level.

How they work:

• Commit to $10/hour of compute spend for 1 or 3 years
• AWS applies the commitment to EC2, Fargate, and Lambda usage automatically
• Covers any instance type, any region, any OS
• Discount rates similar to RIs (30-66% for 1-year, up to 72% for 3-year)

Key advantages for EKS:

• Flexibility across instance families (covers m5, c5, r6i, Graviton, etc.)
• Applies to Fargate pods and Lambda functions
• Automatically adapts as you change instance types (e.g., migrate to Graviton)
• Simpler management (no need to track specific instance types)

EC2 Instance Savings Plans

A middle ground: commit to a dollar amount per hour for a specific instance family (e.g., m5) in a specific region. More flexible than RIs (covers m5.large, m5.xlarge, etc.) but less flexible than Compute Savings Plans.

When to use: If you know you’ll stay within one instance family (e.g., general-purpose m-family) but want size flexibility, EC2 Instance Savings Plans offer slightly higher discounts than Compute Savings Plans.

Feature Comparison Matrix

Feature	Reserved Instances	EC2 Instance Savings Plans	Compute Savings Plans
Discount range (1-year)	30-60%	33-66%	30-66%
Instance type flexibility	No (locked to type)	Yes (within family)	Yes (any type)
Instance family flexibility	No	No	Yes
Region flexibility	No (regional RIs only)	No	Yes
Applies to Fargate	No	No	Yes
Applies to Lambda	No	No	Yes
Covers Spot instances	No	No	No
Change/sell commitment	Yes (RI marketplace)	No	No
Best for	Stable, predictable workloads	Single instance family, flexible sizes	Dynamic, multi-family, or Fargate

Why Compute Savings Plans Win for EKS

EKS workloads are inherently dynamic:

• Autoscaling changes instance counts hourly (HPA scales pods, Cluster Autoscaler/Karpenter scales nodes)
• Karpenter provisions diverse instance types (m5.large one hour, r6i.xlarge the next)
• Teams migrate to new instance families (x86 to Graviton, older to newer generations)
• Some workloads run on Fargate while others use EC2

Reserved Instances lock you into specific instance types that may not match what Karpenter provisions or what your workloads need next month. Compute Savings Plans adapt automatically—they apply to whatever instances your cluster actually uses.

Example scenario:

• Month 1: You run 50× m5.large instances
• Month 2: Karpenter diversifies to m5a.large and m6i.large for better Spot availability
• Month 3: You migrate some workloads to Fargate

With RIs, the m5.large reservation becomes underutilized in month 2-3. With Compute Savings Plans, the $X/hour commitment automatically applies to m5a, m6i, and Fargate without intervention.

Calculating Your Baseline Commitment

The core principle: only commit for your steady-state baseline. Use Spot and On-Demand for everything above baseline.

Step 1: Identify Baseline Usage

Baseline is the minimum compute you run 24/7, even during low-traffic periods.

Pull 30-90 days of historical usage from Cost Explorer:

aws ce get-cost-and-usage \ --time-period Start=2024-11-01,End=2025-02-01 \ --granularity HOURLY \ --metrics UsageQuantity \ --group-by Type=DIMENSION,Key=INSTANCE_TYPE \ --filter '{ "Dimensions": { "Key": "SERVICE", "Values": ["Amazon Elastic Compute Cloud"] } }'

Plot hourly instance-hours. The baseline is the minimum sustained usage (typically the lowest 10th percentile).

Example data:

• Peak: 200 instance-hours/hour (during business hours)
• Off-peak: 80 instance-hours/hour (nights/weekends)
• Absolute minimum: 70 instance-hours/hour

Baseline recommendation: Start with 60-70 instance-hours/hour (80-90% of minimum). This leaves headroom for unexpected dips and ensures you don’t over-commit.

Step 2: Convert to Dollar Commitment

Calculate the On-Demand cost of your baseline usage.

Example: Baseline of 70 instance-hours/hour, average instance type m5.large ($0.096/hour):

Hourly baseline cost = 70 instances × $0.096 = $6.72/hour

Use AWS Savings Plans recommendations tool to see exact discount rates for your usage patterns.

Conservative approach: Start with a commitment covering 50-60% of baseline. For the example above:

Initial commitment = $6.72 × 0.6 = $4.03/hour (~$2,950/month)

After 60-90 days, analyze utilization and purchase additional Savings Plans if you’re consistently exceeding commitment.

Step 3: Account for Spot Usage

Spot instances do NOT count toward Savings Plans or RIs. If 50% of your capacity is Spot, your commitment should only cover the On-Demand/RI-eligible portion.

Example adjustment:

• Total baseline: 70 instance-hours/hour
• Spot percentage: 50%
• On-Demand baseline: 35 instance-hours/hour
• Commitment: 35 × $0.096 × 0.6 = $2.02/hour (~$1,475/month)

This prevents paying for commitments on capacity you’ll run as Spot anyway.

Combining Savings Plans with Spot

The optimal EKS cost strategy layers three pricing models:

1. Compute Savings Plans for baseline On-Demand capacity (30-40% of total)
2. Spot instances for burst and interruptible workloads (50-60% of total)
3. On-Demand for anything above Savings Plan commitment and Spot fallback (10-20%)

Example cluster cost breakdown:

Capacity Type	Percentage	Cost (normalized)	Effective Rate
Savings Plan covered	35%	$3,360	$0.040/hour (60% discount)
Spot instances	55%	$2,640	$0.020/hour (80% discount)
On-Demand (above commitment)	10%	$960	$0.096/hour (no discount)
Total	100%	$6,960	$0.039/hour avg

Compared to 100% On-Demand at $0.096/hour = $18,470/month, this hybrid approach saves 62%.

Coverage Strategy: How Much to Commit

Conservative (Recommended for New Clusters)

• Coverage target: 50-60% of baseline On-Demand usage
• Rationale: Leaves room for workload changes, prevents over-commitment
• Review cadence: Quarterly, increase commitment as usage stabilizes

Moderate (Stable Workloads)

• Coverage target: 70-80% of baseline On-Demand usage
• Rationale: Captures most baseline savings while retaining flexibility
• Review cadence: Quarterly adjustments

Aggressive (Mature, Predictable Fleets)

• Coverage target: 90-100% of baseline On-Demand usage
• Rationale: Maximum discount for highly predictable workloads
• Risk: Over-commitment if baseline drops (e.g., product pivot, migrations)

Warning: Avoid purchasing 3-year commitments until you have 12+ months of stable usage data. Start with 1-year commitments and renew annually.

Monitoring and Adjusting Commitments

Track Utilization Rate

AWS Cost Explorer shows Savings Plans utilization percentage. Target: >95% utilization.

aws ce get-savings-plans-utilization \ --time-period Start=2025-01-01,End=2025-02-01 \ --granularity MONTHLY

• >98% utilization: You may be under-committed; consider increasing
• 85-95% utilization: Healthy range
• <80% utilization: Over-committed; wait for commitment to expire before adding more

Track Coverage Percentage

Coverage shows what percentage of eligible On-Demand usage is covered by Savings Plans.

aws ce get-savings-plans-coverage \ --time-period Start=2025-01-01,End=2025-02-01 \ --granularity MONTHLY

• <50% coverage: Opportunity to purchase more Savings Plans
• 50-70% coverage: Typical for dynamic EKS workloads
• >80% coverage: Risk of over-commitment unless workload is very stable

Set Up Alerts

Create CloudWatch alarms for Savings Plans utilization dropping below 85%:

aws cloudwatch put-metric-alarm \ --alarm-name savings-plan-underutilization \ --metric-name SavingsPlansUtilization \ --namespace AWS/SavingsPlans \ --statistic Average \ --period 86400 \ --threshold 85 \ --comparison-operator LessThanThreshold

Common Mistakes

Purchasing 3-year commitments too early. You lock in prices before optimizing workloads. Start with 1-year, measure savings, then commit longer-term.

Committing for Spot-eligible capacity. Spot doesn’t count toward Savings Plans. Calculate baseline as On-Demand-only usage.

Buying RIs for Karpenter-managed clusters. Karpenter provisions diverse instance types dynamically—RIs lock you into specific types. Use Compute Savings Plans instead.

Not accounting for growth. If you’re scaling 20% per quarter, your baseline will outgrow commitments quickly. Review and adjust quarterly.

Ignoring Fargate in commitment strategy. If you run Fargate pods, Compute Savings Plans apply but RIs don’t. Missing opportunity for savings.

Decision Framework

Choose Compute Savings Plans if:

• You run EKS with Karpenter (dynamic instance provisioning)
• You use Fargate or plan to in the future
• You’re migrating between instance families (e.g., x86 to Graviton)
• You run workloads across multiple regions
• You value flexibility over maximum discount

Choose EC2 Instance Savings Plans if:

• You know you’ll stay within one instance family (e.g., m5 only)
• You want slightly higher discounts than Compute Savings Plans
• You don’t use Fargate

Choose Reserved Instances if:

• You run very stable workloads with fixed instance types
• You want the ability to sell unused capacity in the RI marketplace
• You’re certain instance types won’t change for 1-3 years

Reality for most EKS users: Compute Savings Plans offer the best balance of discount and flexibility.

Implementation Checklist

1. Gather 60-90 days of usage data from Cost Explorer
2. Calculate baseline On-Demand usage (exclude Spot)
3. Start conservative: Purchase Savings Plans covering 50-60% of baseline
4. Monitor utilization weekly for first month, then monthly
5. Adjust quarterly: Increase commitment if utilization >95%
6. Layer Spot aggressively for burst capacity (50-70% of total fleet)
7. Review annually: Consider 3-year commitments only after 12+ months of stable usage

Conclusion

For dynamic EKS workloads, Compute Savings Plans offer the best commitment strategy—they provide flexibility across instance types, families, and services (including Fargate) while delivering discounts comparable to Reserved Instances. Start by identifying your baseline On-Demand usage excluding Spot, commit conservatively at 50-60% of that baseline, and increase gradually based on utilization data. Never commit for Spot-eligible capacity, and avoid 3-year commitments until you have at least 12 months of stable usage patterns. The optimal EKS cost structure layers Compute Savings Plans for baseline, Spot for burst capacity, and On-Demand as overflow—typically achieving 60-70% total savings versus pure On-Demand pricing. Monitor utilization monthly, adjust commitments quarterly, and resist the temptation to over-commit based on peak usage or growth projections.

Comparing AWS container services solely on compute pricing misses the bigger picture. Total Cost of Ownership includes operational labor, training, tooling, and the hidden costs of managing control planes and node infrastructure. A service that costs 20% more per vCPU but requires half the engineering time can deliver better TCO for many teams.

Key Takeaways

• ECS has the lowest per-vCPU cost but requires AWS-specific expertise and limits portability
• EKS provides Kubernetes portability but adds control plane fees ($73/month per cluster) and operational overhead
• Fargate eliminates node management but costs 30-50% more per vCPU and has one-minute minimum billing
• For teams under 5 engineers, Fargate’s labor savings often outweigh higher compute costs
• TCO calculations must include training time, on-call burden, and opportunity cost of infrastructure work

Understanding the Three Options

Amazon ECS (Elastic Container Service)

ECS is AWS’s proprietary container orchestration service. You define task definitions (similar to pod specs), and ECS schedules containers on EC2 instances or Fargate. It integrates deeply with AWS services but doesn’t use Kubernetes.

Key characteristics:

• No control plane fees (ECS itself is free)
• Pay only for EC2 instances or Fargate capacity
• AWS-native tooling (CloudFormation, CDK, CLI)
• No Kubernetes portability

Amazon EKS (Elastic Kubernetes Service)

EKS runs managed Kubernetes control planes. You use standard Kubernetes APIs (kubectl, Helm) to deploy workloads on EC2 nodes or Fargate pods.

Key characteristics:

• $0.10/hour ($73/month) per cluster control plane fee
• Full Kubernetes compatibility and ecosystem
• Can run on EC2 (self-managed), managed node groups, or Fargate
• Multi-cloud and on-prem portability via Kubernetes

AWS Fargate (Serverless Compute for Containers)

Fargate is a serverless compute engine that works with both ECS and EKS. You define task/pod resource requirements and AWS provisions right-sized VMs without exposing EC2 instances.

Key characteristics:

• No node management or Auto Scaling Groups
• Pay per vCPU and GB-RAM with one-minute minimum billing
• Higher per-vCPU cost than EC2
• Works with both ECS and EKS

Direct Compute Cost Comparison

Let’s start with the raw infrastructure costs for running a simple workload: 10 containers, each requiring 1 vCPU and 2GB RAM, running 24/7.

Scenario: 10 vCPU, 20GB RAM Total

ECS on EC2 (2× m5.large: 2 vCPU, 8GB each):

• EC2 cost: $0.096/hour × 2 = $0.192/hour
• Monthly: $0.192 × 730 hours = $140/month
• ECS control plane: $0 (no charge)
• Total: $140/month

EKS on EC2 (2× m5.large):

• EC2 cost: $140/month (same as ECS)
• EKS control plane: $73/month
• Total: $213/month

ECS on Fargate:

• Fargate pricing: $0.04048/vCPU-hour + $0.004445/GB-hour
• Per task: (1 vCPU × $0.04048) + (2GB × $0.004445) = $0.0494/hour
• 10 tasks: $0.494/hour × 730 hours = $361/month
• Total: $361/month

EKS on Fargate:

• Fargate cost: $361/month (same as ECS)
• EKS control plane: $73/month
• Total: $434/month

Summary for this scenario:

• ECS on EC2: $140/month (baseline)
• EKS on EC2: $213/month (+52% vs ECS)
• ECS on Fargate: $361/month (+158% vs ECS on EC2)
• EKS on Fargate: $434/month (+210% vs ECS on EC2)

This is why “Fargate is expensive” is a common complaint—but it ignores operational costs.

The Hidden Costs: Operational Labor

Infrastructure costs are visible in AWS billing. Labor costs are hidden in salaries, on-call rotations, and opportunity cost (engineers managing infrastructure instead of building features).

Task Breakdown by Service

ECS on EC2 operational tasks:

• Provision and patch EC2 instances (AMI updates, security patches)
• Configure Auto Scaling Groups and capacity providers
• Monitor EC2 health and replace failed instances
• Optimize instance types and right-size capacity
• Manage ECS agent updates
• Handle Spot interruptions if using Spot instances
• Estimated hours/week: 4-8 hours for small clusters, 10-20 for large fleets

EKS on EC2 operational tasks:

• All ECS tasks above, plus:
• Manage Kubernetes control plane upgrades (quarterly)
• Configure and maintain autoscalers (Cluster Autoscaler or Karpenter)
• Troubleshoot Kubernetes networking (CNI, kube-proxy, CoreDNS)
• Manage add-ons (AWS Load Balancer Controller, EBS CSI driver, metrics-server)
• Handle RBAC, service accounts, and IAM roles for service accounts (IRSA)
• Monitor and tune pod resource requests/limits
• Estimated hours/week: 8-15 hours for small clusters, 20-40 for large fleets

Fargate (ECS or EKS) operational tasks:

• Define task/pod specs with CPU and memory
• Monitor task execution and logs
• Tune Fargate profiles (EKS only)
• No node management, patching, or autoscaling
• Estimated hours/week: 1-3 hours for small workloads, 5-10 for large

Labor Cost Calculation

Assume a platform engineer costs $150,000/year fully loaded (salary + benefits + overhead):

• Hourly rate: $150,000 / 2080 hours = $72/hour
• Weekly maintenance cost by service:

Small cluster scenario (10-20 nodes or equivalent Fargate capacity):

• ECS on EC2: 6 hours/week × $72 = $432/week = $1,870/month
• EKS on EC2: 12 hours/week × $72 = $864/week = $3,740/month
• Fargate (ECS or EKS): 2 hours/week × $72 = $144/week = $624/month

Total Cost of Ownership: Real Scenarios

Scenario 1: Small Startup (5-person team, 20 containers)

Requirements: 20 vCPU, 40GB RAM total. Team has limited Kubernetes experience. Speed to market is critical.

Monthly costs:

Service	Compute	Control Plane	Labor (partial FTE)	Total TCO
ECS on EC2	$280	$0	$1,870	$2,150
EKS on EC2	$280	$73	$3,740	$4,093
ECS on Fargate	$722	$0	$624	$1,346
EKS on Fargate	$722	$73	$624	$1,419

Winner: ECS on Fargate ($1,346/month). Despite 2.5× higher compute costs, labor savings make it the cheapest option. The team can focus on product instead of infrastructure.

Why not EKS? The team doesn’t need Kubernetes portability yet, and learning Kubernetes would slow feature development by months.

Scenario 2: Mid-Size SaaS (15-person engineering team, multi-cloud strategy)

Requirements: 200 vCPU, 400GB RAM. Running workloads on AWS and GCP. Need portability. Platform team of 3 engineers.

Monthly costs:

Service	Compute	Control Plane	Labor (0.5 FTE dedicated)	Total TCO
ECS on EC2	$2,800	$0	$3,740	$6,540
EKS on EC2	$2,800	$219 (3 clusters)	$7,480	$10,499
EKS on Fargate	$7,220	$219	$1,248	$8,687

Winner: ECS on EC2 ($6,540/month) if staying AWS-only. But if multi-cloud portability is required, EKS on EC2 ($10,499/month) is necessary despite higher cost.

Trade-off: The company values Kubernetes skills (portable across clouds) over short-term cost savings. EKS justifies its premium through reduced vendor lock-in and ability to hire from broader talent pool.

Scenario 3: Enterprise (100-person engineering org, batch + APIs)

Requirements: 2,000 vCPU steady-state, 5,000 vCPU peak (batch jobs). Dedicated 8-person platform team. Optimizing for cost at scale.

Monthly costs (steady-state only):

Service	Compute (with Spot)	Control Plane	Labor (1 FTE)	Total TCO
ECS on EC2	$12,000	$0	$12,500	$24,500
EKS on EC2	$12,000	$365 (5 clusters)	$18,700	$31,065
EKS on Fargate	$72,000	$365	$6,240	$78,605

Winner: ECS on EC2 ($24,500/month). At this scale, compute costs dominate and Fargate’s premium becomes prohibitive. The platform team can absorb operational overhead efficiently.

Why not Fargate? Fargate would cost 3.2× more monthly ($78K vs $24K). The labor savings ($12K/month) don’t justify the compute premium ($60K/month).

Why might they choose EKS anyway? If the company values Kubernetes ecosystem (Helm charts, operators, multi-cloud optionality) and has budget, the $6,500/month premium for EKS on EC2 may be justified strategically.

TCO Factors Beyond Compute and Labor

Training and Onboarding

• ECS: AWS-specific learning curve. Documentation is good but ecosystem is smaller. Harder to hire experienced ECS engineers.
• EKS: Large training investment (CKA/CKAD courses, Kubernetes learning). Easier to hire (huge Kubernetes talent pool).
• Fargate: Minimal learning curve. Developers define resource needs; infrastructure is abstracted.

Typical training costs:

• ECS: $2,000-5,000 per engineer (AWS training, trial-and-error)
• EKS: $5,000-15,000 per engineer (formal Kubernetes training, certifications, ramp-up time)
• Fargate: $500-1,000 per engineer (basic containerization concepts)

Tooling and Ecosystem

• ECS: Integrated with AWS tooling (CloudWatch, X-Ray, Systems Manager). Limited third-party integrations.
• EKS: Massive ecosystem (Helm, operators, service meshes, Prometheus/Grafana, Argo CD). Requires integration effort.
• Fargate: Works with both ECS and EKS ecosystems but limits some advanced features (DaemonSets on EKS Fargate require workarounds).

Cost example: A company using EKS might spend $10,000-50,000/year on tools like Kubecost, Datadog Kubernetes monitoring, or managed service mesh—costs that don’t exist in ECS.

Opportunity Cost

Every hour spent managing infrastructure is an hour not spent building product features. For a startup racing to market, this can be existential.

Example calculation: If managing EKS takes an extra 10 hours/week vs Fargate, that’s 520 hours/year. At $72/hour, that’s $37,440 in opportunity cost—or roughly one engineer-quarter that could have shipped features.

Fargate Nuances That Impact TCO

One-Minute Minimum Billing

Fargate bills with a one-minute minimum, then per-second after. For tasks that run under one minute (like quick CI jobs), you pay for the full minute even if the task finishes in 10 seconds.

Impact: Fargate is poor for very short-lived tasks. A CI pipeline running 1,000 ten-second jobs per day pays for 1,000 minutes (16.7 hours) even though actual compute is 2.8 hours.

Fargate Spot (ECS Only)

ECS supports Fargate Spot (up to 70% discount) for interruptible workloads. EKS on Fargate does not support Spot—you must use EC2 Spot instances.

TCO impact: For batch workloads on ECS, Fargate Spot can bridge the cost gap with EC2, making Fargate viable even at scale.

Fargate Per-Task Overhead

Each Fargate task gets a dedicated VM with its own overhead. For workloads with many tiny containers (microservices with <0.5 vCPU each), bin-packing on EC2 is more efficient than individual Fargate tasks.

Decision Framework

Choose ECS on EC2 if:

• You’re committed to AWS long-term (no multi-cloud needs)
• Cost optimization is the top priority
• You have platform engineering capacity
• You run large-scale, steady-state workloads (>100 vCPU)

Choose EKS on EC2 if:

• You need Kubernetes portability (multi-cloud, hybrid, on-prem)
• You want access to the Kubernetes ecosystem (Helm, operators, service meshes)
• You have or can build Kubernetes expertise
• You run diverse workloads that benefit from Kubernetes features

Choose ECS on Fargate if:

• You have a small team (<10 engineers) without infrastructure specialists
• You’re AWS-committed and don’t need Kubernetes
• Speed to market matters more than cost optimization
• You run long-lived tasks (not sub-minute jobs)

Choose EKS on Fargate if:

• You need Kubernetes but have limited ops capacity
• You’re willing to pay a premium to avoid node management
• Your workloads are variable and you want elastic scaling without autoscaler tuning

Common Mistakes

Choosing EKS because “everyone uses Kubernetes” without evaluating whether you actually need its features. Many teams would be better served by ECS with less complexity.

Avoiding Fargate because “it’s too expensive” based only on compute pricing. For small teams, Fargate’s TCO is often lowest when labor is included.

Starting with EKS on Fargate for cost-sensitive batch workloads. Fargate doesn’t support Spot on EKS—use EC2 with Spot instead.

Ignoring team size in TCO calculations. A 3-person startup has very different TCO than a 100-person platform team.

TCO Calculator Inputs

To calculate your own TCO, gather these inputs:

1. Workload requirements: Total vCPU, RAM, task/pod count, runtime (24/7 vs burst)
2. Team size and composition: Engineers available for platform work, Kubernetes experience level
3. Fully-loaded engineer cost: Salary + benefits + overhead (typically 1.3-1.5× salary)
4. Operational time estimates: Hours/week for infrastructure management per service
5. Strategic factors: Multi-cloud plans, portability needs, ecosystem access
6. Growth projections: Expected scale in 6-12 months (TCO shifts with scale)

Formula:

Monthly TCO = (Compute Cost) + (Control Plane Cost) + (Weekly Ops Hours × 4.33 × Hourly Engineer Rate) + (Training Cost / Amortization Period) + (Tooling Subscriptions)

Conclusion

Total Cost of Ownership for AWS container services depends more on team size and operational maturity than on per-vCPU pricing. Small teams with limited infrastructure expertise get the lowest TCO from Fargate despite its compute premium—the labor savings outweigh infrastructure costs. Mid-size teams benefit from ECS on EC2 when staying AWS-native, or EKS on EC2 when Kubernetes portability matters strategically. Large enterprises running thousands of vCPUs optimize TCO with EC2-based solutions where dedicated platform teams amortize operational overhead across large fleets. Calculate your TCO honestly by including engineer time, training costs, and opportunity cost of infrastructure work. The service with the lowest compute bill often has the highest total cost when you account for the full picture.

Both Karpenter and Cluster Autoscaler scale EKS nodes automatically, but they use fundamentally different approaches. Cluster Autoscaler adjusts existing node groups based on pending pods, while Karpenter provisions right-sized nodes on-demand from a fleet of instance types. Choosing the wrong autoscaler can cost you thousands in wasted capacity or cause availability issues during scale events.

Key Takeaways

• Cluster Autoscaler scales pre-configured node groups; Karpenter provisions diverse instance types dynamically
• Karpenter typically scales faster (30-60 seconds vs 2-5 minutes) and bins pods more efficiently
• Cluster Autoscaler offers mature, predictable behavior; Karpenter provides aggressive consolidation and Spot optimization
• Migration from Cluster Autoscaler to Karpenter requires careful planning to avoid disruption
• Most teams benefit from Karpenter’s cost savings, but Cluster Autoscaler remains valid for stability-first environments

How Each Autoscaler Works

Cluster Autoscaler

Cluster Autoscaler monitors for pending pods that cannot be scheduled due to insufficient resources. When it detects unschedulable pods, it increases the desired capacity of matching Auto Scaling Groups. When nodes become underutilized (typically below 50% for 10+ minutes), it cordons, drains, and terminates them.

The workflow:

1. Pod enters Pending state (no node has capacity)
2. Cluster Autoscaler checks which node group could satisfy the pod’s requirements
3. Increases desired capacity on the matching Auto Scaling Group
4. AWS launches a new EC2 instance (2-4 minutes)
5. Node joins cluster and pod is scheduled

Key constraint: You must pre-define node groups with specific instance types. If your node groups only have m5.large and a pod needs 16GB RAM, Cluster Autoscaler launches m5.large (which has 8GB) and the pod stays Pending.

Karpenter

Karpenter watches for unschedulable pods and directly provisions EC2 instances without Auto Scaling Groups. It evaluates pod requirements (CPU, memory, architecture, zones) and selects the best-fit instance type from a configurable fleet.

The workflow:

1. Pod enters Pending state
2. Karpenter calculates exact resource needs
3. Selects optimal instance type from Provisioner configuration (can choose from dozens of types)
4. Calls EC2 RunInstances directly (30-90 seconds)
5. Node joins and pod schedules immediately

Key advantage: Karpenter can provision an m5.xlarge for one pod and an r6i.2xlarge for another—whatever fits best. It’s not limited to pre-configured groups.

Feature Comparison Matrix

Feature	Cluster Autoscaler	Karpenter
Scaling speed	2-5 minutes (ASG launch time)	30-90 seconds (direct EC2 API)
Instance selection	Pre-configured node groups only	Dynamic from configured fleet
Spot handling	Separate Spot node groups required	Mixed Spot/On-Demand in single Provisioner
Consolidation	Limited (terminates idle nodes)	Aggressive bin-packing and node replacement
Multi-AZ support	Requires per-AZ ASGs for PV locality	Built-in topology awareness
Configuration complexity	Medium (ASG + flags)	Medium-High (Provisioner CRDs)
Maturity	Stable (7+ years)	Rapidly evolving (2021+, GA 2023)
Community support	Large, established	Growing, AWS-backed
Interruption handling	Requires separate termination handler	Built-in Spot interruption handling
Scale-down safety	Configurable thresholds + PDB respect	Consolidation can be aggressive; requires tuning

When to Choose Cluster Autoscaler

Best for:

• Predictable workload patterns where you can define 2-3 node group types that cover all use cases
• Stability over optimization — you prefer well-tested behavior and don’t want cutting-edge features
• Simpler operational model — your team is already familiar with Auto Scaling Groups and prefers that abstraction
• Regulated environments where change control favors mature, widely-adopted tools
• Low Spot usage — if you run mostly On-Demand or Reserved capacity, Cluster Autoscaler’s simpler Spot handling is sufficient

Example configuration for Cluster Autoscaler:

apiVersion: v1 kind: ServiceAccount metadata: name: cluster-autoscaler namespace: kube-system annotations: eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT:role/cluster-autoscaler --- # Deploy via Helm with key flags helm upgrade --install cluster-autoscaler autoscaler/cluster-autoscaler \ --namespace kube-system \ --set autoDiscovery.clusterName=my-cluster \ --set extraArgs.balance-similar-node-groups=true \ --set extraArgs.skip-nodes-with-system-pods=false \ --set extraArgs.scale-down-delay-after-add=10m

Real-world scenario: A financial services company runs stateful workloads with strict compliance requirements. They use three node groups (general-purpose m5.large, memory-optimized r5.xlarge, compute-optimized c5.2xlarge) and scale predictably during business hours. Cluster Autoscaler provides the stability and auditability they need without introducing rapid node churn.

When to Choose Karpenter

Best for:

• Cost optimization priority — you want maximum Spot usage and efficient bin-packing
• Diverse workloads — batch jobs, APIs, ML training, analytics all running in one cluster with different resource profiles
• Fast scaling requirements — sub-minute provisioning matters for your SLAs
• Spot-heavy strategies — you want to maximize Spot coverage with automatic fallback to On-Demand
• Dynamic instance type selection — you don’t want to maintain separate node groups for every workload type

Example Karpenter Provisioner configuration:

apiVersion: karpenter.sh/v1alpha5 kind: Provisioner metadata: name: default spec: requirements: - key: karpenter.sh/capacity-type operator: In values: ["spot", "on-demand"] - key: kubernetes.io/arch operator: In values: ["amd64"] - key: karpenter.k8s.aws/instance-category operator: In values: ["c", "m", "r"] - key: karpenter.k8s.aws/instance-generation operator: Gt values: ["4"] limits: resources: cpu: 1000 memory: 1000Gi providerRef: name: default ttlSecondsAfterEmpty: 30 consolidation: enabled: true --- apiVersion: karpenter.k8s.aws/v1alpha1 kind: AWSNodeTemplate metadata: name: default spec: subnetSelector: karpenter.sh/discovery: my-cluster securityGroupSelector: karpenter.sh/discovery: my-cluster instanceProfile: KarpenterNodeInstanceProfile

This Provisioner allows Karpenter to choose from c5, c6i, m5, m6i, r5, r6i families (generation 5+) across Spot and On-Demand, automatically selecting the best price-performance option.

Real-world scenario: A SaaS company runs microservices with highly variable traffic. Some services need 2GB RAM, others need 32GB. They use Karpenter to provision exactly the right instance types on-demand, achieving 70% Spot coverage with automatic On-Demand fallback. Karpenter’s consolidation feature replaces three underutilized m5.xlarge nodes with one m5.2xlarge, saving 30% on compute costs.

Cost Impact Comparison

Cluster Autoscaler cost characteristics:

• Predictable spending patterns (fixed instance types)
• Potential over-provisioning due to instance type constraints
• Scale-down conservatism reduces waste but may leave idle nodes longer
• Typical savings: 10-30% versus no autoscaling

Karpenter cost characteristics:

• Dynamic right-sizing reduces over-provisioning waste
• Aggressive consolidation increases utilization (often 60-80% vs 40-50%)
• Better Spot diversification improves interruption resilience and savings
• Typical savings: 30-50% versus no autoscaling; 15-25% versus Cluster Autoscaler

Example calculation: A cluster spending $10,000/month on EC2:

• No autoscaling: $10,000/month baseline
• Cluster Autoscaler (20% reduction): $8,000/month
• Karpenter (40% reduction): $6,000/month

Karpenter’s additional savings come from better bin-packing, more aggressive Spot usage, and consolidation replacing multiple small nodes with fewer large ones.

Spot Instance Handling Comparison

Cluster Autoscaler Approach

Create separate Spot and On-Demand node groups. Use node affinity to prefer Spot for tolerant workloads. Requires aws-node-termination-handler DaemonSet for graceful spot interruption handling.

eksctl create nodegroup \ --cluster=my-cluster \ --name=spot-workers \ --spot \ --instance-types=m5.large,m5a.large,m5n.large \ --nodes-min=0 \ --nodes-max=20 \ --node-labels="workload-type=batch" # Must also install termination handler separately kubectl apply -f https://github.com/aws/aws-node-termination-handler/releases/download/v1.19.0/all-resources.yaml

Karpenter Approach

Single Provisioner handles both Spot and On-Demand. Karpenter automatically diversifies across instance types and handles interruptions without separate tooling.

apiVersion: karpenter.sh/v1alpha5 kind: Provisioner metadata: name: spot-optimized spec: requirements: - key: karpenter.sh/capacity-type operator: In values: ["spot"] - key: karpenter.k8s.aws/instance-family operator: In values: ["m5", "m5a", "m5n", "m6i", "m6a"] # Karpenter handles interruptions automatically # No separate termination handler needed

Interruption handling: Karpenter monitors EC2 Spot interruption notices and EventBridge events, automatically cordoning nodes and triggering replacement. Cluster Autoscaler requires aws-node-termination-handler as a separate component.

Consolidation Deep Dive

Consolidation is where Karpenter truly differentiates itself. It actively replaces underutilized nodes to improve packing efficiency.

How Karpenter Consolidation Works

1. Karpenter identifies nodes with low utilization
2. Simulates whether pods could fit on fewer, larger instances
3. If consolidation saves money, cordons the target nodes
4. Provisions replacement node(s)
5. Drains old nodes once replacements are ready

Example scenario:

• Three m5.large nodes (2 vCPU, 8GB each) running at 40% utilization
• Karpenter calculates all pods fit on one m5.2xlarge (8 vCPU, 32GB)
• Provisions m5.2xlarge, migrates pods, terminates the three m5.large
• Cost reduction: $0.096/hour × 3 = $0.288/hour → $0.384/hour = 20% savings + better utilization

Consolidation Risks and Safeguards

Risk: Aggressive consolidation can cause pod churn and temporary unavailability.

Safeguards:

• Set do-not-evict annotation on critical pods
• Use PodDisruptionBudgets to control eviction rate
• Configure ttlSecondsAfterEmpty to delay consolidation
• Monitor consolidation events and rollback if issues arise

# Prevent specific pods from being consolidated apiVersion: v1 kind: Pod metadata: name: critical-database annotations: karpenter.sh/do-not-evict: "true" --- # Control consolidation timing apiVersion: karpenter.sh/v1alpha5 kind: Provisioner metadata: name: default spec: ttlSecondsAfterEmpty: 300 # Wait 5 minutes before consolidating empty nodes ttlSecondsUntilExpired: 604800 # Replace nodes weekly for security patches consolidation: enabled: true

Cluster Autoscaler alternative: No built-in consolidation. Scale-down happens when nodes fall below utilization threshold (default 50%) for 10+ minutes, but it doesn’t actively re-pack pods.

Migration Playbook: Cluster Autoscaler to Karpenter

Migrating requires careful planning to avoid disrupting production workloads.

Phase 1: Preparation (Week 1)

1. Install Karpenter in non-production cluster and test Provisioner configurations
2. Document current node groups — instance types, labels, taints, AZ distribution
3. Identify workload dependencies — which pods require specific node types or zones
4. Set up monitoring — create dashboards for node count, pod scheduling latency, costs

Phase 2: Parallel Operation (Week 2-3)

1. Deploy Karpenter to production but don’t create Provisioners yet
2. Create initial Provisioner that matches your existing node group characteristics
3. Label a subset of workloads (e.g., dev namespace) to prefer Karpenter nodes
4. Run both autoscalers — Cluster Autoscaler manages existing groups, Karpenter handles new workloads

# Gradually migrate workloads with node affinity apiVersion: apps/v1 kind: Deployment metadata: name: test-app spec: template: spec: nodeSelector: karpenter.sh/provisioner-name: default # Prefer Karpenter nodes

Phase 3: Full Migration (Week 4)

1. Expand Karpenter Provisioner capacity limits to handle full cluster load
2. Cordon Cluster Autoscaler managed nodes to prevent new pod scheduling
3. Drain nodes gradually (one AZ at a time to maintain availability)
4. Monitor pod rescheduling — ensure Karpenter provisions nodes successfully
5. Delete old Auto Scaling Groups once all workloads have migrated
6. Uninstall Cluster Autoscaler

Phase 4: Optimization (Week 5+)

1. Enable consolidation in Provisioner (start conservatively)
2. Expand instance type diversity to maximize Spot options
3. Tune ttlSecondsAfterEmpty based on workload patterns
4. Implement do-not-evict annotations for critical workloads
5. Measure cost savings and adjust Provisioner requirements

Rollback Plan

If issues arise during migration:

1. Increase Auto Scaling Group desired capacity back to original levels
2. Remove node selectors that prefer Karpenter
3. Delete Karpenter Provisioners to stop new node creation
4. Re-enable Cluster Autoscaler
5. Drain Karpenter nodes and migrate pods back

Decision Tree

Start here: What is your primary optimization goal?

• Stability and predictability → Choose Cluster Autoscaler
• Maximum cost savings → Choose Karpenter

Do you run diverse workloads with different resource profiles?

• No, 2-3 node types cover everything → Cluster Autoscaler is sufficient
• Yes, wide variation in CPU/memory needs → Karpenter provides better bin-packing

What is your Spot usage target?

• Low (< 30%) → Either works; Cluster Autoscaler is simpler
• High (> 50%) → Karpenter’s diversification and interruption handling shine

How important is sub-minute scaling?

• Not critical → Cluster Autoscaler’s 2-5 minute scale is acceptable
• Essential for SLAs → Karpenter’s 30-90 second provisioning helps

Do you have stateful workloads with AZ-bound PersistentVolumes?

• Yes, many StatefulSets → Both work, but Karpenter’s topology awareness is easier to configure than per-AZ ASGs
• No, mostly stateless → Either works

What is your team’s operational maturity with Kubernetes?

• Early in Kubernetes journey → Start with Cluster Autoscaler, migrate to Karpenter later
• Experienced, comfortable with CRDs and rapid iteration → Karpenter is a good fit

Common Pitfalls

Cluster Autoscaler pitfalls:

• Forgetting –balance-similar-node-groups: Scale-outs concentrate in one AZ, causing imbalance
• Too few instance types in Spot groups: High interruption correlation when single type is reclaimed
• Missing per-AZ ASGs for StatefulSets: Pods stuck Pending after cross-AZ interruptions
• Insufficient IAM permissions: Autoscaler can’t modify ASGs, scaling fails silently

Karpenter pitfalls:

• Overly aggressive consolidation: Pod churn impacts availability; start with consolidation disabled, enable gradually
• No do-not-evict annotations on critical pods: Databases get evicted during consolidation
• Insufficient Provisioner capacity limits: Karpenter provisions unlimited nodes during runaway scaling
• Missing PodDisruptionBudgets: Consolidation violates availability requirements
• Wrong instance families in requirements: Over-provisioning if you include only large instance types

Hybrid Approach: Running Both

Some teams run both autoscalers temporarily during migration or permanently for different workload classes:

• Cluster Autoscaler manages stable, production node groups
• Karpenter handles batch jobs, dev/test, and experimental workloads

Use node selectors and taints to separate workloads clearly. This reduces risk while gaining Karpenter’s benefits for appropriate workloads.

# Production pods stay on Cluster Autoscaler nodes nodeSelector: node-group: production # Batch jobs use Karpenter nodeSelector: karpenter.sh/provisioner-name: batch

Measuring Success

Track these metrics after deploying either autoscaler:

• Node utilization: Target 60-80% average CPU/memory (Karpenter typically achieves higher)
• Pod scheduling latency: Time from Pending to Running (Karpenter faster)
• Scale-up time: Time to provision new capacity when needed
• Spot instance percentage: Karpenter usually achieves higher safe Spot coverage
• Cost per pod: Use Kubecost to measure before/after
• Node churn rate: Consolidation increases churn; monitor for acceptable levels

Expected outcomes:

• Cluster Autoscaler: 10-30% cost reduction, predictable behavior, 2-5 minute scale-up
• Karpenter: 30-50% cost reduction, 30-90 second scale-up, requires tuning for stability

Conclusion

Cluster Autoscaler and Karpenter solve the same problem with different philosophies. Cluster Autoscaler prioritizes stability through pre-configured node groups and conservative scaling, making it ideal for teams wanting predictable, well-tested behavior. Karpenter optimizes for cost and efficiency through dynamic provisioning, aggressive consolidation, and superior Spot handling—delivering 15-25% additional savings but requiring more operational maturity. Most teams benefit from starting with Cluster Autoscaler and migrating to Karpenter once they’re comfortable with autoscaling fundamentals. Use the decision tree to evaluate your priorities: choose Cluster Autoscaler if stability matters most, choose Karpenter if you want maximum optimization and can handle its complexity. Both are valid choices—the wrong decision is running neither and leaving nodes over-provisioned.

You can’t optimize EKS costs without knowing which pods, namespaces, and teams are driving spend. AWS provides native billing tools like Cost Explorer and Cost & Usage Reports, while Kubernetes-native tools like Kubecost and OpenCost offer pod-level attribution. This guide compares all three approaches to help you choose the right visibility strategy for your environment.

Key Takeaways

• AWS Cost Explorer shows EKS spend by service but can’t attribute costs to individual pods or namespaces
• Kubecost integrates with AWS billing to provide per-pod, per-namespace cost allocation and rightsizing recommendations
• OpenCost is the open-source alternative to Kubecost with similar pod-level attribution but fewer enterprise features
• AWS split cost allocation (EKS add-on) bridges the gap by tagging EKS costs in Cost Explorer by cluster and namespace
• Most teams use a combination: Kubecost/OpenCost for engineering decisions and Cost Explorer for finance reporting

The Cost Visibility Problem in EKS

EKS clusters create costs across multiple AWS services:

• EKS control plane ($0.10/hour per cluster)
• EC2 instances (worker nodes)
• EBS volumes (persistent storage)
• Load balancers (ALB/NLB)
• Data transfer (cross-AZ, NAT gateway, internet egress)

AWS billing aggregates these charges at the service level—you see “$5,000 on EC2” but not “which namespace or application consumed those resources.” For multi-team environments, this makes cost accountability impossible.

Kubernetes-native cost tools solve this by mapping resource requests and actual usage to AWS pricing, providing granular attribution down to individual pods.

AWS Cost Explorer: The Finance View

What It Does

Cost Explorer is AWS’s built-in cost analysis tool. It aggregates spend across all services with daily or monthly granularity, supports filtering by tags, and provides basic forecasting.

Strengths

• Authoritative pricing data: Cost Explorer reflects actual AWS billing, including discounts from Savings Plans and Reserved Instances
• Multi-service visibility: See EKS control plane, EC2, EBS, data transfer, and load balancer costs in one place
• Tag-based filtering: Group costs by custom tags like Environment: production or Team: platform
• No installation required: Available in every AWS account at no extra cost
• Budget and anomaly alerts: Set spending thresholds and receive notifications

Limitations

• No pod-level attribution: Can’t map costs to namespaces, deployments, or pods
• Delayed data: Cost Explorer lags 24-48 hours behind actual usage
• Limited rightsizing guidance: AWS Compute Optimizer provides instance-level recommendations but not pod-level request tuning
• Manual correlation required: You must manually match EC2 instance IDs to Kubernetes node names to understand cluster-level spend

How to Use It

Query monthly EKS-related costs via CLI:

aws ce get-cost-and-usage \ --time-period Start=2025-01-01,End=2025-02-01 \ --granularity MONTHLY \ --metrics UnblendedCost \ --group-by Type=DIMENSION,Key=SERVICE \ --filter '{ "Dimensions": { "Key": "SERVICE", "Values": ["Amazon Elastic Kubernetes Service", "Amazon Elastic Compute Cloud"] } }'

Filter by cluster tag if you’ve tagged your node groups:

aws ce get-cost-and-usage \ --time-period Start=2025-01-01,End=2025-02-01 \ --granularity DAILY \ --metrics UnblendedCost \ --group-by Type=TAG,Key=Cluster

Best For

• Finance teams tracking overall AWS spend and budget compliance
• High-level cluster cost trends and service-level breakdowns
• Validating Savings Plans and Reserved Instance coverage

AWS Cost & Usage Reports (CUR): The Data Source

CUR is the most detailed AWS billing export, delivering hourly usage and cost data to S3. It’s the authoritative source both Kubecost and AWS split cost allocation use for accurate pricing.

Strengths

• Hourly granularity: Track costs hour-by-hour instead of daily aggregates
• Resource-level detail: Includes resource IDs, tags, and line-item charges
• Integration-friendly: Kubecost, OpenCost, and third-party FinOps tools all ingest CUR for accurate cost mapping

Limitations

• Raw data format: CUR delivers CSV/Parquet files—you need a tool to parse and visualize them
• Setup required: Must configure S3 bucket, enable resource IDs, and activate cost allocation tags

How to Enable

1. Go to AWS Billing Console → Cost & Usage Reports
2. Create a new report with hourly granularity
3. Enable resource IDs and split cost allocation data
4. Specify an S3 bucket for delivery
5. Configure Kubecost or OpenCost to read from that bucket

Best For

• Feeding accurate AWS pricing into Kubecost or OpenCost
• Building custom FinOps dashboards in Athena, Redshift, or QuickSight
• Validating discounts from Savings Plans and spot usage

Kubecost: The Full-Featured Option

What It Does

Kubecost runs inside your cluster and maps Kubernetes resource requests (CPU, memory, storage, network) to AWS pricing data from the Cost & Usage Report. It provides real-time cost attribution by pod, namespace, deployment, label, and annotation.

Strengths

• Pod-level attribution: See exactly which pods drive costs, with drill-down to container-level granularity
• Rightsizing recommendations: Suggests optimal CPU/memory requests based on actual usage
• Showback and chargeback: Generate per-team cost reports for internal billing
• Multi-cluster support: Aggregate costs across dozens of EKS clusters
• Savings opportunities dashboard: Highlights orphaned volumes, underutilized nodes, and Spot adoption candidates
• Efficiency scoring: Tracks “slack cost” (reserved but unused resources)
• Alert rules: Notify when namespace costs spike or budgets are exceeded

Limitations

• In-cluster footprint: Kubecost runs Prometheus, a cost model, and a frontend—adds ~1-2GB memory overhead
• Pricing model: Free tier covers single clusters; enterprise features (multi-cluster, SSO, long-term retention) require a license (~$500-$5,000+/year depending on scale)
• Initial configuration: Requires CUR integration and proper IAM permissions for accurate pricing

Installation

helm repo add kubecost https://kubecost.github.io/cost-analyzer/ helm upgrade --install kubecost kubecost/cost-analyzer \ --namespace kubecost --create-namespace \ --set kubecostToken="your-token" \ --set prometheus.server.persistentVolume.enabled=true

Access the UI:

kubectl port-forward -n kubecost svc/kubecost-cost-analyzer 9090:9090

Navigate to http://localhost:9090 to see cost breakdowns.

Integrating with CUR

Update Kubecost’s Helm values to point to your CUR S3 bucket:

kubecostProductConfigs: athenaProjectID: "your-aws-account-id" athenaBucketName: "s3://your-cur-bucket" athenaRegion: "us-east-1" athenaDatabase: "athenacurcfn_cur_database" athenaTable: "cur_table_name"

Kubecost will reconcile in-cluster metrics with actual AWS billing, providing accurate per-pod costs including discounts from Savings Plans.

Best For

• Platform teams needing actionable rightsizing and Spot recommendations
• Multi-cluster environments (10+ clusters) with centralized cost visibility needs
• Organizations implementing chargeback to application teams
• Teams willing to invest in a commercial tool for full-featured analytics

OpenCost: The Open-Source Alternative

What It Does

OpenCost is a CNCF sandbox project that provides the same core cost attribution logic as Kubecost—it’s actually the open-source foundation that Kubecost was built on. It maps pod resource requests to cloud pricing and exposes costs via Prometheus metrics and a basic API.

Strengths

• Truly open source: Apache 2.0 license with no vendor lock-in
• Lightweight: Smaller footprint than Kubecost (no frontend, just API and exporter)
• Prometheus integration: Exports cost metrics that you can query in Grafana
• Multi-cloud support: Works with AWS, GCP, Azure, and on-prem pricing
• Free forever: No enterprise tiers or licensing

Limitations

• No built-in UI: Requires Grafana or custom dashboards to visualize data
• Limited features: No automated rightsizing recommendations, no alerting, no multi-cluster aggregation UI
• DIY integration: You manage Prometheus retention, dashboard creation, and CUR integration yourself
• Community support only: No commercial support contracts

Installation

kubectl apply -f https://raw.githubusercontent.com/opencost/opencost/develop/kubernetes/opencost.yaml

Access the API:

kubectl port-forward -n opencost service/opencost 9003:9003

Query namespace costs:

curl http://localhost:9003/allocation/compute \ -d window=7d \ -d aggregate=namespace \ -G

Grafana Dashboard

Import the community OpenCost dashboard (ID: 15474) into Grafana to visualize pod and namespace costs over time.

Best For

• Small teams or startups with existing Prometheus/Grafana stacks
• Organizations that prefer open-source tooling without vendor dependencies
• Environments where custom dashboards and API integrations are sufficient
• Cost-conscious teams that don’t need enterprise support or advanced features

AWS Split Cost Allocation: The Native Bridge

AWS introduced split cost allocation as an EKS add-on that tags EC2 costs in Cost Explorer by cluster, namespace, and workload type.

How It Works

The split cost allocation controller runs in your cluster and periodically reports resource usage to AWS. AWS then tags EC2 instance costs with:

• eks:cluster-name
• eks:namespace
• eks:workload-type (Deployment, StatefulSet, DaemonSet, etc.)

These tags appear in Cost Explorer, allowing you to filter and group EKS costs without installing Kubecost or OpenCost.

Strengths

• Native AWS integration: Works directly in Cost Explorer—no third-party tools
• Finance-friendly: Non-technical stakeholders can query costs in the AWS Console
• Lightweight: Minimal in-cluster footprint (just a controller pod)

Limitations

• EC2 costs only: Doesn’t split EBS, data transfer, or load balancer costs
• No rightsizing recommendations: Just attribution, not optimization guidance
• Delayed visibility: Data appears in Cost Explorer with the usual 24-48 hour lag
• Namespace-level only: Can’t drill down to individual pods or containers

Enabling Split Cost Allocation

1. Install the EKS add-on:

aws eks create-addon \ --cluster-name my-cluster \ --addon-name eks-pod-identity-agent \ --resolve-conflicts OVERWRITE

2. Activate the cost allocation tags in Billing preferences
3. Wait 24-48 hours for data to populate
4. Filter Cost Explorer by eks:cluster-name or eks:namespace

Best For

• Teams already using Cost Explorer who want basic EKS attribution
• Finance departments needing namespace-level visibility without installing cluster tools
• Organizations uncomfortable with third-party agents running in production

Feature Comparison Matrix

Feature	Cost Explorer	Kubecost	OpenCost	Split Allocation
Pod-level attribution	No	Yes	Yes	No
Namespace costs	Via tags	Yes	Yes	Yes
Rightsizing recommendations	Instance-level	Pod-level	No	No
Multi-cluster aggregation	Manual	Yes	DIY	Per cluster
Real-time visibility	No (24-48h lag)	Yes	Yes	No (24-48h lag)
Built-in UI	Yes	Yes	No	Yes (Cost Explorer)
Cost	Free	Free tier + paid	Free	Free
In-cluster footprint	None	Medium	Small	Minimal
CUR integration	Native	Yes	Yes	Native
Spot/Savings Plan discounts	Yes	Yes (with CUR)	Yes (with CUR)	Yes

Recommended Approach: Layered Visibility

Most successful EKS cost optimization programs use a combination:

1. Enable Cost & Usage Reports for authoritative pricing data
2. Deploy Kubecost or OpenCost for engineering teams to see pod-level costs and rightsizing opportunities
3. Activate split cost allocation so finance teams can track namespace costs in Cost Explorer without learning Kubernetes
4. Use Cost Explorer for high-level trends, budget alerts, and Savings Plan management

This layered approach gives engineers actionable optimization data while providing finance teams with familiar AWS billing interfaces.

Decision Framework

Choose Kubecost if:

• You manage 5+ clusters and need centralized cost visibility
• You want automated rightsizing recommendations and Spot adoption guidance
• You need chargeback/showback with per-team cost reports
• You have budget for commercial tooling (~$500-$5,000+/year)

Choose OpenCost if:

• You prefer open-source tools and already run Prometheus/Grafana
• You’re comfortable building custom dashboards and integrations
• You want pod-level costs but don’t need a polished UI
• You have engineering resources to maintain the stack

Choose split cost allocation if:

• You only need namespace-level attribution (not per-pod)
• Finance teams drive cost reviews and prefer Cost Explorer
• You want minimal in-cluster footprint
• You’re uncomfortable installing third-party agents in production

Use Cost Explorer for:

• Overall AWS spend trends and forecasting
• Budget enforcement and anomaly detection
• Savings Plan and Reserved Instance coverage analysis
• Finance-driven reporting and compliance

Conclusion

Cost visibility is the foundation of EKS optimization—you can’t reduce spend without knowing where it’s going. AWS Cost Explorer and split cost allocation provide service and namespace-level visibility that works for finance teams, while Kubecost and OpenCost deliver the pod-level granularity engineering teams need to make rightsizing decisions. For most organizations, the winning strategy combines all three: enable CUR for accurate pricing, deploy Kubecost or OpenCost for engineers, activate split cost allocation for finance, and use Cost Explorer for high-level governance. Start with your existing Prometheus/Grafana stack and OpenCost if you want a lightweight open-source approach, or choose Kubecost if you need a full-featured platform with automated recommendations and multi-cluster support.

While EC2 instance hours dominate EKS bills, storage, networking, and forgotten resources quietly accumulate charges that can represent 20-40% of total spend. These costs are easy to overlook because they appear as small line items spread across multiple services—but they compound quickly in multi-cluster, multi-team environments.

Key Takeaways

• Orphaned EBS volumes from deleted StatefulSets can cost hundreds of dollars monthly
• Unused Application Load Balancers charge hourly fees plus data processing costs
• Cross-AZ data transfer adds $0.01/GB—significant for chatty microservices
• NAT gateway charges $0.045/GB for outbound traffic that could use VPC endpoints
• EBS gp2 volumes cost ~20% more than gp3 for equivalent performance

Orphaned EBS Volumes: The Silent Cost Multiplier

Every StatefulSet with a PersistentVolumeClaim creates an EBS volume. When you delete the StatefulSet, Kubernetes deletes the pod—but the PersistentVolume and underlying EBS volume persist by default.

Over months, these orphaned volumes accumulate. A 100GB gp3 volume costs $8/month. Across 50 forgotten volumes, that’s $400/month for storage you’re not using.

Finding Orphaned Volumes

List all PersistentVolumes in Available state (not bound to a pod):

kubectl get pv --all-namespaces -o wide | grep Available

Cross-reference with AWS to find unattached EBS volumes:

aws ec2 describe-volumes \ --filters Name=status,Values=available \ --region us-east-1 \ --query 'Volumes[*].[VolumeId,Size,CreateTime,Tags]' \ --output table

Look for volumes created months ago with no recent attachment history. These are prime candidates for deletion.

Safe Deletion Process

Before deleting any volume:

1. Verify it’s truly orphaned (no PVC reference, no running pods)
2. Create a snapshot for safety: aws ec2 create-snapshot --volume-id vol-xxxxx --description "backup before deletion"
3. Wait 7-30 days to confirm no one needs it
4. Delete the snapshot after the grace period to avoid ongoing snapshot storage costs

Delete the volume:

aws ec2 delete-volume --volume-id vol-xxxxx

Preventing Future Orphans

Set reclaimPolicy: Delete in your StorageClass so PVs are automatically removed when PVCs are deleted:

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: gp3-delete provisioner: ebs.csi.aws.com parameters: type: gp3 reclaimPolicy: Delete volumeBindingMode: WaitForFirstConsumer

Warning: This makes deletions permanent. For production databases, use reclaimPolicy: Retain and manage cleanup manually with snapshot policies.

Unused Load Balancers

Every Kubernetes Service of type LoadBalancer provisions an AWS Application Load Balancer or Network Load Balancer. ALBs cost $0.0225/hour (~$16/month) plus $0.008/LCU for data processing.

When you delete a Service, the AWS Load Balancer Controller should delete the corresponding ALB—but if the controller isn’t running or the Service has finalizer issues, load balancers can persist indefinitely.

Finding Orphaned Load Balancers

List all Kubernetes Services with external load balancers:

kubectl get svc -A -o json | jq '.items[] | select(.spec.type=="LoadBalancer") | {namespace: .metadata.namespace, name: .metadata.name, ingress: .status.loadBalancer.ingress}'

List all AWS load balancers:

aws elbv2 describe-load-balancers \ --region us-east-1 \ --query 'LoadBalancers[*].[LoadBalancerName,DNSName,CreatedTime,State.Code]' \ --output table

Compare the two lists. Any AWS load balancer without a matching Kubernetes Service is a candidate for deletion.

Consolidating with Ingress

Instead of creating one ALB per Service, use an Ingress controller to share a single ALB across multiple services:

apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: shared-ingress annotations: alb.ingress.kubernetes.io/scheme: internet-facing spec: ingressClassName: alb rules: - host: api.example.com http: paths: - path: / pathType: Prefix backend: service: name: api-service port: number: 80 - host: www.example.com http: paths: - path: / pathType: Prefix backend: service: name: web-service port: number: 80

This provisions a single ALB with host-based routing rules—saving $16/month per consolidated Service.

Cross-AZ Data Transfer Charges

AWS charges $0.01/GB for traffic between availability zones. For a microservice architecture with 10 services each sending 50GB/day to other services, that’s $150/month in transfer fees—just for internal communication.

Measuring Cross-AZ Traffic

Use AWS Cost Explorer to identify cross-AZ charges:

aws ce get-cost-and-usage \ --time-period Start=2025-01-01,End=2025-01-31 \ --granularity DAILY \ --metrics UnblendedCost \ --filter file://filter.json

Filter JSON for cross-AZ usage types:

{ "Dimensions": { "Key": "USAGE_TYPE", "Values": ["DataTransfer-Regional-Bytes"] } }

Reducing Cross-AZ Traffic

See the “5 EKS Networking Tweaks” post for detailed configuration, but the quick fixes are:

• Use internalTrafficPolicy: Local on high-traffic Services
• Enable topology-aware routing with trafficDistribution: PreferClose
• Deploy per-AZ node groups to keep pod-to-pod traffic local
• Use topology spread constraints to ensure even pod distribution

NAT Gateway Egress Costs

Every byte leaving your VPC through a NAT gateway costs $0.045/GB. For clusters that frequently pull container images from ECR or make API calls to AWS services, this adds up fast.

Measuring NAT Gateway Spend

aws ce get-cost-and-usage \ --time-period Start=2025-01-01,End=2025-01-31 \ --granularity DAILY \ --metrics UnblendedCost \ --filter '{ "Dimensions": { "Key": "USAGE_TYPE", "Values": ["NatGateway-Bytes"] } }'

Eliminating NAT Costs with VPC Endpoints

For AWS services like ECR, S3, and DynamoDB, create VPC endpoints to route traffic privately:

aws ec2 create-vpc-endpoint \ --vpc-id vpc-xxxxx \ --service-name com.amazonaws.us-east-1.ecr.api \ --subnet-ids subnet-xxxxx subnet-yyyyy \ --security-group-ids sg-xxxxx aws ec2 create-vpc-endpoint \ --vpc-id vpc-xxxxx \ --service-name com.amazonaws.us-east-1.ecr.dkr \ --subnet-ids subnet-xxxxx subnet-yyyyy \ --security-group-ids sg-xxxxx aws ec2 create-vpc-endpoint \ --vpc-id vpc-xxxxx \ --service-name com.amazonaws.us-east-1.s3 \ --route-table-ids rtb-xxxxx

Interface endpoints cost ~$7/month per AZ, but eliminate NAT charges. Break-even is around 60GB/month of traffic per AZ.

Storage Type Optimization

EBS gp2 volumes were the default for years, but gp3 offers better price-performance:

• gp2: $0.10/GB-month, 3 IOPS per GB baseline
• gp3: $0.08/GB-month, 3000 IOPS baseline (configurable up to 16,000)

For a 1TB volume, switching from gp2 to gp3 saves $20/month—with better performance.

Migrating to gp3

Update your StorageClass:

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: gp3 provisioner: ebs.csi.aws.com parameters: type: gp3 iops: "3000" throughput: "125" volumeBindingMode: WaitForFirstConsumer allowVolumeExpansion: true

For existing volumes, modify them in place:

aws ec2 modify-volume --volume-id vol-xxxxx --volume-type gp3

Monitor the modification:

aws ec2 describe-volumes-modifications --volume-id vol-xxxxx

Snapshot and AMI Cleanup

Old EBS snapshots and unused AMIs accumulate over time—especially from CI/CD pipelines that create custom images.

Finding Old Snapshots

aws ec2 describe-snapshots \ --owner-ids self \ --query 'Snapshots[?StartTime<=`2024-01-01`].[SnapshotId,VolumeSize,StartTime,Description]' \ --output table

Implement a lifecycle policy to automatically delete snapshots older than 90 days (adjust retention based on compliance needs).

Automated Cleanup Workflows

Manual cleanup doesn’t scale. Automate with scheduled scripts or tools:

Lambda Function for Volume Cleanup

Create a Lambda function that runs weekly to:

1. Query available EBS volumes older than 30 days
2. Check for associated snapshots
3. Create final snapshot
4. Delete volume after 7-day grace period
5. Send summary to Slack/email

Cost Anomaly Detection

Enable AWS Cost Anomaly Detection to alert on unexpected spikes:

aws ce create-anomaly-monitor \ --anomaly-monitor Name=eks-cost-monitor,MonitorType=DIMENSIONAL,MonitorDimension=SERVICE aws ce create-anomaly-subscription \ --anomaly-subscription file://subscription.json

This automatically emails you when EBS, EC2, or data transfer costs exceed historical patterns.

Measuring Impact

After implementing cleanup and optimization:

• Track monthly EBS volume count and total GB
• Monitor load balancer count in Cost Explorer
• Measure cross-AZ and NAT gateway data transfer trends
• Document cost per cleanup action (e.g., “deleted 47 orphaned volumes, saved $376/month”)

Typical results from storage and networking cleanup:

• 10-30% reduction in EBS costs (orphaned volumes + gp3 migration)
• 5-15% reduction in data transfer (networking optimizations)
• $15-50/month saved per consolidated load balancer

Conclusion

Hidden costs in EKS often go unnoticed because they appear as small line items—but across dozens of orphaned volumes, unused load balancers, and unoptimized network paths, they compound to significant waste. Implement automated discovery for orphaned EBS volumes and load balancers, migrate to gp3 storage, deploy VPC endpoints for AWS service access, and use topology-aware routing to reduce cross-AZ traffic. These fixes require minimal engineering effort but deliver sustained monthly savings that scale with your cluster growth. Start with a quarterly audit, automate cleanup with Lambda or scheduled jobs, and make cost hygiene part of your regular operational reviews.

Spot instances offer up to 90% savings on EC2 compute costs, but interruptions can cause outages if not handled properly. This guide shows you how to adopt Spot safely by mixing capacity types, implementing termination handlers, and using Kubernetes scheduling primitives to keep critical workloads protected.

Key Takeaways

• Never run critical stateful workloads on Spot without On-Demand fallback capacity
• Spot termination handlers cordon and drain nodes gracefully during 2-minute warning windows
• Node labels, affinity, and PodDisruptionBudgets control which workloads land on Spot
• Start with batch jobs and stateless services before expanding to broader workloads
• Monitor interruption rates and adjust instance diversification strategies accordingly

Understanding Spot Interruptions

AWS can reclaim Spot instances with 2 minutes notice when capacity is needed for On-Demand customers. The interruption frequency varies by instance type and availability zone—some combinations see interruptions weekly, others monthly.

During the 2-minute window, AWS sends a termination notice via EC2 instance metadata and EventBridge. Without handling this signal, pods are abruptly killed mid-request, potentially causing:

• Dropped database connections
• Failed in-flight transactions
• Incomplete batch jobs that must restart from scratch
• Service degradation if too many replicas disappear simultaneously

The safety-first approach treats Spot as supplemental capacity, not primary.

Architecture: Mixed Capacity Strategy

The safest Spot adoption pattern uses separate node groups for different workload classes:

• Essential nodes (On-Demand or Savings Plans): databases, message queues, critical APIs
• Preemptible nodes (Spot): batch jobs, CI/CD, stateless microservices, dev environments

This prevents cascading failures—even if all Spot nodes disappear, essential services remain available.

Creating Node Groups with Lifecycle Labels

Label nodes during bootstrap so you can target them with affinity rules:

eksctl create nodegroup \ --cluster=my-cluster \ --name=ondemand-essential \ --node-type=m5.large \ --nodes=2 \ --nodes-min=2 \ --nodes-max=5 \ --node-labels="kubernetes.io/lifecycle=essential" eksctl create nodegroup \ --cluster=my-cluster \ --name=spot-preemptible \ --node-type=m5.large,m5a.large,m5n.large \ --nodes=3 \ --nodes-min=0 \ --nodes-max=20 \ --spot \ --node-labels="kubernetes.io/lifecycle=preemptible"

Note the instance type diversification in the Spot group—using multiple types across instance families reduces interruption correlation.

Installing the Spot Termination Handler

The AWS Node Termination Handler monitors for Spot interruption notices and gracefully drains nodes before termination:

kubectl apply -f https://github.com/aws/aws-node-termination-handler/releases/download/v1.19.0/all-resources.yaml

This deploys a DaemonSet that:

1. Polls EC2 metadata for termination notices
2. Cordons the node to prevent new pod scheduling
3. Drains existing pods gracefully (respects PodDisruptionBudgets)
4. Allows Kubernetes to reschedule pods on healthy nodes

Verify it’s running on Spot nodes:

kubectl get daemonset -n kube-system aws-node-termination-handler kubectl logs -n kube-system ds/aws-node-termination-handler --tail=100

You should see log entries showing periodic metadata polling and ready state.

Workload Placement: Affinity and Tolerations

Pinning Critical Workloads to On-Demand

Use requiredDuringSchedulingIgnoredDuringExecution to force databases and stateful services onto essential nodes:

apiVersion: apps/v1 kind: StatefulSet metadata: name: postgres spec: template: spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: "kubernetes.io/lifecycle" operator: "In" values: - essential containers: - name: postgres image: postgres:15

This pod will remain in Pending state if no essential nodes are available—preventing it from landing on Spot.

Preferring Spot for Batch Jobs

For fault-tolerant workloads, use preferredDuringScheduling to favor Spot but allow fallback:

apiVersion: batch/v1 kind: Job metadata: name: data-processing spec: template: spec: affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 100 preference: matchExpressions: - key: "kubernetes.io/lifecycle" operator: "In" values: - preemptible containers: - name: processor image: my-batch-job:latest restartPolicy: OnFailure

The job prefers Spot nodes but can schedule on On-Demand if Spot capacity is unavailable. The restartPolicy: OnFailure ensures Kubernetes retries if a Spot interruption occurs mid-job.

Pod Disruption Budgets: Controlling Eviction Rate

PodDisruptionBudgets (PDBs) ensure a minimum number of replicas remain available during voluntary disruptions like node drains:

apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: api-pdb spec: minAvailable: 2 selector: matchLabels: app: api-server

If your API has 5 replicas and a Spot node terminates, the drain operation waits until Kubernetes schedules new pods on healthy nodes before evicting the 3rd pod. This prevents service degradation.

Critical: PDBs only work during graceful drains, not forced terminations. Always run enough replicas on non-Spot nodes to satisfy minAvailable even if all Spot capacity disappears.

Per-AZ Node Groups for Persistent Volumes

EBS volumes are single-AZ resources. If you run Spot nodes in multiple AZs via a single Auto Scaling group, StatefulSets with PersistentVolumeClaims can get stuck:

1. Pod with PVC in us-east-1a runs on a Spot node in us-east-1a
2. That node is interrupted
3. Cluster Autoscaler provisions a new Spot node in us-east-1b
4. Pod remains Pending because the EBS volume can’t attach cross-AZ

Solution: Create one Auto Scaling group per AZ:

eksctl create nodegroup \ --cluster=my-cluster \ --name=spot-us-east-1a \ --node-zones=us-east-1a \ --spot \ --node-labels="topology.kubernetes.io/zone=us-east-1a,kubernetes.io/lifecycle=preemptible" eksctl create nodegroup \ --cluster=my-cluster \ --name=spot-us-east-1b \ --node-zones=us-east-1b \ --spot \ --node-labels="topology.kubernetes.io/zone=us-east-1b,kubernetes.io/lifecycle=preemptible"

Then use Cluster Autoscaler’s --balance-similar-node-groups flag to distribute scale-outs evenly:

--balance-similar-node-groups=true

Alternatively, use Karpenter with AZ-specific Provisioners and let it handle the topology automatically.

Testing Spot Interruptions

Don’t wait for a real interruption to discover your safety mechanisms don’t work. Simulate terminations:

Manual Drain Test

kubectl drain  --ignore-daemonsets --delete-emptydir-data

Watch pod eviction and rescheduling behavior. Verify PDBs are respected and critical services maintain availability.

Chaos Engineering with Spot Interruptions

Use AWS Fault Injection Simulator to trigger real Spot interruptions in a controlled test:

1. Create an FIS experiment template targeting a specific Spot instance
2. Run the experiment during a load test
3. Measure latency spikes and error rates during node drain
4. Verify termination handler logs show proper cordon/drain sequence

Monitoring and Alerting

Track Spot-related metrics to detect issues early:

• Interruption frequency: Log termination handler events to CloudWatch or Prometheus
• Pod eviction rate: Monitor kube_pod_status_phase{phase="Pending"} spikes
• PDB violations: Alert on kube_poddisruptionbudget_status_pod_disruptions_allowed < 1
• Node replacement time: Measure time from termination notice to new node ready

Set up CloudWatch alarms for Spot instance interruptions:

aws cloudwatch put-metric-alarm \ --alarm-name spot-interruption-rate \ --metric-name SpotInstanceInterruption \ --namespace AWS/EC2Spot \ --statistic Sum \ --period 3600 \ --threshold 5 \ --comparison-operator GreaterThanThreshold

Common Pitfalls and How to Avoid Them

Running databases on Spot without fallback. Even with termination handlers, you risk data inconsistency during abrupt shutdowns. Always use On-Demand or Reserved Instances for stateful workloads.

Insufficient replica counts. If you run 3 replicas and all are on Spot, a simultaneous multi-node interruption (rare but possible) violates your PDB. Keep at least minAvailable + 1 replicas with some on On-Demand.

Single instance type in Spot pools. Using only m5.large creates a single point of failure—if that instance type has high interruption rates, your entire Spot fleet churns. Diversify across families: m5.large,m5a.large,m5n.large,m6i.large.

Ignoring PV topology. StatefulSets on Spot require per-AZ Auto Scaling groups or Karpenter with zone-aware provisioning. Otherwise, pods get stuck Pending after AZ-crossing interruptions.

No termination handler. Without aws-node-termination-handler or equivalent, pods are forcefully killed with no grace period—breaking in-flight requests and database transactions.

Progressive Rollout Strategy

Don’t convert your entire cluster to Spot overnight. Use this staged approach:

1. Week 1: Add a small Spot node group (10% of capacity) for dev/test namespaces
2. Week 2: Deploy batch jobs and CI/CD runners to Spot with affinity rules
3. Week 3: Move stateless APIs with >5 replicas and PDBs to prefer Spot
4. Week 4: Increase Spot percentage to 50-70% of total capacity
5. Ongoing: Monitor interruption rates and adjust instance diversification

At each stage, run load tests and chaos experiments before proceeding.

Expected Savings

Spot pricing varies by instance type and region, but typical savings range from 60-90% versus On-Demand. For a cluster spending $10,000/month on EC2:

• 50% Spot adoption at 70% discount: $3,500/month savings
• 70% Spot adoption at 70% discount: $4,900/month savings

Combine with Savings Plans on your On-Demand baseline for additional 20-40% savings on the remaining 30-50% of capacity.

Conclusion

Spot instances are the single highest-impact cost lever in EKS, but only when implemented with proper safety guardrails. Never run critical stateful workloads on Spot alone—always maintain On-Demand fallback capacity. Install the termination handler, use node affinity to control placement, protect services with PodDisruptionBudgets, and create per-AZ node groups for workloads with persistent volumes. Start with batch jobs and stateless services, measure interruption impact, and expand gradually. With these practices, you can safely capture 60-90% compute savings while maintaining production reliability.

Optimizing EKS costs requires a structured approach that balances quick wins with sustainable practices. This 30/60/90 day plan walks you through measurement, right-sizing, autoscaling, and Spot adoption—the four levers that collectively reduce EKS spend by 50-80% according to real-world implementations.

Key Takeaways

• Week 1-2 focuses on cost visibility: deploy Kubecost and enable AWS Cost & Usage Reports
• Week 3-6 tackles right-sizing using VPA recommendations and actual resource usage data
• Week 7-10 implements autoscaling with HPA and Cluster Autoscaler or Karpenter
• Week 11-14 introduces Spot instances with safety guardrails for non-critical workloads
• Monthly reviews and cleanup automation sustain savings long-term

Days 1-14: Establish Cost Visibility

You can’t optimize what you can’t measure. The first two weeks focus entirely on instrumentation—no optimization yet.

Deploy Kubecost or OpenCost

Kubecost provides per-pod, per-namespace cost attribution by mapping Kubernetes resource requests to AWS pricing data.

helm repo add kubecost https://kubecost.github.io/cost-analyzer/ helm upgrade --install kubecost kubecost/cost-analyzer \ --namespace kubecost --create-namespace \ --set kubecostToken="your-token-here"

For OpenCost (the open-source alternative):

kubectl apply -f https://raw.githubusercontent.com/opencost/opencost/develop/kubernetes/opencost.yaml

Access the Kubecost UI:

kubectl port-forward -n kubecost svc/kubecost-cost-analyzer 9090:9090

Navigate to http://localhost:9090 to see cost breakdowns by namespace, deployment, and pod.

Enable AWS Cost & Usage Reports

Cost & Usage Reports (CUR) provide the authoritative source of AWS pricing data. Kubecost integrates with CUR for accurate cost allocation.

1. Go to AWS Billing Console → Cost & Usage Reports
2. Create a new report with hourly granularity
3. Enable resource IDs and split cost allocation
4. Configure S3 bucket for report delivery
5. Update Kubecost configuration to point to your CUR S3 bucket

Tag Everything

Tags enable cost allocation by team, environment, and application. Apply tags to:

• EC2 instances (via node group tags)
• EBS volumes (via StorageClass parameters)
• Load balancers (via Service annotations)

Example node group tags in eksctl:

nodeGroups: - name: production-nodes tags: Environment: production Team: platform CostCenter: engineering

Activate cost allocation tags in AWS Billing preferences so they appear in Cost Explorer.

Baseline Current Costs

Document your starting point:

aws ce get-cost-and-usage \ --time-period Start=2025-01-01,End=2025-01-31 \ --granularity DAILY \ --metrics UnblendedCost \ --group-by Type=DIMENSION,Key=SERVICE

Record EC2, EBS, data transfer, and EKS control plane costs. This becomes your benchmark for measuring progress.

Days 15-45: Right-Size Resources

Right-sizing eliminates the gap between reserved resources (requests) and actual usage—the single biggest cost waste in most clusters.

Install Metrics Server

Metrics Server provides the foundation for autoscaling and right-sizing decisions:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Verify it’s working:

kubectl top nodes kubectl top pods --all-namespaces

Analyze Resource Slack

Compare pod requests to actual usage:

kubectl get pods --all-namespaces -o custom-columns=\ 'NAMESPACE:.metadata.namespace,\ NAME:.metadata.name,\ CPU_REQ:.spec.containers[*].resources.requests.cpu,\ MEM_REQ:.spec.containers[*].resources.requests.memory'

In Kubecost, navigate to the “Savings” tab to see rightsizing recommendations. Look for pods with <20% CPU utilization or excessive memory requests.

Deploy Vertical Pod Autoscaler (Recommendation Mode)

VPA analyzes historical usage and recommends optimal requests and limits:

git clone https://github.com/kubernetes/autoscaler.git cd autoscaler/vertical-pod-autoscaler ./hack/vpa-up.sh

Create a VPA in recommendation-only mode:

apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: name: my-app-vpa spec: targetRef: apiVersion: "apps/v1" kind: Deployment name: my-app updatePolicy: updateMode: "Off"

After 24-48 hours, check recommendations:

kubectl describe vpa my-app-vpa

Apply recommendations incrementally—reduce requests by 10-30% initially and monitor for OOMKilled events or CPU throttling.

Set Resource Limits and Quotas

Prevent future over-provisioning with LimitRanges and ResourceQuotas:

apiVersion: v1 kind: LimitRange metadata: name: default-limits namespace: production spec: limits: - default: cpu: 500m memory: 512Mi defaultRequest: cpu: 100m memory: 128Mi type: Container

Days 46-75: Implement Autoscaling

Autoscaling eliminates manual capacity management and ensures you pay only for what you use.

Configure Horizontal Pod Autoscaler

HPA scales pod replicas based on CPU, memory, or custom metrics:

apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: my-app-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: my-app minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70

Monitor HPA decisions:

kubectl get hpa --watch

Deploy Cluster Autoscaler or Karpenter

Cluster Autoscaler scales node groups based on pending pods. It’s mature and works well with managed node groups.

Create an IAM role with autoscaling permissions and annotate the ServiceAccount:

apiVersion: v1 kind: ServiceAccount metadata: name: cluster-autoscaler namespace: kube-system annotations: eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT:role/cluster-autoscaler

Deploy Cluster Autoscaler with the --balance-similar-node-groups flag to distribute scale-outs evenly across AZs:

helm repo add autoscaler https://kubernetes.github.io/autoscaler helm upgrade --install cluster-autoscaler autoscaler/cluster-autoscaler \ --namespace kube-system \ --set autoDiscovery.clusterName=my-cluster \ --set extraArgs.balance-similar-node-groups=true

Karpenter is a newer alternative that provisions right-sized nodes faster and supports more aggressive consolidation. It’s ideal for dynamic workloads.

Choose based on your operational maturity—Cluster Autoscaler for stability, Karpenter for optimization.

Test Autoscaling Behavior

Generate load to trigger scaling:

kubectl run -i --tty load-generator --rm --image=busybox --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://my-app; done"

Watch HPA scale pods and Cluster Autoscaler provision nodes. Verify that scale-down happens after the load subsides (default: 10 minutes).

Days 76-90: Introduce Spot Instances

Spot instances can reduce compute costs by up to 90%, but they require careful handling to avoid disruptions.

Create Mixed Node Groups

Start with a small Spot node group for non-critical workloads:

eksctl create nodegroup \ --cluster=my-cluster \ --name=spot-nodes \ --node-type=m5.large \ --nodes=3 \ --nodes-min=1 \ --nodes-max=10 \ --spot \ --node-labels="kubernetes.io/lifecycle=preemptible"

Keep a separate On-Demand node group for critical workloads:

eksctl create nodegroup \ --cluster=my-cluster \ --name=ondemand-nodes \ --node-type=m5.large \ --nodes=2 \ --node-labels="kubernetes.io/lifecycle=essential"

Install Spot Termination Handler

AWS sends a 2-minute warning before reclaiming Spot instances. A termination handler cordons and drains nodes gracefully:

kubectl apply -f https://github.com/aws/aws-node-termination-handler/releases/download/v1.19.0/all-resources.yaml

Verify it’s running:

kubectl get daemonset -n kube-system | grep aws-node-termination-handler

Use Node Affinity to Pin Critical Pods

Ensure databases and stateful workloads stay on On-Demand nodes:

affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: "kubernetes.io/lifecycle" operator: "In" values: - essential

For batch jobs, prefer Spot:

affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 100 preference: matchExpressions: - key: "kubernetes.io/lifecycle" operator: "In" values: - preemptible

Set Pod Disruption Budgets

PDBs prevent too many pods from being evicted simultaneously:

apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: my-app-pdb spec: minAvailable: 2 selector: matchLabels: app: my-app

Ongoing: Sustain and Improve

Cost optimization isn’t a one-time project. Establish monthly rituals:

Monthly Cost Review

• Review Kubecost “Savings” tab for new rightsizing opportunities
• Check for orphaned EBS volumes and unused load balancers
• Analyze Cost Explorer for anomalies and trends
• Adjust Savings Plans coverage based on baseline usage

Automate Cleanup

Find and delete orphaned volumes:

aws ec2 describe-volumes \ --filters Name=status,Values=available \ --query 'Volumes[*].[VolumeId,Size,CreateTime]' \ --output table

Schedule non-production cluster shutdowns using kube-downscaler:

helm repo add kube-downscaler https://charts.kiwigrid.com helm upgrade --install kube-downscaler kube-downscaler/kube-downscaler \ --set env.DEFAULT_UPTIME="Mon-Fri 08:00-18:00 America/New_York"

Measuring Success

Track these metrics monthly:

• Total EKS spend (EC2 + EBS + data transfer + control plane)
• Cost per pod (from Kubecost)
• Node utilization (target: >60% average CPU/memory)
• Spot instance percentage (target: 50-70% for tolerant workloads)
• Orphaned resource count (target: zero)

Expect 15-25% savings from right-sizing alone, 10-20% from autoscaling, and 30-50% from Spot adoption—compounding to 50-80% total reduction when combined.

Conclusion

This 90-day plan provides a structured path from measurement to meaningful savings. Start with visibility in weeks 1-2, tackle right-sizing in weeks 3-6, implement autoscaling in weeks 7-10, and carefully introduce Spot in weeks 11-14. The key is incremental progress with validation at each step—don’t skip measurement, don’t apply VPA in auto mode without testing, and don’t put critical workloads on Spot without fallback capacity. By day 90, you’ll have the foundation for sustainable cost optimization that adapts as your cluster grows.

Network configuration choices in Amazon EKS directly impact your AWS bill through cross-AZ data transfer charges, NAT gateway costs, and load balancer processing fees. By optimizing how traffic flows between pods, nodes, and external services, you can reduce these often-overlooked expenses by 40% or more without sacrificing performance or availability.

Key Takeaways

• Cross-AZ data transfer and NAT gateway egress are hidden cost drivers in EKS clusters
• Service internalTrafficPolicy: Local keeps traffic node-local and eliminates cross-AZ hops
• AWS Load Balancer Controller IP mode reduces unnecessary kube-proxy routing
• VPC endpoints for ECR and S3 eliminate NAT gateway charges for image pulls
• Topology-aware routing automatically prefers same-zone endpoints when available

Why EKS Network Costs Matter

When you run Kubernetes on EKS, compute costs get most of the attention. But for microservice-heavy workloads with frequent pod-to-pod communication or large container images, networking can consume 15-30% of your total AWS spend.

The primary culprits are:

• Cross-AZ data transfer: AWS charges $0.01/GB for traffic between availability zones
• NAT gateway processing: $0.045/GB for outbound internet traffic
• Load balancer data processing: ALBs charge per GB processed
• Container registry pulls: Repeated ECR image downloads across NAT or internet gateways

The good news? Most of these costs are controllable through configuration, not infrastructure changes.

Tweak #1: Enable Service internalTrafficPolicy: Local

By default, Kubernetes Services distribute traffic to any healthy pod across your entire cluster. If you have a three-AZ deployment, this means a pod in us-east-1a might send requests to a backend in us-east-1b—triggering cross-AZ charges.

The fix: Set internalTrafficPolicy: Local on chatty internal services to route only to pods on the same node.

apiVersion: v1 kind: Service metadata: name: orders-service namespace: ecommerce spec: selector: app: orders type: ClusterIP internalTrafficPolicy: Local ports: - protocol: TCP port: 3003 targetPort: 3003

Important: This only works when you have pod replicas on every node (or use DaemonSets). If a node lacks a local endpoint, traffic will be dropped. Use topologySpreadConstraints to ensure even distribution:

topologySpreadConstraints: - maxSkew: 1 topologyKey: "topology.kubernetes.io/zone" whenUnsatisfiable: ScheduleAnyway labelSelector: matchLabels: app: orders

Expected savings: For services handling 100GB/day of internal traffic across zones, this eliminates $1/day ($30/month) in transfer fees per service.

Tweak #2: Use Topology-Aware Routing

Kubernetes 1.30+ introduced trafficDistribution: PreferClose, which automatically routes traffic to same-zone endpoints when available, falling back to other zones only when necessary.

apiVersion: v1 kind: Service metadata: name: payments-api spec: trafficDistribution: PreferClose selector: app: payments ports: - port: 8080

Alternatively, use the annotation-based approach for earlier Kubernetes versions:

metadata: annotations: service.kubernetes.io/topology-mode: Auto

The Kubernetes control plane uses EndpointSlice hints to guide kube-proxy toward local endpoints. This balances availability (cross-zone failover still works) with cost savings.

Gotcha: Topology hints require relatively even pod distribution. If you have 10 pods in us-east-1a and 2 in us-east-1b, hints may not activate. Monitor with:

kubectl get endpointslices -n ecommerce -o yaml

Look for endpoints[].hints.forZones to verify hint assignment.

Tweak #3: Switch Load Balancers to IP Mode

The AWS Load Balancer Controller supports two target modes: instance (default) and ip. In instance mode, the ALB sends traffic to NodePorts, and kube-proxy forwards it to the final pod—often crossing AZs.

In IP mode, the ALB registers pod IPs directly, eliminating the extra kube-proxy hop and reducing cross-AZ traffic.

apiVersion: v1 kind: Service metadata: name: frontend annotations: service.beta.kubernetes.io/aws-load-balancer-type: "external" service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip" spec: type: LoadBalancer selector: app: frontend ports: - port: 80 targetPort: 8080

Trade-off: IP mode requires the AWS Load Balancer Controller (install via Helm or EKS add-on). It also changes how source IP preservation works—test before applying to production.

Expected savings: Reduces cross-AZ hops for external-facing services. For a service processing 500GB/month, this can save $5-15/month per load balancer.

Tweak #4: Deploy VPC Endpoints for ECR and S3

Every time a node pulls a container image from ECR, the traffic flows through your NAT gateway by default—costing $0.045/GB. For clusters that scale frequently or use large images, this adds up fast.

Solution: Create VPC Interface Endpoints for ECR and a Gateway Endpoint for S3 (ECR stores layers in S3).

aws ec2 create-vpc-endpoint \ --vpc-id vpc-xxxxx \ --service-name com.amazonaws.us-east-1.ecr.api \ --subnet-ids subnet-xxxxx subnet-yyyyy \ --security-group-ids sg-xxxxx aws ec2 create-vpc-endpoint \ --vpc-id vpc-xxxxx \ --service-name com.amazonaws.us-east-1.ecr.dkr \ --subnet-ids subnet-xxxxx subnet-yyyyy \ --security-group-ids sg-xxxxx aws ec2 create-vpc-endpoint \ --vpc-id vpc-xxxxx \ --service-name com.amazonaws.us-east-1.s3 \ --route-table-ids rtb-xxxxx

Cost: Interface endpoints cost ~$7/month per AZ, but eliminate NAT charges. If you pull 100GB/month of images, you save $4.50/month in NAT fees per AZ—break-even is around 60GB.

Bonus tip: Use ECR replication to keep images in the same region, avoiding cross-region transfer fees entirely.

Tweak #5: Optimize NAT Gateway Placement

A single NAT gateway in one AZ forces all outbound traffic from other zones to cross AZ boundaries—doubling your costs (cross-AZ transfer + NAT processing).

Best practice: Deploy one NAT gateway per availability zone and configure route tables so each private subnet routes to its local NAT.

aws ec2 describe-nat-gateways \ --filter Name=vpc-id,Values=vpc-xxxxx \ --query 'NatGateways[*].[NatGatewayId,SubnetId,State]' \ --output table

Verify your route tables point to the correct local NAT:

aws ec2 describe-route-tables \ --filters Name=vpc-id,Values=vpc-xxxxx \ --query 'RouteTables[*].{ID:RouteTableId,Routes:Routes[?GatewayId!=null]}' \ --output table

Trade-off: Multiple NAT gateways increase fixed costs ($0.045/hour each = ~$32/month per AZ), but eliminate cross-AZ transfer fees on outbound traffic. This pays off when you have >70GB/month of egress per AZ.

Measuring Your Network Cost Savings

After implementing these changes, validate the impact using AWS Cost Explorer:

aws ce get-cost-and-usage \ --time-period Start=2025-01-01,End=2025-01-31 \ --granularity DAILY \ --metrics UnblendedCost \ --group-by Type=DIMENSION,Key=USAGE_TYPE \ --filter file://filter.json

Look for usage types like:

• DataTransfer-Regional-Bytes (cross-AZ)
• NatGateway-Bytes (NAT processing)
• LoadBalancerUsage (ALB/NLB data processing)

You can also use tools like Kubecost to attribute network costs to specific namespaces and services, making it easier to identify which workloads benefit most from optimization.

Common Gotchas

internalTrafficPolicy: Local can drop traffic if pods aren’t evenly distributed. Always combine with topology spread constraints and test failover scenarios.

Topology hints don’t work with wildly uneven pod counts. If one zone has 90% of your pods, Kubernetes won’t activate hints to avoid overloading that zone.

VPC endpoints have a per-AZ cost. They’re worth it for high image-pull workloads but can increase costs for small clusters. Calculate your break-even point.

IP mode load balancers change IP preservation behavior. Test SSL passthrough and X-Forwarded-For headers before switching production services.

Conclusion

Network optimization in EKS isn’t about choosing between cost and performance—it’s about making traffic flow intelligently. By keeping traffic local when possible, eliminating unnecessary hops through kube-proxy and NAT gateways, and using VPC endpoints for AWS services, you can cut network-related costs by 40% while often improving latency. Start with internalTrafficPolicy: Local on high-traffic internal services, add VPC endpoints for ECR, and measure the results in Cost Explorer. These changes require no additional infrastructure—just smarter configuration.