Kubernetes vs Serverless: A Practical Decision Framework

Choose serverless (AWS Lambda, Google Cloud Run, Azure Functions or Container Apps) when traffic is spiky or idle much of the day, requests are short, and your team has no one who wants to own a platform. Choose Kubernetes when load is sustained, connections are long-lived, you need GPUs or sidecars, or enough teams share the platform to justify a group that runs it. The deciding variables are utilization, request duration, latency tolerance and who carries the pager for the platform, not fashion. Most production systems end up hybrid, and serverless containers such as Cloud Run and Fargate are where many teams should start.

Key takeaways

• This is a spectrum, not a binary: functions, serverless containers, managed node modes (GKE Autopilot, EKS Auto Mode) and standard Kubernetes each move a different amount of work off your team.
• Hard limits decide many cases outright. Lambda caps a standard invocation at 15 minutes and 10,240 MB of memory; Cloud Run caps a request at 60 minutes.
• Per-request billing wins at low and spiky utilization and loses at high sustained utilization, especially for I/O-bound work. The model below lets you find your own crossover.
• The biggest Kubernetes cost is people: upgrades three times a year, networking, policy and platform on-call.
• Serverless moves failure modes rather than removing them: concurrency exhaustion, connection storms against databases, duplicate events and runaway costs.
• Standardize what crosses the boundary (tracing, deployment pipeline, IaC) so a hybrid estate does not become two estates.

What you actually stop operating

The useful comparison is not "servers or no servers". It is which layers your team owns.

Every row you hand to the provider is a row nobody on your team has to upgrade at 2 a.m. Every row you keep is a knob you can tune. That is the whole trade-off, and the rest of this guide is about pricing it.

The middle columns are the ones teams skip. Cloud Run, Fargate and Azure Container Apps run an ordinary container image, scale on requests, and leave you no cluster to upgrade. AWS also blurred the line from the other side with Lambda Managed Instances, which runs Lambda functions on EC2 capacity you choose and can bill under Savings Plans. If you already build containers (our Docker production guide covers doing that well), a serverless container platform is often the lowest-risk first step.

Hard limits that decide for you

Check these before any cost analysis. If a limit rules an option out, the spreadsheet does not matter.

Two consequences come up constantly. A request that might run longer than 15 minutes cannot be a standard Lambda function; break it into steps (Step Functions, Lambda durable functions, a queue) or run it as a container. And because a Lambda execution environment serves one request at a time, an I/O-bound function waiting 180 ms on a database is billed for those 180 ms on every request, while a container instance can serve dozens of such requests concurrently.

Cost: a model you can rerun

Lambda's bill is two numbers. At the published us-east-1 list price for x86, that is $0.20 per million requests plus $0.0000166667 per GB-second, before the monthly free tier. Arm (Graviton) is about 20 percent cheaper per GB-second.

lambda_monthly = requests / 1e6 * 0.20
               + requests * avg_billed_seconds * memory_gb * 0.0000166667
 
avg_concurrency = requests * avg_billed_seconds / seconds_in_month

Worked through for an API at 200 ms average duration and 512 MB:

Now add the parts people forget. Behind API Gateway, HTTP APIs list at $1.00 per million requests for the first 300 million each month and REST APIs at $3.50 per million, so the gateway can cost more than the compute. Functions in a VPC that reach the internet pay NAT gateway processing charges. Logs are billed per GB ingested.

The container side of the model is simpler:

container_monthly = instances_needed * price_per_instance_hour * hours
instances_needed  = peak_concurrency / requests_per_instance_at_target_utilization
                    (rounded up, plus headroom and one spare per zone)

On EKS, add $0.10 per cluster-hour for the control plane in standard support, roughly $73 a month, before any nodes. Then add the line most comparisons omit: the engineering time to run the platform. A cluster that needs a fraction of an engineer's time every week costs more than the nodes for most small teams.

How to read the result. At 10 million requests the function costs less than one small always-on instance, so serverless wins unless something else rules it out. At an average concurrency in the dozens, with requests that mostly wait on I/O, a few container instances with request concurrency usually cost less than paying per GB-second, and the gap grows with volume. Our cloud cost optimization guide covers how to commit-discount whichever side you land on.

Latency and scaling behavior

Cold starts. When no warm execution environment is free, the platform creates one: fetch the code or image, start the runtime, run your initialization. How long that takes depends on the runtime, package size and what your init code does, so measure your own. Mitigations, in order of cost: keep packages small and move heavy setup out of the request path; use Lambda SnapStart (Java, Python and .NET) to restore from a snapshot; set provisioned concurrency for Lambda or minimum instances for Cloud Run, which means paying for idle capacity to remove the effect.

Scale-up speed. Lambda can add up to 1,000 execution environments per function every 10 seconds, which absorbs most bursts. Kubernetes scales in two stages: the HorizontalPodAutoscaler adds pods within seconds if nodes have room, and the node autoscaler (Cluster Autoscaler or Karpenter) adds nodes, which takes minutes. Teams on Kubernetes keep headroom or overprovision with low-priority placeholder pods for this reason.

Steady-state latency. A warm container that holds database connections, caches and JIT-compiled code is hard to beat for consistent tail latency under sustained load. That is where Kubernetes, or Cloud Run with minimum instances, has a structural advantage over functions.

A decision tree

The tree is deliberately biased toward serverless containers when the answer is unclear. They run the same image you would deploy to Kubernetes, so moving later is a deployment change, not a rewrite.

The same service, three ways

Kubernetes. The minimum we consider production-ready for a stateless API: resource requests, honest probes, zone spreading, an autoscaler and a disruption budget.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-api
  namespace: orders
spec:
  selector:
    matchLabels:
      app: orders-api
  strategy:
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 25%
  template:
    metadata:
      labels:
        app: orders-api
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: ScheduleAnyway
          labelSelector:
            matchLabels:
              app: orders-api
      containers:
        - name: api
          image: ghcr.io/acme/orders-api@sha256:REPLACE_WITH_DIGEST
          ports:
            - name: http
              containerPort: 8080
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              memory: 256Mi
          readinessProbe:
            httpGet:
              path: /ready
              port: http
            periodSeconds: 5
          livenessProbe:
            httpGet:
              path: /live
              port: http
            periodSeconds: 10
            failureThreshold: 3
          securityContext:
            runAsNonRoot: true
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: orders-api
  namespace: orders
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: orders-api
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: orders-api
  namespace: orders
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: orders-api

The memory limit equals the request so the pod is never killed for using memory it was promised. There is no CPU limit on purpose: CPU limits cause throttling even when the node has idle cores, and the request already guarantees a fair share. The Kubernetes security guide covers the policy layer this example leaves out.

Cloud Run. The same image, no cluster:

gcloud run deploy orders-api \
  --image=us-docker.pkg.dev/acme-prod/apps/orders-api@sha256:REPLACE_WITH_DIGEST \
  --region=us-central1 \
  --cpu=1 --memory=512Mi \
  --concurrency=80 \
  --min-instances=1 --max-instances=50 \
  --timeout=60s \
  --service-account=orders-api@acme-prod.iam.gserviceaccount.com \
  --no-allow-unauthenticated

--min-instances=1 keeps one warm instance to absorb the first request after idle. --max-instances is your cost ceiling and your database's protection, so set it from what the database can accept, not from hope.

Lambda behind a queue. The event-driven side of the same system, in Terraform:

resource "aws_lambda_function" "thumbnailer" {
  function_name = "thumbnailer"
  role          = aws_iam_role.thumbnailer.arn
  package_type  = "Image"
  image_uri     = "${aws_ecr_repository.thumbnailer.repository_url}@${var.image_digest}"
  architectures = ["arm64"]
  memory_size   = 1024
  timeout       = 60
 
  reserved_concurrent_executions = 100
 
  environment {
    variables = {
      OUTPUT_BUCKET = aws_s3_bucket.thumbnails.bucket
    }
  }
}
 
resource "aws_lambda_event_source_mapping" "uploads" {
  event_source_arn        = aws_sqs_queue.uploads.arn
  function_name           = aws_lambda_function.thumbnailer.arn
  batch_size              = 10
  function_response_types = ["ReportBatchItemFailures"]
 
  scaling_config {
    maximum_concurrency = 50
  }
}

reserved_concurrent_executions both guarantees and caps this function's share of the account's concurrency. maximum_concurrency on the queue mapping throttles consumption before it reaches that cap, and ReportBatchItemFailures retries only the failed messages in a batch instead of the whole batch.

Hybrid is the normal end state

Most mature systems land here: core APIs on containers (Kubernetes or serverless containers), event glue and scheduled work on functions, state in managed databases and queues. Patterns for the function side are in our serverless architecture patterns guide.

Hybrid works when three things are shared across the boundary:

• One trace. Instrument both sides with OpenTelemetry and propagate W3C trace context through queues as message attributes, so a request that crosses from a pod into a function is one trace, not two mysteries.
• One delivery path. The same pipeline builds the image, scans it and deploys it, whether the target is a Deployment, a Cloud Run service or a Lambda function.
• One IaC repository. Functions, queues, clusters and IAM in the same Terraform or equivalent, reviewed the same way.

If you run Kubernetes and want event-driven scaling without functions, KEDA scales Deployments from queue depth, stream lag and dozens of other sources, including down to zero. Knative Serving adds request-driven scale to zero for HTTP. Both give a serverless experience on a cluster you still operate.

What goes wrong in production

On serverless:

• One function starves the account. Lambda concurrency is a Regional pool shared by every function. A runaway consumer can take all of it and throttle your user-facing API. Set reserved concurrency on anything critical and cap the noisy consumers.
• Connection storms. A burst that scales a function to hundreds of environments opens hundreds of database connections. Relational databases fall over long before Lambda does. Put RDS Proxy or another pooler in front, cap concurrency to what the database can take, or use a datastore designed for it.
• Duplicate side effects. Asynchronous invocations are retried on error (twice by default), queues deliver at least once, and clients retry on timeouts. Make handlers idempotent with an idempotency key stored alongside the side effect.
• Mismatched timeouts. API Gateway REST APIs default to a 29-second integration timeout. A function with a 60-second timeout behind it keeps running and billing after the client has received an error. Align timeouts from the edge inward, each layer shorter than the one in front of it.
• Recursive triggers and runaway bills. A function that writes to the bucket that triggers it can loop. Use separate buckets or prefixes, set budgets with alerts, and cap concurrency so a loop is limited in cost.

On Kubernetes:

• Missing requests. Pods without resource requests get packed onto nodes until something is OOMKilled or starved. Require requests through admission policy.
• No PodDisruptionBudget. A node upgrade drains every replica of a service at once. A PDB is a few lines and prevents a self-inflicted outage.
• The upgrade treadmill. Kubernetes ships three minor releases a year, and EKS standard support for each lasts 14 months before paid extended support begins. A cluster nobody upgrades becomes a cluster nobody can upgrade. Budget the time every quarter.
• Autoscaling lag. HPA reacts in seconds, but new nodes take minutes. Spiky traffic needs headroom, faster node provisioning, or the spiky part moved to a serverless service.
• Paying for requests nobody uses. Requests set high "to be safe" are billed whether used or not. Review actual usage against requests regularly; this is where most Kubernetes cost hides.

Frequently Asked Questions

Is serverless cheaper than Kubernetes?

Serverless is usually cheaper for spiky or low-volume workloads because you pay nothing while idle and run no cluster. As average utilization rises, per-request billing overtakes the cost of always-on containers, especially for functions that spend most of their time waiting on I/O. Run the numbers for your own traffic, and include API gateway, NAT and the engineering time a cluster needs.

When should you not use Kubernetes?

Skip Kubernetes when you have a few services, a small team with no one to own the platform, and traffic that a managed serverless or serverless container service handles within its limits. The cluster itself is rarely the problem; upgrades, networking, security policy and on-call for the platform are. Adopt it when the workload or the number of teams justifies that ownership.

Can serverless handle high traffic?

Yes, within its quotas. AWS Lambda has a default quota of 1,000 concurrent executions per Region, which can be raised (new accounts start lower), and each function can add up to 1,000 execution environments every 10 seconds. The usual bottleneck is not Lambda but what it calls, such as a relational database that cannot accept thousands of new connections.

What is the difference between Cloud Run and Kubernetes?

Cloud Run runs your container and handles scaling, including to zero, with no nodes or cluster to manage, and it serves many concurrent requests per instance. Kubernetes gives you full control over scheduling, networking, sidecars, stateful workloads and GPUs, and in exchange you own the cluster. Cloud Run is a good default for stateless HTTP services; Kubernetes earns its place when you need what Cloud Run does not offer.

Can you run serverless on Kubernetes?

Yes. Knative Serving gives Kubernetes request-driven autoscaling with scale to zero, and KEDA scales Deployments from queue length, stream lag and other event sources. You get a serverless developer experience, but your team still operates the cluster underneath.

How much do cold starts matter?

Cold starts matter for latency-sensitive, user-facing requests on functions that sit idle, and barely matter for background jobs. Their length depends on runtime, package size and initialization code. Provisioned concurrency, Lambda SnapStart and minimum instances on Cloud Run remove most of the effect, at the cost of paying for idle capacity.

Need help choosing your compute platform?

We help teams pick the compute model that fits their traffic and their people, then build and run it: Kubernetes platforms, serverless systems, and the hybrids in between. If we built it, we are on the pager for it. The first call is with the engineer who would do the work, not a salesperson.

• Cloud Architecture and Infrastructure
• Startup Tech Consulting
• Talk to an engineer