Skip to content
Certification Guide

Task 2.1 · 19 min read

Scaling, Caching and the Edge: Load Balancers, Stateless Tiers and CloudFront

Services in this lesson

  • Elastic Load Balancing (ELB)in scope
  • Amazon CloudFrontin scope
  • Amazon ElastiCachein scope
  • Amazon EC2 Auto Scalingin scope
  • Amazon RDSin scope
  • Amazon API Gatewayin scope

What the exam tests

  • Choosing between horizontal scaling (more copies) and vertical scaling (a bigger copy), and making a tier stateless so it can scale out.
  • Picking the load balancer from protocol and feature words. Application Load Balancer means HTTP routing. Network Load Balancer means TCP or UDP, extreme throughput and static IP addresses.
  • Choosing a caching layer and strategy: Amazon CloudFront at the edge, API Gateway response caching, Amazon ElastiCache with lazy loading or write-through, and expiry times.
  • Using CloudFront to speed up static and dynamic content: origin access control for S3, expiry times, invalidation versus versioned file names, and origin failover.
  • Knowing when read replicas solve the problem (read-heavy databases) and when they don't (write bottlenecks, failover).

The core concept

A web application grows by adding identical copies of each layer, and it stays fast by answering repeated requests from a cache. The exam rewards the answer that scales out the right layer and caches as close to the user as the data allows.

An analogy: a sandwich shop at lunchtime

Picture a sandwich shop when the lunch rush hits:

  • The shop opens more identical counters instead of hiring one super-fast cook. That's horizontal scaling. A faster cook is vertical scaling, and there's a limit to how fast one person gets.
  • A host at the door sends each customer to a free counter and skips any counter whose cook is out sick. That's a load balancer.
  • Orders are pinned to a shared board, not remembered by one cashier. Any counter can finish any order, and closing a counter loses nothing. That's a stateless tier.
  • Popular sandwiches sit pre-made in a case by the door, so most customers never reach a counter. That's a cache.

Where the analogy breaks: a pre-made sandwich is gone once sold, but one cached response can be served to any number of users until it expires.

Walk two readers through it

The walkthrough follows two readers of Harbor Daily, a news site whose app runs in one AWS Region.

Enlarge
Scale out stateless copies, then cache whatever repeats at each layer.

Wen reads Harbor Daily from Sydney. Ines, an editor, works for the site and is signed in to leave comments.

  1. Wen's request reaches a CloudFront edge location. Amazon CloudFront is a content delivery network (CDN): a set of edge locations around the world that keep cached copies of responses. Sydney has one close to Wen.
  2. The photo on the page is already cached. It stays cached until its TTL (time to live, how long a cached copy is kept) runs out. If it weren't cached, CloudFront would fetch it from the origin, the place the real file lives. Here that's an Amazon S3 bucket, which accepts only CloudFront through origin access control (OAC).
  3. Wen's /api/headlines call misses the cache. CloudFront forwards it to the Application Load Balancer (ALB), the single entry point for application traffic. The ALB runs a health check (a regular test request) on every app copy and sends requests only to healthy ones.
  4. A listener rule routes the call. A listener is the port and protocol the ALB accepts traffic on. A listener rule is its instruction, such as "requests for /api/* go to the API copies." The ALB picks one of several identical Amazon EC2 instances (virtual servers) in that group.
  5. The app copy needs the top stories. It asks Amazon ElastiCache, an in-memory cache, first. On a miss it queries a read replica, a read-only copy of the database, and stores the answer in ElastiCache. Filling the cache only on a miss is called lazy loading.
  6. Ines posts a comment. The write goes to the primary database, the only copy that accepts writes. The primary copies the change to the read replica with asynchronous replication: in the background, after the write is confirmed. So the replica lags slightly behind.
  7. Ines's next click lands on a different app copy. She stays signed in, because her session lives in ElastiCache, not in the memory of the copy that served her before.
  8. Traffic drops in the evening. Amazon EC2 Auto Scaling removes two app copies. Nobody is signed out, because no copy held anything a later request needed.

Keep Ines's session in one copy's memory instead, and step 7 signs her out, because her next click lands on another copy. Step 8 would then also sign out anyone whose copy was removed.

The rules the exam tests

  • Scale out first. Horizontal scaling adds copies and spreads load across them. It has a much higher ceiling (account quotas still apply) and removes single points of failure.
  • Scaling out needs stateless copies. Stateless means nothing a later request needs is stored on the copy itself. The exam prefers horizontal scaling whenever a component can run as identical copies.
  • Scaling up has a ceiling. Vertical scaling moves to a bigger instance. It stops at the largest size, usually needs a restart, and still leaves one copy.
  • A load balancer is the front door. It gives clients one endpoint, health-checks every copy and sends traffic only to healthy ones. EC2 Auto Scaling adds and removes copies behind it (policies are in 3.2).
  • Pick the load balancer by layer. Layer 7 means the balancer reads the HTTP request, so it can route by path or host. Layer 4 means it sees only connections and ports.
  • Layer 4 carries more than web traffic. TCP and UDP are the two basic ways to send data over a network. TLS is the encryption behind HTTPS.
  • Every cache hit is a request the next tier never sees. Put the cache as close to the user as the data allows.
  • Match the cache to the data. Shared content goes to CloudFront. Repeated API responses go to the API Gateway stage cache, a cache Amazon API Gateway keeps for one deployed stage of a REST API. Hot database reads and sessions go to ElastiCache.
  • Read replicas scale reads only. A replica is a full read-only copy, not a cache, and it can lag. It takes read load off the primary like a cache does. Writes and failover need other answers.
Deep diveScaling up, and why sticky sessions aren't the fix

Vertical scaling is still the right answer when software can't run as more than one copy, such as a legacy app. You accept the size ceiling and the single copy.

Sticky sessions tell the load balancer to keep sending a user to the same copy. They only pin the user to that copy. When that copy goes away, the session goes with it, so treat sticky sessions as a stopgap, not the design.

The real fix is to move sessions out to ElastiCache or Amazon DynamoDB (a key-value database). Then Auto Scaling can remove any instance safely.

Deep diveLoad balancer details the exam checks
  • An ALB works at layer 7, the application layer. It reads HTTP, HTTPS and gRPC (a remote procedure call protocol), so it can route on path, host, headers, query strings and HTTP method.
  • ALB targets can be instances, IP addresses or AWS Lambda functions.
  • An NLB (Network Load Balancer) works at layer 4, the transport layer. It handles TCP, UDP and TLS at very high throughput and routes on protocol and port only.
  • NLB targets can be instances, IP addresses or an ALB.
  • Fixed IP addresses: an NLB has one static IP address in each Availability Zone (AZ) it uses. An AZ is an isolated group of data centers in a Region.
  • Elastic IPs on an NLB: an internet-facing NLB can take an Elastic IP address in each subnet mapping. An Elastic IP is a static public IPv4 address allocated to your account.
  • An ALB has no fixed IP address. Clients reach it by its DNS name.
  • Cross-zone load balancing spreads traffic evenly across targets in every AZ. For an ALB it is always on at the load balancer level, and you can turn it off per target group. It is off by default for an NLB.
Deep diveCloudFront freshness and failover
  • TTL (time to live) is how long CloudFront keeps a cached copy before checking the origin again. The default TTL is 24 hours unless your cache policy sets another value.
  • The origin can set freshness with a Cache-Control header. A minimum TTL above 0 beats the origin. CloudFront still caches for that long, even if the origin sends no-cache.
  • A cache key is what CloudFront uses to tell cached objects apart. A versioned file name such as app.v42.js is a new cache key, so there's nothing stale to remove.
  • An invalidation removes cached copies before their TTL ends.
  • An origin group pairs a primary and a secondary origin. CloudFront fails over to the secondary when the primary returns a status code you chose, such as 500, 502, 503 or 504.
  • Connection failures and timeouts count only through those codes. A failed connection triggers failover only if 503 is selected. A timeout triggers it only if 504 is selected.
  • Failover applies only to GET, HEAD and OPTIONS requests.

Decision guide

Decision guide
If the question says…Reach for…Because…
"route /api and /images to different services", "host-based routing", HTTP/HTTPS, containers or Lambda targetsApplication Load BalancerLayer 7 rules route on path, host and headers. Targets can be instances, IPs or Lambda
"TCP or UDP", "millions of requests per second", "ultra-low latency", "preserve the client IP"Network Load BalancerLayer 4 balancing built for very high throughput on TCP, UDP and TLS
"partners must allowlist fixed IP addresses"Network Load Balancer with an Elastic IP per Availability ZoneAn internet-facing NLB takes an Elastic IP per subnet. An ALB is reached by DNS name
"users are logged out when instances scale in"Move sessions to ElastiCache or DynamoDBStateless copies can be removed safely. Sticky sessions still lose the session with the target
"legacy app can't run as multiple copies", "needs more memory now"Vertical scaling: a larger instance typeThe only lever when software can't be distributed. Accept the ceiling and the single copy
"global users", "static assets load slowly", "reduce load on the origin"CloudFront in front of S3 (with origin access control) or the ALBEdge locations serve cached copies until the TTL expires. Only misses reach the origin
"origin must keep serving when the primary origin returns errors", "failover without changing the URL"CloudFront origin group: a primary and a secondary origin, with failover status codesFails over on status codes you choose. Applies to GET, HEAD and OPTIONS requests only
"same database queries repeated thousands of times", "read-heavy", "microsecond latency"ElastiCache with lazy loading (cache-aside) and a TTLCaches only requested data, survives a cache node failure, and the TTL limits staleness
"cached data must never be stale after an update"Write-through caching, plus a TTLEvery write updates the cache, so reads are fresh. The TTL clears unread entries
"identical API responses for many callers", REST APIAPI Gateway stage cacheCaches GET responses for 300 seconds by default, up to 3,600, cutting backend calls
"reporting queries slow down the transactional database"A read replica that serves the reporting queriesAsynchronous copies take read load off the primary. More in 3.3
"updated CSS and JavaScript must reach users immediately after each release"Versioned file names (or an invalidation)A new name is a new cache key. An invalidation removes copies before they expire
"non-HTTP traffic", "static anycast IPs", "fast regional failover" for global usersAWS Global Accelerator (3.4)Fixed static IPs, routed over the AWS global network to healthy endpoints. No caching

Correct pattern

REFERENCE PATTERN: one ALB, two services, stateless copies. Path rules send /api/* to an API target group (the set of copies serving one service) and everything else to the web target group.

Each group health-checks its own path. An Auto Scaling group or Amazon ECS (Elastic Container Service) service registers copies into it.

ApiRule:
  Type: AWS::ElasticLoadBalancingV2::ListenerRule
  Properties:
    ListenerArn: !Ref HttpsListener
    Priority: 10
    Conditions:
      - Field: path-pattern
        PathPatternConfig:
          Values: ["/api/*"]
    Actions:
      - Type: forward
        TargetGroupArn: !Ref ApiTargetGroup

ApiTargetGroup:
  Type: AWS::ElasticLoadBalancingV2::TargetGroup
  Properties:
    VpcId: !Ref Vpc
    Protocol: HTTP
    Port: 8080
    TargetType: ip # ECS tasks on Fargate register by IP
    HealthCheckPath: /api/health

# The listener's default action forwards everything else to WebTargetGroup.

REFERENCE PATTERN: lazy loading with a TTL. Read from the cache first. On a miss, read the database and write the result back to the cache with an expiry.

get_product(id):
    item = cache.get("product:" + id)
    if item exists:
        return item                       # cache hit: database untouched
    item = db.query("SELECT ... WHERE id = ?", id)
    cache.set("product:" + id, item, ttl = 300 seconds)
    return item

Lazy loading caches only what someone asked for, and it survives an empty or failed cache node. The cost is a slower first read and possibly stale data until the TTL ends.

Write-through updates the cache whenever the database changes, so data is fresh. The cost is that every write happens twice, and the cache fills with items that may never be read. Combining write-through with a TTL gets most of both.

Traps

TRAPExam traps
The trapWhy it failsCorrect pattern
Adding instances behind a load balancer while sessions live in each instance's memoryUsers bounce between copies and lose sessions, and scale-in deletes themMove sessions to ElastiCache or DynamoDB. Sticky sessions are a stopgap, not the design
Network Load Balancer to route /orders and /users to different servicesNLB feels like the faster upgrade. It works at layer 4 and can't see HTTP pathsApplication Load Balancer with path-pattern listener rules
Application Load Balancer when partners need fixed IPs in their firewallALB is the default web answer, but it is reached only by DNS nameNetwork Load Balancer with Elastic IPs (or Global Accelerator static anycast IPs, taught in 3.4)
"Make it highly available" answered with a bigger instanceIt adds capacity, but it's still one instance in one AZ: a single point of failureCopies across AZs behind a load balancer (2.2)
Read replica to fix a write bottleneck or to provide automatic failoverSounds like more capacity. Replicas serve only reads, writes still hit the primary, and lag is normalScale the primary or redesign writes (3.3). Use Multi-AZ for failover (2.2)
Caching per-user, rapidly changing responses at CloudFront to cut origin loadAny cache sounds like a win. Responses unique per user or per second rarely get a cache hitCache what repeats across users. Send personalized paths straight to the origin with caching disabled
Minimum TTL above 0 on a behavior whose origin sends no-cacheSeems a safe floor. CloudFront then caches for the minimum TTL even though the origin said not toLeave minimum TTL at 0 where the origin controls freshness with Cache-Control

Compare

Compare
AspectApplication Load BalancerNetwork Load Balancer
OSI layer7 (HTTP, HTTPS, gRPC)4 (TCP, UDP, TLS)
Routing decisionsPath, host, headers, query strings, methodsProtocol and port only
TargetsInstances, IP addresses, Lambda functionsInstances, IP addresses, an ALB
Fixed IP addressesNo; reached by DNS nameOne static IP per Availability Zone; Elastic IPs if internet-facing
Cross-zone load balancingAlways on; can be turned off per target groupOff by default
Stem keywords"path-based", "microservices", "HTTP headers", "Lambda targets""TCP/UDP", "static IP", "millions of requests", "low latency"
Compare
AspectCloudFrontAPI Gateway cacheElastiCacheRead replica
Where it sitsEdge locations near usersIn front of the API's backend integrationBeside the application, inside the VPCBeside the primary database
What it cachesWhole HTTP responses: files, pages, API GETsREST API method responses (GET by default)Anything the code puts there: query results, sessionsA full, read-only copy of the database
Who decides freshnessTTLs and origin Cache-Control (default TTL 24 hours unless the cache policy sets another)Stage TTL (default 300 s, max 3,600 s)Your code: TTL per key, lazy loading or write-throughAsynchronous replication lag
Code changes neededNone for cacheable contentNoneYes: the application reads and writes the cacheYes: send reads to the replica endpoint

Worked exam scenario

Exam scenario

A news website runs on Amazon EC2 instances in an Auto Scaling group behind an Application Load Balancer. Article pages and images are the same for every reader. Readers are spread across several continents. When a major story breaks, traffic rises sharply, the instances hit 100% CPU before new ones finish launching, and readers far from the AWS Region report slow page loads. Which solution will MOST improve performance for readers while reducing load on the instances?

  1. A.Change the instance type in the launch template to a larger size.
  2. B.Create an Amazon RDS read replica and point the article queries at it.
  3. C.Put an Amazon CloudFront distribution in front of the Application Load Balancer with a cache policy that caches article pages and images.
  4. D.Put AWS Global Accelerator in front of the Application Load Balancer.
Show the walkthrough
  1. Find what repeats. Pages and images are identical for every reader, so they can be cached. Readers are global, so the cache belongs at the edge.
  2. Eliminate A. Larger instances raise each copy's ceiling, but every request still reaches the origin. Distant readers still wait on a round trip across the world.
  3. Eliminate B. Nothing in the stem points to the database. The instances' CPU is the bottleneck, and a replica doesn't help distant readers.
  4. Eliminate D. Global Accelerator speeds the network path to the ALB, but it doesn't cache. Every request still lands on the instances.
  5. Choose C. CloudFront answers repeated requests from edge locations near readers. That cuts latency and absorbs the spike before it reaches the Auto Scaling group.

Answer: C

Check your understanding

Question 1: A compliance rule says TLS traffic must be decrypted only on the application instances, never on the load balancer. Which option fits?
Question 2: A stateless web tier runs on one large EC2 instance. Which change makes it both highly available and able to grow with demand?
Question 3: Which TWO statements about read replicas are correct? (Choose TWO.)
Question 4: After each release, users keep getting old JavaScript from CloudFront for up to a day. What is the cleanest fix?

Build it

The lab for task 2.1 is the Loose coupling lab. It builds the queue-and-topic layer from 2.1a.

Its queues let a worker tier like the one in this lesson scale out on backlog instead of on request rate. Treat it as the back half of the stateless-copies pattern.

Key takeaways

  • Scale out before you scale up. Identical stateless copies behind a load balancer remove the ceiling and the single point of failure. Scale up only software that can't be distributed.
  • Use an ALB for HTTP routing by path, host or header. Use an NLB for TCP or UDP, extreme throughput and fixed IP addresses.
  • Cache at the layer closest to the user that the data allows: CloudFront for shared content, the API Gateway cache for repeated API responses, ElastiCache for hot data and sessions.
  • Lazy loading tolerates cache failures and some staleness. Write-through keeps data fresh at the cost of extra writes. A TTL balances both.
  • Read replicas scale reads, not writes or failover.
  • Scale out stateless copies behind the right load balancer, then cache whatever repeats: CloudFront for shared content, an in-memory cache for hot data, read replicas for read-heavy databases.

Hands-on lab: Fan Out Orders with SNS, SQS and a Dead-Letter Queue

Sources

Last reviewed