Skip to content
Certification Guide

Task 3.2 · 21 min read

High-Performing and Elastic Compute

Services in this lesson

  • Amazon EC2in scope
  • Amazon EC2 Auto Scalingin scope
  • AWS Auto Scalingin scope
  • AWS Batchin scope
  • AWS Lambdain scope
  • Amazon CloudWatchin scope

What the exam tests

  • Choosing the compute shape from how work arrives. The options are long-running servers (Amazon EC2), short event-driven functions (AWS Lambda), or queued jobs that run to completion (AWS Batch). Containers go to Amazon ECS (Elastic Container Service) or Amazon EKS (Elastic Kubernetes Service).
  • Reading the bottleneck in the stem: CPU, memory, GPU, local disk, or network between nodes. Then picking the EC2 instance family that removes it, or sizing a CPU-bound Lambda function's memory.
  • Choosing the Auto Scaling policy and its metric. Target tracking on a per-instance metric is the default. Scheduled or predictive scaling covers known patterns. A warm pool covers slow boots, and queue backlog per instance covers workers.
  • Decoupling tiers with a queue so each one scales on its own signal, and placing tightly coupled nodes close together for low latency. Tightly coupled nodes constantly exchange data with each other while they work.

This lesson owns EC2 instance families, Auto Scaling, and compute placement for performance. Related topics live elsewhere:

The core concept

Fast compute comes from three choices in order, plus placement for tightly coupled nodes. First pick the compute shape: the kind of service that fits how the work arrives. Then pick the instance family from the resource that runs out first, and let Auto Scaling add and remove instances as load changes.

An analogy: a call center

Picture a call center. Most calls are routine, but some need a specialist who speaks another language. The manager hires specialists for the skill that is short, not just more of the same staff.

To decide how many agents to staff, the manager watches calls waiting per agent, not the total in the queue. Twenty calls waiting means panic for two agents and calm for twenty.

Where it breaks: a new agent is useful from the first call, but a new server spends a while booting. Auto Scaling covers this with a set warmup time, during which the new server's usage doesn't count.

Walk one sample through it

The diagram reads top to bottom: the shape of the work, then the family, then the scaling policy. The walkthrough follows one upload through each choice.

Enlarge
Pick the shape, then the family, then the scaling policy that matches how load changes.

Brightwave Labs is a genomics analytics startup. Tobias runs its AWS platform, and Leila is a scientist uploading a DNA sample file for analysis.

  1. Leila opens the web portal. The portal runs all day, so it lives on Amazon EC2 (Elastic Compute Cloud) instances, which are virtual servers.
    • An Application Load Balancer (ALB) spreads her requests across the portal's instances.
  2. The portal runs in an Auto Scaling group. An Auto Scaling group is a fleet built from a launch template: a saved recipe for each instance.
    • The group has a minimum, a maximum and a desired capacity, the number it aims to run now.
    • Its instances spread over subnets in several Availability Zones (AZs), each one or more separate data centers in a Region.
  3. The portal scales on requests per instance. Tobias set a target tracking policy: keep about 1,000 requests per target (here, each instance).
    • Auto Scaling creates the Amazon CloudWatch alarms, rules that fire when a metric crosses a threshold. It then moves the desired capacity to hold the target.
    • Adding instances is scaling out. Removing them is scaling in.
  4. Leila's upload becomes a job on a queue. The portal puts a message on an Amazon SQS (Simple Queue Service) queue. A separate worker tier pulls messages from it.
  5. The workers run out of memory first. Aligning a genome needs far more RAM than CPU. Tobias picks the R family (memory optimized) for the worker instances.
  6. The workers scale on backlog per instance. Their policy tracks messages waiting divided by running workers. When a partner lab uploads a hundred samples, the backlog per worker jumps and new workers launch.
  7. New worker instances boot before they help. A booting worker shows busy CPU, which could trigger extra scale-outs.
    • Default instance warmup is a set time that starts when the instance enters the InService state.
    • Until it ends, the worker's usage data stays out of the group's combined metrics.
  8. Each morning, scientists log in at 09:00. Tobias adds a scheduled action that raises the portal's desired capacity just before that time.
  9. Once a month, Brightwave reprocesses its archive. That is thousands of independent jobs that each run to completion. Tobias hands them to AWS Batch instead of growing the worker tier.

The spike of uploads in step 6 grew the workers without touching the portal. That is the payoff of a queue between tiers: each tier scales on its own signal.

The rules the exam tests

  • Long-running servers, or software moved as is, go on EC2.
  • Short event-driven bursts go on Lambda. Each invocation must finish within 15 minutes.
  • Packaged containers go on ECS or EKS, running on EC2 or on AWS Fargate (serverless container capacity).
  • Jobs that run to completion go on AWS Batch. Simulations and nightly renders are typical, and Batch schedules them on EC2 or Fargate.
  • The first letter of an instance type names the family. C is compute, R and X are memory, P and G are accelerators, I and D are local storage. Accelerators are chips such as GPUs (graphics processing units) that do parallel math.
  • Two more families round it out. M is a balanced mix, and T is burstable CPU.
  • The number after the family is the generation. In m6i.large, 6 is the generation, i means an Intel processor, and large is the size.
  • Letters after the generation number add options. The letter g means Graviton, AWS's own Arm-based processor. The letter n means extra network and Amazon EBS (Elastic Block Store) bandwidth.
  • The letter d means local instance store disks. Instance store is disk on the host itself.
  • Target tracking is the default scaling answer. You name a metric and a value, such as 50 percent average CPU or 1,000 requests per target.
  • Dynamic policies react to live metrics. Target tracking, step scaling and simple scaling are the dynamic policies.
  • The metric must change in proportion to the instance count. Backlog per instance works for queue workers. The raw queue length does not.
  • Match the stem to the tool. A known time needs a scheduled action. A learned daily or weekly pattern needs predictive scaling, which only scales out.
  • Slow boots and load balancers have their own fixes. Slow boots need a warm pool. A group behind a load balancer needs ELB (Elastic Load Balancing) health checks.
  • Three placement strategies. Cluster suits tightly coupled HPC (high performance computing). Partition suits big data stores such as HDFS (Hadoop Distributed File System). Spread suits a few critical instances such as primary and standby.
Deep diveBurstable T instances: standard and unlimited mode

A T instance has a baseline level of CPU. While it runs below that baseline, it earns CPU credits.

It spends credits to burst above the baseline during spikes. In standard mode, the instance is held to its baseline once its credits run out.

Unlimited mode keeps bursting after the credits are gone. It bills surplus credits when average CPU runs above the baseline.

Deep divePredictive scaling

Predictive scaling builds a forecast from up to 14 days of metric history. By default it launches capacity at the start of each forecast hour.

A pre-launch time starts instances earlier, so they are ready when load arrives. In forecast only mode, the policy forecasts but doesn't scale.

Predictive scaling only scales out. A forecast of lower load doesn't terminate instances, so keep a dynamic policy for scale-in.

Deep diveWarm pools

A warm pool holds pre-initialized instances, stopped, running or hibernated. They join the group fast instead of booting from scratch.

Deep diveHealth checks

The default EC2 health check only confirms that the instance is running. ELB health checks also replace instances whose application stops answering.

Deep diveLambda memory and CPU

Lambda has no instance size to choose. You set memory from 128 MB to 10,240 MB, and CPU is allocated in proportion to it.

At about 1.8 GB of memory, a function gets one full vCPU (virtual CPU). Raising memory is the fix for a slow, CPU-bound function.

More memory never lifts the 15-minute maximum per invocation. Long jobs belong on AWS Batch, Fargate or EC2.

The other Lambda limits are taught in 2.1b Serverless and containers.

Deep diveAWS Auto Scaling scaling plans

AWS Auto Scaling is a separate service from EC2 Auto Scaling. A scaling plan configures scaling for several resource types in one application from one place.

Those resources include Auto Scaling groups, ECS services, DynamoDB tables and Aurora replicas. For EC2 alone, scaling policies on the group are enough.

The walkthrough kept every instance where Auto Scaling put it. Some stems also care about which physical hardware the instances share.

Where the instances sit: placement groups

A placement group tells EC2 how to place a set of instances on the underlying hardware. There are three strategies.

  • Cluster packs instances close together in one AZ. The segment has high bisection bandwidth, meaning fast links between every node. It gives the lowest latency for tightly coupled HPC.
  • Partition splits instances into up to seven partitions per AZ, each on its own racks. A rack failure stays inside one partition.
  • Spread puts each instance on distinct hardware. A group allows at most seven running instances per AZ.
Deep divePlacement group details

A cluster placement group can't span AZs, and its size is limited by capacity in that AZ. Not every instance type is supported in one: t3.micro, for example, is not.

The recommended setup is one current-generation instance type, launched in a single request. Add enhanced networking or an Elastic Fabric Adapter (EFA).

EFA adds low-latency traffic between instances for MPI (Message Passing Interface) and NCCL (NVIDIA Collective Communications Library). Both are libraries that HPC and machine learning (ML) training jobs use to talk across nodes.

A partition group can span several AZs and is limited only by account quotas. The application can read which partition each instance is in.

Partition groups suit HDFS (Hadoop Distributed File System), HBase, Cassandra and Kafka. Spread groups suit primary and standby servers, or quorum nodes.

Decision guide

Terms used in the table below:

  • Instance store is disk physically attached to the host. Its data doesn't survive an instance stop.
  • Metric math combines CloudWatch metrics with an expression, such as one metric divided by another.
  • Amazon CloudWatch is the AWS monitoring service. It holds the metrics and alarms that scaling policies use.
Decision guide
If the question says…Reach for…Because…
Batch encoding, scientific modeling or a game server is limited by CPUCompute optimized (C family)The highest ratio of vCPU to memory, for CPU-bound work
An in-memory cache, real-time analytics or a large database runs out of RAMMemory optimized (R family, or X for very large memory)More memory per vCPU. Adding CPU won't help a memory-bound workload
Model training, inference or graphics renderingAccelerated computing (P or G family)GPUs do parallel math a CPU would take far longer to finish
Very high random I/O on local disks for a NoSQL store or data warehouseStorage optimized (I or D family)Large, fast instance store. Its data is lost when the instance stops, so replicate
Low average CPU with short spikes, such as a small web server or a dev boxBurstable (T family)Baseline CPU plus credits to burst. Standard mode throttles to baseline when credits run out
Web tier load rises and falls through the dayTarget tracking on average CPU or ALBRequestCountPerTargetAuto Scaling holds the metric near the target and manages the alarms
Workers pull from an Amazon SQS queue and must keep up with itTarget tracking on backlog per instance (metric math)Backlog per worker changes in proportion to worker count. The raw total doesn't
Traffic jumps every weekday at 09:00 and the stem names the timeScheduled scaling actionRaises the desired capacity at the known time, before the load arrives
A daily or weekly pattern, and instances take minutes to initializePredictive scaling, with dynamic scaling for the restLaunches capacity ahead of the forecast. It only scales out, so keep a dynamic policy
Instances need many minutes to boot before they can serve, and spikes are unpredictableWarm pool on the Auto Scaling groupPre-initialized instances join fast instead of booting from scratch
New instances report high CPU while booting and cause extra scale-outsDefault instance warmup on the groupInstances don't count toward the group's metrics until the warmup ends
Tightly coupled HPC nodes exchange lots of traffic and need the lowest latencyCluster placement group in one AZ, enhanced networking, EFANodes share one high-bandwidth segment. EFA speeds MPI and NCCL traffic
HDFS, HBase or Cassandra nodes must not share racksPartition placement groupUp to seven partitions per AZ, each on its own racks
A handful of critical instances must never fail togetherSpread placement groupEach instance on distinct hardware, at most seven running per AZ per group
Thousands of independent jobs are submitted, queued and run to completionAWS BatchManaged queues, scheduling and compute on EC2 or Fargate. No batch software to run
A Lambda function is CPU-bound and slowRaise the function's memory settingCPU grows with memory. About 1.8 GB gives one full vCPU
One plan must scale an Auto Scaling group, ECS services, DynamoDB tables and Aurora replicasAWS Auto Scaling scaling planOne place to scale several resource types. For EC2 alone, group policies suffice

Correct pattern

REFERENCE PATTERN: a web tier and a worker tier that scale independently, each on the metric that matches its own load.

# 1. Web tier: an Auto Scaling group across three AZs behind an ALB.
#    Default instance warmup keeps booting instances out of the metrics.
aws autoscaling create-auto-scaling-group \
  --auto-scaling-group-name web-asg \
  --launch-template LaunchTemplateName=web-lt,Version='$Latest' \
  --min-size 2 --max-size 12 --desired-capacity 2 \
  --vpc-zone-identifier "<subnet-a>,<subnet-b>,<subnet-c>" \
  --target-group-arns <web-target-group-arn> \
  --health-check-type ELB --health-check-grace-period 120 \
  --default-instance-warmup 120

# Target tracking on requests per target: the metric falls as instances are added.
aws autoscaling put-scaling-policy --auto-scaling-group-name web-asg \
  --policy-name web-requests --policy-type TargetTrackingScaling \
  --target-tracking-configuration '{
    "TargetValue": 1000,
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ALBRequestCountPerTarget",
      "ResourceLabel": "<alb-resource-label>/<target-group-resource-label>"
    }}'
{
  "TargetValue": 100,
  "CustomizedMetricSpecification": {
    "Metrics": [
      {
        "Id": "m1",
        "ReturnData": false,
        "MetricStat": {
          "Metric": {
            "Namespace": "AWS/SQS",
            "MetricName": "ApproximateNumberOfMessagesVisible",
            "Dimensions": [{ "Name": "QueueName", "Value": "<jobs-queue>" }]
          },
          "Stat": "Sum"
        }
      },
      {
        "Id": "m2",
        "ReturnData": false,
        "MetricStat": {
          "Metric": {
            "Namespace": "AWS/AutoScaling",
            "MetricName": "GroupInServiceInstances",
            "Dimensions": [
              { "Name": "AutoScalingGroupName", "Value": "worker-asg" }
            ]
          },
          "Stat": "Average"
        }
      },
      { "Id": "e1", "Expression": "m1 / m2", "ReturnData": true }
    ]
  }
}

The JSON above is the worker tier's target tracking configuration. You pass it with put-scaling-policy --target-tracking-configuration file://worker.json.

The target of 100 is the delay the business accepts divided by the time one message takes. At 10 seconds of acceptable delay and 0.1 seconds per message, each worker can hold 100 messages.

Why it works: the web tier scales on requests per instance, and the worker tier on work waiting per worker. A spike of uploads grows the queue and the workers without touching the web tier. ELB health checks replace instances that are running but no longer answer.

Traps

TRAPExam traps
The trapWhy it failsCorrect pattern
Scaling queue workers on ApproximateNumberOfMessagesVisible aloneThe total ignores how many workers exist, so target tracking over- or under-provisions while the queue drainsDivide the backlog by GroupInServiceInstances with metric math and track backlog per instance.
Scaling on CPU when the stem says memory is the bottleneckEC2 doesn't publish memory by default. CPU can stay low while instances run out of RAMPublish memory with the CloudWatch agent and scale on it, or move to a memory optimized family.
Picking simple scaling with a long cooldown for spiky trafficSimple scaling waits out the cooldown (300 seconds by default) before reacting againTarget tracking, or step scaling when the response must grow with the size of the breach.
Expecting predictive scaling to also remove capacityPredictive scaling only scales out. Lower forecasts don't terminate instancesPair predictive scaling with a dynamic (target tracking) policy that handles scale-in.
Leaving the default EC2 health check behind a load balancerIt only checks the instance is running, so a hung application stays in serviceTurn on ELB health checks for the group, with a grace period that covers boot time.
Running sustained, heavy CPU on a T instance in standard modeOnce credits are spent, it's held at baseline, just when load is highestA C or M instance for steady high CPU. Keep T for low average with occasional bursts.
Spreading a cluster placement group across AZs, or filling it with T instancesA cluster group lives in one AZ, and some types aren't supported (t3.micro, for example)One AZ, one current-generation instance type launched in a single request, with enhanced networking or EFA.
Using a bigger instance to fix a function that hits Lambda's 15-minute limitLambda has no instance size. Memory raises CPU, but the 15-minute maximum staysSplit the work, or run long jobs on AWS Batch, Fargate or EC2.

The CloudWatch agent is software you install on an instance to collect extra metrics, such as memory use. A cooldown is a pause after a scaling activity before a simple scaling policy can act again.

Compare

Compare
AspectTarget trackingStep scalingScheduled actionPredictive scaling
TriggerA metric drifts from your target valueA CloudWatch alarm you create, with breach stepsA date, time or recurring scheduleA forecast built from up to 14 days of metric history
You manage the alarmsNo, Auto Scaling creates themYesNo alarmsNo alarms
Scales inYes, graduallyYes, with a separate policyYes, to the capacity you setNo, pair it with a dynamic policy
Reacts or anticipatesReactsReactsAnticipates a time you knowAnticipates a pattern it learns
Reach for it whenThe default for most web and worker tiersBigger breaches need bigger jumpsThe stem names the time of the peakDaily or weekly cycles plus slow-starting instances
Compare
AspectCluster placement groupPartition placement groupSpread placement group
GoalLowest latency, highest throughputContain hardware failures to one partitionKeep each critical instance on its own hardware
ScopeOne Availability ZoneSeveral AZs, up to seven partitions per AZSeveral AZs, up to seven running instances per AZ
SizeLimited by capacity in that AZLimited only by account quotasSmall groups of critical instances
Typical workloadHPC, MPI, ML training across nodesHDFS, HBase, Cassandra, KafkaPrimary and standby servers, quorum nodes

Worked exam scenario

Exam scenario

A company runs an internal expense-reporting application on two m6i.large instances behind an Application Load Balancer. Over the past month, average CPU utilization has been about 6 percent, with short spikes to about 60 percent for a few minutes when employees submit reports at the end of each day. Memory use is low and steady. The company wants to reduce compute cost without slowing down the end-of-day spikes. Which change is MOST cost-effective?

  1. A.Move to c6i.large compute optimized instances so the spikes finish faster.
  2. B.Move to t3.large burstable instances that build CPU credits between spikes.
  3. C.Move to r6i.large memory optimized instances and run only one of them.
  4. D.Keep m6i.large and add target tracking on average CPU at 50 percent.
Show the walkthrough
  1. Mark the requirements: low average CPU, short spikes a few minutes long, low memory, and lower cost without slower spikes.
  2. Name the pattern: a low baseline with occasional bursts is what the T family is built for. Below its baseline the instance earns CPU credits, and it spends them during the spikes.
  3. A buys more CPU per instance than a workload averaging 6 percent ever uses. It doesn't cut cost for this pattern. Eliminate.
  4. C pays for memory the application doesn't need. Dropping to one instance also removes the second instance the tier relies on. Eliminate.
  5. D keeps the same always-on instances and only adds a third during spikes, so cost can only go up. Eliminate.
  6. B matches the instance to the load shape: credits earned during the quiet day cover the short end-of-day bursts. Answer: B.

Answer: B

Check your understanding

Question 1: A self-managed Redis cache on EC2 is evicting keys because the instance runs out of memory. CPU averages 15 percent. Which change fits best?
Question 2: A predictive scaling policy in forecast and scale mode adds capacity at the start of each forecast hour, but new instances need about 10 minutes to boot, so the first minutes of each morning peak are slow. Which setting fixes this?
Question 3: Which TWO statements about EC2 placement groups are true? (Choose TWO.) (Choose TWO.)
Question 4: A Lambda function that resizes images at 512 MB is CPU-bound and takes 8 seconds per image. What should you try first?

Build it

Do the elastic compute lab. You build an Auto Scaling group with a target tracking policy, load its instances and watch it scale out to its maximum. Then you see a cluster placement group refuse a burstable instance.

Key takeaways

  • The shape of the work picks the service: EC2 for servers, Lambda for short events, ECS or EKS for containers, AWS Batch for queued jobs.
  • The resource that runs out first picks the EC2 family. C is for CPU, R or X for memory, P or G for accelerators, I or D for local disk, M or T for balanced.
  • Target tracking is the default policy. Scheduled actions cover known times, predictive scaling covers learned patterns (scale-out only), and warm pools and instance warmup cover slow boots.
  • Scale queue workers on backlog per instance, and turn on ELB health checks for any group behind a load balancer.
  • Cluster placement for low-latency node-to-node traffic, partition for rack-aware big data, spread for a few critical instances.
  • Pick the compute shape from how work arrives and the instance family from what runs out first, then scale on a per-instance metric, not a raw total.

Hands-on lab: Target Tracking and Placement Limits

Sources

Last reviewed