Skip to content
Certification Guide

Lab · Task 3.1 · about 50 minutes

Block Versus Shared File Storage

Services in this lab

  • Amazon EBSin scope
  • Amazon EFSin scope
  • Amazon EC2in scope
  • AWS Systems Managerin scope
  • IAMin scope

Lesson for this lab: High-Performing and Scalable Storage

Goal

Attach a gp3 volume to one instance, watch an attach across Availability Zones fail, raise the volume's IOPS while it stays mounted, and share one EFS file system between instances in two Availability Zones.

Before you start

  • An AWS account you can experiment in, ideally a sandbox account with a budget alert. Don't use the root user; sign in with an IAM Identity Center user or an IAM role that can manage EC2, EBS, EFS and IAM roles.
  • AWS CLI v2 configured for that identity, and one Region chosen for the whole lab (for example us-east-1). Every command below uses the CLI's default Region.
  • The Session Manager plugin for the AWS CLI, so aws ssm start-session works from your terminal. You can also connect from the console with EC2 > Instances > Connect > Session Manager.
  • The Region's default VPC, which has a public subnet in each Availability Zone. You will connect to instances with Systems Manager Session Manager, so no inbound ports are opened.

Resources this lab creates (the clean-up list): an IAM role and instance profile, two security groups, two EC2 instances, one EBS volume, one EFS file system with two mount targets.

Steps

0 of 9 steps done

  1. Create an instance role that lets Session Manager reach the instances. Amazon Linux 2023 ships with the SSM Agent preinstalled.

    cat > trust.json <<'EOF'
    {"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}
    EOF
    aws iam create-role --role-name lab31-ssm-role --assume-role-policy-document file://trust.json
    aws iam attach-role-policy --role-name lab31-ssm-role \
      --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
    aws iam create-instance-profile --instance-profile-name lab31-ssm-profile
    aws iam add-role-to-instance-profile --instance-profile-name lab31-ssm-profile --role-name lab31-ssm-role
    
  2. One group for the instances (no inbound rules) and one for EFS that allows NFS (TCP 2049) only from the instance group.

    VPC=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true --query 'Vpcs[0].VpcId' --output text)
    EC2_SG=$(aws ec2 create-security-group --group-name lab31-ec2 --description "lab 3.1 instances" --vpc-id $VPC --query GroupId --output text)
    EFS_SG=$(aws ec2 create-security-group --group-name lab31-efs --description "lab 3.1 EFS" --vpc-id $VPC --query GroupId --output text)
    aws ec2 authorize-security-group-ingress --group-id $EFS_SG --protocol tcp --port 2049 --source-group $EC2_SG
    
  3. Pick two default subnets in different Availability Zones and launch a small Amazon Linux 2023 instance in each.

    read SUBNET_A AZ_A <<<$(aws ec2 describe-subnets --filters Name=vpc-id,Values=$VPC Name=default-for-az,Values=true \
      --query 'sort_by(Subnets,&AvailabilityZone)[0].[SubnetId,AvailabilityZone]' --output text)
    read SUBNET_B AZ_B <<<$(aws ec2 describe-subnets --filters Name=vpc-id,Values=$VPC Name=default-for-az,Values=true \
      --query 'sort_by(Subnets,&AvailabilityZone)[1].[SubnetId,AvailabilityZone]' --output text)
    AMI=resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64
    launch() {
      aws ec2 run-instances --image-id $AMI --instance-type t3.micro --subnet-id $1 \
        --security-group-ids $EC2_SG --iam-instance-profile Name=lab31-ssm-profile \
        --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=lab31}]' \
        --query 'Instances[0].InstanceId' --output text
    }
    INST_A=$(launch $SUBNET_A)
    INST_B=$(launch $SUBNET_B)
    aws ec2 wait instance-running --instance-ids $INST_A $INST_B
    

    Console: EC2 > Launch instances, choose Amazon Linux 2023 and t3.micro, pick a subnet in each AZ, the lab31-ec2 security group and the lab31-ssm-profile IAM instance profile. If the launch says the instance profile is invalid, IAM has not finished propagating it; wait a few seconds and run it again.

  4. Create a 20 GiB gp3 volume with the included baseline performance (3,000 IOPS, 125 MiB/s). gp3 allows up to 500 provisioned IOPS per GiB, so 20 GiB leaves room for the 6,000 IOPS you set later. Console: EC2 > Volumes > Create volume, type gp3, in the first instance's Availability Zone.

    VOL=$(aws ec2 create-volume --availability-zone $AZ_A --volume-type gp3 --size 20 \
      --tag-specifications 'ResourceType=volume,Tags=[{Key=Name,Value=lab31}]' --query VolumeId --output text)
    aws ec2 wait volume-available --volume-ids $VOL
    
  5. EXPECTED FAILURE. Try to attach the volume to INST_B, which runs in $AZ_B.

    aws ec2 attach-volume --volume-id $VOL --instance-id $INST_B --device /dev/sdf
    

    You should see an error saying the volume and the instance are in different Availability Zones. This is the trap from the lesson: an EBS volume is a disk for instances in its own AZ, not shared storage for a fleet. To use its data in another AZ you would snapshot it and create a new volume there.

  6. aws ec2 attach-volume --volume-id $VOL --instance-id $INST_A --device /dev/sdf
    

    A new instance takes a minute or two to register with Systems Manager, and start-session fails with TargetNotConnected until it does. Run this until it lists both instance IDs, then start the session:

    aws ssm describe-instance-information --filters Key=InstanceIds,Values=$INST_A,$INST_B \
      --query 'InstanceInformationList[].[InstanceId,PingStatus]' --output text
    aws ssm start-session --target $INST_A
    

    Inside the session on INST_A, find the new disk with lsblk (on Nitro instances it appears as an NVMe device such as nvme1n1), then format and mount it:

    lsblk
    sudo mkfs -t xfs /dev/nvme1n1
    sudo mkdir -p /data && sudo mount /dev/nvme1n1 /data
    echo "written before the change" | sudo tee /data/before.txt
    

    Leave the session open.

  7. From your own terminal, modify the volume with Elastic Volumes (console: EC2 > Volumes > Actions > Modify volume). No detach and no restart are needed.

    aws ec2 modify-volume --volume-id $VOL --iops 6000 --throughput 250
    aws ec2 describe-volumes-modifications --volume-ids $VOL \
      --query 'VolumesModifications[0].[ModificationState,TargetIops,TargetThroughput]'
    

    The state moves from modifying to optimizing to completed. Back in the session on INST_A, run cat /data/before.txt and df -h /data at any point: the file system stays mounted and readable throughout.

  8. FS=$(aws efs create-file-system --performance-mode generalPurpose --throughput-mode elastic \
      --encrypted --tags Key=Name,Value=lab31 --query FileSystemId --output text)
    aws efs describe-file-systems --file-system-id $FS --query 'FileSystems[0].LifeCycleState'
    

    When the state is available, add one mount target per subnet:

    aws efs create-mount-target --file-system-id $FS --subnet-id $SUBNET_A --security-groups $EFS_SG
    aws efs create-mount-target --file-system-id $FS --subnet-id $SUBNET_B --security-groups $EFS_SG
    aws efs describe-mount-targets --file-system-id $FS --query 'MountTargets[].[AvailabilityZoneName,LifeCycleState]'
    

    Wait until both mount targets show available.

  9. On each instance (use aws ssm start-session --target <instance-id>), install the EFS client and mount the file system. Replace <file-system-id> with the value of $FS.

    sudo dnf -y install amazon-efs-utils
    sudo mkdir -p /shared
    sudo mount -t efs -o tls <file-system-id> /shared
    

    On INST_A, write a file:

    echo "hello from $(hostname)" | sudo tee /shared/from-a.txt
    

    On INST_B, in the other Availability Zone, read it with cat /shared/from-a.txt. Both instances see the same files, which no single EBS gp3 volume could give you.

Verify

  • aws ec2 describe-volumes --volume-ids $VOL --query 'Volumes[0].[VolumeType,Iops,Throughput,Attachments[0].InstanceId]' returns gp3, 6000, 250 and the ID of INST_A.
  • In the session on INST_A, cat /data/before.txt prints written before the change.
  • On INST_B, cat /shared/from-a.txt prints the hello line with INST_A's host name.
  • aws efs describe-file-systems --file-system-id $FS --query 'FileSystems[0].[PerformanceMode,ThroughputMode]' returns generalPurpose and elastic.

Clean up

Delete everything in this order. Each step depends on the one before it.

If you are in a new terminal and the shell variables are gone, look the IDs up by the lab31 tag and group names first:

read INST_A INST_B <<<$(aws ec2 describe-instances --filters Name=tag:Name,Values=lab31 Name=instance-state-name,Values=pending,running,stopping,stopped --query 'Reservations[].Instances[].InstanceId' --output text)
VOL=$(aws ec2 describe-volumes --filters Name=tag:Name,Values=lab31 --query 'Volumes[0].VolumeId' --output text)
FS=$(aws efs describe-file-systems --query "FileSystems[?Name=='lab31'].FileSystemId | [0]" --output text)
EFS_SG=$(aws ec2 describe-security-groups --filters Name=group-name,Values=lab31-efs --query 'SecurityGroups[0].GroupId' --output text)
EC2_SG=$(aws ec2 describe-security-groups --filters Name=group-name,Values=lab31-ec2 --query 'SecurityGroups[0].GroupId' --output text)
  1. On both instances, unmount the shares: sudo umount /shared, and on INST_A also sudo umount /data. Exit the sessions.

  2. Detach and delete the EBS volume:

    aws ec2 detach-volume --volume-id $VOL
    aws ec2 wait volume-available --volume-ids $VOL
    aws ec2 delete-volume --volume-id $VOL
    
  3. Delete the EFS mount targets, then the file system:

    for MT in $(aws efs describe-mount-targets --file-system-id $FS --query 'MountTargets[].MountTargetId' --output text); do
      aws efs delete-mount-target --mount-target-id $MT
    done
    until [ -z "$(aws efs describe-mount-targets --file-system-id $FS --query 'MountTargets[].MountTargetId' --output text)" ]; do
      sleep 10
    done
    aws efs delete-file-system --file-system-id $FS
    
  4. Terminate both instances and wait for them: aws ec2 terminate-instances --instance-ids $INST_A $INST_B then aws ec2 wait instance-terminated --instance-ids $INST_A $INST_B.

  5. Delete the security groups (EFS group first, because it references the instance group): aws ec2 delete-security-group --group-id $EFS_SG then aws ec2 delete-security-group --group-id $EC2_SG.

  6. Remove the IAM pieces:

    aws iam remove-role-from-instance-profile --instance-profile-name lab31-ssm-profile --role-name lab31-ssm-role
    aws iam delete-instance-profile --instance-profile-name lab31-ssm-profile
    aws iam detach-role-policy --role-name lab31-ssm-role --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
    aws iam delete-role --role-name lab31-ssm-role
    
  7. Check that nothing billable remains. Each command should return an empty list (terminated instances may still be listed for a short while with state terminated):

    aws ec2 describe-instances --filters Name=tag:Name,Values=lab31 Name=instance-state-name,Values=pending,running,stopping,stopped --query 'Reservations[].Instances[].InstanceId'
    aws ec2 describe-volumes --filters Name=tag:Name,Values=lab31 --query 'Volumes[].VolumeId'
    aws efs describe-file-systems --query "FileSystems[?Name=='lab31'].FileSystemId"
    aws ec2 describe-security-groups --filters Name=group-name,Values=lab31-ec2,lab31-efs --query 'SecurityGroups[].GroupId'
    

    Remove the local trust.json file you created in step 1.

Sources

Last reviewed