I package a few things here and there in Fedora and one of my latest packages is efs-utils. AWS offers a mount helper for their Elastic File System (EFS) product on GitHub.

In this post, I’ll explain how to:

1. Launch a Fedora instance on AWS EC2
2. Install efs-utils and launch the watchdog service
3. Create an EFS volume in the AWS console
4. Mount the EFS volume inside the Fedora instance

Let’s go! 🚀

Wait, what is EFS? #

When you launch a cloud instance (virtual machine) on most clouds, you have different storage options available to you:

• Block storage: You can add partitions to this storage, create filesystems, or even use LVM. It looks like someone plugged in a disk to your instance. You get full control over every single storage block on the volume. An example of this is Elastic Block Storage (EBS) on AWS.

• Object storage: Although you can’t mount object storage (typically) within your instance, you can read/write objects to this storage via an API. You can upload nearly any type of file you can imagine as an object and then download it later. Objects can also have little bits of metadata attached to them and some of the metadata include prefixes which give a folder-like experience. AWS S3 is a good example of this.

• Shared filesystems: This storage shows up in the instance exactly as it sounds: you get a shared filesystem. If you’re familiar with NFS or Samba (SMB), then you’ve used shared filesystems already. They give you much better performance than object storage but offer less freedom than block storage. They’re also great for sharing the same data between multiple instances.

Using EFS is almost like having someone else host a network accessible storage (NAS) device within your cloud deployment.

Launching Fedora #

Every image in AWS has an AMI ID attached to it and you need to know the ID for the image you want in your region. You can find these quickly for Fedora by visiting the Fedora Cloud download page. Look for AWS in the list, click the button on that row, and you’ll see a list of Fedora AMI IDs. Click the rocket (🚀) for your preferred region and you’re linked directly to launch that instance in AWS!

I’m clicking the launch link for us-east-2 (Ohio)¹. To finish quickly, I’m choosing all of the default options and using a spot instance (look inside Advanced details at the bottom of the page).

Wait for the instance to finish intializing and access it via ssh:

$ ssh fedora@EXTERNAL_IP
[fedora@ip-172-31-2-38 ~]$ cat /etc/fedora-release 
Fedora release 38 (Thirty Eight)

Success! 🎉

Prepare your security group #

Before leaving the EC2 console, you need to make a note of the security group that you used for this instance. That’s because EFS uses security groups to guard access to volumes. Follow these steps to find it:

1. Click Instances on the left side of the EC2 console.
2. Click on the row showing the instance we just created.
3. In the bottom half of the screen, click the Security tab.
4. Look for Security groups in the security details and copy the security group ID for later.

It should be in the format sg-[a-f0-9]*.

If you click the security group name (after saving it), you’ll see the inbound rules associated with that security group. By default, items in the same security group can’t talk to each other. We need to allow that so our EFS mount will work later.

Click Edit inbound rules and do the following:

1. Click Add rule.
2. Choose All traffic in the Type column. (You can narrow this down further later.)
3. In the source box, look for the security group you just created along with your EC2 instance. If you took the default during the EC2 launch process, it might be named launch-wizard-[0-9]+.
4. Click Save rules.

Installing efs-utils #

Let’s start by getting the efs-utils package onto our new Fedora system:

$ sudo dnf -qy install efs-utils
Installed:
  efs-utils-1.35.0-2.fc38.noarch

The package includes some configuration, a watchdog, and a mount helper:

$ rpm -ql efs-utils
/etc/amazon
/etc/amazon/efs
/etc/amazon/efs/efs-utils.conf
/etc/amazon/efs/efs-utils.crt
/usr/bin/amazon-efs-mount-watchdog
/usr/lib/systemd/system/amazon-efs-mount-watchdog.service
/usr/sbin/mount.efs
/usr/share/doc/efs-utils
/usr/share/doc/efs-utils/CONTRIBUTING.md
/usr/share/doc/efs-utils/README.md
/usr/share/licenses/efs-utils
/usr/share/licenses/efs-utils/LICENSE
/usr/share/man/man8/mount.efs.8.gz
/var/log/amazon/efs

Let’s get the watchdog running so we have that ready later. The watchdog helps to build and tear down the encrypted connection when you mount and unmount an EFS volume:

$ sudo systemctl enable --now amazon-efs-mount-watchdog.service
Created symlink /etc/systemd/system/multi-user.target.wants/amazon-efs-mount-watchdog.service → /usr/lib/systemd/system/amazon-efs-mount-watchdog.service.
$ systemctl status amazon-efs-mount-watchdog.service
● amazon-efs-mount-watchdog.service - amazon-efs-mount-watchdog
     Loaded: loaded (/usr/lib/systemd/system/amazon-efs-mount-watchdog.service; enabled; preset: disabled)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
     Active: active (running) since Wed 2023-09-13 18:43:46 UTC; 5s ago
   Main PID: 1258 (amazon-efs-moun)
      Tasks: 1 (limit: 4385)
     Memory: 13.3M
        CPU: 76ms
     CGroup: /system.slice/amazon-efs-mount-watchdog.service
             └─1258 /usr/bin/python3 /usr/bin/amazon-efs-mount-watchdog

Sep 13 18:43:46 ip-172-31-2-38.us-east-2.compute.internal systemd[1]: Started amazon-efs-mount-watchdog.service - amazon-efs-mount-watchdog.

Setting up an EFS volume #

Start by going over to the EFS console and do the following:

1. Click File systems in the left navigation bar

2. Click the orange Create file system button at the top right

   A modal appears with a box for the volume name and a VPC selection. Select an easy to remember name (I’m using testing-efs-for-blog-post) and select a VPC. If you’re not sure what a VPC is or which one to use, use the default VPC since that’s likely where your instance landed as well.

3. Click Create.

There’s a delay while the filesystem initializes and you should see the filesystem show Available with a green check mark after about 30 seconds. Click on the filesystem you just created from the list and you’ll see the details page for the filesystem.

Security setup #

EFS volumes come online with the default security group attached and that’s not helpful. From the EFS filesystem details page, click the Network tab and then click Manage.

For each availability zone, go to the Security groups column and add the security group that your instance came up with in the first step. In my case, I accepted the defaults from EC2 and ended up with a launch-wizard-1 security group. Remove the default security group from each. Click Save.

Mounting time #

You should still be on the filesystem details page from the previous step. Click Attach at the top right and a modal will appear with mount instructions. The first option should use the EFS mount helper!

For me, it looks like sudo mount -t efs -o tls fs-0baabc62763375bb1:/ efs

Go back to your Fedora instance, create a mount point, and create the volume:

$ sudo mkdir /mnt/efs
$ sudo mount -t efs -o tls fs-0baabc62763375bb1:/ /mnt/efs
$ df -hT | grep efs
127.0.0.1:/    nfs4      8.0E     0  8.0E   0% /mnt/efs

We did it! 🎉

We see 127.0.0.1 here because efs-utils uses stunnel to handle the encryption between your instance and the EFS storage system.

The disk was mounted by root, so we can add a -o user=fedora to give our Fedora user permissions to write files:

$ umount /mnt/efs
$ sudo mount -t efs -o user=fedora,tls fs-0baabc62763375bb1:/ /mnt/efs
$ touch /mnt/efs/test2.txt
$ stat /mnt/efs/test2.txt
  File: /mnt/efs/test2.txt
  Size: 0         	Blocks: 8          IO Block: 1048576 regular empty file
Device: 0,54	Inode: 17657675890899444015  Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/  fedora)   Gid: ( 1000/  fedora)
Context: system_u:object_r:nfs_t:s0
Access: 2023-09-13 19:14:23.308000000 +0000
Modify: 2023-09-13 19:14:23.308000000 +0000
Change: 2023-09-13 19:14:23.308000000 +0000
 Birth: -

Also, efs-utils uses encrypted communication by default, which is great. There may be some situations where you don’t need encrypted communications or you don’t want the overhead. In that case, drop the -o tls option from the mount command and you’ll mount the volume unencrypted.

$ sudo umount /mnt/efs
$ sudo mount -t efs -o user=fedora fs-0baabc62763375bb1:/ /mnt/efs
$ df -hT | grep efs
fs-0baabc62763375bb1.efs.us-east-2.amazonaws.com:/ nfs4      8.0E     0  8.0E   0% /mnt/efs

Extra credit #

You can get fancy with access points that allow you to carve up your EFS storage and only let certain instances mount certain parts of the filesystem. So instance A might only be able to mount /files/hr while instance B can only mount /documents.

It would also be a good idea to take an inventory of your security groups and ensure the least amount of instances can reach your EFS volume as possible. Much of the work I did in this post was just for testing. A good plan might be to make a security group for your EFS volume and only allow inbound traffic from security groups which should access it. That would allow you to gather up all of your instances into different security groups and limit access.

Also, be aware of the EFS pricing! 💸

You are billed not only for how much storage you use, but also on requests. Different requests are priced differently depending on access frequency. Backups are also enabled by default at $0.05/GB-month!