Mounting an AWS S3 Bucket on Amazon EC2 Linux Using S3FS

Introduction

In this detailed guide, we'll walk you through the process of mounting an AWS S3 bucket on an Amazon EC2 Linux instance using S3FS. This hands-on project will help you understand key AWS services such as S3, EC2, and S3FS, along with using the AWS CLI. Let's dive in!

What You'll Learn

Creating an IAM user with the necessary permissions
Launching and connecting to an EC2 instance
Installing S3FS on your EC2 instance
Configuring S3FS to mount your S3 bucket
Automating the mount process

Step-by-Step Solution

Step 1 : Create an IAM User

First, you need an IAM (Identity and Access Management) user to manage access to your AWS resources. Here’s how you do it:

Log in to the AWS Management Console:
- Open your web browser and go to the AWS Management Console.
Navigate to the IAM Dashboard:
- Click on "Services" at the top left.
- Under "Security, Identity, & Compliance," select "IAM."
Create a New User:
- Click "Users" in the left sidebar.
- Click "Add user."
- Enter a username (e.g., s3fs-user).
- Select "Programmatic access" to allow API access.
Set Permissions:
- Click "Next: Permissions."
- Select "Attach existing policies directly."
- Search for and select AmazonS3FullAccess and AmazonEC2FullAccess.
- Click "Next: Tags" (optional), then "Next: Review," and finally "Create user."
Download Credentials:
- Download the .csv file containing your access key ID and secret access key. This file is essential for accessing AWS services programmatically.

Step 2 : Set Up Your EC2 Instance

Launch an EC2 Instance:
- Go to the EC2 Dashboard.
- Click "Launch Instance."
- Choose an Amazon Machine Image (AMI), such as Amazon Linux 2.
- Select an instance type (e.g., t2.micro), which is part of the free tier.
- Configure instance details and add storage as needed.
- Add tags (optional) to identify your instance.
- Configure security groups to allow SSH (port 22) and HTTP (port 80) access.
- Review and launch your instance.
- Download the key pair (.pem file) that you'll use to connect to your instance.
Connect to Your EC2 Instance:
- Open a terminal on your local machine.
- Use the SSH command to connect:
```
  ssh -i /path/to/your-key.pem ec2-user@your-ec2-public-ip
```

Step 3: Install S3FS on Your EC2 Instance

Update the Package List:
- Once connected to your EC2 instance, update the package list:
```
  sudo yum update -y
```

Install Dependencies:

Install the necessary dependencies for S3FS:

  sudo yum install -y git gcc automake autoconf fuse fuse-devel libcurl-devel libxml2-devel openssl-devel mailcap

Install S3FS:

Clone the S3FS GitHub repository:

  git clone https://github.com/s3fs-fuse/s3fs-fuse.git

Change to the S3FS directory and compile the package:

  cd s3fs-fuse
  ./autogen.sh
  ./configure
  make
  sudo make install

Step 4 : Configure S3FS

Create a Credentials File:

Store your AWS credentials in a file:

  echo "your-access-key-id:your-secret-access-key" > ~/.passwd-s3fs
  chmod 600 ~/.passwd-s3fs

Mount the S3 Bucket:

Create a directory to mount the S3 bucket:
```
  sudo mkdir /mnt/s3bucket
```

Mount the S3 bucket using S3FS:

  s3fs your-bucket-name /mnt/s3bucket -o passwd_file=~/.passwd-s3fs

Step 5 : Verify the Mount

Check the Mounted Directory:
- List the contents of the mounted directory to verify:
```
  ls /mnt/s3bucket
```
- If the mount is successful, you should see the contents of your S3 bucket.

Step 6 : Automate the Mount (Optional)

Edit the fstab File:

To ensure the S3 bucket is mounted automatically at boot, add an entry to the /etc/fstab file:

  echo "s3fs#your-bucket-name /mnt/s3bucket fuse _netdev,passwd_file=/home/ec2-user/.passwd-s3fs 0 0" | sudo tee -a /etc/fstab

Conclusion

Congratulations! You've successfully mounted an AWS S3 bucket on an Amazon EC2 Linux instance using S3FS. This setup allows you to treat your S3 bucket as a local filesystem, making it easier to manage files and data.