s3fs allows you to mount an Amazon S3 bucket as a file system so you can access it like any other file.
Because there are no official packages available, you will need to compile s3fs yourself. Luckily this is very easy.
Installing the build dependencies
If you are on Ubuntu:
sudo apt-get install automake autotools-dev g++ git libcurl4-gnutls-dev libfuse-dev libssl-dev libxml2-dev make pkg-config
If you are running CentOS:
sudo yum install automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel
If you are running another operating system, please refer to the s3fs wiki.
Build the s3fs source code and install it
cd ~ git clone https://github.com/s3fs-fuse/s3fs-fuse.git cd s3fs-fuse ./autogen.sh ./configure make sudo make install
Set up user permissions
If you need a read-only set up, you can skip this step. If you need write permission for PHP, you have to add the correct user to the fuse user group. On most systems, this is either www-data or apache. If you are unsure which user to assign, consult with your system administrator.
To add the www-data user to the group, simply run:
sudo addgroup www-data fuse
Mounting your S3 bucket
Store your login credentials where s3fs can find them (make sure to replace with your correct ID and access key):
echo "<AWS Access Key ID>:<AWS Secret Access Key>" | sudo tee /etc/passwd-s3fs sudo chmod 600 /etc/passwd-s3fs
Mount S3 bucket
Now mount the bucket:
sudo mkdir /mnt/mys3bucket /var/cache/s3cache sudo s3fs -o use_cache=/var/cache/s3cache,allow_other,nonempty joomlatools-s3fuse-test /mnt/mys3bucket
Test it by creating a file:
echo "Test" | sudo tee /mnt/mys3bucket/test.txt
Verify if the file shows up on your Amazon S3 bucket through the Amazon web console.
If you get errors or the file is not being synced, take a look at your system log (on Ubuntu:
/var/log/syslog) to see what s3fuse is saying.
Mount S3 bucket after reboot
Add an entry to the
joomlatools-s3fuse-test /mnt/mys3bucket fuse.s3fs _netdev,uid=33,gid=33,umask=007,use_cache=/var/cache/s3cache,allow_other,nonempty,retries=5 0 0
Move your files to the S3 bucket
joomlatools-files directory on S3 and give it the necessary permissions:
mkdir /mnt/mys3bucket/joomlatools-files sudo chown --reference=/var/www/yourwebsite.com/joomlatools-files/ /mnt/mys3bucket/joomlatools-files sudo chmod --reference=/var/www/yourwebsite.com/joomlatools-files/ /mnt/mys3bucket/joomlatools-files
joomlatools-files directory contents to S3:
sudo rsync -a /var/www/yourwebsite.com/joomlatools-files/ /mnt/mys3bucket/joomlatools-files/
Backup the original directory and replace it with a symlink to your mounted S3 bucket:
cd /var/www/yourwebsite.com/ mv joomlatools-files joomlatools-files-backup ln -s /mnt/mys3bucket/joomlatools-files joomlatools-files
Test your DOCman installation. Remove the
joomlatools-files-backup directory if everything works.
Your files are now being stored on Amazon S3!
Frequently asked questions
Will the mounted S3 bucket take up much disk space?
No. Because s3fs is a virtual file system, it will only download the files when they are accessed. These will be stored in a temporary directory which s3fs cleans up automatically.
If you want to reduce the number of downloads, you can use a local cache to keep files around on your disk. For more information, please refer to the s3fs docs.