Amazon S3

Want to store DOCman files on Amazon S3? This is easily done by using s3fs.

s3fs allows you to mount an Amazon S3 bucket as a file system so you can access it like any other file.

Installation

Because there are no official packages available, you will need to compile s3fs yourself. Luckily this is very easy.

Installing the build dependencies

If you are on Ubuntu:

sudo apt-get install automake autotools-dev g++ git libcurl4-gnutls-dev libfuse-dev libssl-dev libxml2-dev make pkg-config

If you are running CentOS:

sudo yum install automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel

If you are running another operating system, please refer to the s3fs wiki.

Build the s3fs source code and install it

cd ~
git clone https://github.com/s3fs-fuse/s3fs-fuse.git
cd s3fs-fuse
./autogen.sh
./configure
make
sudo make install

Mounting your S3 bucket

Store credentials

Store your login credentials where s3fs can find them (make sure to replace with your correct ID and access key):

echo "<AWS Access Key ID>:<AWS Secret Access Key>" | sudo tee /etc/passwd-s3fs
sudo chmod 600 /etc/passwd-s3fs

Mount S3 bucket

Now mount the bucket:

sudo mkdir /mnt/mys3bucket /var/cache/s3cache
sudo s3fs -o use_cache=/var/cache/s3cache,allow_other,nonempty joomlatools-s3fuse-test /mnt/mys3bucket

Test it by creating a file:

echo "Test" | sudo tee /mnt/mys3bucket/test.txt

Verify if the file shows up on your Amazon S3 bucket through the Amazon web console.

If you get errors or the file is not being synced, take a look at your system log (on Ubuntu: /var/log/syslog) to see what s3fuse is saying.

Mount S3 bucket after reboot

Add an entry to the /etc/fstab file:

joomlatools-s3fuse-test /mnt/mys3bucket fuse.s3fs _netdev,uid=33,gid=33,umask=007,use_cache=/var/cache/s3cache,allow_other,nonempty,retries=5 0 0

Move your files to the S3 bucket

Create directory

Create the joomlatools-files directory on S3 and give it the necessary permissions:

mkdir /mnt/mys3bucket/joomlatools-files
sudo chown --reference=/var/www/yourwebsite.com/joomlatools-files/ /mnt/mys3bucket/joomlatools-files
sudo chmod --reference=/var/www/yourwebsite.com/joomlatools-files/ /mnt/mys3bucket/joomlatools-files

Copy files

Copy the joomlatools-files directory contents to S3:

sudo rsync -a /var/www/yourwebsite.com/joomlatools-files/ /mnt/mys3bucket/joomlatools-files/

Add symlink

Backup the original directory and replace it with a symlink to your mounted S3 bucket:

cd /var/www/yourwebsite.com/
mv joomlatools-files joomlatools-files-backup
ln -s /mnt/mys3bucket/joomlatools-files joomlatools-files

Test your DOCman installation. Remove the joomlatools-files-backup directory if everything works.

Your files are now being stored on Amazon S3!