SmoothLinux Blog

Increasing Riofs Performance

18 November, 2014 | riofs

Riofs for me was a huge improvement over s3fs. S3fs is OK as a web server backend storage as the web server knows were to look for the files. However for me when it came down to listing out the directory content s3fs was just way to slow for me to use. So I started my search for something else that could meet my needs for quick directory content listing and Riofs stepped in to meet that challenge.

Warning!!! Riofs currently does not support Directory Renaming. This is one huge draw back for me has I am having to set this up for others to use not just me. In some environments limitation is a deal breaker.

Step 1: Installing Riofs From Git On Debian 7 (Wheeze)

Packages that needs to be installed before getting started:

apt-get install git build-essential gcc make automake fuse autoconf libtool pkg-config intltool libglib2.0-dev libfuse-dev libxml2-dev libevent-dev libssl-dev libmagic-dev

Getting the source from github: This will clone riofs into a a folder name: riofs

git clone https://github.com/skoobe/riofs.git

cd into folder riofs and run the following commands below. The commands below will compile and install riofs into /usr/local/bin/riofs. The most important options is the --with-libmagic. Without this option riofs will not be able to guess the content-type and it will put all your files on the S3 mount as binary/octet-stream. Which is very bad for images and video as the browser will just pop up and ask the end user to download the file instead of rendering it in the browser.

./autogen.sh
./configure --prefix=/usr/local --with-libmagic=/usr/lib/x86_64-linux-gnu
sudo make
sudo make install

Make sure you see the following lines below are in the config output to verify that libmagic is enabled and will be build into riofs. If not fix this before you start using riofs. This was a bought lesson for me.

/usr/lib/x86_64-linux-gnu
checking magic.h usability... yes
checking magic.h presence... yes
checking for magic.h... yes
checking for magic_open in -lmagic... yes

Step 2: Riofs Configuration And Mounting S3 Bucket

Edit: /etc/fuse.conf and add: user_allow_other

The default riofs.conf.xml file is located in: /usr/local/etc/riofs/riofs.conf.xml

Personally I install this file in the root directory to protect it like so:

install -dv /root/.config/riofs/
cp /usr/local/etc/riofs/riofs.conf.xml /root/.config/riofs/
chmod 600 /root/.config/riofs/riofs.conf.xml

Now to mount the store with the default configuration

/usr/local/bin/riofs -c /root/.config/riofs/riofs.conf.xml -l /var/log/riofs.log -o "allow_other" 
bucketname /mountpoint

So the first time I mounted the bucket using riofs. I was really impressed with the speed at which it would return directory content listing but when I uploaded a file via FTP it would upload about 5mb to 10mb and then the transfer speed would slow. After a few seconds the transfer speed would pick back up again this continued this up and down behaviour till the transfer was over. I thought to myself another slow fuse mount.

So I started digging through the configuration options and the following configuration really helped me with transfer speed:

<part_size type="uint">5242880</part_size>

Changed 5242880 to 1073741824 = 1GB resolved the issues of the slow upload speeds.

Permission issues: Setting this helped me avoid permission issues from the web server and vsftpd server on directory listing. This sets everything to 777 on my system. So this may not be something everyone needs so use with caution.

<dir_mode type="int">-1</dir_mode>
<file_mode type="int">-1</file_mode>

I changed -1 on both lines to 511

I also increased the size of the cache directory to fit my partition size that I would like to grow my tmp out to. If your system has a small tmp space you can run into issues here by changing this from the default value.

<cache_dir_max_size type="uint">1073741824</cache_dir_max>

Changed this from 1073741824 to 10737418240 = 10GB

After making the needed changes. You will need to umount the bucket. I always kill off the process id as riofs does not do this on its own just to be safe.

Optional Step Here

If you want log output from riofs you will need change to value below to 1.

<level type="int">0</level>

Change this from 0 to 1

Currently to my knowledge riofs can not be mounted via fstab. So my work around for this is to mount riofs via /etc/rc.local .

Add the following to /etc/rc.local and it should be mounted at boot:

/usr/local/bin/riofs -c /root/.config/riofs/riofs.conf.xml -l /var/log/riofs.log -o "allow_other" bucketname /mountpoint

 

Comments

comments powered by Disqus