Redis No Space Left On Device
Redis throw a no disk space error with 40GB free on the hard drive and more then 3GB of ram free.
Here is the error:
Write error saving DB on disk: No space left on device
The redis error log was nice enough to give me a kernel tweak to add:
Line from the error log:
WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
You can read more about what the kernel tweak does here Redis FAQ or you can blindly trust that it will not break your system.
Even after a reboot to set the kernel tweak redis still would not start due to database corruption.
The redis-check-aof command is needed to repair the database:
Redis database storage location (ubuntu): cd /var/lib/redis Command to repair the database: redis-check-aof --fix appendonly.aof Starting redis to make sure the repair worked: service redis start