15 Feb SSH rsync backups to a ReadyNas – blank media usage reports solved

Before I start, I should warn you, this may be a rare issue that only effects a handful of customers – in particular those that use older NAS hardware. However, through diagnosing this issue I’ve learned much more about how the media usage report works with BackupAssist rsync jobs and thought it would be useful to share my findings.

Background

One of our own rsync jobs is configured to back up an MDaemon mail server once a day using the “basic” scheme. The destination for this job is a remote Netgear Duo (V1) NAS and so the job is configured using rsync over SSH. This job has run for months without issue and updates the backup every day with any changes in the MDaemon server. The only issue was that in the reports, the media usage section was always blank.

When you watch the job running in the monitor window, just before it finishes you can see the following error:-

ls: unrecognized option `–time-style=+’
Try `ls –help’ for more information.

0 0 0 0 Tuesday
0 0 0 0 Monday
0 0 0 0 Sunday
0 0 0 0 Saturday
0 0 0 0 Friday
0 0 0 0 Thursday
0 0 0 0 Wednesday

So whats happening here?

The media usage report is generated by running a special bash script on the rsync server after the rsync job has completed. This script adds up the space used by all the files within the backup folder previously written to.

In effect, this is a clever use of the Linux ‘ls’ command with includes some specific extensions to catalogue each file in the backup and how much space it actually uses on the disk as well as the file size. Starting at the root folder of the backup, the script recurses through each sub folder one by one adding up all the files it finds, keeping a running total.

Once it has trawled through the full backup it has a value for the total space represented by the files, as well as the actual disk space used at the block level. From these two figures it can easily work out the space saved by BackupAssist’s single instance store feature.

This information’s useful as it shows how much actual disk space is used on the rsync server and not just the size that each backup folder represents.

The problem is, to get this information a special ‘ls’ command extension is required (–time-style=), and it is this that unfortunately was not supported by my old Netgear ReadyNas Duo.

To fix this problem and get my media usage reports to work was actually relatively straight forward, but does require you to change core Linux utilities on the NAS that have not been updated by Netgear and therefore would more than likely void your warranty. If you are happy to do this read on, but for legal reasons we cannot be held responsible for any damage you may cause.

How did I fix it?

I opened an SSH session to the NAS, and edited the Linux sources file using the ‘vi’ text editor to include the following aditional debian sources..

# vi /etc/apt/sources.list

deb http://www.readynas.com/packages readynas/

deb http://archive.debian.org/debian-security sarge/updates main contrib non-free
deb-src http://archive.debian.org/debian-security sarge/updates main contrib non-free

deb http://archive.debian.org/backports.org sarge-backports main contrib non-free
deb-src http://archive.debian.org/backports.org sarge-backports main contrib non-free

deb http://archive.debian.org/debian sarge main contrib non-free
deb-src http://archive.debian.org/debian sarge main contrib non-free

Next I ran an apt-get update to refresh the package list.

Finally I updated the coreutils package that includes the ‘ls’ command to the latest version supported by my device using the command apt-get install coreutils

Now my ‘ls’ command supports the `–time-style=‘ extension and the media reports have started to appear as expected.

One side effect is that these reports now add about 50 mins to the job time as the NAS is not the most powerful device and takes some time to add up all these files, but it’s still within an hour so I decided I was prepared to live with that.