Help needed - Backup N2100 to USB-HDD

[nospam]

Hi

I've read that it seems somehow to be possible to backup selected folders on the N2100 to a USB-HDD with a cron-job without having any other machine running. What do I need? Rsync-module ist just the client, right? Does it work with dd? Does anyone have a detailed HowTo for a Linux-NooB with some basic scripts how to deal with that? Is it possible to do regular backups in the night (full, inkremental or differencial (correct english??).

Thanx for advice

rolf

[slowstudy]

Hi Rolf,

Yep, it's not difficult to do what you want. I'm new to Linux, but thanks to this forum I was able to set up an automatic backup system on the n2100 using just the SSH and USBCopy modules.

As I didn't have to actually create a custom module, I thought that documenting my steps might be of assistance other newbies. All steps were carried out from Win XP Pro using WinSCP and WinRAR.

I've tried to keep each step reasonably detailed in order to keep things simple for other first-timers. Veterans, maybe I could ask you to chime in if there's anything incorrect, unnecessary, or risky in the approach I took.

Steps:

A. Collect modules and WinSCP
1. Download USBCopy from "http://www.onbeat.dk/thecus/index.php/N2100_Module:_USBCopy"
2. Download SSH from "http://onbeat.dk/thecus/index.php/N2100_Module:_SSH"
3. Download WinSCP from "http://winscp.net/eng/index.php" (freeware)
4. Download 40-day free WinRAR trial from "http://www.win-rar.com/" (Note: I'm sure there are probably freeware alternatives.)
5. Create "Restore Point" on Windows and Install WinSCP and WinRAR.

B. Extract the modules USBCopy and SSH using WinRAR
1. Launch WinRAR.
2. For each of the downloaded module files, right-click on the file and select "Extract to the selected folder"
3. Repeat process until you see a file with .mod filetype. (You may need to add the .tgz filetype before attempting to extract with WinRAR.)

C. Upload USBCopy and SSH modules to n2100
1. Login to Thecus Web console.
2. Navigate to "System" --> "Module Mgmt"
3. Click on "Browse" button to identify the .mod file to upload.
4. Click "Install"

D. Enable, and initialize modules
1. Select both modules and click "Enable" button.
2. Click on SSH module.
3. Click "Backup root"
4. Click "Initialize Root Password"
5. Click yes to "Flush SSH (host) keys?" (I believe this helps protect against man-in-the-middle attacks.)
6. If desired, also click yes to "Allow SSH user logins?"
7. Reboot the n2100 for good measure ("System" --> "Reboot and shutdown").

E. Login to Thecus via SSH
1. Launch WinSCP, enter Thecus IP in "Host name" and "root" in "User name"
2. Choose "SCP" under "File protocol" and click on "Save" button and save as "NAS". (SFTP would work as well.)
3. Go to "Stored sessions" and double-click on "NAS"
4. Enter the temporary password "irresistible" associated with the SSH module.

F. Change password and date/time
1. Type Ctrl + T to open a session terminal.
2. Enter "passwd" which will allow you to assign a new root password.
3. Enter "date" to check if current system clock data is correct.
4. If not, correct by typing "date [MMDDhhmmYYYY.ss]" ( Example: "date 022721112008.00" for Feb 27 9:11:00PM 2008)

G. Add automatic backup command to crond file
1. Backup existing crond file by entering "cp /app/cfg/crond.conf /app/cfg/crond.conf.bak"
2. In WinSCP file menu window, navigate to /app/cfg/crond.conf
3. Right click on file to open visual editor.
4. Add "15 23 * * * /raid/module/USBCopy/bin/rsync -rvt --delete /raid/PLACEHOLDER /raid/usbhdd/usb3/1" to the end of the file. ("15 23" means run command at 23:15/11:15PM)
5. Click on "Save" and "Exit" icons in WinSCP.
6. Now enter "crontab /app/cfg/crond.conf -u root" and close out of WinSCP.
7. The automatic backup will copy the contents of the folder PLACEHOLDER to the FAT32-formatted hard drive plugged into USB port 3 (bottom port at back of Thecus) at 11:15PM each night.

E. Conserve system resources (disable modules)
1. Login to Thecus Web console.
2. Navigate to "System" --> "Module Mgmt"
3. Select both the USBCopy and SSH modules and click "Disable" button. (Neither module needs to be enabled for the automated backup to run.)

Observations:
1. Backup transfer occurred at 2.3 MB/s for a 500MB file -- not impressive, but adequate (RAM upgrade may help).
2. While I haven't explored it, the UTILS module also contains rsync and could be used in place of the USBCopy module with path alterations.
3. By default, rysnc only copies files from the source that are different in date, time or size from what' s already in the destination folder.
4. The "-delete" control for rysnc ensures that any file that is deleted at the source will be removed from the destination as well.

Alternative:
While I haven't tested it, I expect it's possible to skip the installation of the USBCopy module and just place the rsync binary into a subfolder of the raid directory. (I like having the USBCopy module installed in case I need to perform quick copies from the Thecus console, but other users might wish to implement automated backups with as small a system footprint as possible.)

Hope the above is helpful. I'd like to thank "mshapf," "Marty," "Lzimmerma," "getmythe," "ktulaman," "werner," "tyz," "ryebank," and N2100Owner for their contributions on this topic. Most of all, thanks to those involved in setting up and maintaining the "ThecusWiki" -- it's a great resource!

[dudda]

I followed the instructions but when logging in via ssh I always get connection refused

any ideas to what could cause this?

[slowstudy]

dudda,

Not sure why the connection is being refused.

Here are a few suggestions:
1. To be safe, make sure your n2100 has the most recent firmware installed, v. 2.01.09 (sometimes referred to as 2.1.09).
2. Make sure the SSH module has been successfully installed, enabled, and initialized in "Module Mgmt." Remember to reboot.
3. For SSH login, make sure you're using the same IP that you used to login to the Thecus Web console.
4. Also, make sure that you are logging in as "root" with password, "irresistible."
5. If logging in with WinSCP doesn't work, try downloading Putty from www.chiark.greenend.org.uk/~sgtatham/putty/ . Here's an excellent step-by-step tutorial on setting up your SSH connection with Putty, www.jfitz.com/tips/putty_config.html . FYI if you stick the Putty.exe in /Program Files/Putty, WinSCP will recognize it and allow you to call it up from its GUI.

Good luck -- let us know how it goes.

[dudda]

sorted the problem. I had the old sftp module which was conflicting with the ssh module

[slowstudy]

Glad you were able to find that conflict between modules.

FYI, I've been testing the above auto-backup approach for over a week without any surprises.

Please let us know if you experience any problems going forward. Thanks.

[dudda]

I think you can get away without the usb copy module and use utilities instead. I have done it that way, see what happens tonight..

[dudda]

I used this and it worked a charm

"15 23 * * * /raid/module/UTILS/system/bin/rsync -rvt --delete /raid/PLACEHOLDER /raid/usbhdd/usb3/1"

saves the need for usb copy then

[slowstudy]

Thanks, dudda -- that's good to know!

I suppose the simplest approach is just to copy the rsync binary via SSH onto the RAID directory and adjust the path appropriately in the cron job.

Of possible interest: Changing your rsync invocation in the cron job to

*/5 * * * * /raid/module/UTILS/system/bin/rsync -rvt --delete /raid/PLACEHOLDER /raid/usbhdd/usb3/1"

will automatically backup every 5 minutes, instead of once a day.

The advantage is that at that frequency you've almost implemented a file replication system, as opposed to simple backup. (Nice, should your N2100 array fail mid-day in a heavy work cycle.)

Question: With the above approach, does anyone know what will happen if rsync has to copy a file(s) that requires more than 5 minutes to transfer to the USB drive (i.e., where the next automatic invocation of rsync would occur before the previous rsync operation has completed)?

[mercedes]

I followed the instruction but found that some files were backup and some didn't at all. Typical error is like this:

rsync: mkstemp "filename" failed: No such file or directory (2)

I did a google search and found that it might be caused by upper-case directories being processed as lower-case directories, when rsync tried to recursively copy the files, it couldn't find them and complained with an error message.

www.linuxforums.org/forum/linux-newbie/55198-rsync-complains-no-such-file-directory.html

This problem can however be solved by formatting the USB drive with ext3 instead of with FAT32, but this makes Windows incapable of reading the drive unless you install ext3 reader.

Is there any ways to fix this bug if I want to keep the drive in FAT32? Will there be an updated rsync version that I copy into the NAS?

I can rename the folders but it just doesn't seem to be a complete solution!

Thanks
Mercedes

[mercedes]

Just an update.

I can now backup data to an USB drive formatted in FAT32 after installing the META module and activating the "mount_any_usbdrive.sh" script.

Thanks getmythe for his help.