rsync stalls, especially over ssh

[fabi]

Hi

I use the rsync module to backup my files. Sometimes, especially with big files over ssh, rsync just stops in the middle of the file without doing anything anymore.

I found this post and patch: marc.info/?l=rsync&m=116932447411768&w=2

Would it be possible to build an rsync binary with this patch and use it instead of the one in the module? Or would anybody capable of building modules consider to build an updated rsync?

Thanks

[omega]

Hi fabi,

In my RSYNC module I'm using rsync version 2.6.9 which is the actual version available.

The patch you're mentioning isn't compatible with that version and when I'm looking at the source code I have the feeling that the modifications the patch provides is already built in.

I think your problem must be related to something different.

Can you tell us how exactly you're doing the rsync over ssh transfer?
Then maybe I can do the same in order to to investigate what's going on.

Andreas

[fabi]

Actually I also had this stall sometimes without ssh but much less so. With ssh it was almost repeatable. The command line itself doesn't seem that special, I took most of it from other examples on the net.

This is from a script called by cron:

RSYNC_ARGS="-rptgo --progress --rsync-path=\"/raid/data/scripts/rsync\""

SSH="ssh -l sys -ax -i /raid/data/configs/serverkey -o ClearAllForwardings=yes -o StrictHostKeyChecking=no"

$RSYNC_BIN $RSYNC_ARGS --delete -e "$SSH" -l $LOCALSOURCE $BACKUPSERVER:dest 2>&1


I tried the actual version from cvs (3.0.0cvs), compiled it on another system and just copied it on the raid (therefore the --rsync-path). With this version I never had a stall anymore.

[omega]

So you mean that when using the cvs version of rsync, all your problems are gone?

Andreas

[fabi]

I had a small problem and a big problem.

The big problem was the stalling and that seems to be fixed as I never had one again after using the cvs version.

The small problem was that 2.6.9 wasn't able to sync a folder at once because there were too many files to build up the file list. So I changed that to sync several subfolders. That may also be obsolete now as the cvs version has an incremental file list. But I didn't change my scripts as they were running fine.

So yes, the cvs rsync solved my problems. The only thing is that I'm a bit uncomfortable using a developer version on a production server. As soon as rsync 3.0.0 comes out I may switch to that in hope that it is a bit better tested.

[omega]

Fabi,

I'm just curious and therefore I ask: why are you not using the rsync daemon on the N5200. Using it you don't need the SSH transport and I expect it to be faster.

I'm using the rsync daemon for my own backups and I never seen any stalls.

Andreas

[fabi]

We do use the rsync daemon for intranet backups, but we do also backups over internet so ssh was a must. We don't want the box any more open as needed.

Speaking of the devil: Seems like I just had a stall again, this time using rsync with ssh over internet and firewalls. Maybe it's a timing problem. But still the new version is much better than 2.6.9. If this only happens once every now and then I can live with the rsync timout option, I think.

[fabi]

Little update: I didn't have a stall yesterday, it was just synching a 1.4 GB file over a 1MB line, so that obviously took some time. I just misinterpreted it as stall. So the rsync cvs version still didn't have a stall.