FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 09-26-2012, 03:33 AM
Emmett Culley
 
Default BackupPC and rsync

I have been using BackupPC via rsync daemons on the target machines and all has been working well. At least until about the time rsync went to version 3.0.6.

Since then backups have been failing for some targets. The failure is a TCP ZeroWindow issue. It looks like the server rsync process cannot empty it's buffer soon enough and eventually the process fails.

I really do not know if it is rsync version related, because I never looked at the packet stream until I started having failures.

BackupPC was running on a VM host and the failing machine was one of the guest VMs. I tried upgrading rsync to 3.0.9 and that didn't help.

So I moved the BackupPC set up to a different server (no VM). Not all backups are working. though I still see some ZeroWindow packets along with a few retransmision packets. But the backup completes on all machines, including the VM that was failing before.

I've googled rsync zerowindow and found some references, but none seemed to apply to my situation.

Anybody else having a similar experience?

Emmett
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 09-26-2012, 06:05 PM
Les Mikesell
 
Default BackupPC and rsync

On Tue, Sep 25, 2012 at 10:33 PM, Emmett Culley <emmett@webengineer.com> wrote:
> I have been using BackupPC via rsync daemons on the target machines and all has been working well. At least until about the time rsync went to version 3.0.6.
>
> Since then backups have been failing for some targets. The failure is a TCP ZeroWindow issue. It looks like the server rsync process cannot empty it's buffer soon enough and eventually the process fails.
>
> I really do not know if it is rsync version related, because I never looked at the packet stream until I started having failures.
>
> BackupPC was running on a VM host and the failing machine was one of the guest VMs. I tried upgrading rsync to 3.0.9 and that didn't help.
>
> So I moved the BackupPC set up to a different server (no VM). Not all backups are working. though I still see some ZeroWindow packets along with a few retransmision packets. But the backup completes on all machines, including the VM that was failing before.
>
> I've googled rsync zerowindow and found some references, but none seemed to apply to my situation.
>
> Anybody else having a similar experience?

I found an old mention of turning off TCP window scaling solving a
problem with rsync (not involving backuppc). Seems unlikely but
might be worth a try: sysctl -w net.ipv4.tcp_window_scaling=0.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 09-27-2012, 01:35 AM
Emmett Culley
 
Default BackupPC and rsync

On 09/26/2012 11:05 AM, Les Mikesell wrote:
> On Tue, Sep 25, 2012 at 10:33 PM, Emmett Culley <emmett@webengineer.com> wrote:
>> I have been using BackupPC via rsync daemons on the target machines and all has been working well. At least until about the time rsync went to version 3.0.6.
>>
>> Since then backups have been failing for some targets. The failure is a TCP ZeroWindow issue. It looks like the server rsync process cannot empty it's buffer soon enough and eventually the process fails.
>>
>> I really do not know if it is rsync version related, because I never looked at the packet stream until I started having failures.
>>
>> BackupPC was running on a VM host and the failing machine was one of the guest VMs. I tried upgrading rsync to 3.0.9 and that didn't help.
>>
>> So I moved the BackupPC set up to a different server (no VM). Not all backups are working. though I still see some ZeroWindow packets along with a few retransmision packets. But the backup completes on all machines, including the VM that was failing before.
>>
>> I've googled rsync zerowindow and found some references, but none seemed to apply to my situation.
>>
>> Anybody else having a similar experience?
>
> I found an old mention of turning off TCP window scaling solving a
> problem with rsync (not involving backuppc). Seems unlikely but
> might be worth a try: sysctl -w net.ipv4.tcp_window_scaling=0.
>

I tried that on the target VM and it didn't make any difference. Then I tried it on the BackupPC server and that didn't make any difference either.

Thanks for the suggestion anyway.

Emmett

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 09-27-2012, 02:25 AM
Gordon Messmer
 
Default BackupPC and rsync

On 09/25/2012 08:33 PM, Emmett Culley wrote:
> Since then backups have been failing for some targets. The failure
> is a TCP ZeroWindow issue. It looks like the server rsync process
> cannot empty it's buffer soon enough and eventually the process
> fails.

If the rsync server sends a TCP ZeroWindow packet to a client, that
probably indicates that the server is either hung or looped. Either
way, it has stopped reading data from the socket, and the kernel sends
ZeroWindow to the client's host to indicate that the client should stop
sending data.

You may have found a bug in rsync that the developers need to resolve.
Have you tried attaching strace to the rsync server process when it hangs?
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 09:45 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org