Weird routing / arp / ppp problem - low upload after debian upgrade
Hi,
After upgrade from old patched etch, my clients cannot browse internet anymore* (upload is ok but download not bigger than* few kbps ) - problem occurs randomly - other services that use small packets like voip work perfectly.
Here's my detailed problem.
I have pppoe concentrator serving several hundreds of computers . Every user can have public IP (directly on the pppoe tunnel without snat/dnat) or snated private IP. Those clients with public IP are proxy_arp'ed so world can see them. Incoming traffic goes on imq0 and outgoing on eth0 - traffic shaping looks fine
This is typical example of firewall rule generated for public IP :
for private IP we have almost the same but there's SNAT in the iptables part. Every client has the same formula for generating iptables firewall.
My problem is following - totally random clients are having problems with download. If I use mikrotik bandwidth tester from internet to their computer it gives transfer like Xmbits upload (from their side) and 10-15kbps in direction to the client. Problem ONLY occurs when they are behind their client router. If they connect via pppoe directly to my server - problem disappeares.
Bandwidth tester uses big packets so they are fragmented. If I use packets like ping - they have nice transfer and everything is reachable from them.* The problem on the side of client looks like they can browse internet but google.com loads for like 20 minutes but voip works fine. Moreover if i take client router and place it in the other place of my "lan" (my lan is 100% bridged with mikrotik) , it usually works.
What i've triple checked :
- generation of iptables/tc rules
- pppoe MTU (1480 or 1492 - both working )
- mss - path mtu discovery packets are not blocked, everything looks fine
What i suspect :
- some arp problem maybe ?
Problem began right after i've changed my old Etch server on 2.6.15 witch patched iptables and kernel with patch-o-matic into clean 2.6.32 squeeze with everything from apt. My sysctl.conf along with pppoe-server-options is attached at the end of this message.
I've done also tcpdump sniff on the clients interface many times and nothing drags my attention.
Typical arp entry for snated IP is :
? (10.100.0.25) at <incomplete> on eth1
? (10.100.0.25) at <from_interface> PERM PUB on eth1
Typical arp entry for public IP looks like :
? (217.17.10.250) at <from_interface> PERM PUB on eth0
Any clues will be VERY appreciated.
debian-firewall , please Cc to me as I'm not subscribed please.
--
Wojciech Ziniewicz
http://www.rfc-editor.org/rfc/rfc2324.txt
12-08-2010, 03:47 PM
Brett Parker
Weird routing / arp / ppp problem - low upload after debian upgrade
On 08 Dec 16:26, Wojciech Ziniewicz wrote:
> Hi,
> After upgrade from old patched etch, my clients cannot browse internet
> anymore (upload is ok but download not bigger than few kbps ) - problem
> occurs randomly - other services that use small packets like voip work
> perfectly.
Between that system and the outside world, is there another
router/firewall?
My initial guess would be that you've hit the tcp window scale problem,
you can (quickly) check this by doing:
sysctl net.ipv4.tcp_window_scaling=0
On the box that they're going through - if that works then you've got a
box between that and the internet that doesn't watch the window scaling
flag as it goes past, and therefore mangles packets later on because it
doesn't know that they can get through.
--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20101208164714.GD4830@sommitrealweird.co.uk">http://lists.debian.org/20101208164714.GD4830@sommitrealweird.co.uk
12-08-2010, 05:09 PM
Wojciech Ziniewicz
Weird routing / arp / ppp problem - low upload after debian upgrade
> After upgrade from old patched etch, my clients cannot browse internet
> anymore *(upload is ok but download not bigger than *few kbps ) - problem
> occurs randomly - other services that use small packets like voip work
> perfectly.
Between that system and the outside world, is there another
router/firewall?
*there's only my router with BGP session acting as a gateway
*
My initial guess would be that you've hit the tcp window scale problem,
you can (quickly) check this by doing:
* *sysctl net.ipv4.tcp_window_scaling=0
I did some tests with both settings :
1.
telneting on a host behind my client's router :
listening on ppp296, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
--- from now on my router tries to get response from the box behind firewall
18:58:40.056068 IP 1.mydomain.com.3718 > 10.100.0.194.telnet: Flags [P.], seq 28:37, ack 46, win 1452, length 9
18:58:40.284102 IP 1.mydomain.com.3718 > 10.100.0.194.telnet: Flags [P.], seq 28:37, ack 46, win 1452, length 9
syn with reset all the time - totally no connectivity.
so with tcp scaling on my server we have packets going thru client's nat but big packets cannot go thru . on the other hand when I turn tcp window scaling to "on" i can't even connect (reset + syn all the time), but icmp goes thruough both in 1 and 2 case
frankly i have no clue why O_o
--
Wojciech Ziniewicz
http://www.rfc-editor.org/rfc/rfc2324.txt
Weird routing / arp / ppp problem - low upload after debian upgrade
