When you have ethernet interface and gre tunnel in linux box, you use both interface for passing traffic or maybe your gre act as your default gateway, you will face problem can’t browse to some maybe most site. You can’t upload the attachment to the server, but you still could do browsing. When you dump the packet (by using tcpdump), you will see following messages:
21:02:26.126022 IP (tos 0xc0, ttl 64, id 12001, offset 0, flags [none], proto 1, length: 576) 202.169.xxx.7 > 202.169.yyy.0: icmp 556: 188.8.131.52 unreachable – need to frag (mtu 1476) for IP (tos 0×0, ttl 62, id 26504, offset 0, flags [DF], proto 6, length: 1500) 202.169.yyy.0.6344 > 184.108.40.206.https: P 0:1460(1460) ack 1 win 64637
You have problem different MTU size between GRE tunnel interface and ethernet interface. Gre tunnel has MTU size 1476, meanwhile ethernet has default MTU 1500. When your router receive packet from your client that have length 1500 bytes need to be routed through gre interface, your router send icmp messages to the client that your packet need to fragment to fit MTU size of gre tunnel.
Ok, let’s we overcome this problem.
First thing, we should know about TCP handshaking process.
1. at first time when open connection, the client send SYN to the server. Inside SYN packet, there is MSS (maximum segment size) to inform the server how much maximum packet could be received by client in a single segment. Usually MSS is MTU size minus 40 byte header of tcp/ip. If the client use ethernet, so maximum MSS is 1500 – 40 = 1460.
2. In response, the server replies back with SYN+ACK. The replied packet contains MSS also that inform how much largest size packet could be received by the server in a single segment.
3. After receiving SYN+ACK, client and server just send ACK without MSS information inside the packet .
here I capture real packet, from server side (in this case is yahoo) replies SYN+ACK to client (the packet
21:46:53.289455 IP (tos 0×0, ttl 57, id 56012, offset 0, flags [DF], proto 6, length: 60) 220.127.116.11.http > 202.169.yyy.0.59830: S [tcp sum ok] 2406052800:2406052800(0) ack 3906615889 win 65535 <mss 1460,nop,nop,timestamp 2020243263 676441344,sackOK,eol>
After client receiving the syn+ack, the client sent the largest packet to the server, at here the client send packet with length 1486 byte in a segment.
21:50:52.840159 IP (tos 0×0, ttl 64, id 57867, offset 0, flags [DF], proto 6, length: 52) 18.104.22.168.52743 > 22.214.171.124.http: . [tcp sum ok] 4150686945:4150686945(0) ack 1332179280 win 5840 <nop,nop,timestamp 676501358 2020482795>
21:50:52.840574 IP (tos 0×0, ttl 64, id 57868, offset 0, flags [DF], proto 6, length: 1486) 126.96.36.199.52743 > 188.8.131.52.http: P 0:1434(1434) ack 1 win 5840 <nop,nop,timestamp 676501358 2020482795>
When router receives above packet, and after process the routing table. The packet should pass through gre interface. Meanwhile gre interface have MTU size 1476 byte. So, it’s impossible to forward the above packet to gre interface. So router send icmp unreachable message to client
21:54:14.150394 IP (tos 0xc0, ttl 64, id 12116, offset 0, flags [none], proto 1, length: 576) 202.169.aaa.7 > 202.169.yyy.0: icmp 556: 184.108.40.206 unreachable – need to frag (mtu 1476) for IP (tos 0×0, ttl 63, id 17304, offset 0, flags [DF], proto 6, length: 1486) 202.169.yyy.0.33674 > 220.127.116.11.http: P 1:1435(1434) ack 1 win 5840 <nop,nop,timestamp 676551634 617980472>
So, we already examined the whole process. The solution is here
1. Use PMTUD (PAth MTU Discovery)
If the router block icmp, this is useless. And not all router support PMTUD
2. Alter MSS Information in three way handshaking
I love this way, at here we alter MSS in SYN packet and in SYN+ACK packet.
iptables -t filter -I FORWARD -p tcp –tcp-flags SYN,RST,ACK SYN -j TCPMSS –set-mss 1400
iptables -t filter -I FORWARD -p tcp –tcp-flags SYN,RST,ACK SYN,ACK -j TCPMSS –set-mss 1400
Filed under: Sistem Operation