[ovs-discuss] OVS 2.9.0 native firewall drops empty payload TCP packets

Zhang, Jing C. (Nokia - CA/Ottawa) jing.c.zhang at nokia.com
Mon Aug 27 13:20:15 UTC 2018


We have customers reporting sluggish HTTP download with OVS 2.9.0. After debugging, we find the issue is OVS 2.9.0 native firewall drops TCP control packets (TCP ACK) with empty payload. The issue can be avoided by either reverting back to the legacy Linux bridge firewall or enabling TCP timestamp at both HTTP server and client sides.
$ ovs-vswitchd --version
ovs-vswitchd (Open vSwitch) 2.9.0
DPDK 17.11.0

openvswitch-2.9.0-3.el7.x86_64
The issue can be easily reproduced with scapy.
This is an example empty payload TCP packet received on the physical NIC port:

# tcpdump tcp -i ens1f0 -nevvXX
tcpdump: listening on ens1f0, link-type EN10MB (Ethernet), capture size 262144 bytes

19:05:52.354860 fa:16:3e:37:a1:4f > fa:16:3e:f8:db:64, ethertype 802.1Q (0x8100), length 64: vlan 628, p 0, ethertype IPv4, (tos 0x0, ttl 63, id 1, offset 0, flags [none], proto TCP (6), length 40)
    135.255.159.203.64766 > 10.0.1.7.http: Flags [S], cksum 0x5f5e (correct), seq 100, win 8192, length 0
        0x0000:  fa16 3ef8 db64 fa16 3e37 a14f 8100 0274  ..>..d..>7.O...t
        0x0010:  0800 4500 0028 0001 0000 3f06 48fe 87ff  ..E..(....?.H...
        0x0020:  9fcb 0a00 0107 fcfe 0050 0000 0064 0000  .........P...d..
        0x0030:  0000 5002 2000 5f5e 0000 0000 0000 0000  ..P..._^........

Debugging shows those packets can arrive on the tap interface if the firewall is skipped like the following:

ovs-ofctl add-flow br-int "table=81, priority=101, ct_state=-trk,reg5=0x3 actions=output:3"

# tcpdump tcp -i tapeda6e769-ea -nevvXX
tcpdump: listening on tapeda6e769-ea, link-type EN10MB (Ethernet), capture size 262144 bytes

17:26:47.091233 fa:16:3e:37:a1:4f > fa:16:3e:f8:db:64, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 63, id 1, offset 0, flags [none], proto TCP (6), length 40)


I have been digging the related code areas but unable to find the obvious packet drop due to empty payload. Please advise of the root cause, or give pointers, I am looking for a quick local fix first.

No issue with firewall rules themselves, I copy paste here the rules leading to the packet drop on a lab I use for debugging, to help explain the context:
...
cookie=0x3671c32b581ad607, duration=211770.989s, table=0, n_packets=619026, n_bytes=168758644, idle_age=14, hard_age=65534, priority=3,in_port=3,dl_vlan=697 actions=mod_vlan_vid:1,resubmit(,60)
...
cookie=0x3671c32b581ad607, duration=211769.188s, table=81, n_packets=611880, n_bytes=168298703, idle_age=19, hard_age=65534, priority=90,ct_state=-trk,ip,reg5=0x1 actions=ct(table=82,zone=NXM_NX_REG6[0..15])
...
cookie=0x3671c32b581ad607, duration=211769.188s, table=60, n_packets=619006, n_bytes=168757352, idle_age=14, hard_age=65534, priority=90,dl_vlan=1,dl_dst=fa:16:3e:9a:b1:5c actions=load:0x1->NXM_NX_REG5[],load:0x1->NXM_NX_REG6[],strip_vlan,resubmit(,81)



Thanks a lot.

Jing Zhang




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20180827/6374f8c9/attachment.html>


More information about the discuss mailing list