...
- 2022-09-07 krowe: Doing tcpdumps of iperf3 tests between ingress_sbox namespaces shows that the TCP iperf3 packets are being NATed into UDP packets. So I used iperf3 from across na-arc nodes (not in the ingress_sbox namespaces)
- iperf3 -B <LOCAL IP> -c <REMOTE IP> -u -b 2000000000 -t 100
Table5: iperf3 UDP to/from hosts (% packet loss) na-arc-1
(naasc-vs-4)
na-arc-2
(naasc-vs-4)
na-arc-3
(naasc-vs-3)
na-arc-4
(naasc-vs-4)
na-arc-5
(naasc-vs-5)
na-arc-1 na-arc-2 na-arc-3 na-arc-4 na-arc-5
- iperf3 -B <LOCAL IP> -c <REMOTE IP> -u -b 2000000000 -t 100
- 2022-09-08 krowe: I have tested the other overlay networks (production_agent_network 10.0.1.0/24 and production_default 10.0.2.0/24) and they perform similarly to the ingress overlay network 10.0.0.0/24.
- 2022-09-09 krowe: na-arc-6 is now online served from naasc-vs-2. Here are the iperf3 tests from ingress_sbox to ingress_sbox. When throughput is slow (Kb/s) I see that the congestion window size is reduced from about 1MB to about 2.73KB.
Table6: iperf3 TCP throughput from/to ingress_sbox (Mb/s) na-arc-1
(naasc-vs-4)
na-arc-2
(naasc-vs-4)
na-arc-3
(naasc-vs-3)
na-arc-4
(naasc-vs-4)
na-arc-5
(naasc-vs-5)
na-arc-6
(naasc-vs-2)
na-arc-1 3920 2300 4200 3110 3280 na-arc-2 3950 2630 4000 3350 3530 na-arc-3 0.2 0.3 0.2 2720 2810 na-arc-4 3860 3580 2410 3390 3290 na-arc-5 0.2 0.2 2480 0.2 2550 na-arc-6 0.005 0.005 2790 0.005 3290 - 2022-09-09 krowe: The ingress network (docker mesh) that I have been testing using the ingress_sbox namespace uses a veth interface (this is like a pipe) that connects to its corrosponding veth interface in another namespace on the same host which connects to a vxlan over a bridge in that second namespace. vxlan is a tunneling protocol that uses UDP over port 4789. This is why I am seeing my TCP packets turn into UDP packets. Using tcpdump in the ingress_sbox to watch iperf TCP traffic going from na-arc-2 to na-arc-3 looks clean. Watching traffic going from na-arc-3 to na-arc-2, which is slow (32KB/s), shows lots of TCP Retransmission and TCP Out-Of-Order packets.
Comparisons
naasc-vs-3, 4, 5
...