In which Tom doesn't understand networks
Jan. 12th, 2019 12:36 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
fivemackMy model of the world is that, for most network-like purposes, computers are infinitely fast at computing and bottlenecked entirely by the difficulty of sending bytes across wires.
I now have two machines with 10GBASE-T network cards. I've plugged a cat6 cable between them, and done
ifconfig eno2 172.26.128.2
route add -host 172.26.128.1 dev eno2
on machine OAK and
ifconfig enp101s0 172.26.128.1
route add -host 172.26.128.2 dev enp101s0
on machine PINEAPPLE.
If I run 'netperf -H 172.26.128.1' on OAK I get 9411.48Mbits/sec throughput, which suggests at least something is running at 10Gb/s or so. Similarly 'netperf -H 172.26.128.2' on PINEAPPLE says 9402.15Mbits/sec.
But rsync between the 10Gbit interfaces on the hosts runs at the same 80Mbyte/sec that it did over gigabit ethernet and two switches. rcp is a bit faster (100Mbyte/sec versus 65), but I was hoping for high three figures.
OAK and PINEAPPLE both have fast NVMe storage which can be read and written at 2Gbyte/sec or so.
How can I get a file between these machines at at least half the wire speed? I feel I am missing some critical fact about networking in the modern age.
I now have two machines with 10GBASE-T network cards. I've plugged a cat6 cable between them, and done
ifconfig eno2 172.26.128.2
route add -host 172.26.128.1 dev eno2
on machine OAK and
ifconfig enp101s0 172.26.128.1
route add -host 172.26.128.2 dev enp101s0
on machine PINEAPPLE.
If I run 'netperf -H 172.26.128.1' on OAK I get 9411.48Mbits/sec throughput, which suggests at least something is running at 10Gb/s or so. Similarly 'netperf -H 172.26.128.2' on PINEAPPLE says 9402.15Mbits/sec.
But rsync between the 10Gbit interfaces on the hosts runs at the same 80Mbyte/sec that it did over gigabit ethernet and two switches. rcp is a bit faster (100Mbyte/sec versus 65), but I was hoping for high three figures.
OAK and PINEAPPLE both have fast NVMe storage which can be read and written at 2Gbyte/sec or so.
How can I get a file between these machines at at least half the wire speed? I feel I am missing some critical fact about networking in the modern age.
