Categories
Mastering Development System & Network

linux 10gbe ixgbe (intel X550T) slow at 420mbyte/s max, p2p lan cat8 1m, tried ethtool and ifconfig options to no avail

first time dealing with 10gbe.

I have two boxes, one is older with i7 2.8ghz from 2009 generation, pcie 2.0 and I just put the Intel X550T there to make use of absolutely similar but builtin adapter on my newer box with AMD EPYC gen 1.

I’m dling file from tmpfs on either box and never go above 415mbyte/s which is about half of potential bandwidth there, right?

I tried most stuff from https://www.kernel.org/doc/ols/2009/ols2009-pages-169-184.pdf with a little improvement from 393mbyte/s to current 415. that’s the result of me setting mtu 9000, txqueuelen 10000.

Also tried this to no improvements over current result: https://darksideclouds.wordpress.com/2016/10/10/tuning-10gb-nics-highway-to-hell/

And a few other pages out there on google, most stating same options.

I don’t see a 100% load burst on i7 box when I dl file, it’s around 10% across the cores. So CPU shouldn’t be a bottleneck, cable(it’s 1meter cat8, with a nice thick shielding) shouldn’t be a bottleneck and ram speed and pcie 2.0 speed shouldn’t be.
iperf also reports same speed.

Tried these tests in both directions.

There’s gotta be something obvious that I’m missing, right?

Pretty sure it doesn’t matter but it’s lfs on i7 and debian 10 on epyc, both sharing same sysctl settings and ethtool-controlled options.

lspci -n for it on both computers: 62:00.0 0200: 8086:1563 (rev 01) so it’s similar device. I bought this particular one for compatibility but didn’t expect these issues.

Maybe someone here would suggest something, I’m out of ideas.

TIA

Leave a Reply

Your email address will not be published. Required fields are marked *