View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 30 posts ] 
Go to page 1, 2  Next

Print view Previous topic   Next topic  
Author Message
Search for:
PostPosted: Wed Dec 13, 2006 2:31 pm 
Offline
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1995
Location: /dev/null
**This assumes you have a NIC that can support jumbo frames and that you're using a network that can as well (i.e. have a router or switch that can handle them or are using a crossover cable).

Thanks to tjc who helped out with this as well. The info contained in this thread is distilled out of several other ones I started a while ago here and here if you want to read all the discussion.

To enable 4k jumbo frames on your LINUX box, do this:
Code:
# ifconfig eth0 mtu 4000


To enable 9k jumbo frames on your LINUX box, do this:
Code:
# ifconfig eth0 mtu 9000


(If you have a Windows box, you can change the MTU size from within the netcard driver.)

Each of these are only temporary utilization of JFs and your mtu will revert back to the standard value at the next system boot. Once you've tested the new frame size and are happy with the results, you can make it permanent by adding some lines to your /etc/network/interfaces. As always BACKUP files you are going to edit first like this:
Code:
# cp /etc/network/interfaces /etc/network/interfaces_virgin


Here is my /etc/network/interfaces:
Code:
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
# automatically added when upgrading
auto lo eth0
iface lo inet loopback

iface eth0 inet static
address   192.168.0.4
network   192.168.0.0
netmask   255.255.255.0
broadcast   192.168.0.255
gateway   192.168.0.1
mtu   4000


Note that you have to switch to static IP mode to use this. Someone out there may know how to do so in dhcp mode, but I never could get it to work. (Obviously, you'll need to substitute in your own values for the setup.)

To active the new changes without a reboot, simply do a:
Code:
# ifdown -a
#ifup -a


Enjoy!

---- Supplemental Information ----

Do jumbo frames really speed up file transfers? YES, they do in my experience. I have some older hardware (mythtv is running on a 1900+ athlon nforce2 based board and my win box is a 3200+ nforce2 based board) which may explain the giant boost of larger frame size.

Here are the results of my tests moving a large file and several smaller (~150 meg or so each) files from the win to myth and from myth to win via Samba. I've concluded that my network HUGELY benefits from a 4k MTU size. Here are the data:

Test 1/Crossover Cable (large file, 1,048,522 kb) via Samba
Code:
mtu=1500   time (sec)   Kb/sec
myth to win   48         21,844
win to myth   81         12,945
      
mtu=4000      
myth to win   44         23,830
win to myth   41         25,574
      
mtu=9000      
myth to win   49         21,398
win to myth   41         25,574

4k vs. standard   % Change
myth to win         9%   
win to myth         98%   

9k vs. standard      
myth to win         -2%   
win to myth         98%


Test 2/Crossover Cable (several small files, 1,283,439 kb total) via Samba
Code:
mtu=1500   time (sec)   Kb/sec
myth to win   59         21,753
win to myth   94         13,654
      
mtu=4000      
myth to win   51         25,165
win to myth   46         27,901
      
mtu=9000      
myth to win   57         22,516
win to myth   49         26,193

4k vs. standard   % Change   
myth to win         16%
win to myth         104%

9k vs. standard   
myth to win         4%
win to myth         92%


I've done the same test using my switch and the numbers are similar.

UPDATE - 29-Sep-07

win pc = Intel Q6600/P965 based machine
mythtv pc = AMD Ahtlon XP 1900+/nforce2 based machine

Here are the results of my tests moving a 3x large file (R5F1 iso) from the new pc to the mythtv pc and vise-versa. Here you see jumbo frames actually hurt the throughput when going to the myth box, but help it coming from the mythbox. Here are the data:

Large files through a switch
Code:
mtu=1500   time (sec)   Kb/sec
win to lin   98         17,333
lin to win   85         20,096
      
mtu=4000      
win to lin   126         13,571
lin to win   71         24,079
      
mtu=9000      
win to lin   151         11,314
lin to win   90         18,972

4k vs. standard   % Change
win to lin       -29 %   
lin to win        16 %
9k vs. standard      
win to lin       -54 %   
lin to win       -6 %


Conclusion: Results hardware specific. Enable them, test them, and see which is setting is best for your hardware.

EDIT: 15-Feb-09

I got an average speed of 26.6 MB/s on a 30+ GB xfer of approx. 1.1 GB files over this setup (4k jumbo frames). The slowest drive in the link was a Western Digital WD3200JB - 4-year-old, EIDE 320 GB (ATA100/7200 RPM/8 MB cache).

Image

Burst speeds (i.e. 300-400 meg files) through the network are about 80 % of the burst speed drive-to-drive which isn't too bad (approx. 30 MB/s network vs. approx 38 MB/s drive-to-drive).

_________________
Retired KM user (R4 - R6.04); friend to LH users.


Last edited by graysky on Sun Feb 15, 2009 2:19 pm, edited 7 times in total.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jul 14, 2007 4:54 am 
Offline
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1995
Location: /dev/null
Wow, posted about 8 months ago and no replies... has anyone found this useful?

_________________
Retired KM user (R4 - R6.04); friend to LH users.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jul 14, 2007 7:45 am 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Here's the script fragment I use to enable it on my box after every upgrade. It may very well be based on the information in this posting.

Code:
# enable jumbo frames
/sbin/ifconfig eth0 mtu 9000
grep 'MTU=9000' /etc/sysconfig/network-scripts/ifcfg-eth0 >/dev/null ||
echo 'MTU=9000' >>/etc/sysconfig/network-scripts/ifcfg-eth0
grep 'mtu 9000' /etc/init.d/bootmisc.sh ||
cat <<EOF >>/etc/init.d/bootmisc.sh
# enable jumbo frames
/sbin/ifconfig eth0 mtu 9000
EOF
grep 'mtu 9000' /etc/network/interfaces >/dev/null ||
sed -si.orig -e '/gateway/a\
        mtu 9000' /etc/network/interfaces

ifdown eth0
ifup eth0


Top
 Profile  
 
 Post subject:
PostPosted: Sun Jul 15, 2007 3:59 am 
Offline
Joined: Sat Feb 26, 2005 9:05 am
Posts: 160
Location: Ipswich, UK
last time I tried this, I had some stability issues that seemed to be due to different devices not supporting jumbo frames.

I've tried it again today, and it all looks good :-) so I'll stick with it for now and see how it goes.

So yeah, I've found it useful, thanks :-)

_________________
Mephi
-----------
R5.5
Opteron 2.Ghz
1Gb Ram, 200Gb Maxtor PATA
Avermedia 771 DVB-T
Hauppauge Nova-S2 (waiting to be installed)
MS MCE2 Remote


Top
 Profile  
 
 Post subject:
PostPosted: Sun Jul 15, 2007 1:49 pm 
Offline
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1995
Location: /dev/null
Cool man, glad it's working out for you. Did you do any before/after speed comparisons?

_________________
Retired KM user (R4 - R6.04); friend to LH users.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Sep 28, 2007 10:32 pm 
Offline
Joined: Fri Oct 20, 2006 12:04 pm
Posts: 905
Location: LA, CA
Thought I'd give this a try and I get this...

Code:
ifconfig eth0 mtu 4000
SIOCSIFMTU: Invalid argument


any ideas? I must be going something dumb.

Code:
ifconfig
eth0      Link encap:Ethernet  HWaddr 00:18:F3:45:3D:2C
          inet addr:192.168.1.13  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::218:f3ff:fe45:3d2c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1355065 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1050939 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1690348957 (1.5 GiB)  TX bytes:542604571 (517.4 MiB)
          Interrupt:16

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:566 errors:0 dropped:0 overruns:0 frame:0
          TX packets:566 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:58999 (57.6 KiB)  TX bytes:58999 (57.6 KiB)


Top
 Profile  
 
 Post subject:
PostPosted: Sat Sep 29, 2007 2:47 am 
Offline
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1995
Location: /dev/null
@TMS - I'm guessing either your driver or NIC doesn't support jumbo frames. If both did, and your switch/router didn't, I think you would lose connectivity whereas here you're getting an error.

1) What is the brand/model of your NIC?
2) What driver is it using?

_________________
Retired KM user (R4 - R6.04); friend to LH users.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Sep 29, 2007 3:31 am 
Offline
Joined: Sat Feb 26, 2005 9:05 am
Posts: 160
Location: Ipswich, UK
I had exactly that issue with a DLink GbE card. The driver or the card wouldn't support jumbo frames.

_________________
Mephi
-----------
R5.5
Opteron 2.Ghz
1Gb Ram, 200Gb Maxtor PATA
Avermedia 771 DVB-T
Hauppauge Nova-S2 (waiting to be installed)
MS MCE2 Remote


Top
 Profile  
 
 Post subject:
PostPosted: Sat Sep 29, 2007 4:41 am 
Offline
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1995
Location: /dev/null
I re-did some throughput tests using a much faster pc on the win32 side and the same mythbox pc. Now jumbo frames actually hurt the throughput in some cases so I switched back down to the standard MTU size (see my first post).

_________________
Retired KM user (R4 - R6.04); friend to LH users.


Last edited by graysky on Sat Sep 29, 2007 9:36 am, edited 1 time in total.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Sep 29, 2007 5:03 am 
Offline
Joined: Sat Feb 26, 2005 9:05 am
Posts: 160
Location: Ipswich, UK
With a 9k frame, throughput should be increased a bit because there is 5 less headers to send (In comparison with 1.5k frames) but the main performance increase is due to there being five less headers to calculate. This means that all nodes (PCs, Switches, Routers) have less processing to do.

_________________
Mephi
-----------
R5.5
Opteron 2.Ghz
1Gb Ram, 200Gb Maxtor PATA
Avermedia 771 DVB-T
Hauppauge Nova-S2 (waiting to be installed)
MS MCE2 Remote


Top
 Profile  
 
 Post subject:
PostPosted: Sat Sep 29, 2007 6:39 am 
Offline
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1995
Location: /dev/null
Perhaps in theory... I will say that CPU usage does increase as the frame size increases at least with older [slower] machines. My mythtv box is powered by an aged Athlon XP 1900+ for example.

_________________
Retired KM user (R4 - R6.04); friend to LH users.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Sep 29, 2007 7:52 am 
Offline
Joined: Wed Nov 16, 2005 8:55 pm
Posts: 1381
Location: Farmington, MI USA
Too Many Secrets wrote:
Thought I'd give this a try and I get this...

Code:
ifconfig eth0 mtu 4000
SIOCSIFMTU: Invalid argument


any ideas? I must be going something dumb.

Code:
ifconfig
eth0      Link encap:Ethernet  HWaddr 00:18:F3:45:3D:2C
Looks like you are using the onboard NIC on your M2NPV-VM? And therefore the forcedeth driver?

forcedeth supported Jumbo frames as of 0.36, but I haven't found anything that tells me that the onboard NIC supports it...


Top
 Profile  
 
 Post subject:
PostPosted: Sat Sep 29, 2007 10:17 am 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
The advantage to large frame size is the same as the advantage to large packet "windows" in a protocol. It cuts delays due to stalls when the maximum number of unacknowledged packets have been sent.

Basically as the communication media gets faster and faster stalls become a bigger and bigger issue. This is why when sending things on some ultra high speed networks they just transmit the whole thing 3 time right out of the box. Simply because it's faster than waiting fro the turn around time on the acknowledgment.

This is obviously a simplification, the full story on the information/communications theory reasons are the contents of an advanced IS course or text book.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Sep 29, 2007 10:05 pm 
Offline
Joined: Fri Oct 20, 2006 12:04 pm
Posts: 905
Location: LA, CA
slowtolearn wrote:
Too Many Secrets wrote:
Thought I'd give this a try and I get this...

Code:
ifconfig eth0 mtu 4000
SIOCSIFMTU: Invalid argument


any ideas? I must be going something dumb.

Code:
ifconfig
eth0      Link encap:Ethernet  HWaddr 00:18:F3:45:3D:2C
Looks like you are using the onboard NIC on your M2NPV-VM? And therefore the forcedeth driver?

forcedeth supported Jumbo frames as of 0.36, but I haven't found anything that tells me that the onboard NIC supports it...


Correct assumption, but I haven't checked the driver.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Sep 29, 2007 10:19 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
OBTW - Large frames may not be a win if your network has a very high error rate. Say you've got bad cables, lots of interference, or an iffy router. Otherwise they should markedly improve your throughput.

I know that on my household LAN they were the difference between machine to machine copies being network bound without and disk bound with. It's kind of freaky when you can copy files from machine to machine faster than you can copy them on the same machine. :-D


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 30 posts ] 
Go to page 1, 2  Next



All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu