Performance Testing: the Network Tester

Linksys says the SD2008 switch has a 8Gbps nonblocking fabric, and this number is also highlighted in the BCM5388 asic brochure. Let's put this claim to test. David
vs. Goliath -- Linksys SD2008 next to the Agilent N2X

An Agilent N2X network tester was used to generate traffic through the switch and analyze performance. The N2X I have access to has 8 GigE copper ports so the physical connection was trivial: just plug in the Linksys and get started.

Setup, ready, go

Ingress
port
Egress
port
15
26
37
51
62
73
For my first shot at testing, only 6 of the GigE ports were available. I used ports 1,2,3 which I assume are wired directly to the 5388's integrated PHYs, and ports 5,6,7 which logically should go in the 5464 PHYs. This was done to test both the integrated switch fabric and the inter-asic communication path.

To verify that I was sending traffic to the correct ports, I manually set all source and destination MAC addresses in a single stream per port, and verified that I was hitting the right ports by generating a distinctive amount of traffic per stream as can be seen in Figure 1.

Various packets size from 46 to 1500 bytes (6 ports)

Note: the N2X calculates ip throughput, so in the tests keep in mind the Ethernet II encapsulation overhead: one has to remove 14 bytes of header and 4 bytes of FCS = 18 bytes per frame. A 64-bytes frame carries only a 46 bytes ip packet. Throughput improves with larger packets sizes (less overhead).

Packets
size
Frames
size
Tx/Rx
pps
Throughput
Mbps
46641,488,095761.90
64821,225,490803.92
250268434,028930.56
1500151881,275987.00
The Linksys SD2008 delivers. The results show that the switch performs as advertised. With the smallest packet size (46 bytes) each port can send and recieve almost 1.5 Million packets per second (Mpps). IP packets throughput is not phenomenal, but this is explained by the Ethernet encapsulation overhead. Figures 2-5 are showing the raw results.

With 1500 bytes packets, the SD2008 delivers 987 Mbps per port, probably more than your average computer can handle. That's full-duplex, so you can multiply the figures by 2 since many vendors advertize their products capabilities that way.

During the tests, the Linksys SD2008 never dropped a packet, as can be seen in the results below, even at the maximum fill rate of 17.86 Mpps (using only 6 ports).

Various packets size from 46 to 1500 bytes (8 ports)

A few days later all 8 GigE ports were available so I redid the tests. This time I wired 1--8, 2--7, 3--4 and 5--6 in order to measure latency within the fabric and accross to the other PHYs.

Packets
size
Tx/Rx (pps)
per port
Tx/Rx (pps)
combined
Throughput (Mbps)
per port
Throughput (Mbps)
combined
Latency
(µs)
461,488,09523,809,529761.9012,190.482.50
641,225,49019,607,844803.9212,862.742.65
128753,01212,048,193879.5214,072.293.17
256425,1706,802,720931.9714,911.564.19
512227,2733,636,366963.6415,418.196.23
1000120,4241,926,784980.7315,691.7210.13
150081,2751,300,399987.0015,792.0514.15

Again, the results speak for themselves. I produced more data for the 8-port test than for the 6-port, so I emphasized the three most important figures. The fill rate is exceptionnal with 23.8 million packets per second (Mpps). The combined throughput is 15.72 Gb/s while the smallest delay observed was 2.5µs.

Jumbo frames support, or lack thereof

Packets
size
Frames
size
Tx/Rx
pps
Throughput
Mbps
-153680,334987.14
-153700
To my dismay, the SD2008 cannot switch Ethernet frames greater than 1536 bytes. Traffic consisting of 1537 bytes frames is dropped, period. The Broadcom 5388 ASIC product brief mentions that jumbo frames are implemented on silicon, but for whatever reason Linksys has not enabled this feature. Figures 6-7 show traffic sent at 1536 and 1537 bytes.

1536 bytes for an Ethernet frame is still good, as it leaves room enough for the most popular L2 encapsulation protocols if ever needed on a switch this size.

Update: since I put this page up, I got confirmation from Linksys insiders that the SD2008-v1 does not support jumbo frames. I was also told that this features would be enabled on the SD2008-v2. This product isn't FCS as of writing, so wait and see.

Screenshots (6 ports)

MAC addresses and ports assignment verification 46 bytes packets 64 bytes packets 250 bytes packets 1500 bytes packets 1536 bytes packets, still OK 1537 bytes packets, failure

Last words

Pros:

Cons:

Future research

Other products in the $100 price range recently caught my attention. These are:

Right now I have no need for more Gigabit ports at home, but who knows. You never know what might happen in the future, perhaps even just for fun.

<< Back to Part 1: Opening the Switch
<< Back to Part 2: Replacing the Fan

gaetan@soltesz.net