Xnet SJF Report 2005 - Booth Setup 919B & 2158

SC05 Xnet Super Jumbo Frames (SJF) Links

Bill Rutherford, Lixin Liu, Randy Ottman, Martin Siegert, Arnold Sodder and Brian Corrie arrived around noon or so on Sunday. All exhibitor badges were arranged efficiently in short order. Both router were shipped to the correct booths. The BigBangwidth optical switch was shipped to 2158. One of the first factors noted was the yellow SMF spools for all the dark fibre connections. Apparently this was not noted until the last minute giving us little time to adapt. The routed connection was of orange MMF cable allowing direct connection to a server for testing. The routed connection was working in short order. In response to the SMF dark fibre a somewhat hasty note sent to Loki Jorgenson by Bill Rutherford resulting in three 10G LR NICs with SC connectors appearing on Monday night. The X8 router was set up by Randy and Arnold in 2158 by Sunday evening proceeding smoothly despite the absence of racks. The 2158 power connections for the router were of correct disposition and location. Two AX/4000 XLP and two server blades were unloaded and carried by hand to 2158 from Martin's vehicle. Apparently some GBICs/SFPs which might have been used for SMF were not allowed entry from Canada with Brian Corrie who came in a separate vehicle. The ER16 router was setup Monday morning, resting on the floor in 919B as no plaform of sufficient substance could be located to mount it's considerable mass. The 919B power connections for the router were of correct disposition and location. Loki Jorgenson arrived Monday morning (wearing a kilt). Gary Finley arrived Monday noon. A Freeman order was placed on Monday by Paul Daspit for swap of 2 x 6 table in 919B for 2 x 3 table, due to space limitations. Also an order was placed by Paul Love for a blue cloth for the X8 pedestal in 2158. The AX/4000 XLP 1G cards were moved to a single chassis and located in 919B with a successful test run on the ER16 at 64000 bytes after config by Arnold. The AX/4000 control software was copied to Loki's notebook and the automated powerpoint finalized. The Xnet sign for 919B was mounted with velcro tape in the same locale as the 919A signage. This completed the main exhibit for 919B. The lightpath connection to Nortel/540 was routed through the X8 using an LX SFP with a LC-SC jumper adapter, of which 3 were kindly loaned us by SCInet. Considerable troubleshooting of the fibre connection by Nortel, Thomas Tam, Arnold Sodder Lixin Liu and Martin Siegert finally saw it working by Tuesday afternoon via the X8, with successful demos thereafter. All parties are to be congratulated on the difficult task of getting the lightpath to work properly in such a short time frame. The server blade for 919B was opened and the 1G NIC swapped out for the 10G NIC. The SMF dark fibre SC connectors were connected to the NIC using the DFB pair corresponding to DFB in 2158. The server blade for 2158 was opened and the 1G NIC swapped out for the 10G NIC. The SMF dark fibre SC connectors were connected to the NIC using the DFB pair corresponding to DFB in 919B. It was noted that the link light came on and assumed a valid circuit was present. Subsequent efforts revealed the link not to be working over the DFB link or the DFC link and both servers were moved to 2158 where they were connected via short SC-SMF-SC jumpers. Investigation revealed a working link via jumper but not over the DFB/DFC links. A trouble ticket was issued to investigate on Tuesday. A SCInet team was on site shortly after the ticket was issued and efficiently investigated the DFB and DFC links. The losses measured were apparently in the 4 db range due to numerous intermediate connectors, however more than adequate for the link. The DFB and DFC were connected in 919B providing a loopback to both machines in 2158. Subsequent investigations seemed to verify the 10G NICs could communicate over a short jumper but not over the longer loopback link for some reason. The empty AX/4000 chassis in 2158 was swapped out by Greg Goddard for a chassis with two 10G Generator/Analyzer pairs. The model of these was of WAN only type thus prohibiting testing of the 10G ports on the X8, however enabling testing over the BigBangwidth Optical Switch setup by Gary Finley.

Xnet SJF Report 2005 - Exhibit Performance

Both the Xnet displays in 919B and 2158 attracted reasonable attention. The 919B exhibit operated continuously for the entire period alternately manned by Loki Jorgenson, Bill Rutherford, Thomas Tam(short duration) and Randy Ottman. The 2158 Xnet exhibit ran from 1:30 to 4:30 on Tuesday and from 2:30 to 5:30 on Wednesday and was manned by Bill Rutherford. Randy Ottman manned 919B on Thursday from 10:00 to 4:00. The location of the Xnet SJF booth was ideal as a relatively continuous stream of interested parties were engaged. Gary Finley fielded questions at 2158 when the Xnet demo was not running.

Xnet SJF Report 2005 - Committee Recommendations

Overall the process of setting up the exhibits was handled in an exemplary and professional manner. The unfortunate change in fibre type for the interbooth runs was compounded by lack of available adapters and somewhat unexpected difficulties with the fibre and 10G NICs. Considerable extra time was spent by Lixin, Martin and Loki troubleshooting the 10G NICs. We made a valiant effort to adapt and partially succeeded. Clearly it might be wise to double check the fibre type in future.

Xnet SJF Report 2005 - Research Follow-up

It is our intention to complete the 16k testing initiated at SC 05 (preliminarily at SFU at 1G) and extend it to the wide area. There is an interesting observed behavior at 12K exhibited by the 1G Intel cards that we are investigating. As well, we wish to move to an environment where super-jumbo is more relevant - non-LAN latencies (> 10 ms). Some discussion wrt a 64k path of significant round trip time to provide the critical validation of the ability of larger packet size to compensate for reduced throughput indicated that the limiting factor of NLR to support such a test may be the 15808 MTU of 10,000 bytes. As such it is an open question as to how best to proceed with a 64k test over a large latency. Clearly the potential availability of 2 ER16 routers, possibly with both 1G and 10G interfaces at some considerable distance linked by a non blocking 64k circuit would be ideal. This of course assumes the availability of 64k NICs at some point in the future.

The Xnet SJF project team would like to acknowledge the generous participation and supporting infrastructure from Enterasys, Spirent, BCNET, CANARIE, CA*net4, HEPnet Canada, IRMACS, Netera, Simon Fraser University, WestGrid, University of Alberta Subatomic Physics and University of Victoria Physics.