Raw Ethernet
There have been many efforts aimed at developing alternative ways for communicating over Ethernet networks besides using the IP, TCP, and UDP protocols. Several of these methods continue to be developed. On July 20, 2004, Simone Saravalli posted that he was interested in studying raw Ethernet communication on Intel P4's. Robert Brown then asked if Simone was interested in using raw Ethernet packets on a flat bridged network for point-to-point communications. He thought such a project was beneficial as a learning tool for understanding networks and networking protocols. He suggested that Simone could probably skip the IP protocol, but would have to deal with some of the functions that TCP provides, such as, reliable packet transmission, sequencing, check summing, etc.
Several people posted some ideas for getting started including taking a look at the GAMMA Project and HyperSCSI projects.
Simone responded that he was indeed studying the effects of raw Ethernet on point-to-point communication. He reported that throughput was good, but that reliability was an issue because of the lack of the things that TCP provides. Robert Brown responded that the project may have the potential to provide some good GPL drivers that may reduce latency for network communication. He also thought Simone would have to develop a library that emulates the standard socket libraries and transport layer, perhaps using the arp tables to provide a mapping between hostname and Ethernet number on the flat network.
Daniel Ridge suggested adding your node number into the MAC address of the NIC (most NICs can be programmed with a specific MAC) and then don't worry about the hostname to MAC mapping (this ignores the IANA rules). Gerry Creager thought that Simone should just use IPv6 and encode the node number. Robert thought that this would not be a good idea because it would limit the networking to a private flat network. He went into some depth to explain why including some old war stories.
Tim Mattox posted to suggest that if you want to change the MAC address of a NIC, then you should make sure you set the Locally Administered bit in the MAC. If you do this and clear the Group bit, then you will never conflict with a factory set MAC. Tim went on to say that as long as you have a firewall or router between your modified MAC then you will never have to worry about an outside conflict.
Daniel Ridge jumped back in to say that he thought these types of projects were good because they take commodity networking components and then ask the question, " I wonder what you get if you think about them differently." He also thought that this kind of project was perfect for clusters that have a private network. He also pointed out that the ifconfig command can change the MAC address of a NIC as well.
John Hearns posted that if you wanted to do this sort of thing with clusters that use PXE booting, then you might have to do some fancy footwork to boot first and then change the MAC address for the private network. Tim Mattox posted that he has been experimenting with changing MAC addresses for some time with his FNN (Flat Network Neighborhood) cluster configuration. He had some very good insights into changing MAC's and then rebooting. He also pointed out that the next version of the Warewulf Cluster Distribution will support multiple MAC's per node (very useful in this situation).
Beowulf: Good Grid Computing Classes?
There seems to be a definite lack of formal courses where one can learn cluster design, administration, and programming. On August 11, 2004, Michael Jastram asked about courses that would address grid computing and other aspects of clustering. His employer had a tuition assistance program and Michael wanted to take advantage of that program.
Brent Clements suggest the Linux Institute to see what they might offer. Dean Johnson suggested the old fashion approach of " just do it." This method is quite simple -- buy/find a couple of machines, connect them and start trying the software and asking questions. Dean had some very good ideas about trying things to learn as much as you can. Robert agreed with Dean's ideas (and comments about getting hardware past the "management").
Michael Hanulec mentioned that the Cornell Theory Center taught courses on Windows clusters. He went on to suggest using either Oscar, Rocks, or Warewulf to get started with cluster management tools.
Robert thought that subscribing to I
Finally, Glen Gardner echoed the sentiment of others that not many courses existed for this kind of thing. Sounds like a business opportunity to me! Update: The Advanced Research Computing (ARC) team at Georgetown University are running cluster courses. They have successfully run Introduction to Beowulf Design, Planning, Building and Administering trainings several times.
| Sidebar Two: Technology Mentioned In This Column |
This article was originally published in ClusterWorld Magazine. It has been updated and formatted for the web. If you want to read more about HPC clusters and Linux you may wish to visit Linux Magazine.
Jeff Layton has been a cluster enthusiast since 1997 and spends far too much time reading mailing lists. He can found hanging around the Monkey Tree at ClusterMonkey.net (don't stick your arms through the bars though).
