As the Head Monkey around here, I often get questions sent to me about clusters. I thought it might make sense to share some of the questions (and answers). Please feel free to comment and offer your insights as well. Here is today's question.

Question: I am having trouble finding current information on hyper-threading and clusters, or even hyper-threading and heavy loads that is not at least 2 years out of date. I am about to overhaul our engineering cluster and would really like to find some information on the current state of things before I enable the HT ability on the cluster. Read on for my answer.

By the way, I'm open to other questions as well. Head on over to my contact page and drop me line. I'll also try an reach out to some of the more seasoned cluster jocks for answers as well. And finally, you can always post and search the adept Beowulf Mailing List. Oh, and try and keep the questions about HPC clusters.

Answer: Hyper-threading is not really used on newer chips. It was used on the Pentium 4 line, but the newer multi-core Pentium M series (Merom, Conroe, and Woodcrest) do not have hyper-threading. There is some talk by Intel that the new Nehalem will again use hyper-threading.

In term of HPC, the consensus is that you want to turn off hyper-threading on compute nodes. The reason is that Linux will "see" two CPUs for each processor (The P4 is a single core). But it is not really two processors and you should not overload the processor with two "heavy" compute jobs because they share the same FPU.

Hyper-threading seems to work best when you have more threads than cores, and the threads are waiting on IO, that way the second virtual processor can "fill in the gaps". If the head node is serving NFS to the nodes, it may make sense to turn on hyper-threading (on the head node).

Further Information:

• Intel Hyper-threading Page
• Wikipedia - Hyper-threading