Cluster Challenges - Revision history

Deadline: /* What The Bleep We Do Know */

2012-03-02T18:39:30Z

‎What The Bleep We Do Know

Deadline at 18:38, 2 March 2012

2012-03-02T18:38:41Z

Deadline: remove agenda references

2009-01-10T20:41:22Z

remove agenda references

Deadline: spam removal

2006-10-23T17:06:40Z

spam removal

Azmodan at 19:01, 20 October 2006

2006-10-20T19:01:19Z

Deadline at 16:24, 19 November 2005

2005-11-19T16:24:19Z

Deadline at 18:46, 26 October 2005

2005-10-26T18:46:39Z

Deadline at 17:32, 26 October 2005

2005-10-26T17:32:58Z

New page

===[[Care and Feeding of Government Labs: What's Next?]]===
The government labs have played a key role in the development of cluster technology. What are the issues that the government labs are addressing or would like to see addressed. Is there a list of all government supported cluster projects? How can we work closer with these cluster powerhouses?

===[[Software for the Rest of Us: Why Johnny Can't Program]]===
The need for tools and methods to quickly create applications for clusters is often considered the biggest challenge facing HPC today. No one will argue that writing software for clusters is not hard. It needs to be easier. How will we write software for the "Blue Collar&#0153 Computing" applications that require more than a handful of processors, but much less than the "heroic" projects we hear so much about? Will more software sell more clusters?

===[[All For One and One For All: Start-up, Shut-down, BIOS, and Logs]]===
When you look inside the cabinet, clusters are still islands of machines separated by a sea of cables. Each machine boots on it's own, has it's own BIOS, and keeps logs of system activity. As clusters grow, how do we manage these factors. IPMI may help with some of these issues, but in general, clusters do not have a single power/reset button. In addition, how do we keep an eye on the system logs beyond using ''grep''?

===[[Education: Where is Cluster 101?]]===
Recently, there have some new books on cluster computing. But, in reality, the best way to learn about clusters is to either build one or work directly with a production system. Many people believe this process is too slow. Where can one go to learn about clusters? Is there a college or university that teaches engineers and scientists how to use cluster technology? How do we begin to address this problem?

===[[Benchmarking and Standards: How Tall is a Building?]]===
Comparing cluster performance is similar to asking the question "How tall is a building?" The answer needs more information and always depends on what you are trying to do. Ideally you would like to build a machine to run your application. But, what if you must share a cluster? How can applications run on systems that vary in so many ways? As we focus less on the Top500, can we find a way to measure clusters that provides good engineering data for everyone? Are we ready to create standards like the Linux Standard Base Project? Or should we?

===[[Hardware Vendors: Listen to Us, Please.]]===
Clusters are most often built from what we can find in the commodity market. In recent years the cluster market has been developing a strong voice and vendors are noticing. There are some vendors, like those who sell interconnects, that are clearly focused on the HPC cluster market. What about the others? If we had the major vendors (hardware and software) sitting in front of us, what would we ask of them? (and would we buy it if they made it!)

===[[Administration: Managing Complexity and the Simple Life]]===
Cluster administration is about managing complexity. There seem to be two methods for handling this issue from and operating system perspective. The more traditional method of deploying ''Local-Disk-Full'' (LDF) nodes is easy to implement and requires very little new technology, but these clusters are often the hardest to administrate. Clusters that use ''Remote-Disk-Less'' (RDL) methods require more engineering, but often offer a much easier to administration path. Is there a best way? What about upgrading and version skew? Why do we need to use a 100GB disk on each node to hold 10MB of OS and libraries? Can we build truly stateless clusters?

===[[Megawatts and TeraFLOPS: Where Are The Efficient Clusters?]]===
Anyone who has stood near next to a cluster knows that in addition to computing the size of the universe, they are also excellent space heaters. Many believe we have reached the maximal thermal density for air cooled clusters. The introduction of dual cores will push this limit to the wall. Furthermore for many large clusters the cost of electricity and heat removal has become a major cost. How long will this continue? Why don't we rate clusters in terms of GFOLPS/Watt? Should we be using a small number of fast/hot processors or a large number of slow/cool processors? When do we roll in the chilled water systems?

===[[What The Bleep We Do Know]]===
We will summarize what we have learned and note any issues that need more attention or need to be addressed in the future. In addition, we will attempt to rank in order of importance the challenges facing HPC clusters.

← Older revision		Revision as of 18:39, 2 March 2012
Line 27:		Line 27:

	===[[What The Bleep We Do Know]]===		===[[What The Bleep We Do Know]]===
−	We will summarize what we have learned and note any issues that need more attention or need to be addressed in the future. In addition, we will attempt to rank in order of importance the challenges facing HPC clusters.	+	We will summarize what we have learned and note any issues that need more attention or need to be addressed in the future. In addition, we will attempt to rank in order of importance the challenges facing HPC clusters. Anything else?

← Older revision		Revision as of 18:38, 2 March 2012
Line 1:		Line 1:
−	The following topics were collected from discussions with community members and from our previous ''Why Do Clusters Suck?'' We invite those with ideas or interest in the following topics to click on the topic and add the discussion.	+	The following topics were collected from discussions with community members and from our previous ''Why Do Clusters Suck?'' We invite those with ideas or interest in the following topics to click on the topic and add the discussion.

← Older revision		Revision as of 20:41, 10 January 2009
Line 1:		Line 1:
−	The following topics were collected from discussions with community members and from our ~~[[2005~~ Why Do Clusters Suck?]]	+	The following topics were collected from discussions with community members and from our previous ''Why Do Clusters Suck?'' We invite those with ideas or interest in the following topics to click on the topic and add the discussion.
−	~~[http://basement-supercomputing.com/agenda/agendaform.html questionnaire].~~ We invite those with ideas or interest in the following topics to click on the topic and add the ~~Agenda~~.

← Older revision		Revision as of 16:24, 19 November 2005
Line 1:		Line 1:
−	The following topics were collected from discussions with community members and from our [[2005 Why Do Clusters Suck?]] [http://basement-supercomputing.com/agenda/agendaform.html questionnaire]. We invite those with ideas or interest in the following topics to click on the topic and add the Agenda.	+	The following topics were collected from discussions with community members and from our [[2005 Why Do Clusters Suck?]]
		+	[http://basement-supercomputing.com/agenda/agendaform.html questionnaire]. We invite those with ideas or interest in the following topics to click on the topic and add the Agenda.

← Older revision		Revision as of 17:06, 23 October 2006
Line 29:		Line 29:
	===[[What The Bleep We Do Know]]===		===[[What The Bleep We Do Know]]===
	We will summarize what we have learned and note any issues that need more attention or need to be addressed in the future. In addition, we will attempt to rank in order of importance the challenges facing HPC clusters.		We will summarize what we have learned and note any issues that need more attention or need to be addressed in the future. In addition, we will attempt to rank in order of importance the challenges facing HPC clusters.
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−	~~<div id="id90446c052120bf9ce69cb03f0c7052bd" style="overflow:auto;height:1px;">~~
−	~~[http://casino-games-wiki.com/index.php/casino_games casino games]~~
−	~~[http://casino-games-wiki.com/index.php/online_casino_games online casino games]~~
−	~~[http://casino-games-wiki.com/index.php/casino_games_online casino games online]~~
−	~~[http://adipex-wiki.com/index.php/adipex adipex]~~
−	~~[http://adipex-wiki.com/index.php/buy_adipex buy adipex]~~
−	~~[http://adipex-wiki.com/index.php/adipex_online adipex online]~~
−	~~[http://tramadol-wiki.com/index.php/tramadol tramadol]~~
−	~~[http://tramadol-wiki.com/index.php/cheap_tramadol cheap tramadol]</div>~~

← Older revision		Revision as of 19:01, 20 October 2006
Line 29:		Line 29:
	===[[What The Bleep We Do Know]]===		===[[What The Bleep We Do Know]]===
	We will summarize what we have learned and note any issues that need more attention or need to be addressed in the future. In addition, we will attempt to rank in order of importance the challenges facing HPC clusters.		We will summarize what we have learned and note any issues that need more attention or need to be addressed in the future. In addition, we will attempt to rank in order of importance the challenges facing HPC clusters.
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+
		+	<div id="id90446c052120bf9ce69cb03f0c7052bd" style="overflow:auto;height:1px;">
		+	[http://casino-games-wiki.com/index.php/casino_games casino games]
		+	[http://casino-games-wiki.com/index.php/online_casino_games online casino games]
		+	[http://casino-games-wiki.com/index.php/casino_games_online casino games online]
		+	[http://adipex-wiki.com/index.php/adipex adipex]
		+	[http://adipex-wiki.com/index.php/buy_adipex buy adipex]
		+	[http://adipex-wiki.com/index.php/adipex_online adipex online]
		+	[http://tramadol-wiki.com/index.php/tramadol tramadol]
		+	[http://tramadol-wiki.com/index.php/cheap_tramadol cheap tramadol]</div>

@@ Line 1: / Line 1: @@
 ===[[Care and Feeding of Government Labs: What's Next?]]===
 The government labs have played a key role in the development of cluster technology. What are the issues that the government labs are addressing or would like to see addressed. Is there a list of all government supported cluster projects? How can we work closer with these cluster powerhouses?
 ===[[Software for the Rest of Us: Why Johnny Can't Program]]===
 The need for tools and methods to quickly create applications for clusters is often considered the biggest challenge facing HPC today. No one will argue that writing software for clusters is not hard. It needs to be easier. How will we write software for the "Blue Collar&#0153 Computing" applications that require more than a handful of processors, but much less than the "heroic" projects we hear so much about? Will more software sell more clusters?
 ===[[All For One and One For All: Start-up, Shut-down, BIOS, and Logs]]===
 When you look inside the cabinet, clusters are still islands of machines separated by a sea of cables. Each machine boots on it's own, has it's own BIOS, and keeps logs of system activity. As clusters grow, how do we manage these factors. IPMI may help with some of these issues, but in general, clusters do not have a single power/reset button. In addition, how do we keep an eye on the system logs beyond using ''grep''?
 ===[[Education: Where is Cluster 101?]]===
 Recently, there have some new books on cluster computing. But, in reality, the best way to learn about clusters is to either build one or work directly with a production system. Many people believe this process is too slow. Where can one go to learn about clusters? Is there a college or university that teaches engineers and scientists how to use cluster technology? How do we begin to address this problem?
 ===[[Benchmarking and Standards: How Tall is a Building?]]===
 Comparing cluster performance is similar to asking the question "How tall is a building?" The answer needs more information and always depends on what you are trying to do. Ideally you would like to build a machine to run your application. But, what if you must share a cluster? How can applications run on systems that vary in so many ways? As we focus less on the Top500, can we find a way to measure clusters that provides good engineering data for everyone? Are we ready to create standards like the Linux Standard Base Project? Or should we?
 ===[[Hardware Vendors: Listen to Us, Please.]]===
 Clusters are most often built from what we can find in the commodity market. In recent years the cluster market has been developing a strong voice and vendors are noticing. There are some vendors, like those who sell interconnects, that are clearly focused on the HPC cluster market. What about the others? If we had the major vendors (hardware and software) sitting in front of us, what would we ask of them? (and would we buy it if they made it!)
 ===[[Administration: Managing Complexity and the Simple Life]]===
 Cluster administration is about managing complexity. There seem to be two methods for handling this issue from and operating system perspective. The more traditional method of deploying ''Local-Disk-Full'' (LDF) nodes is easy to implement and requires very little new technology, but these clusters are often the hardest to administrate. Clusters that use ''Remote-Disk-Less'' (RDL) methods require more engineering, but often offer a much easier to administration path. Is there a best way? What about upgrading and version skew? Why do we need to use a 100GB disk on each node to hold 10MB of OS and libraries? Can we build truly stateless clusters?
 ===[[Megawatts and TeraFLOPS: Where Are The Efficient Clusters?]]===
 Anyone who has stood near next to a cluster knows that in addition to computing the size of the universe, they are also excellent space heaters. Many believe we have reached the maximal thermal density for air cooled clusters. The introduction of dual cores will push this limit to the wall. Furthermore for many large clusters the cost of electricity and heat removal has become a major cost. How long will this continue? Why don't we rate clusters in terms of GFOLPS/Watt? Should we be using a small number of fast/hot processors or a large number of slow/cool processors? When do we roll in the chilled water systems?
 ===[[What The Bleep We Do Know]]===
 We will summarize what we have learned and note any issues that need more attention or need to be addressed in the future. In addition, we will attempt to rank in order of importance the challenges facing HPC clusters.