As one of the last three readers of Cluster Monkey, do not despair. I apologize for the rather low frequency of content over the last ten months. It is entirely my fault. I have been so busy with some other projects, Cluster Monkey got lost. I have not had time to post interesting news or write any articles. This situation is about to change.
So What Have I Been Doing?
I have been working in three main areas, Multi-core Cluster Design, Hadoop/Spark, and 3D Printing. There are other things, but these are bubbling to the top right now. I'll mention a few things about each of these topics. Look for more in-depth coverage in the coming months.- Multi-Core Cluster Design - Ever since multi-core hit the scene, I have been interested in how these processors can be effectively used in HPC. In the past, I have written several Benchmarking A Multi-Core Processor For HPC and Counting Your Effective HPC Cores. I have continued testing and now have a pile of benchmark results for I7/i5 Haswell, Sky Lake, Coffee Lake, plus some Xeon numbers as well (also have older Sandy and Ivy Bride data). Interesting results. Coming soon.
- Hadoop/Spark - A lot going on here. Coming from an HPC perspective, there are many things I have learned about how Hadoop does clusters. Along the way I have been teaching and writing about Hadoop and Spark. One of the impediments has been providing a platform to try programming examples. I have recently put a some work into a Linux Hadoop Minimal virtual machine. The machine is available here. My goal is to provide a low resource usable environment for programmers to learn Hadoop and Spark.
- 3D Printing - A while back I started the Limulus Project to create a small low noise/power/heat that provided cluster HPC performance. While working on the project, I began to believe the impediment to desk-side cluster was not so much high tech, but mostly a low tech packaging issue. Indeed, all that was really needed were the right metal brackets and you could pack a bunch of motherboards into (big) standard cases. Indeed, small desk-side HPC and Hadoop clusters are available from Basement Supercomputing (Full disclosure, I work for Basement). Here is the interesting part, specialized sheet metal brackets are expensive and have long lead times (and design changes can render existing parts useless). My research has found that all of these custom parts can be 3D printed. I have a great story to tell about this. I can now fit anything in any box given heat and power restrictions because I "own" the bracket design and manufacturing process--much like Linux and open software where I can "own" the creation and implementation process. Think about it.
We Are Looking for Writers/Editors!
If you have any interest in writing or editing for ClusterMonkey, please contact me. We can't help you buy that new vacation home, but we can offer a hearty thank-you and a shot at increasing you HPC street cred. (As an aside, we do get occasional marketing fluff disguised as articles thrown our way. These are quickly returned with a well crafted "let down" letter.)I am interested in HPC/Hadoop & Spark/Machine Learning ideas, opinions, projects, benchmarks, and those things that may be off the beaten track. In particular, if you want to promote and new (or old) idea, share some experiences, or even get up on your soap box and rant about some cluster topic, this is your place. I like to post useful things that users, admins and those people called "devops" like to hear about. Now back to putting together my Julia course (maybe should be another bullet point!)
