Addison Snell of Intersect360 Research presented the firm’s findings on HPC Software Environments in 2014 to the members of the HPC500, an elite user group of organizations that represent a worldwide, diverse collection of established HPC professionals from a cross-section of academic, government, and commercial organizations.

The presentation drew on findings from several different reports. Intersect360 Research recently released two HPC User Site Census reports, one on applications and one on middleware. The reports examined the supplier, products, and primary usage of the application software and middleware reported at a number of sites over the previous year. The presentation also included findings from the research firm’s annual HPC user budget map, which tracks spending patterns and shows a percent of spending by category, as well as from a special study on “The Big Data Opportunity for HPC,” which surveyed both HPC and non-HPC enterprises (through a partnership with Gabriel Consulting) on Big Data applications and infrastructures.

Intersect360 Research segmented their research this year into High Performance Business Computing (HPBC) and High Performance Technical Computing (HPTC) super-segments. About three-fourths of applications surveyed were HPTC packages, such as structural mechanics, computational fluid dynamics, and weather/environmental modeling. The remaining one-fourth were HPBC packages, including financial analysis, business intelligence, and MapReduce-based applications. More than half of HPBC packages were business intelligence applications.

Eighty percent of the applications — a mix of in-house, packaged, and open-source applications — were mentioned only once in the HPC Applications survey, indicating the diversity of application needs and the experimental nature of the HPC market. This trend also held true with middleware, where 62% of applications were only mentioned once. The amount of unique middleware packages tracked more than tripled over the previous year with the addition of more than 340 packages, for a total of 496 packages.

Open Source emerged as an ever more important component of the HPC software landscape, especially among commercial HPC users. Open Source was the leading source of applications with 38% of software mentions, its rise driven largely by pricing models and new categories.

The increasing dominance of Open Source presents a challenge for Independent Software Vendors (ISVs), who are already struggling with the inevitable growth of multi-core and many core systems. These systems have caused stress on their business model due to the evolution of new processing paradigms and the resulting changes in licensing models for ISVs. This is especially visible in the Computational Fluid Dynamics space, where Open Foam is the fastest growing application in the HPC market, challenging the leading ISV package, Fluent by ANSYS.

Traditional market thinking is that software in commercial HPC is dominated by commercial ISVs, but it is instead split almost equally with one-third apiece for in-house, commercial ISVs, and open source solutions. This differs between vertical markets, of course — for example, biosciences tends to use more commercial applications, whereas energy slightly favors in-house applications. HPC usage in commercial markets overall is growing, especially as compared to the public sector.

Big Data provides a breakout opportunity for HPC, and is currently comprised primarily of in-house applications. For non-HPC sites (businesses who don’t think of themselves as using HPC) the use of in-house applications is diminishing as Open Source solutions are increasing, due to the growing acceptability of Hadoop-based platforms.

Over time, more commercially available applications are expected, which will result in fewer in-house and open-source applications. This is a familiar pattern also seen over the decade-long span of ERP adoption, which began as high-priced consulting for large enterprise and ended with any enterprise organization being able to purchase and implement an existing package.

Last month Intersect360 Research was commissioned by the U.S. Council on Competitiveness to survey 100+ HPC-using companies in industries such as manufacturing, energy, finance, pharmaceuticals and entertainment. The interviews formed the basis of Solve. The Exascale Effect: The Benefits of Supercomputing Investment for U.S. Industry, a publication of the High Performance Computing (HPC) Initiative.

The findings in Solve that pertain to software focused on the barriers to HPC scalability. Many of those surveyed thought they could use 10x greater scalability, and they cited scalability of software — especially in regards to new models of parallelism (multi-core and many core) — as the major limitation to achieving this goal.

The presentation concluded with a lively Q&A session with HPC500 members. The HPC500 is comprised of the foremost people who bring high performance computing technology to bear on challenging problems in science, engineering, and business. HPC500 members are leading organizations at the vanguard of their areas of specialization, spanning geographies, budget sizes, and application areas. To apply for inclusion please visit:

You have no rights to post comments


Login And Newsletter

Create an account to access exclusive content, comment on articles, and receive our newsletters.


This work is licensed under CC BY-NC-SA 4.0

©2005-2023 Copyright Seagrove LLC, Some rights reserved. Except where otherwise noted, this site is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International. The Cluster Monkey Logo and Monkey Character are Trademarks of Seagrove LLC.