Article Index

Digging for Data

As the amount of on-line financial data is increasing each year, data mining has become one method for learning trading rules from this data. Once a trading rule has been established from historical data, it can assist with forecasting S&P 500, exchange rates, stock directions, and rating stocks for portfolio. Rules are generated by a number of ways including statistical analysis, neural networks, ruled-based systems , decision-tree systems , and fuzzy-logic methods. Clearly the large the amount of data and large amount of processing requires high end resources like clusters. Parallel data mining can sometimes be done using SQL, but more often advanced tools are used to look for relationships that are not apparent in the data. For those interested, there are open source applications available from the Australian National University that provide tools to do parallel data mining on Linux clusters (see Resources).

Tic, Tic, Tic

High performance databases are another area where clusters can provide answers to hard questions. An interesting application in this area is the use of the Kdb from Kx Systems to scan stock ticks. Every time a stock is traded the price is recorded on the "ticker". What once was a mechanical printer and paper tape is now a huge database which can easily exceed four million trades each day for just the NYSE and the NASD alone. Searching this data is monumental task and is important to "time series analysis" where a stock's past series of trade and quote prices can assist in future trades.

Kx has, in the past, demonstrated Kdb running on a Linux cluster consisting of 50 CPUs, 50 Gigabytes of RAM, and 300 GB of storage. They loaded 2 years of NYSE tick data (2.5 billion trades and quotes) onto the cluster and where able to archive a sub-second query response rate on all publicly traded stocks. In addition, multi-dimensional aggregations were produced in 5 to 20 seconds.

Quantlib Anyone?

If you are interested in learning how to become an options trading baron or would like to play with some computational finance tools you can take a look at Quantlib. QuantLib is library for modeling, trading, and risk management in real-life. It is released under the modified BSD License. QuantLib is written in C++ with a clean object model, and is then exported to different languages such as Python, Ruby, and Scheme. An initial Excel add-in is also available. There are ports to the .NET framework in C#. Bindings to other languages (including Java), and ports to Gnumeric, Matlab/Octave, S-PLUS/R, Mathematica, COM/CORBA/SOAP architectures, FpML, are under consideration.

Quantlib offer tools that can be used for building your own applications. Some of the components include, Lattice methods, finite differences, Monte Carlo, Short-rate models, Currencies and FX (exchange) rates, and Instruments and pricers. There are also source code example applications using the Quantlib library.

Only the Beginning

Clusters are only now coming onto their own in the finance industry. In addition to being the workhorse of derivative pricing and risk calculation for large and small institutions, they are finding their way into many other areas that include, desktop analysis, real time trading monitoring, data mining, large database analysis, neural nests, and genetic algorithms. As clusters continue to grow and develop they will continue to earn their keep and provide the financial markets levels of analysis never thought possible.

This article was originally published in ClusterWorld Magazine. It has been updated and formated for the web. If you want to read more about HPC clusters and Linux you may wish to visit Linux Magazine.

Sidebar One: The Black-Scholes Formula

You will ofter hear the Black-Scholes equation mentioned when people are discussing the finance markets. The Black-Scholes equation is the workhorse of the finance community and provides a method to determine how much a call option is worth at any given time. A call option is a contract between two parties that allows the buyer the right but not the obligation to buy an agreed quantity of a particular commodity or financial instrument at an agreed upon price for an agreed upon time.

The power of the Black-Scholes model is that it lets you calculate the value of an option at any given time. Using Black-Scholes the price of the call option is based on a fraction of the stock's current price minus a fraction of the exercise price. These fractions depend on five factors; the price of the stock; the exercise price of the option; the risk-free interest rate; the time to maturity of the option, and volatility of the underlying stock price. The last factor is the only one that is unobservable.

Financial institutions use the Black-Scholes and other methods to calculate the value of options from which they can understand the risk and set a price to help manage the portfolio.

Using the Black-Scholes equation requires the solution of a set of partial differential equations by means of numerical integration. Depending on the number of assets taken into account, obtaining a solution can be highly computationally intensive.

Sidebar Two: Resources
Computational Financial Derivatives Laboratory

Financial Engineering News


Cornell Theory Center

Kx Systems

KD Nuggets (Data Mining)

Data Mining


The QuantNotes

Douglas Eadline is the swinging Head Monkey at

You have no rights to post comments


Login And Newsletter

Create an account to access exclusive content, comment on articles, and receive our newsletters.


Creative Commons License
©2005-2019 Copyright Seagrove LLC, Some rights reserved. Except where otherwise noted, this site is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License. The Cluster Monkey Logo and Monkey Character are Trademarks of Seagrove LLC.