[Beowulf] ...Re: Benchmark results

Josip Loncaric josip at lanl.gov
Thu Jan 8 16:43:30 EST 2004

Jim Lux wrote:

> At 11:13 AM 1/8/2004 -0700, Josip Loncaric wrote:
>> "On Efficiently Implementing Global Time for Performance Evaluation on 
>> Multiprocessor Systems," Eric Maillet and Cecile Tron, Journal of 
>> Parallel and Distributed Computing, No. 28, pp. 84-93 (1995).
> What was the measurement/integration interval for their frequency 
> measurements?  Robert (and I, and others) are interested in stability 
> over fairly short time scales (milliseconds?).

Alas, their sampling interval was about 6 seconds.  They had 500 message 
exchanges in 50 minutes.

However, drift due to independent random events should grow like 
sqrt(time), so in absolute terms it should be lower at smaller time 
scales (even though it may be larger in relative terms).  In other 
words, if you had 6 microseconds of uncompensated drift in 6 seconds 
(1ppm), and you assume that the uncompensated part was due to a random 
walk process, you'd see about 0.06 microseconds of drift in 0.6 
milliseconds.  This is not bad in absolute terms, even though in 
relative terms it looks bad (100ppm).

>> Presumably, clock drift coefficients could be correlated to 
>> motherboard temperature measurements.  Over time, one should be able 
>> to construct a linearized model of clock dynamics, linking the local 
>> clock drift, motherboard temperature, and possibly even power supply 
>> voltages.  This should overcome the main sources of timing uncertainty.
> I wouldn't be too sure that you could build a suitable model.  One could 
> certainly make a very accurate predictor in the long term sense: it 
> would very accurately model things like how many cycles of the processor 
> clock there are in every 10 or 60 second interval. But at shorter time 
> scales, the clocks MIGHT be pretty bad, with random jumps and stuff. I 
> don't know, but since it's relevant to my current research, I'm going to 
> find out, starting today..

Hmmm...  I'm no expert, but I believe that typical clock oscillators 
become quite good when temperature compensation is used.  I am not sure 
if there is any significant supply voltage dependence, particularly 
since there may be some voltage stabilization in the clock circuit.  I 
assume we're talking about a crystal oscillator plus clock multipliers; 
there may be significant phase jitter but ultimately one cares only 
about cycle counting.  These days, there can be over 3000 cycles in a 

If good quality synchronization pulses can be provided every second, 
then microsecond requirement means that one can afford 1 ppm of 
uncompensated clock drift.  Over a year, the uncompensated drift could 
be 31.5 seconds.  Clock in my car does almost as well, despite being 
subjected to 60 deg. C temperature swings and variable accelerations.

Perhaps I'm overly optimistic, but writing a kernel module to achieve 
cluster-wide microsecond-level clock synchronization just might be 


Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list