Experimental Physics and
Industrial Control System

But -- if the slowdown is due to limited cache space -- why would the
throughput improve as you process bigger arrays (larger than 4k - I
would expect the throughput to stay low once your cache is thrashing)?
This doesn't make sense. I'd also argue that the cache doesn't really
give you a big performance boost in this 'stream processing' scenario.
The cache speeds things up if you repeatedly access the same data
over and over again but that's typically not the case when you process
a stream of data.
Maybe some TCP issue? I would try e.g., reading from a file
or some other kind of non-TCP data source.

FWIW
- Till


On 06/19/2012 09:54 AM, Andrew Johnson wrote:

Hi Dirk,

On 2012-06-19 Dirk Zimoch wrote:

I am testing the performance of array parsing with StreamDevice on a
Linux system. I am reading strings like "3.1415,3.1415,...\n" into a
waveform record with FTVL=DOUBLE. The input is via TCP from localhost.

I found a very strange duration/size relationship:
size       1     duration:      423     time/element:  423.0
size       2     duration:      234     time/element:  117.0

...

size     256     duration:      397     time/element:    1.6
size     512     duration:      552     time/element:    1.1
size    1024     duration:    40023     time/element:   39.1
size    2048     duration:    41450     time/element:   20.2

...

size  524288     duration:   275591     time/element:    0.5
size 1048576     duration:   518404     time/element:    0.5

All times are in 1e-6 seconds. Time is measured on the server. The clock
starts before sending the string and stops after the waveform sends back
an acknowledge.

Protocol: {in "%f"; out "%(NORD)d";}

I understand that for small arrays, the time is dominated by overhead
and stays more or less constant.

But does anyone have an idea why is there this strange performance drop
for sizes between 1024 and 4096 elements?

I would second Martin Konrad's cache suggestion, your string data above takes
up 7 characters per element and the double version is 8 bytes per element, so
with 512 elements they each fit into a 4KB MMU page (or straddle two pages).
When you double the number of elements you need 4 pages to store it all, and
that may be where you run out of cache.  Try setting FTVL=FLOAT and see if the
break moves upwards at all (you might need to measure at size=768 to detect a
change though).  If that doesn't make any difference it could also be a effect
on the machine at the other end of the TCP socket.

- Andrew

Replies:

Re: Strange performance results -- any ideas? Dirk Zimoch

References:

Strange performance results -- any ideas? Dirk Zimoch

Re: Strange performance results -- any ideas? Andrew Johnson

Navigate by Date:

Prev: Re: Strange performance results -- any ideas? Andrew Johnson

Next: RE: BOY XY graph tick marks [SEC=UNCLASSIFIED] Chen, Xihui

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <2012>  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 

Navigate by Thread:

Prev: Re: Strange performance results -- any ideas? Andrew Johnson

Next: Re: Strange performance results -- any ideas? Dirk Zimoch

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <2012>  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024