Hi Dirk,
On 2012-06-19 Dirk Zimoch wrote:
I am testing the performance of array parsing with StreamDevice on a
Linux system. I am reading strings like "3.1415,3.1415,...\n" into a
waveform record with FTVL=DOUBLE. The input is via TCP from localhost.
I found a very strange duration/size relationship:
size 1 duration: 423 time/element: 423.0
size 2 duration: 234 time/element: 117.0
...
size 256 duration: 397 time/element: 1.6
size 512 duration: 552 time/element: 1.1
size 1024 duration: 40023 time/element: 39.1
size 2048 duration: 41450 time/element: 20.2
...
size 524288 duration: 275591 time/element: 0.5
size 1048576 duration: 518404 time/element: 0.5
All times are in 1e-6 seconds. Time is measured on the server. The clock
starts before sending the string and stops after the waveform sends back
an acknowledge.
Protocol: {in "%f"; out "%(NORD)d";}
I understand that for small arrays, the time is dominated by overhead
and stays more or less constant.
But does anyone have an idea why is there this strange performance drop
for sizes between 1024 and 4096 elements?
I would second Martin Konrad's cache suggestion, your string data above takes
up 7 characters per element and the double version is 8 bytes per element, so
with 512 elements they each fit into a 4KB MMU page (or straddle two pages).
When you double the number of elements you need 4 pages to store it all, and
that may be where you run out of cache. Try setting FTVL=FLOAT and see if the
break moves upwards at all (you might need to measure at size=768 to detect a
change though). If that doesn't make any difference it could also be a effect
on the machine at the other end of the TCP socket.
- Andrew