RE: V4 question.

[Mark Rivers <rivers@cars.uchicago.edu>]

My understanding is that Bruno Martins from BNL has already prototyped replacing NDArrays with V4 NTNDArrays and using these as the communication mechanism between drivers and plugins.

ADCore milestone Release 5-0 (https://github.com/areaDetector/ADCore/milestone/8) contains the following:

*****
This release may change the NDArray and NDArrayPool classes to use shared pointers. This could either be done by changing the pData and pAttributeList pointers in NDArray into std::tr1:shared_ptr or by replacing NDArray with V4 NTNDArray. The latter would make V4 a requirement for ADCore.
*****

Mark


-----Original Message-----
From: tech-talk-bounces@aps.anl.gov [mailto:tech-talk-bounces@aps.anl.gov] On Behalf Of Andrew Johnson
Sent: Tuesday, May 09, 2017 10:56 AM
To: Shankar, Murali; EPICS tech-talk
Subject: Re: V4 question.

Hi Murali,

On 05/08/2017 08:17 PM, Shankar, Murali wrote:
>     I have a complicated EPICS V4 question; I am not too sure if the
> question is even a correct one nor if I understand the issue completely. 

Please ask these kinds of questions on either the tech-talk or core-talk
mailing lists. You probably won't be the only person looking for answers
like this, and by using the lists the answers get archived and are thus
searchable in the future. Also your questions will get seen by more
people who may be able to help answer them.

>   We are trying to do some design on detector software; folks have heard
> that EPICS V4 tries to be zero copy and therefore we are trying to see
> if we can useit in this case. This situation is slightly different from
> the normal EPICS V4 usecase. Instead of "areadetector sendsimages to
> processing plugins across the wire"; we can use shared memory so in this
> case so this would be "areadetector sends images to processing plugins
> across shared memory".
> 
>   Assume I have a shared memory situation; where one process is writing
> a PVStructure into shared memory and another one is reading it perhaps
> on another machine. If I am somehow memory mapping the shared memory
> into the same locations in the two address spaces, could I then cast
> this memory location into the same PVStructure on the reader and read
> the pvfields without paying the price of de-serialization? Would this be
> true if I restrict the processes on each side to C++/Python? I do see
> that the API tries to be zero-copy as much as possible. For example, 
> 
> /From //PVScalarArray//:://getAs/
> /    Fetch the current value and convert to the requested type. A copy
> is made if the requested type does not match the element type. If the
> types do match then no copy is made. /
> 
>   My interpretation is that this could potentially work even with plain
> old C structs but with pvData we'd get better introspection and
> other features. I know there could be potentially other issues with this
> approach; but at a high level, would this even be feasible? 
> 
> Any help is appreciated.

I'm not sure that I can answer definitively, but my understanding of C++
and the pvData code is that I think you might have difficulties using
shared memory to transfer pvData objects between processes using shared
memory at the moment. I haven't looked, but I don't know that the pvData
code does anything special when it needs to allocate or free memory;
persuading it to do so only from a memory pool that is shared between
multiple processes may be tricky, or even impossible. I've never written
code that uses the Linux/SYSV shared memory functionality though, so I'm
really not the right person to answer this.

Is there no way you can use multiple threads instead of processes? In
Linux the line between the two is really very thin... The pvData and
pvAccess code was designed for communicating between threads, but not
for multiple processes, so it would make a lot more sense to use that
instead of jumping through hoops to try and get it to work across
process boundaries.

Hopefully someone else with more knowledge of this will also be able to
chip in with some answers.

- Andrew


-- 
Arguing for surveillance because you have nothing to hide is no
different than making the claim, "I don't care about freedom of
speech because I have nothing to say." -- Edward Snowdon