RE: Writing areadetector driver- Getting array data to NDArray properly?

[Mark Rivers <rivers@cars.uchicago.edu>]

> PV: Varian:image1:ArrayData scanAdd: I/O Intr not valid (no get_ioint_info) 
> I must be missing something in my code. 

I don't think you are missing anything in your code.  That message is because the device support does not support I/O Intr scanning.  But the standard asyn device support that should be being used for that record DOES support I/O Intr scanning.  That leads me to think that you may have a typo in your startup script?

Mark

________________________________________
From: Gomella, Andrew (NIH/NHLBI) [F] [andrew.gomella@nih.gov]
Sent: Friday, June 13, 2014 4:42 PM
To: Hu, Yong; Mark Rivers; tech-talk@aps.anl.gov
Subject: RE: Writing areadetector driver- Getting array data to NDArray properly?

Sorry accidentally sent in the middle of typing...

I have been so busy figuring out how to properly write AD drivers that I have not tested it until now. I do not believe it's a problem with anything except for my code itself. I just noticed that hidden in the output from my lengthy st.cmd boot I have this message:

PV: Varian:image1:ArrayData scanAdd: I/O Intr not valid (no get_ioint_info)

I must be missing something in my code. This is my first areadetctor driver so it is not a surprise. Since I am putting so much work into this driver I have to wonder if anyone else out there in the future may find a use for any Varian Paxscan detectors? I am guessing it unlikely considering they are purpose built hospital style detectors (I learned  this as soon as I realized there is no way to set the exposure length or gain on this panel)

Thanks,
Andrew
________________________________________
From: Gomella, Andrew (NIH/NHLBI) [F]
Sent: Friday, June 13, 2014 5:37 PM
To: Hu, Yong; Mark Rivers; tech-talk@aps.anl.gov
Subject: RE: Writing areadetector driver- Getting array data to NDArray properly?

Hi Yong and Mark,

Thank you very much for the help with this. I think I much better understand the problems now.

I have been so busy figuring out how to properly write AD drivers that I have not tested it until now. I do not believe it's a problem with anything except for my code itself. I just noticed that hidden in the output from my lengthy st.cmd boot I have this message:

________________________________________
From: Hu, Yong [yhu@bnl.gov]
Sent: Friday, June 13, 2014 3:03 PM
To: Mark Rivers; Gomella, Andrew (NIH/NHLBI) [F]; tech-talk@aps.anl.gov
Subject: Re: Writing areadetector driver- Getting array data to NDArray properly?

Hello Andrew G,

Just want to share my two-cent on the settings of EPICS_CA_MAX_ARRAY_BYTES:

1. All CA clients (ImageJ, EDM, CSS, camonitor, etc.) can not read the whole detector image data (waveform record) successfully if EPICS_CA_MAX_ARRAY_BYTES is set to a not-enough value on the IOC (CA server) side. This is why your system did not work at your first try.

2. On the other hand, as Mark suggested, it is better not to set EPICS_CA_MAX_ARRAY_BYTES too big (300 MB) on an IOC side although it works. It will be problematic for the IOC running on 32-bit Linux machine (no PAE, max. 4GB per process) if there are tens of CA clients are trying to display the detector image concurrently. Once the IOC reaches the limit of max-memory per process, no more CA client can display the image successfully.

3. Recently, there was a discussion about auto-configuration of EPICS_CA_MAX_ARRAY_BYTES on core-talk: http://www.aps.anl.gov/epics/core-talk/2014/msg00105.php. I am working on the auto-configuration of EPICS_CA_MAX_ARRAY_BYTES on the CA server side, setting EPICS_CA_MAX_ARRAY_BYTES to a value being just-enough value, not small and not too big.

One suggestion: it is always a good practice to use the command-line tool caget to test and debug big waveform data. In your case, if you use the command "caget areadetector-array-data-record > test.txt", you may see some waring messages indicating something wrong with settings of EPICS_CA_MAX_ARRAY_BYTES on the CA client side or the server side.

HTH,

Yong

NSLS-II Controls Group


-----Original Message-----
From: Mark Rivers <rivers@cars.uchicago.edu>
Date: Sunday, June 8, 2014 11:48 PM
To: "Gomella, Andrew (NIH/NHLBI) [F]" <andrew.gomella@nih.gov>
Cc: "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>
Subject: RE: Writing areadetector driver- Getting array data to
NDArray properly?

OK, that's progress.

You probably should not increase EPICS_CA_MAX_ARRAY_BYTES to 300000000.
That value is actually used to allocate arrays in Channel Access whenever
the required size is > 16KB.  So you are allocating arrays of nearly 300MB
each time an array needs to be transferred.

But it is important the that ImageJ process also has a size that is larger
than the largest image you will be transferring.

Mark

________________________________________
From: Gomella, Andrew (NIH/NHLBI) [F] [andrew.gomella@nih.gov]
Sent: Sunday, June 08, 2014 9:20 PM
To: Mark Rivers
Cc: tech-talk@aps.anl.gov
Subject: RE: Writing areadetector driver- Getting array data to NDArray
properly?

Hi Mark,

Thank you very much for the quick response. I modified my code to read how
you have it. I also realized that my NDArraySize was off by a factor of 2
(did not take into account sizeof(USHORT)).

The AD Tiff plugin now works (earlier I would get a pop up dialog- I
forget exactly what it said but something along the lines of "Error
opening tiff". )

ImageJ Viewer still isn't working but I was just using it as a test to see
if my driver was working. It is the latest version (same as one currently
on github). I checked in my ioc and EPICS_CA_MAX_ARRAY_BYTES is 300000000
(i bumped it up a factor of 10 earlier to be sure).

I restarted ImageJ but it still doesnt work, but getting the IJ viewer
working is low priority seeing how much work I have left in this driver. I
am just happy to have the arrays reading out now.

Thanks!

Andrew



________________________________________
From: Mark Rivers [rivers@cars.uchicago.edu]
Sent: Sunday, June 08, 2014 9:43 PM
To: Gomella, Andrew (NIH/NHLBI) [F]; tech-talk@aps.anl.gov
Subject: RE: Writing areadetector driver- Getting array data to NDArray
properly?

Hi Andrew,

Note that while what you are doing should work (unless there is a problem
I am missing), it is not the most efficient way to do it.  You can avoid
allocating 2 arrays and the memcpy operation if you change the order of
operations as follows:

this->pArrays[0] = this->pNDArrayPool->alloc(2, dims, NDUInt16, 0, NULL);
pImage = this->pArrays[0];
vip_get_image(this->ModeNumber, VIP_CURRENT_IMG_RAW, this->sensorWidth,
this->sensorHeight,
                     (USHORT *)pImage->pData);

So you pass the NDArray data pointer directly to their DLL.

Mark


________________________________________
From: tech-talk-bounces@aps.anl.gov [tech-talk-bounces@aps.anl.gov] on
behalf of Gomella, Andrew (NIH/NHLBI) [F] [andrew.gomella@nih.gov]
Sent: Sunday, June 08, 2014 7:12 PM
To: tech-talk@aps.anl.gov
Subject: Writing areadetector driver- Getting array data to NDArray
properly?

Hi Everyone,

I am writing a driver for a Varian 3024m flat panel detector. I am having
trouble in my driver getting the data from the panel into NDArrays.

To get the data from the panel it is fairly simple, I pass a pointer to
the dll (with the call vip_get_image) and it populates it with  ushorts.
It looks like this:


USHORT *pFrame = new USHORT[this->sensorSize];
vip_get_image(this->ModeNumber, VIP_CURRENT_IMG_RAW, this->sensorWidth,
this->sensorHeight, pFrame);

Next I attempt to get this data to NDPluginStdArrays with:
(This is just one of many attempts, I tried my best to follow examples
from other areadetector drivers, mainly Prosilica and PerkinElmer).


NDArray *pImage;
dims[0] = this->sensorWidth;
dims[1] = this->sensorHeight;

if (this->pArrays[0])
       this->pArrays[0]->release();

this->pArrays[0] = this->pNDArrayPool->alloc(2, dims, NDUInt16, 0, NULL);
pImage = this->pArrays[0];
pImage->getInfo(&arrayInfo);
memcpy(pImage->pData, pFrame, arrayInfo.totalBytes);


pImage->getInfo(&arrayInfo);
setIntegerParam(NDArraySize,  arrayInfo.totalBytes);
setIntegerParam(NDArraySizeX, arrayInfo.xSize);
setIntegerParam(NDArraySizeY, arrayInfo.ySize);
setIntegerParam(NDDataType, dataType);
setIntegerParam(NDColorMode, colorMode);

getIntegerParam(NDArrayCallbacks, &arrayCallbacks);

if (arrayCallbacks) {

    this->unlock();
    doCallbacksGenericPointer(pImage, NDArrayData, 0);
    this->lock();
}


Testing this, Tiff plugin gives me an error about not being able to open
it and the Imagej Plugin just shows a black window with a  "UpdateImage
got exception: pendIO timed out" error. My EPICS_MAX_CA_ARRAY_BYTES is set
to 30MB, and all appropriate array callback/ image callbacks are enabled
for this test. System is Windows x64, i7 processor, 16gb ram. Area
detector 1-9-1.

My other attempts, which there are too numerous to mention here result in
the app crashing.

Any help is greatly appreciated.

Thanks,
Andrew Gomella

Imaging Physics Lab, NIH