| 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 <2022> 2023 2024 2025 | Index | 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 <2022> 2023 2024 2025 |
| <== Date ==> | <== Thread ==> |
|---|
| Subject: | Re: NDArrayPool:reserve ERROR, reference count = 0, should be = 1 |
| From: | Mark Rivers via Tech-talk <tech-talk at aps.anl.gov> |
| To: | "Daykin, Evan" <daykin at frib.msu.edu>, Michael Davidsaver <mdavidsaver at gmail.com> |
| Cc: | "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov> |
| Date: | Tue, 3 May 2022 18:16:45 +0000 |
|
Hi Evan,
> Thanks, that was the problem. Setting copyArray=true makes the issue go away. Are there any adverse side-effects
of doing so?
This argument to NDPluginDriver::endProcessCallbacks() is documented here:
It says:
Your driver is calling endProcessCallbacks with a new NDArray that processCallbacks() created. Thus you must set copyArray=false, as you were originally doing.
NDPluginDriver::endProcessCallbacks(outputArray, false, true);
NDArrayPool:reserve ERROR, reference count = 0, should be >= 1, pArray=0x7f453c001d80 Thread SimDetTask (0x5621bbbb0c80) can't proceed, suspending. Dumping a stack trace of thread 'SimDetTask': [ 0x7f45538ffab3]: /lib/x86_64-linux-gnu/libCom.so.3.15.9(epicsStackTrace+0x73) [ 0x7f45538f0216]: /lib/x86_64-linux-gnu/libCom.so.3.15.9(cantProceed+0xc6) [ 0x7f4553197b08]: /lib/x86_64-linux-gnu/libADBase.so.3.11(_ZN11NDArrayPool7reserveEP7NDArray+0x78) [ 0x7f455305db37]: /lib/x86_64-linux-gnu/libNDPlugin.so.3.11(_ZN14NDPluginDriver21beginProcessCallbacksEP7NDArray+0x367) [ 0x7f455375df1e]: /home/daykin/git/areadetector-temperature/lib/linux-x86_64/libNDPluginTemperature.so(_ZN19NDPluginTemperature16processCallbacksEP7NDArray+0xae) [ 0x7f455305dd53]: /lib/x86_64-linux-gnu/libNDPlugin.so.3.11(_ZN14NDPluginDriver14driverCallbackEP8asynUserPv+0x1d3) [ 0x7f4553851155]: /lib/x86_64-linux-gnu/libasyn.so.4.38(_ZN14asynPortDriver25doCallbacksGenericPointerEPvii+0x1f5) [ 0x7f4553809974]: /usr/lib/epics/lib/linux-x86_64/libsimDetector.so(_ZN11simDetector7simTaskEv+0x4e4) [ 0x7f45538fa0bb]: /lib/x86_64-linux-gnu/libCom.so.3.15.9(epicsSnprintf+0x7bb) [ 0x7f4553387ea7]: /lib/x86_64-linux-gnu/libpthread.so.0(start_thread+0xd7) [ 0x7f455349edef]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)
From this stack trace it looks like beginProcessCallbacks is being called from the simDetTask. I think this means you have set callbacksBlock=Enable for this plugin. Is that correct?
Otherwise the task with the error should be your plugin tasks, not the simDetTask. Is there a reason you set callbacksBlock? That should not be a problem, but it is unusual and I am just curious.
I have a couple of comments on your processCallbacks function.
You are measuring the elapsed time for execution.
epicsTimeStamp after; epicsTimeGetCurrent(&after); double delta = epicsTimeDiffInSeconds(&after, &before); cout<<"Took "<<delta<<" s"<<endl; setDoubleParam(runTimeIdx, delta);
But the base class already does this for you, so you don't should not need to do this.
You are allocating a new inputArray with NDArrayPool->convert(), converting to NDUInt16. But you already know that the input array (pArray) has type NDUInt16, so why make a new array?
You can just pass pArray to doTemperatureConversion, as long as you don't modify that array.
this->pNDArrayPool->convert(pArray,&(this->inputArray), NDUInt16);
More importantly you have allocated this->inputArray, but you have never released it. This means you have a memory leak. Once you are done with this->inputArray you should call this->inputArray->release().
But as I said above I am not sure you need to create this array at all.
You are only allocating this->outputArray if it is currently NULL. You then pass this->outputArray to doTemperatureConversion(). What happens if the new input array has different
dimensions from the one that was used when creating this->outputArray? If the new array is larger then you will probably get an access violation unless doTemperatureConversion() is checking the dimensions of the output array.
This does not solve your original problem. For some reason the NDArray being passed to NDPluginDriver::beginProcessCallbacks() has a reference count of 0. That should never happen,
because if the reference count is 0 then it should be in the free list and not in active use.
You should switch the flag to endProcessCallbacks back to the correct value of false and then try to track down the problem.
Mark
From: Daykin, Evan <daykin at frib.msu.edu>
Sent: Tuesday, May 3, 2022 11:52 AM To: Michael Davidsaver <mdavidsaver at gmail.com> Cc: Mark Rivers <rivers at cars.uchicago.edu>; tech-talk at aps.anl.gov <tech-talk at aps.anl.gov> Subject: RE: NDArrayPool:reserve ERROR, reference count = 0, should be = 1 Thanks, that was the problem. Setting copyArray=true makes the issue go away. Are there any adverse side-effects of doing so?
-----Original Message----- From: Michael Davidsaver <mdavidsaver at gmail.com> Sent: Tuesday, May 3, 2022 12:10 PM To: Daykin, Evan <daykin at frib.msu.edu> Cc: Mark Rivers <rivers at cars.uchicago.edu>; tech-talk at aps.anl.gov Subject: Re: NDArrayPool:reserve ERROR, reference count = 0, should be = 1 [EXTERNAL] This email originated from outside of FRIB On 5/3/22 08:52, Daykin, Evan via Tech-talk wrote: > Hm… I am not explicitly calling release() anywhere. This is my processCallbacks function- everything else in my plugin, AFAIK, doesn’t access any shared resources. I have half a memory that NDPluginDriver::endProcessCallbacks() "steals"* a reference to the NDArray when copyArray=false. * https://docs.python.org/3/c-api/intro.html#reference-count-details > void NDPluginTemperature::processCallbacks(NDArray *pArray){ > > static const char *functionName = "processCallbacks"; > > NDArrayInfo_t arrayInfo; > > if(pArray->dataType != NDUInt16){ > > asynPrint(this->pasynUserSelf, ASYN_TRACE_ERROR, > > "%s:%s: Only UInt16 supported.", driverName, functionName); > > return; > > } > > std::string lastCalFileName = this->calibrationFileName; > > getStringParam(this->calibrationFileNameIdx, this->calibrationFileName); > > if(this->calibrationFileName != lastCalFileName){ > > this->processCalibrationFile(); > > } > > int arrayCallbacks; > > getIntegerParam(NDArrayCallbacks, &arrayCallbacks); > > if(arrayCallbacks==1){ > > epicsTimeStamp before; > > epicsTimeGetCurrent(&before); > > NDPluginDriver::beginProcessCallbacks(pArray); > > this->pNDArrayPool->convert(pArray,&(this->inputArray), NDUInt16); > > if(NULL == this->outputArray){ > > this->pNDArrayPool->convert(pArray,&(this->outputArray),NDUInt16); > > } > > //unlock while the plug-and-chug happens. No shared resources are accessed at this time. > > inputArray->getInfo(&arrayInfo); > > this->unlock(); > > this->doTemperatureConversion(this->inputArray, this->outputArray, &arrayInfo); > > this->lock(); > > setIntegerParam(NDArraySizeX, (int)outputArray->dims[arrayInfo.xDim].size); > > setIntegerParam(NDArraySizeY, (int)outputArray->dims[arrayInfo.yDim].size); > > epicsTimeStamp after; > > epicsTimeGetCurrent(&after); > > double delta = epicsTimeDiffInSeconds(&after, &before); > > cout<<"Took "<<delta<<" s"<<endl; > > setDoubleParam(runTimeIdx, delta); > > NDPluginDriver::endProcessCallbacks(outputArray, false, true); > > callParamCallbacks(); > > } > > } > > *From:*Mark Rivers <rivers at cars.uchicago.edu> > *Sent:* Tuesday, May 3, 2022 11:37 AM > *To:* tech-talk at aps.anl.gov; Daykin, Evan <daykin at frib.msu.edu> > *Subject:* Re: NDArrayPool:reserve ERROR, reference count = 0, should be = 1 > > *[EXTERNAL] This email originated from outside of FRIB* > > Hi Evan, > > That error probably means you have called NDArray::release() on an array whose reference count is already 0. > > Most plugins don't need to call release() because it is handled in the base class NDPluginDriver::endProcessCallbacks(). > > Mark > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > > *From:*Tech-talk <tech-talk-bounces at aps.anl.gov <mailto:tech-talk-bounces at aps.anl.gov>> on behalf of Daykin, Evan via Tech-talk <tech-talk at aps.anl.gov <mailto:tech-talk at aps.anl.gov>> > *Sent:* Tuesday, May 3, 2022 10:20 AM > *To:* tech-talk at aps.anl.gov <mailto:tech-talk at aps.anl.gov> <tech-talk at aps.anl.gov <mailto:tech-talk at aps.anl.gov>> > *Subject:* NDArrayPool:reserve ERROR, reference count = 0, should be = 1 > > Hi, > > I am hoping there’s an AreaDetector plugin maven here… > > > I have written a small AD plugin intended to convert our raw camera images into a temperature heatmap, based on a calibrated gain, emissivity and exposure time. To test it, I set up a SimDetector in “peak” mode. I connect my plugin to this simDetector image. I can successfully capture and convert 3 frames. On the fourth frame, my test dies with the following error. Are there any common pitfalls that might cause this? > > NDArray.uniqueId=1 > > Do temperature conversion start > > Took 0.0937809 s > > NDArray.uniqueId=2 > > Do temperature conversion start > > Took 0.0655851 s > > NDArray.uniqueId=3 > > Do temperature conversion start > > Took 0.0642867 s > > NDArray.uniqueId=4 > > NDArrayPool:reserve ERROR, reference count = 0, should be >= 1, pArray=0x7f453c001d80 > > Thread SimDetTask (0x5621bbbb0c80) can't proceed, suspending. > > Dumping a stack trace of thread 'SimDetTask': > > [ 0x7f45538ffab3]: /lib/x86_64-linux-gnu/libCom.so.3.15.9(epicsStackTrace+0x73) > > [ 0x7f45538f0216]: /lib/x86_64-linux-gnu/libCom.so.3.15.9(cantProceed+0xc6) > > [ 0x7f4553197b08]: /lib/x86_64-linux-gnu/libADBase.so.3.11(_ZN11NDArrayPool7reserveEP7NDArray+0x78) > > [ 0x7f455305db37]: /lib/x86_64-linux-gnu/libNDPlugin.so.3.11(_ZN14NDPluginDriver21beginProcessCallbacksEP7NDArray+0x367) > > [ 0x7f455375df1e]: /home/daykin/git/areadetector-temperature/lib/linux-x86_64/libNDPluginTemperature.so(_ZN19NDPluginTemperature16processCallbacksEP7NDArray+0xae) > > [ 0x7f455305dd53]: /lib/x86_64-linux-gnu/libNDPlugin.so.3.11(_ZN14NDPluginDriver14driverCallbackEP8asynUserPv+0x1d3) > > [ 0x7f4553851155]: /lib/x86_64-linux-gnu/libasyn.so.4.38(_ZN14asynPortDriver25doCallbacksGenericPointerEPvii+0x1f5) > > [ 0x7f4553809974]: /usr/lib/epics/lib/linux-x86_64/libsimDetector.so(_ZN11simDetector7simTaskEv+0x4e4) > > [ 0x7f45538fa0bb]: /lib/x86_64-linux-gnu/libCom.so.3.15.9(epicsSnprintf+0x7bb) > > [ 0x7f4553387ea7]: /lib/x86_64-linux-gnu/libpthread.so.0(start_thread+0xd7) > > [ 0x7f455349edef]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) > > *Evan Daykin* > > Controls Engineer > > Facility for Rare Isotope Beams > > Michigan State University > > 640 South Shaw Lane > > East Lansing, MI 48824, USA > > Tel. 517-908-7678 > > Email: daykin at frib.msu.edu <mailto:mccausey at frib.msu.edu> > > > > *cid:[email protected]* > |