Experimental Physics and Industrial Control System
Maybe it helps, after yet another crash after longer time the
attribute in question is the same:
#2 0x00007ffff776bff7 in NDFileHDF5AttributeDataset::createHDF5Dataset (
this=this@entry=0x7ffb2ffb2e10) at ../NDFileHDF5AttributeDataset.cpp:99
99 H5Pset_fill_value(cparm_, datatype_, ptrFillValue_);
(gdb) p *this
$3 = {
_vptr.NDFileHDF5AttributeDataset = 0x7ffff799c5b0 <vtable for
NDFileHDF5AttributeDataset+16>, name_ = "CameraModel", dsetName_ =
"CameraModel", file_ = 72057594037982897,
type_ = NDAttrString, groupName_ = "/entry/instrument/NDAttributes",
dataset_ = -1,
dataspace_ = -1, memspace_ = -1, datatype_ = 216172782122083269,
cparm_ = 720575940385270143, filespace_ = -1, ptrFillValue_ = 0x7ffb2fff4f30,
dims_ = 0x7ffb1c058030, offset_ = 0x7ffb1c0c3cb0, chunk_ = 0x7ffb2ff89a80,
maxdims_ = 0x7ffb2feedf00, virtualdims_ = 0x0, elementSize_ = 0x7ffb1c0c3cd0,
isUndefined_ = false, rank_ = 1, nextRecord_ = 0, extraDimensions_ = 0,
whenToSave_ = hdf5::OnFrame}
(gdb)
/Hinko
On Fri, Nov 17, 2017 at 3:42 PM, Hinko Kocevar <[email protected]> wrote:
> Hi Mark,
>
> I finally got the machine and time to debug further.
>
> On Tue, Nov 14, 2017 at 1:15 PM, Mark Rivers <[email protected]> wrote:
>> Hi Hinko,
>>
>>
>> Can you also print the contents of "this" in NDFileHDF5AttributeDataset::createHDF5Dataset()? That will tell us the name of the attribute, which may be helpful.
>>
>
> Here it is:
>
> (gdb) frame 2
> #2 0x00007ffff776bff7 in NDFileHDF5AttributeDataset::createHDF5Dataset (
> this=this@entry=0x7ffecc07bcb0) at ../NDFileHDF5AttributeDataset.cpp:99
> 99 H5Pset_fill_value(cparm_, datatype_, ptrFillValue_);
> (gdb) p this
> $1 = (NDFileHDF5AttributeDataset * const) 0x7ffecc07bcb0
> (gdb) p *this
> $2 = {
> _vptr.NDFileHDF5AttributeDataset = 0x7ffff799c5b0 <vtable for
> NDFileHDF5AttributeDataset+16>, name_ = "CameraModel", dsetName_ =
> "CameraModel", file_ = 72057594037929019,
> type_ = NDAttrString, groupName_ = "/entry/instrument/NDAttributes",
> dataset_ = -1,
> dataspace_ = -1, memspace_ = -1, datatype_ = 216172782113947691,
> cparm_ = 720575940379397441, filespace_ = -1, ptrFillValue_ = 0x7ffed7ffcfd0,
> dims_ = 0x7ffed7ed2180, offset_ = 0x7ffecc0611d0, chunk_ = 0x7ffecc011ea0,
> maxdims_ = 0x7ffed7d5f2e0, virtualdims_ = 0x0, elementSize_ = 0x7ffecc0611f0,
> isUndefined_ = false, rank_ = 1, nextRecord_ = 0, extraDimensions_ = 0,
> whenToSave_ = hdf5::OnFrame}
> (gdb)
>
>
>>
>> Can you also provide some more information on the crash.
>>
>> - Does this occur each time the file is written, or only once in a while?
>
> No, not each time.
> This time, it took less than a minute @ 10 files per second.
>
>>
>> - If once in a while what is the frequency? 1%, .1%, etc.?
>
> Now I'm running again and I have already saved >5000 files and still no crash..
>
>> - Can you send tell me the XML file you are using to define the NDAttributes, and for each of those that are PVAttributes also tell me the EPICS record type?
>
> You can find the file attached.
>
> Thanks,
> Hinko
>
>>
>> Mark
>>
>>
>> ________________________________
>> From: Hinko Kocevar <[email protected]>
>> Sent: Tuesday, November 14, 2017 2:57 AM
>> To: Mark Rivers
>> Cc: [email protected]
>> Subject: Re: areaDetector and HDF5 file saving segafaults
>>
>> Hi Mark,
>>
>> thanks for your time.
>> I have not yet had the time to reproduce this with SimDetector - on my
>> TODO list!
>>
>>
>> Here is the gdb with some variables printed out. I can't see any null
>> pointers or similar low hanging fruit.
>>
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> [Switching to Thread 0x7ffef4b68700 (LWP 26708)]
>> 0x00007ffff35d7e08 in __memcpy_ssse3_back () from /lib64/libc.so.6
>> Missing separate debuginfos, use: debuginfo-install
>> bzip2-libs-1.0.6-13.el7.x86_64 elfutils-libelf-0.168-8.el7.x86_64
>> elfutils-libs-0.168-8.el7.x86_64 glib2-2.50.3-3.el7.x86_64
>> glibc-2.17-196.el7.x86_64 glibmm24-2.50.0-1.el7.x86_64
>> libattr-2.4.46-12.el7.x86_64 libblkid-2.23.2-43.el7.x86_64
>> libcap-2.22-9.el7.x86_64 libffi-3.0.13-18.el7.x86_64
>> libgcc-4.8.5-16.el7.x86_64 libmount-2.23.2-43.el7.x86_64
>> libselinux-2.5-11.el7.x86_64 libsigc++20-2.10.0-1.el7.x86_64
>> libstdc++-4.8.5-16.el7.x86_64 libusbx-1.0.20-1.el7.x86_64
>> libuuid-2.23.2-43.el7.x86_64 libxml2-2.9.1-6.el7_2.3.x86_64
>> ncurses-libs-5.9-14.20130511.el7_4.x86_64 pcre-8.32-17.el7.x86_64
>> readline-6.2-10.el7.x86_64 systemd-libs-219-42.el7_4.1.x86_64
>> xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-17.el7.x86_64
>> (gdb) bt
>> #0 0x00007ffff35d7e08 in __memcpy_ssse3_back () at /lib64/libc.so.6
>> #1 0x00007ffff6371211 in H5Pset_fill_value (plist_id=<optimized out>,
>> type_id=216172782118131140, value=0x7ffcf7feffc0) at ../H5Pdcpl.c:3217
>> #2 0x00007ffff776bff7 in
>> NDFileHDF5AttributeDataset::createHDF5Dataset()
>> (this=this@entry=0x7ffcf7da1fe0) at
>> ../NDFileHDF5AttributeDataset.cpp:99
>> #3 0x00007ffff776c648 in
>> NDFileHDF5AttributeDataset::createDataset(int)
>> (this=this@entry=0x7ffcf7da1fe0, user_chunking=<optimized out>) at
>> ../NDFileHDF5AttributeDataset.cpp:75
>> #4 0x00007ffff776529e in NDFileHDF5::createAttributeDataset(NDArray*)
>> (this=this@entry=0xca2500, pArray=pArray@entry=0x7fff580aed70) at
>> ../NDFileHDF5.cpp:2535
>> #5 0x00007ffff776717f in NDFileHDF5::openFile(char const*, int,
>> NDArray*) (this=0xca2500, fileName=0x7ffef4b675d0
>> "/tmp/spectra1_14109.h5", openMode=<optimized out>,
>> pArray=0x7fff580aed70)
>> at ../NDFileHDF5.cpp:268
>> #6 0x00007ffff775820d in NDPluginFile::openFileBase(int, NDArray*)
>> (this=this@entry=0xca2500, openMode=openMode@entry=2,
>> pArray=pArray@entry=0x7fff580aed70) at ../NDPluginFile.cpp:73
>> #7 0x00007ffff7758ee4 in NDPluginFile::writeFileBase()
>> (this=this@entry=0xca2500) at ../NDPluginFile.cpp:230
>> #8 0x00007ffff77597f8 in NDPluginFile::processCallbacks(NDArray*)
>> (this=0xca2500, pArray=0x7fff580aed70) at ../NDPluginFile.cpp:751
>> #9 0x00007ffff770871e in NDPluginDriver::processTask()
>> (this=0xca2500) at ../NDPluginDriver.cpp:467
>> #10 0x00007ffff48d44c9 in epicsThreadCallEntryPoint(void*)
>> (pPvt=0x7fff80000dc0) at ../../../src/libCom/osi/epicsThread.cpp:85
>> #11 0x00007ffff48d9f6c in start_routine (arg=0x7fff80000cd0) at
>> ../../../src/libCom/osi/os/posix/osdThread.c:403
>> #12 0x00007ffff2f66e25 in start_thread () at /lib64/libpthread.so.0
>> #13 0x00007ffff358534d in clone () at /lib64/libc.so.6
>>
>>
>> (gdb) frame 1
>> #1 0x00007ffff6371211 in H5Pset_fill_value (plist_id=<optimized out>,
>> type_id=216172782118131140, value=0x7ffcf7feffc0) at ../H5Pdcpl.c:3217
>> 3217 HDmemcpy(fill.buf, value, (size_t)fill.size);
>> (gdb) p fill
>> $1 = {sh_loc = {type = 0, file = 0x0, msg_type_id = 0, u = {loc =
>> {index = 0, oh_addr = 18446744073709551615}, heap_id = {id =
>> "\000\000\000\000\000\000\000", val = 0}}}, version = 2, type =
>> 0x7ffcec682ee0,
>> size = 256, buf = 0x7ffcf7f13da0, alloc_time = H5D_ALLOC_TIME_LATE,
>> fill_time = H5D_FILL_TIME_IFSET, fill_defined = false}
>> (gdb) p fill.buf
>> $2 = (void *) 0x7ffcf7f13da0
>> (gdb) p value
>> $3 = (const void *) 0x7ffcf7feffc0
>> (gdb) p fill.size
>> $8 = 256
>> (gdb) x/256 fill.buf
>> 0x7ffcf7f13da0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13da8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13db0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13db8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13dc0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13dc8: 0x11 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13dd0: 0x10 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13dd8: 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13de0: 0x00 0xd5 0x6a 0xec 0xfc 0x7f 0x00 0x00
>> 0x7ffcf7f13de8: 0x18 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13df0: 0x01 0xff 0xff 0xff 0xff 0xff 0xff 0xff
>> 0x7ffcf7f13df8: 0x44 0x45 0x41 0x44 0x42 0x45 0x45 0x46
>> 0x7ffcf7f13e00: 0x21 0x52 0x42 0x00 0x00 0x00 0x00 0x03
>> 0x7ffcf7f13e08: 0x01 0x00 0x00 0x00 0x01 0x00 0x00 0x00
>> 0x7ffcf7f13e10: 0x80 0x11 0x6d 0xec 0xfc 0x7f 0x00 0x00
>> 0x7ffcf7f13e18: 0x42 0x45 0x45 0x46 0x44 0x45 0x41 0x44
>> 0x7ffcf7f13e20: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13e28: 0xa5 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13e30: 0x48 0x35 0x4d 0x4d 0xff 0x7f 0x00 0x00
>> 0x7ffcf7f13e38: 0xa0 0x3a 0x6b 0xec 0xfc 0x7f 0x00 0x00
>> 0x7ffcf7f13e40: 0xf0 0xc6 0xbe 0xf7 0xfc 0x7f 0x00 0x00
>> 0x7ffcf7f13e48: 0x60 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13e50: 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13e58: 0x44 0x45 0x41 0x44 0x42 0x45 0x45 0x46
>> 0x7ffcf7f13e60: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13e68: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13e70: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13e78: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13e80: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13e88: 0x20 0xc7 0xbe 0xf7 0xfc 0x7f 0x00 0x00
>> 0x7ffcf7f13e90: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7f13e98: 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff
>> (gdb) x/256 value
>> 0x7ffcf7feffc0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7feffc8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7feffd0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7feffd8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7feffe0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7feffe8: 0x11 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7fefff0: 0x10 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7fefff8: 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0000: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0008: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0010: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0018: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0020: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0028: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0030: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0038: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0040: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0048: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0050: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0058: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0060: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0068: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0070: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0078: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0080: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0088: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0090: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0098: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff00a0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff00a8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff00b0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff00b8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>>
>>
>> (gdb) frame 2
>> #2 0x00007ffff776bff7 in
>> NDFileHDF5AttributeDataset::createHDF5Dataset
>> (this=this@entry=0x7ffcf7da1fe0) at
>> ../NDFileHDF5AttributeDataset.cpp:99
>> 99 H5Pset_fill_value(cparm_, datatype_, ptrFillValue_);
>> (gdb) p cparm_
>> $4 = 720575940382417266
>> (gdb) p datatype_
>> $5 = 216172782118131140
>> (gdb) p ptrFillValue_
>> $6 = (void *) 0x7ffcf7feffc0
>> (gdb) x/256 ptrFillValue_
>> 0x7ffcf7feffc0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7feffc8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7feffd0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7feffd8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7feffe0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7feffe8: 0x11 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7fefff0: 0x10 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7fefff8: 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0000: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0008: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0010: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0018: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0020: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0028: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0030: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0038: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0040: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0048: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0050: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0058: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0060: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0068: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0070: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0078: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0080: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0088: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0090: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff0098: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff00a0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff00a8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff00b0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> 0x7ffcf7ff00b8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
>> (gdb)
>>
>>
>> Hope this helps,
>> Hinko
>>
>> On Mon, Nov 13, 2017 at 1:46 PM, Mark Rivers <[email protected]> wrote:
>>> Hi Hinko,
>>>
>>>
>>> Since you are using busy 1-6-1 I don't think the crash is due to the busy record. It does not have the bug which was causing crashes for me.
>>>
>>>
>>> This is the code where it is crashing in H5Pdcpl.c
>>>
>>> /* Set the fill value */
>>> if(NULL == (fill.type = H5T_copy(type, H5T_COPY_TRANSIENT)))
>>> HGOTO_ERROR(H5E_PLIST, H5E_CANTCOPY, FAIL, "can't copy datatype")
>>> fill.size = (ssize_t)H5T_get_size(type);
>>> if(NULL == (fill.buf = H5MM_malloc((size_t)fill.size)))
>>> HGOTO_ERROR(H5E_RESOURCE, H5E_CANTINIT, FAIL, "memory allocation failed for fill value")
>>> HDmemcpy(fill.buf, value, (size_t)fill.size); CRASHES HERE
>>>
>>> That code is being called from here in NDFileHDF5AttributeDataset::createHDF5Dataset():
>>>
>>>
>>> asynStatus NDFileHDF5AttributeDataset::createHDF5Dataset()
>>> {
>>> asynStatus status = asynSuccess;
>>>
>>> cparm_ = H5Pcreate(H5P_DATASET_CREATE);
>>>
>>> H5Pset_fill_value(cparm_, datatype_, ptrFillValue_); CRASHES IN CALL FROM HERE
>>>
>>>
>>>
>>> This is the private data in the NDFileHDF5AttributeDataset class:
>>>
>>> private:
>>> asynStatus createHDF5Dataset();
>>> asynStatus configureDims(int user_chunking);
>>> asynStatus configureDimsFromDataset(bool multiframe, int extradimensions, int *extra_dims, int *user_chunking);
>>> asynStatus typeAsHdf();
>>> void extendDataSet();
>>> void extendDataSet(hsize_t *offsets);
>>> void extendIndexDataSet(hsize_t offset);
>>>
>>> std::string name_; // Name of the attribute
>>> std::string dsetName_; // Name of the dataset to store
>>> hid_t file_; // File handle
>>> NDAttrDataType_t type_; // NDAttribute type
>>> std::string groupName_; // Name of the parent group
>>> hid_t dataset_; // Dataset handle
>>> hid_t dataspace_;
>>> hid_t memspace_;
>>> hid_t datatype_;
>>> hid_t cparm_;
>>> hid_t filespace_;
>>> void *ptrFillValue_;
>>> hsize_t *dims_;
>>> hsize_t *offset_;
>>> hsize_t *chunk_;
>>> hsize_t *maxdims_;
>>> hsize_t *virtualdims_;
>>> hsize_t *elementSize_;
>>> bool isUndefined_;
>>> int rank_; // number of dimensions
>>> int nextRecord_;
>>> int extraDimensions_;
>>> hdf5::When_t whenToSave_;
>>>
>>> };
>>>
>>> I have 2 suggestions to track down the problem:
>>>
>>> - Use gdb to examine the values of the variables in H5Pdcpl.c and the NDFileHDF5AttributeDataset private member data when it crashes
>>>
>>> or
>>>
>>> - Use asynPrint to print the values every time that function is called, and route the asynTrace output to a file, since it may be large.
>>>
>>>
>>> Have you tried to reproduce it with the simDetector?
>>>
>>>
>>> Mark
>>>
>>>
>>>
>>>
>>>
>>> ________________________________
>>> From: Hinko Kocevar <[email protected]>
>>> Sent: Monday, November 13, 2017 2:32 AM
>>> To: Mark Rivers
>>> Cc: [email protected]
>>> Subject: Re: areaDetector and HDF5 file saving segafaults
>>>
>>> Hi Mark,
>>>
>>> On Thu, Nov 9, 2017 at 1:06 PM, Mark Rivers <[email protected]> wrote:
>>>> Hi Hinko,
>>>>
>>>>> I'm using Single saving mode, saving one image per acquisition at ~10 images per second.
>>>>
>>>> How is the file saving being done? Are you setting the AutoSave PV to 1? If so, what version of the "busy" module are you using? I have recently seen a segfault when using AutoSave at much higher frame rates with the TIFF file plugin, and the same problem might well occur with the HDF5 plugin. I understand the problem, it has to do with the busy record asyn device support not correctly handling callbacks when they occur rapidly. However, I don't think this happens with any official release of the busy module, only with changes made on the master branch. I am actively working on a fix for that problem.
>>>
>>> Yes, I'm using autosave=1. Busy version is 1-6-1.
>>>
>>>>
>>>> It seems unlikely that hardware problems would cause the error with the stack trace you sent, but it's always possible I suppose.
>>>>
>>>> There was also a change in ADCore R3-0 to fix a problem storing NDAttributes in HDF5 files when those attributes were coming from EPICS PVs that were disconnected. You are using that fixed version. Is it possible that one of your attributes is coming from a disconnected EPICS PV? If so, perhaps I introduced a bug.
>>>
>>> I think all PVs should be connected - I'm only accessing PVs local to the IOC.
>>>
>>> Thanks,
>>> Hinko
>>>
>>>>
>>>> Mark
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ________________________________
>>>> From: Hinko Kocevar <[email protected]>
>>>> Sent: Thursday, November 9, 2017 1:52 AM
>>>> To: Mark Rivers
>>>> Cc: [email protected]
>>>> Subject: Re: areaDetector and HDF5 file saving segafaults
>>>>
>>>> Hi Mark,
>>>>
>>>> Image size is 1936 x 1216, data type is uint16 (~2.3MB). Camera is AlliedVision Manta G-235B, controlled with aravis+aravisGigE.
>>>> I'm saving some attributes - see the attached XML file.
>>>> I'm using Single saving mode, saving one image per acquisition at ~10 images per second.
>>>> It happens quite often, if I leave if running for couple of minutes I get the segfault.
>>>> I have not yet tried reproducing with simDetector - will do.
>>>> I'm using HDF5 ver 1.10.1 that comes with ADSupport.
>>>>
>>>> The thing is that I have not noticed this problems before, this now happens on a remote machine at an experiment we are conducting. It could be the hardware..
>>>>
>>>> Thank you,
>>>> Hinko
>>>>
>>>> On Tue, Nov 7, 2017 at 2:59 PM, Mark Rivers <[email protected]<mailto:[email protected]>> wrote:
>>>> Hi Hinko,
>>>>
>>>>
>>>> Also, what version of HDF5 are you using, the one from ADSupport (and if so what version) or a system library version?
>>>>
>>>>
>>>> Mark
>>>>
>>>>
>>>>
>>>> ________________________________
>>>> From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> on behalf of Hinko Kocevar <[email protected]<mailto:[email protected]>>
>>>> Sent: Tuesday, November 7, 2017 3:58 AM
>>>> To: [email protected]<mailto:[email protected]>
>>>> Subject: areaDetector and HDF5 file saving segafaults
>>>>
>>>> Hi,
>>>>
>>>> I'm experiencing segmentation faults that trace to HDF5 file saving plugin. This is with ADAravis camera, 10 images per second.
>>>>
>>>> EPICS base is 3.15.4, AreaDetector is 3-1.
>>>>
>>>> epics>
>>>> Program received signal SIGSEGV, Segmentation fault.
>>>> [Switching to Thread 0x7fffed55a700 (LWP 27283)]
>>>> 0x00007ffff35d7d89 in __memcpy_ssse3_back () from /lib64/libc.so.6
>>>> (gdb) bt
>>>> #0 0x00007ffff35d7d89 in __memcpy_ssse3_back () at /lib64/libc.so.6
>>>> #1 0x00007ffff6371211 in H5Pset_fill_value (plist_id=<optimized out>, type_id=216172782117023038, value=0x7ffd67ff2fb0) at ../H5Pdcpl.c:3217
>>>> #2 0x00007ffff776bff7 in NDFileHDF5AttributeDataset::createHDF5Dataset() (this=this@entry=0x7ffd5c007460) at ../NDFileHDF5AttributeDataset.cpp:99
>>>> #3 0x00007ffff776c648 in NDFileHDF5AttributeDataset::createDataset(int) (this=this@entry=0x7ffd5c007460, user_chunking=<optimized out>)
>>>> at ../NDFileHDF5AttributeDataset.cpp:75
>>>> #4 0x00007ffff776529e in NDFileHDF5::createAttributeDataset(NDArray*) (this=this@entry=0x939530, pArray=pArray@entry=0x7fff10968800)
>>>> at ../NDFileHDF5.cpp:2535
>>>> #5 0x00007ffff776717f in NDFileHDF5::openFile(char const*, int, NDArray*) (this=0x939530, fileName=0x7fffed5595d0 "/home/dev/DTUdata/image1_20896.h5", openMode=<optimized out>, pArray=0x7fff10968800) at ../NDFileHDF5.cpp:268
>>>> #6 0x00007ffff775820d in NDPluginFile::openFileBase(int, NDArray*) (this=this@entry=0x939530, openMode=openMode@entry=2, pArray=pArray@entry=0x7fff10968800) at ../NDPluginFile.cpp:73
>>>> #7 0x00007ffff7758ee4 in NDPluginFile::writeFileBase() (this=this@entry=0x939530) at ../NDPluginFile.cpp:230
>>>> #8 0x00007ffff77597f8 in NDPluginFile::processCallbacks(NDArray*) (this=0x939530, pArray=0x7fff10968800) at ../NDPluginFile.cpp:751
>>>> #9 0x00007ffff770871e in NDPluginDriver::processTask() (this=0x939530) at ../NDPluginDriver.cpp:467
>>>> #10 0x00007ffff48d44c9 in epicsThreadCallEntryPoint(void*) (pPvt=0x7fffd4001400) at ../../../src/libCom/osi/epicsThread.cpp:85
>>>> #11 0x00007ffff48d9f6c in start_routine (arg=0x7fffd4001310) at ../../../src/libCom/osi/os/posix/osdThread.c:403
>>>> #12 0x00007ffff2f66e25 in start_thread () at /lib64/libpthread.so.0
>>>> #13 0x00007ffff358534d in clone () at /lib64/libc.so.6
>>>>
>>>> Thanks,
>>>> Hinko
>>>>
>>>> --
>>>> .. the more I see the less I believe.., AE AoR
>>>>
>>>>
>>>>
>>>> --
>>>> .. the more I see the less I believe.., AE AoR
>>>
>>>
>>>
>>> --
>>> .. the more I see the less I believe.., AE AoR
>>
>>
>>
>> --
>> .. the more I see the less I believe.., AE AoR
>
>
>
> --
> .. the more I see the less I believe.., AE AoR
--
.. the more I see the less I believe.., AE AoR
- Replies:
- RE: areaDetector and HDF5 file saving segafaults Mark Rivers
- References:
- areaDetector and HDF5 file saving segafaults Hinko Kocevar
- Re: areaDetector and HDF5 file saving segafaults Mark Rivers
- Re: areaDetector and HDF5 file saving segafaults Hinko Kocevar
- Re: areaDetector and HDF5 file saving segafaults Mark Rivers
- Re: areaDetector and HDF5 file saving segafaults Hinko Kocevar
- Re: areaDetector and HDF5 file saving segafaults Mark Rivers
- Re: areaDetector and HDF5 file saving segafaults Hinko Kocevar
- Re: areaDetector and HDF5 file saving segafaults Mark Rivers
- Re: areaDetector and HDF5 file saving segafaults Hinko Kocevar
- Navigate by Date:
- Prev:
Re: areaDetector and HDF5 file saving segafaults Hinko Kocevar
- Next:
RE: areaDetector and HDF5 file saving segafaults Mark Rivers
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
<2017>
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: areaDetector and HDF5 file saving segafaults Hinko Kocevar
- Next:
RE: areaDetector and HDF5 file saving segafaults Mark Rivers
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
<2017>
2018
2019
2020
2021
2022
2023
2024