Experimental Physics and
Industrial Control System

Hi, 

We are trying to do this also at APS and are quite frustrated! Below is an email from my colleague to Mark R, but going to post to tech-talk for wider help in case people haven't left for ICALEPCS yet. ...

 .... 

I am getting ready to punt on this and return to el6, but I thought that I would ping you on it before I do... We have andor SDK 3.13.30034.0 on gypsum.xray at Sector 8. Gypsum is 3.10.0-862.el7.x86_64 el7.5. This is the kernel that the tech talk thread from 8/5 from the Diamond guys noted. I tried 3.10 and the camera, frame grabber not USB, was not recognized using ...examples/listdevices/listdevices. I tried 3.14 and the camera was not recognized. Currently on 3.13 the camera is recognized with listdevices and "image" takes a single image. So I'm convinced that the SDK is working, but we are still getting error here:

andor3Config("ANDOR", 0, 0, 0, 0, 0, 100)

2019/09/30 05:49:40.403 andor3:connectCamera: unable to open camera 0

2019/09/30 05:49:40.403 andor3:andor3:  camera connection failed (10)

Do you have any ideas? I looked at the release notes, but I was uncertain if any of the changes should affect he andor app. 

Thanks,

Troy

---

From: [email protected] <[email protected]> on behalf of Pearson, Matthew R. via Tech-talk <[email protected]>
Sent: Monday, August 5, 2019 4:38 PM
To: [email protected] <[email protected]>; [email protected] <[email protected]>
Subject: Re: [EXTERNAL] updating ADAndor from RHEL6 to RHEL7

 

Hi,

Update for this thread. After discussing with Andor we’ve resolved the issue with the Zyla CameraLink/BifFlow. The important information is that SDK 3.14 only works with kernel version 3.10.0-862.6.3.el7.x86_64 (RHEL 7.5) or below. The more recent 3.10.0-957.10.1.el7.x86_64 does not work. The kernel crashes and the machine reboots (I pasted the kernel stack trace below).

In addition, I also had to manually remove the libatdevapogee.so library from /usr/local/lib. That caused a SIGABRT (due to a free(): invalid pointer) during the call to AT_InitialiseLibrary() in the andor3::andor3 constructor (but strangely the Andor example programs worked fine).

I also found another issue that may catch people out. The SDK seg faults if the log files in /tmp/ are owned by a different user. So to fix that just remove the Andor files from /tmp/.

Cheers,
Matt

usercopy: kernel memory overwrite attempt detected to ffff92451bf9fc10 (<process stack>) (16 bytes)
[  496.694653] ------------[ cut here ]------------
[  496.694703] kernel BUG at mm/usercopy.c:72!
[  496.694747] invalid opcode: 0000 [#1] SMP
[  496.694800] Modules linked in: binfmt_misc nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache bitflow(POE) v4l2_common videodev ip6table_filter ip6_tables iptable_nat nf_nat_ipv4 nf_nat nf_log_ipv4 nf_log_common xt_LOG ipt_REJECT nf_reject_ipv4 xt_comment nf_conntrack_ipv4 nf_defrag_ipv4 xt_hashlimit xt_addrtype xt_multiport xt_conntrack nf_conntrack iptable_filter iTCO_wdt iTCO_vendor_support intel_pmc_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr i2c_i801 sg mei_me mei joydev ie31200_edac ipmi_si ipmi_devintf ipmi_msghandler pinctrl_sunrisepoint video nfsd pinctrl_intel acpi_pad acpi_power_meter auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif
[  496.695830]  crct10dif_generic ast drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm igb megaraid_sas crct10dif_pclmul crct10dif_common crc32c_intel ptp pps_core dca drm_panel_orientation_quirks i2c_algo_bit i2c_hid dm_mirror dm_region_hash dm_log dm_mod
[  496.696168] CPU: 3 PID: 16971 Comm: image Kdump: loaded Tainted: P           OE  ------------   3.10.0-957.21.3.el7.x86_64 #1
[  496.696273] Hardware name: Supermicro Super Server/X11SSW-F, BIOS 2.2 05/23/2018
[  496.696345] task: ffff924573a5c100 ti: ffff92451bf9c000 task.ti: ffff92451bf9c000
[  496.696416] RIP: 0010:[<ffffffff8ec3e6e7>]  [<ffffffff8ec3e6e7>] __check_object_size+0x87/0x250
[  496.696514] RSP: 0018:ffff92451bf9fad8  EFLAGS: 00010246
[  496.696567] RAX: 0000000000000063 RBX: ffff92451bf9fc10 RCX: 0000000000000000
[  496.696635] RDX: 0000000000000000 RSI: ffff9245b78d3898 RDI: ffff9245b78d3898
[  496.696704] RBP: ffff92451bf9faf8 R08: 0000000000000000 R09: 00000000ffffffff
[  496.696772] R10: 0000000000000404 R11: 0000000000000001 R12: 0000000000000010
[  496.696840] R13: 0000000000000000 R14: ffff92451bf9fc20 R15: ffff92451bf9fd00
[  496.696923] FS:  00007f580e047740(0000) GS:ffff9245b78c0000(0000) knlGS:0000000000000000
[  496.697000] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  496.698566] CR2: 00007f580e03a000 CR3: 0000000ffa2fe000 CR4: 00000000003607e0
[  496.699134] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  496.699705] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  496.700276] Call Trace:
[  496.700843]  [<ffffffffc09b8231>] bflki_copy_from_user+0x21/0x40 [bitflow]
[  496.701414]  [<ffffffffc09d566e>] bitflow_user_DMA+0x9be/0x3020 [bitflow]
[  496.701984]  [<ffffffff8ebc09f2>] ? get_page_from_freelist+0x412/0xa70
[  496.702551]  [<ffffffff8eadccbe>] ? account_entity_dequeue+0xae/0xd0
[  496.703115]  [<ffffffffc09b849e>] ? bflki_spin_unlock_irqrestore+0xe/0x10 [bitflow]
[  496.703678]  [<ffffffffc09b8e5e>] ? BFmutexLock+0x5e/0x100 [bitflow]
[  496.704236]  [<ffffffff8ec3e82a>] ? __check_object_size+0x1ca/0x250
[  496.704791]  [<ffffffffc09bd867>] ? bflki_private_ioctl+0x22c7/0x3a80 [bitflow]
[  496.705344]  [<ffffffffc09b72e2>] ? bflki_ioctl+0xd2/0x6d0 [bitflow]
[  496.705892]  [<ffffffff8ec33022>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[  496.706439]  [<ffffffffc09576cd>] ? v4l2_ioctl+0x11d/0x160 [videodev]
[  496.706984]  [<ffffffff8ec569d0>] ? do_vfs_ioctl+0x3a0/0x5a0
[  496.707524]  [<ffffffff8f170628>] ? __do_page_fault+0x228/0x4f0
[  496.708059]  [<ffffffff8ec56c71>] ? SyS_ioctl+0xa1/0xc0
[  496.708589]  [<ffffffff8f175ddb>] ? system_call_fastpath+0x22/0x27
[  496.709114] Code: 45 d1 48 c7 c6 45 be 47 8f 48 c7 c1 50 53 48 8f 48 0f 45 f1 49 89 c0 4d 89 e1 48 89 d9 48 c7 c7 40 22 48 8f 31 c0 e8 60 e2 51 00 <0f> 0b 0f 1f 80 00 00 00 00 48 c7 c0 00 00 a0 8e 4c 39 f0 73 0d
[  496.710265] RIP  [<ffffffff8ec3e6e7>] __check_object_size+0x87/0x250
[  496.710808]  RSP <ffff92451bf9fad8>


> On Jul 16, 2019, at 8:23 AM, [email protected] wrote:
>
> Hi Matthew,
>
>
> It sounds like you made it further than me with the bitflow driver! I am having issues with version 3.13.30034 of the SDK and bitflow version 9.05 on RHEL7. When I run the listdevices example program I only find the two simulation cameras.
>
>
> In the output of dmesg I can see that the camera is assigned to /dev/video0 when I load the kernel module, but I cannot seem to communicate with it.
>
>
> /tmp/atdebug.log also shows some potentially relevant output for my issue:
>
>
> ../../regcam/cldevice/bitflowcilx/bitflowapilinux_impl.cpp. 237 (PID:25227 TID:1066743616): 16/07/19 13:15:07:    apiCiSysBrdInfo:    CiSysBrdInfo Error=(-29999) parameter out of range
> ../../regcam/cldevice/bitflowcilx/cldevice_bitflowlinux.cpp. 427 (PID:25227 TID:1066743616): 16/07/19 13:15:07:    getDeviceVideoNumber:    Error CiSysBrdInfo (VFG Board Number=1 . . .), Err=(1)
> ../../regcam/cldevice/bitflowcilx/bitflowapilinux_impl.cpp. 237 (PID:25227 TID:1066743616): 16/07/19 13:15:07:    apiCiSysBrdInfo:    CiSysBrdInfo Error=(-29999) parameter out of range
> ../../regcam/cldevice/bitflowcilx/cldevice_bitflowlinux.cpp. 427 (PID:25227 TID:1066743616): 16/07/19 13:15:07:    getDeviceVideoNumber:    Error CiSysBrdInfo (VFG Board Number=2 . . .), Err=(1)
>
>
> Did you experience anything similar?
>
>
> Kind regards,
>
> Ben
>
> ________________________________
> From: Pearson, Matthew R. <[email protected]>
> Sent: 15 July 2019 21:13:01
> To: Bradnick, Ben (Tessella,RAL,TEC)
> Cc: Cobb, Tom (DLSLtd,RAL,LSCI); [email protected]
> Subject: RE: updating ADAndor from RHEL6 to RHEL7
>
> Hi Ben,
>
> Thanks for the info.
>
> We also have a Zyla sCMOS camera (CameraLink based, Bitflow card), so I'm trying to get that working on RHEL7 too.
>
> We have SDK3 3.12 (bitflow driver 9.05). I managed to get the kernel module built, and the example 'listdevices' program can see the camera ok. But the example 'image' program crashes the kernel and the machine reboots.
>
> I've requested the latest SDK 3.14 from Andor, and I'll give that a go. Andor usually ask you to update to the latest before they provide help anyway.
>
> Cheers,
> Matt
>
>> -----Original Message-----
>> From: [email protected] <[email protected]>
>> Sent: Monday, July 15, 2019 11:42 AM
>> To: Pearson, Matthew R. <[email protected]>
>> Cc: [email protected]; [email protected]
>> Subject: [EXTERNAL] Re: updating ADAndor from RHEL6 to RHEL7
>>
>> Hi Matt,
>>
>>
>> We are successfully running ADAndor on RHEL 7.6 with an iXon Ultra 897 (USB
>> 2.0 interface). We are using version 2.102.30034.0 of the SDK.
>>
>>
>> The issues Tom mentions are for an Andor Zyla CMOS detector using a Bitflow
>> framegrabber with ADAndor3. We needed to use a later version of the RHEL7
>> kernel to build the kernel module for the framegrabber, but we still have issues
>> communicating with the camera.
>>
>> Kind regards,
>> Ben
>>
>> ________________________________
>> From: Cobb, Tom (DLSLtd,RAL,LSCI)
>> Sent: 09 July 2019 10:40
>> To: Pearson, Matthew R.; [email protected]
>> Cc: Bradnick, Ben (Tessella,RAL,TEC)
>> Subject: RE: updating ADAndor from RHEL6 to RHEL7
>>
>> Hi Matt,
>>
>> Ben (copied) has seen some Andor issues both installing the kernel driver and
>> then getting ADAndor to see any cameras on RHEL7. I don't know how far he
>> got, and he's away for another week, but it would be interesting to compare
>> notes.
>>
>> Thanks,
>> Tom
>>
>>> -----Original Message-----
>>> From: [email protected] <[email protected]> On
>>> Behalf Of Pearson, Matthew R. via Tech-talk
>>> Sent: 08 July 2019 23:53
>>> To: [email protected] Talk <[email protected]>
>>> Subject: updating ADAndor from RHEL6 to RHEL7
>>>
>>>
>>> Hi,
>>>
>>> Has anyone experience updating the ADAndor support from RHEL6 to RHEL7?
>>>
>>> We have an Andor iKon-M (USB2.0 based), that we used with ADAndor on
>>> RHEL6 using SDK 2 version 2.93 (locally installed, not using the
>>> version distributed with ADAndor).
>>>
>>> The same SDK version doesn't seem to work on RHEL7. I can send details
>>> if anyone is interested. It may just be too old though.
>>>
>>> There are two more modern SDK versions currently distributed with
>>> ADAndor (at least, the shared libraries). Is anyone using either of
>>> these versions (2.99 and 2.102) on RHEL7?
>>>
>>> Cheers,
>>> Matt
>>>
>>>
>>> Data Acquisition and Controls Engineer Spallation Neutron Source Oak
>>> Ridge National Lab
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> This e-mail and any attachments may contain confidential, copyright and or
>> privileged material, and are for the use of the intended addressee only. If you
>> are not the intended addressee or an authorised recipient of the addressee
>> please notify us of receipt by returning the e-mail and do not use, copy, retain,
>> distribute or disclose the information in or attached to the e-mail.
>> Any opinions expressed within this e-mail are those of the individual and not
>> necessarily of Diamond Light Source Ltd.
>> Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments
>> are free from viruses and we cannot accept liability for any damage which you
>> may sustain as a result of software viruses which may be transmitted in or with
>> the message.
>> Diamond Light Source Limited (company no. 4375679). Registered in England
>> and Wales with its registered office at Diamond House, Harwell Science and
>> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
>
>
> --
> This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
> Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
> Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
>