Re: Allied Vision camera oddity

["Cobb, Tom \(DLSLtd,RAL,LSCI\) via Tech-talk" <tech-talk at aps.anl.gov>]

That is strange.  I can think of 3 possibilities:
- Power glitch caused both beam dump and camera failure

- Beam dump caused radiation spike that killed cameras

- Cameras are really smart and know that they don't need to work when there's no beam.

We see this too with some of our diagnostics cameras in the ring (AVT Manta), sometimes when we get a beam dump the cameras stop responding and need a power cycle to recover. We suspect that some volatile memory is being corrupted, but the power cycle reloads the FPGA firmware from flash so fixes it. We power them from an IP connected power strip now so we can remotely power cycle them.

Thanks,

Tom

---

From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Mark Rivers via Tech-talk <tech-talk at aps.anl.gov>
Sent: 18 September 2021 02:58
To: John Dobbins <john.dobbins at cornell.edu>
Cc: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: Re: Allied Vision camera oddity

 

Hi John,


> These three cameras just stopped again coincident with a beam loss.

> It turns out that the previous acquisition halt was also coincident with a beam loss.


That is strange.  I can think of 3 possibilities:
- Power glitch caused both beam dump and camera failure
- Beam dump caused radiation spike that killed cameras
- Cameras are really smart and know that they don't need to work when there's no beam.

Mark


________________________________
From: John Dobbins <john.dobbins at cornell.edu>
Sent: Friday, September 17, 2021 8:22 PM
To: Mark Rivers
Subject: Re: Allied Vision camera oddity

update:

These three cameras just stopped again coincident with a beam loss. It turns out that the previous acquisition halt was also coincident with a beam loss.

John Dobbins

________________________________
From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Friday, September 17, 2021 1:34 PM
To: John Dobbins <john.dobbins at cornell.edu>
Cc: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: RE: Allied Vision camera oddity


Hi John,



When the cameras stop do they respond to a ping?  If not then they would appear to be really dead and need to be power-cycled.



Ø  If I power cycle a camera which is successfully acquiring, acquisition will stop but I can recover it by pressing 'Stop' followed by 'Start' in the MEDM screen.



But when you do that the EPICS output PVs will in general not agree with the camera values (gain, exposure time, etc.).  There is a fix for that in the upcoming ADCore R3-12, but currently you really do need to restart the IOC to have the camera and EPICS be in sync for those types of settings.



I have attached the 3 slides from my presentation at the 2021 EPICS Collaboration Meeting that describe this new feature to restore camera settings without restarting the IOC.



Mark





From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of John Dobbins via Tech-talk
Sent: Friday, September 17, 2021 12:13 PM
To: tech-talk at aps.anl.gov
Subject: Allied Vision camera oddity



All,



We are using Allied Vision Mako G-319B cameras with ADVimba. (Firmware 00.01.54.21000, SDK 1.8.0, Driver 1.1.0 ADCore 3.7.0)



An odd behavior we have observed is that rarely image acquisition stops. The PV 'Acquire' reports Acquire but the ArrayCounter_RBV stops incrementing.



We have nine cameras on a dedicated subnet connected to the host. Recently, three cameras stopped acquisition simultaneously ( based on the time-stamp of ArrayData).  I then started the Allied Vision Vimba viewer, and these three cameras are not found. I power cycle the cameras and they now appear in the Vimba viewer.  However, the IOC cannot reestablish image acquisition. For that I have to restart the IOC.



Each camera is connected to a separate IOC on the host (which happens to be a clustered host).



My speculation is that some sort of glitch on the subnet confuses the cameras which then need to be power cycled to recover.  I don't understand why the IOC needs restarting.  If I power cycle a camera which is successfully acquiring, acquisition will stop but I can recover it by pressing 'Stop' followed by 'Start' in the MEDM screen.



Thoughts?



Not an urgent problem because it happens so rarely.  An annoyance because the camera needs to be power cycled and the IOC restarted.



Regards,



John Dobbins



Research Support Specialist

Cornell High Energy Synchrotron Source

Cornell University



www.chess.cornell.edu<http://www.chess.cornell.edu>

 

-- 

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom