EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  <20212022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  <20212022  2023  2024 
<== Date ==> <== Thread ==>

Subject: Re: Allied Vision camera oddity
From: John Dobbins via Tech-talk <tech-talk at aps.anl.gov>
To: "Cobb, Tom (DLSLtd,RAL,LSCI)" <tom.cobb at diamond.ac.uk>, "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
Date: Thu, 23 Sep 2021 23:26:07 +0000
Thanks for the feedback Tom.

As our cameras are PoE, I was able to write a script to ssh to the switch and power cycle the camera port.

John


From: Cobb, Tom (DLSLtd,RAL,LSCI) <tom.cobb at diamond.ac.uk>
Sent: Monday, September 20, 2021 4:16 AM
To: John Dobbins <john.dobbins at cornell.edu>; Mark Rivers <rivers at cars.uchicago.edu>
Cc: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: Re: Allied Vision camera oddity
 
That is strange.  I can think of 3 possibilities:
- Power glitch caused both beam dump and camera failure
- Beam dump caused radiation spike that killed cameras
- Cameras are really smart and know that they don't need to work when there's no beam.
We see this too with some of our diagnostics cameras in the ring (AVT Manta), sometimes when we get a beam dump the cameras stop responding and need a power cycle to recover. We suspect that some volatile memory is being corrupted, but the power cycle reloads the FPGA firmware from flash so fixes it. We power them from an IP connected power strip now so we can remotely power cycle them.

Thanks,
Tom



From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Mark Rivers via Tech-talk <tech-talk at aps.anl.gov>
Sent: 18 September 2021 02:58
To: John Dobbins <john.dobbins at cornell.edu>
Cc: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: Re: Allied Vision camera oddity
 
Hi John,


> These three cameras just stopped again coincident with a beam loss.

> It turns out that the previous acquisition halt was also coincident with a beam loss.


That is strange.  I can think of 3 possibilities:
- Power glitch caused both beam dump and camera failure
- Beam dump caused radiation spike that killed cameras
- Cameras are really smart and know that they don't need to work when there's no beam.

Mark


________________________________
From: John Dobbins <john.dobbins at cornell.edu>
Sent: Friday, September 17, 2021 8:22 PM
To: Mark Rivers
Subject: Re: Allied Vision camera oddity

update:

These three cameras just stopped again coincident with a beam loss. It turns out that the previous acquisition halt was also coincident with a beam loss.

John Dobbins

________________________________
From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Friday, September 17, 2021 1:34 PM
To: John Dobbins <john.dobbins at cornell.edu>
Cc: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: RE: Allied Vision camera oddity


Hi John,



When the cameras stop do they respond to a ping?  If not then they would appear to be really dead and need to be power-cycled.



Ø  If I power cycle a camera which is successfully acquiring, acquisition will stop but I can recover it by pressing 'Stop' followed by 'Start' in the MEDM screen.



But when you do that the EPICS output PVs will in general not agree with the camera values (gain, exposure time, etc.).  There is a fix for that in the upcoming ADCore R3-12, but currently you really do need to restart the IOC to have the camera and EPICS be in sync for those types of settings.



I have attached the 3 slides from my presentation at the 2021 EPICS Collaboration Meeting that describe this new feature to restore camera settings without restarting the IOC.



Mark





From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of John Dobbins via Tech-talk
Sent: Friday, September 17, 2021 12:13 PM
To: tech-talk at aps.anl.gov
Subject: Allied Vision camera oddity



All,



We are using Allied Vision Mako G-319B cameras with ADVimba. (Firmware 00.01.54.21000, SDK 1.8.0, Driver 1.1.0 ADCore 3.7.0)



An odd behavior we have observed is that rarely image acquisition stops. The PV 'Acquire' reports Acquire but the ArrayCounter_RBV stops incrementing.



We have nine cameras on a dedicated subnet connected to the host. Recently, three cameras stopped acquisition simultaneously ( based on the time-stamp of ArrayData).  I then started the Allied Vision Vimba viewer, and these three cameras are not found. I power cycle the cameras and they now appear in the Vimba viewer.  However, the IOC cannot reestablish image acquisition. For that I have to restart the IOC.



Each camera is connected to a separate IOC on the host (which happens to be a clustered host).



My speculation is that some sort of glitch on the subnet confuses the cameras which then need to be power cycled to recover.  I don't understand why the IOC needs restarting.  If I power cycle a camera which is successfully acquiring, acquisition will stop but I can recover it by pressing 'Stop' followed by 'Start' in the MEDM screen.



Thoughts?



Not an urgent problem because it happens so rarely.  An annoyance because the camera needs to be power cycled and the IOC restarted.



Regards,



John Dobbins



Research Support Specialist

Cornell High Energy Synchrotron Source

Cornell University



www.chess.cornell.edu<http://www.chess.cornell.edu>










 

-- 

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 


References:
Allied Vision camera oddity John Dobbins via Tech-talk
RE: Allied Vision camera oddity Mark Rivers via Tech-talk
Re: Allied Vision camera oddity Mark Rivers via Tech-talk
Re: Allied Vision camera oddity Cobb, Tom (DLSLtd,RAL,LSCI) via Tech-talk

Navigate by Date:
Prev: Re: Ava in caQtDM Johnson, Andrew N. via Tech-talk
Next: EPICS 7-0-6 Release Notes Link error Matt Rippa via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  <20212022  2023  2024 
Navigate by Thread:
Prev: Re: Allied Vision camera oddity Cobb, Tom (DLSLtd,RAL,LSCI) via Tech-talk
Next: Install EPICS on debian11 dennis roy via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  <20212022  2023  2024 
ANJ, 23 Sep 2021 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·