1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 <2023> 2024 | Index | 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 <2023> 2024 |
<== Date ==> | <== Thread ==> |
---|
Subject: | RE: Andor3 IOC detects and connects to Marana, can get frames, but frames appear noisy/corrupted |
From: | "Pearson, Matthew via Tech-talk" <tech-talk at aps.anl.gov> |
To: | Mark Rivers <rivers at cars.uchicago.edu>, "Wlodek, Jakub" <jwlodek at bnl.gov>, "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov> |
Date: | Wed, 29 Nov 2023 20:23:35 +0000 |
Hi, It’s been a while since I worked with these cameras, but there might be a debug log file at: /tmp/atdebug.log which might give hints about the issue. Or, at least provide additional info if you need to contact Andor. There should also be an example command line C++ program bundled with the SDK that you could compile and run, that should allow you to connect and acquire and save an image, just to rule out any
software issues. Cheers, Matt From: Tech-talk <tech-talk-bounces at aps.anl.gov>
On Behalf Of Mark Rivers via Tech-talk If it was previously working OK and you did not move the card, then it is unlikely to be a PCIe bandwidth issue. It seems more likely a hardware problem. When my card was in an x4 slot it had horizontal black stripes that are mainly visible when the field of view is changing. If the image is static then they are not seen. Mark From: Wlodek, Jakub <jwlodek at bnl.gov>
Hi Mark, I believe this is the Cyton CXP2 card:https://www.bitflow.com/products/coaxpress/cyton-cxp2/ Is there a way to confirm that the card is dropping packets? According to
37:00.0 Unassigned class [ff04]: BitFlow Inc Device 7002 (rev 01) Physical Slot: 2 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 30 NUMA node: 0 Region 0: Memory at e4000000 (32-bit, non-prefetchable) [size=1M] Region 1: Memory at e3000000 (32-bit, non-prefetchable) [size=16M] Capabilities: [50] MSI: Enable- Count=1/4 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [78] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [80] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 75.000W DevCtl: CorrErr- NonFatalErr+ FatalErr+ UnsupReq- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 256 bytes, MaxReadReq 4096 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- LnkCap: Port #1, Speed 5GT/s, Width x8, ASPM L0s, Exit Latency L0s unlimited ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp- LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s (ok),
Width x8 (ok) TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP- LTR- 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- TPHComp- ExtTPHComp- AtomicOpsCap: 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, AtomicOpsCtl: ReqEn- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [100 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- Kernel driver in use: bitflow Thanks for the help, Jakub
From: Mark Rivers <rivers at cars.uchicago.edu> Which BitFlow card is this? I have seen stripes on their Claxon CoaxPress card when it was plugged into an x4 rather than x8 PCIe slot. They are due to dropped packets. Mark From: Tech-talk <tech-talk-bounces at aps.anl.gov>
On Behalf Of Wlodek, Jakub via Tech-talk Hi all, Recently I configured an Andor Marana detector utilizing the Andor3 areaDetector driver and the bitflow kernel driver included with the
andor3 SDK. This has been working fine and collecting data for ~6 months. At one point, after the detector had been running for a long time, the beamline reported that the IOC was displaying images that appeared corrupted/didn't make sense, but stopping/starting
acquisition got back the expected readout, and they ran OK for another several months. Now, however, this same corrupted image issue has re-emerged, but this time a stop/start does not fix it. In fact, a camera power cycle,
IOC reboot, kernel driver rebuild and reload, and IOC server reboot do not appear to have solved the problem. The IOC can detect and connect to the camera, all firmware/serial number information is correct, and it gets images of the correct size, but they
do not appear to be valid images. The different PreAmp Gain modes produce different issues, please take a look at the linked github gist for screenshots: The 12 bit mode exhibits vertical stripe artifacts, while the 16 bit HDR mode exhibits horizontal lines that cross the image. The overall
average pixel value in the 16 bit mode with no beam used to be ~100 counts, it is now ~35 counts, and spikes to 200 when the artifacts appear. I don't see any error messages that would explain this in the IOC shell, and during acquisition the detector outputs
the correct framerate and image dimensions. The image also does not appear to be influenced by outside stimuli i.e. flashlight/beam. Has anyone seen this on an Andor Marana, or Andor camera in general before? If so, is there a software based fix for this or is this a hardware
related issue? My next troubleshooting steps will probably be to pull out the PCI card and test the setup on a Windows machine with a shorter cable (the bitflow software on windows lets you collect images outside of the Andor SDK), but if this is some kind
of configuration issue I'm missing I'd rather ask first before I pull the card out of the server in the rack. Thanks in advance for the advice! Jakub |