EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  <20212022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  <20212022  2023  2024 
<== Date ==> <== Thread ==>

Subject: RE: QuadEM operation with parameters "Queued"
From: Iain Marcuson via Tech-talk <tech-talk at aps.anl.gov>
To: Mark Rivers <rivers at cars.uchicago.edu>
Cc: "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
Date: Wed, 27 Oct 2021 21:05:29 +0000

I have found a solution.  I see from various logs that network commands weren’t making their way out.  I eventually observed that a setIntegerParam() invocation would clear the queue.  I now perform a write to a dummy PV on a periodic basis, and this causes the message to flow from the queue.

 

From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Iain Marcuson via Tech-talk
Sent: Thursday, October 21, 2021 2:55 PM
To: Mark Rivers <rivers at cars.uchicago.edu>
Cc: tech-talk at aps.anl.gov
Subject: RE: QuadEM operation with parameters "Queued"

 

I wrapped the send/receive with semaphores, so that should stop any clobbering.  I still observed the weird behavior on PC1.  I then pointed EPICS to a different computer (PC2) running the middle layer, whereupon I was not seeing the message backlog.  I am wondering if either there was a change to the middle layer on PC2 that hasn’t been reflected to PC1, or if messages on PC1 were too fast, being localhost and a faster machine.  I did not see these problems on PC2.

 

From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Thursday, October 21, 2021 12:19 PM
To: Iain Marcuson <iain.marcuson at sydortechnologies.com>
Cc: tech-talk at aps.anl.gov
Subject: RE: QuadEM operation with parameters "Queued"

 

You can add some asynPrint statements with before the member buffer is written to and see if one thread is clobbering the string before it is written.  Use ASYN_TRACEINFO_THREAD to see what thread is calling it.

 

Mark

 

 

From: Iain Marcuson <iain.marcuson at sydortechnologies.com>
Sent: Thursday, October 21, 2021 11:08 AM
To: Mark Rivers <rivers at cars.uchicago.edu>
Cc: tech-talk at aps.anl.gov
Subject: RE: QuadEM operation with parameters "Queued"

 

I took a look, and it seems that I only call unlock() in a different thread that reads the data over its own socket, versus the command handling thread(s?) where I don’t do any locking.

 

From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Thursday, October 21, 2021 12:03 PM
To: Iain Marcuson <iain.marcuson at sydortechnologies.com>
Cc: tech-talk at aps.anl.gov
Subject: RE: QuadEM operation with parameters "Queued"

 

  • I am assuming I am not doing ASYN_CANBLOCK, as I see no references to it in the code

 

Actually the base drvQuadEM class sets ASYN_CANBLOCK in its constructor, so you don’t need to worry about it.

 

One thing that could cause a problem is if your driver is calling unlock() after it writes to the member buffer but before the pasynOctetSyncIO functions are called.  Then another thread could get access to your member char buffer.

 

Mark

 

 

From: Iain Marcuson <iain.marcuson at sydortechnologies.com>
Sent: Thursday, October 21, 2021 9:34 AM
To: Mark Rivers <rivers at cars.uchicago.edu>
Cc: tech-talk at aps.anl.gov
Subject: RE: QuadEM operation with parameters "Queued"

 

> asyn does queue both write and read operations if you create your driver with the ASYN_CANBLOCK flag.  Did you do that?

 

I am assuming I am not doing ASYN_CANBLOCK, as I see no references to it in the code

 

> How does your polling record work, i.e. is it going through your quadEM driver and then the underlying drvAsynIPPort driver?

 

The record has a SCAN field of 1 second.  When it is triggered, it invoke ::writeInt32(), which sets the output buffer to the empty string and then calls the function writeReadMeter() to write and read from the hardware.  writeReadMeter() skips the write on an empty string, and then reads the input buffer until empty.

 

> Changing parameter A and having no message sent does not make sense to me unless the asyn queueRequest is timing out.  Are there any messages on the IOC console when this happens?

 

I am having trouble reproducing the problem, but I don’t recall any messages like that.

 

> > All writes, which includes the 1 Hz record mentioned above, write to the same shared buffer that is then sent, so might there be contention there?

 

> Does your polling record write to a record which then results in your driver's writeOctet or similar method being called?  If so, then it should be OK because if you set ASYN_CANBLOCK then both the drvAsynIPPort driver lock, and the asynPortDriver lock for your quadEM driver will be held when writeOctet is called.  If you then write to the shared buffer and immediately call the underlying drvAsynIPPort->write() function then it should not be possible for 2 threads to be trying to use the shared buffer at the same time.

 

The polling record writes to a member char array and ultimately invokes pasynOctetSyncIO->write() and pasynOctetSyncIO->read().  There is no writing to another record.

 

 

Mark

 

 


From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Iain Marcuson via Tech-talk <tech-talk at aps.anl.gov>
Sent: Thursday, October 21, 2021 8:52 AM
To:
tech-talk at aps.anl.gov
Subject: QuadEM operation with parameters "Queued"

 

I am working on writing a new IOC for QuadEM device, and the IOC mostly works.  The IOC talks over TCP via asyn to a middle software layer that talks to the hardware.  The middle layer can send unsolicited messages to EPICS, so I implemented polling by having a record scan once a second and read the port.  Sometimes, however, I get a case where EPICS seems to be queueing writes, so I would get

Change parameter A -> No message send

Change parameter B -> Send A

Change parameter C -> Send B

 

All writes, which includes the 1 Hz record mentioned above, write to the same shared buffer that is then sent, so might there be contention there?

 

Thank you,

 

Iain Marcuson

Software Engineer, Sydor Technologies

585.278.1168 | www.SydorTechnologies.com

Skype: iain.marcuson at sydorinstruments.com

 

 

This message has been scanned for malware by Forcepoint. www.forcepoint.com

 

Click here to report this email as spam.


References:
QuadEM operation with parameters "Queued" Iain Marcuson via Tech-talk
Re: QuadEM operation with parameters "Queued" Mark Rivers via Tech-talk
RE: QuadEM operation with parameters "Queued" Iain Marcuson via Tech-talk
RE: QuadEM operation with parameters "Queued" Mark Rivers via Tech-talk
RE: QuadEM operation with parameters "Queued" Iain Marcuson via Tech-talk
RE: QuadEM operation with parameters "Queued" Mark Rivers via Tech-talk
RE: QuadEM operation with parameters "Queued" Iain Marcuson via Tech-talk

Navigate by Date:
Prev: Re: synApps motor build Siddons, David via Tech-talk
Next: IOC in Docker not responding to caget from another host Paduan Donadio, Marcio via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  <20212022  2023  2024 
Navigate by Thread:
Prev: RE: QuadEM operation with parameters "Queued" Iain Marcuson via Tech-talk
Next: Waveform Generator Proteus Model P2588D -- EPICS Device support Luchini, Kristi L. via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  <20212022  2023  2024 
ANJ, 27 Oct 2021 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·