I have found a solution. I see from various logs that network commands weren’t making their way out. I eventually observed that a setIntegerParam() invocation would clear
the queue. I now perform a write to a dummy PV on a periodic basis, and this causes the message to flow from the queue.
From: Tech-talk <tech-talk-bounces at aps.anl.gov>
On Behalf Of Iain Marcuson via Tech-talk
Sent: Thursday, October 21, 2021 2:55 PM
To: Mark Rivers <rivers at cars.uchicago.edu>
Cc: tech-talk at aps.anl.gov
Subject: RE: QuadEM operation with parameters "Queued"
I wrapped the send/receive with semaphores, so that should stop any clobbering. I still observed the weird behavior on PC1. I then pointed EPICS to a different computer (PC2)
running the middle layer, whereupon I was not seeing the message backlog. I am wondering if either there was a change to the middle layer on PC2 that hasn’t been reflected to PC1, or if messages on PC1 were too fast, being localhost and a faster machine.
I did not see these problems on PC2.
You can add some asynPrint statements with before the member buffer is written to and see if one thread is clobbering the string before it is written. Use ASYN_TRACEINFO_THREAD
to see what thread is calling it.
Mark
I took a look, and it seems that I only call unlock() in a different thread that reads the data over its own socket, versus the command handling thread(s?) where I don’t do
any locking.
- I am assuming I am not doing ASYN_CANBLOCK, as I see no references to it in the code
Actually the base drvQuadEM class sets ASYN_CANBLOCK in its constructor, so you don’t need to worry about it.
One thing that could cause a problem is if your driver is calling unlock() after it writes to the member buffer but before the pasynOctetSyncIO functions are called. Then another thread could get access to your member char buffer.
Mark
> asyn does queue both write and read operations if you create your driver with the ASYN_CANBLOCK flag. Did you do that?
I am assuming I am not doing ASYN_CANBLOCK, as I see no references to it in the code
> How does your polling record work, i.e. is it going through your quadEM driver and then the underlying drvAsynIPPort driver?
The record has a SCAN field of 1 second. When it is triggered, it invoke ::writeInt32(), which sets the output buffer to the empty string and then calls the function writeReadMeter() to write and read from the hardware. writeReadMeter() skips the write
on an empty string, and then reads the input buffer until empty.
> Changing parameter A and having no message sent does not make sense to me unless the asyn queueRequest is timing out. Are there any messages on the IOC console when this happens?
I am having trouble reproducing the problem, but I don’t recall any messages like that.
> > All writes, which includes the 1 Hz record mentioned above, write to the same shared buffer that is then sent, so might there be contention there?
> Does your polling record write to a record which then results in your driver's writeOctet or similar method being called? If so, then it should be OK because if you set ASYN_CANBLOCK then both the drvAsynIPPort driver lock, and
the asynPortDriver lock for your quadEM driver will be held when writeOctet is called. If you then write to the shared buffer and immediately call the underlying drvAsynIPPort->write() function then it should not be possible for 2 threads to be trying to
use the shared buffer at the same time.
The polling record writes to a member char array and ultimately invokes pasynOctetSyncIO->write() and pasynOctetSyncIO->read(). There is no writing to another record.
Mark
I am working on writing a new IOC for QuadEM device, and the IOC mostly works. The IOC talks over TCP via asyn to a middle software layer that talks to the hardware. The middle layer can send unsolicited messages to EPICS, so
I implemented polling by having a record scan once a second and read the port. Sometimes, however, I get a case where EPICS seems to be queueing writes, so I would get
Change parameter A -> No message send
Change parameter B -> Send A
Change parameter C -> Send B
All writes, which includes the 1 Hz record mentioned above, write to the same shared buffer that is then sent, so might there be contention there?
Thank you,
Iain Marcuson
Software Engineer, Sydor Technologies
585.278.1168 |
www.SydorTechnologies.com
Skype: iain.marcuson at sydorinstruments.com
This message has been scanned for malware by Forcepoint.
www.forcepoint.com
Click
here to report this email as spam.