EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  <20202021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  <20202021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: Re: StreamDevice, prevent records from getting "stuck"
From: Mark Davis via Tech-talk <tech-talk at aps.anl.gov>
To: tech-talk at aps.anl.gov, "Sobhani, Bayan" <bsobhani at bnl.gov>
Date: Tue, 14 Apr 2020 12:04:40 -0400
Regarding the "noise" characters:

If the "random characters" tend to occur between normal traffic (i.e. when a device is "driving" the signal lines) or at the end of normal transmissins, that could be an indication that the RS-485 connection is not properly terminated and/or biased.  I strongly recommend using digital scope to monitor the voltage levels of the signal wires to see if they are at a sufficient voltage level when nothing is driving the line.

When we where having this sort of problem a couple of years ago with a couple of devices using RS-485 (or RS-422), I read several different how-to's and explanations regarding how RS-485/RS-422 connections work and should be set up.  Unfortunately, there is one aspect of these descriptions where people seem to be roughly evenly split:  Should have/do you need a common reference for the signal voltages.  II suppose this is understandable, since what I read says the original standard said that insuring a "common return path" was optional, but the the newer ANSI/TIA/EIA-485 standard REQUIRES it for proper operation.

In our case, the problem was caused by not having the signal wires sufficiently biased to keep the voltage difference > 200mV when no device is driving them.  In our case the Moxas we use for the affected devices include pull up/down resistors that can be enabled and proved sufficient to satisfy this requirement.

If it would be at all helpful, I have some notes I took when learning about all this as well as still pictures and some video of what I found when diagnosing such problems for a couple of the devices at our lab.

Mark Davis
NSCL/FRIB Control Systems Software Engineer
davism50 at msu.edu

On 4/6/2020 7:48 PM, Sobhani, Bayan via Tech-talk wrote:
Maybe the reason for the nQueue being 5 is because there are multiple PVs associated with each device.

These are RS-485 temperature controllers that are connected to a moxa. I am not sure if "noise" is the correct word but some of these devices seem to be prone to sending random characters at times. For some of these devices changing the cabling seems to help. Other times it seems to be the device itself.


From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Monday, April 6, 2020 7:35:29 PM
To: Sobhani, Bayan <bsobhani at bnl.gov>
Cc: tech-talk <tech-talk at aps.anl.gov>
Subject: RE: StreamDevice, prevent records from getting "stuck"
 
Note that asynReport said "nQueued 5".  That suggests the port was busy while 5 more I/O requests were queued up.

This looks like a device that is not responding.

Your original message said
    `sometimes when the signal is noisy, the PV gets "stuck"`

What do you mean by "signal is noisy".  The TCP messages should not be "noisy".

Mark


-----Original Message-----
From: Sobhani, Bayan <bsobhani at bnl.gov>
Sent: Monday, April 6, 2020 5:57 PM
To: Mark Rivers <rivers at cars.uchicago.edu>
Cc: tech-talk <tech-talk at aps.anl.gov>
Subject: RE: StreamDevice, prevent records from getting "stuck"

The command I use to configure the port is drvAsynIPPortConfigure. For some reason I had some trouble reproducing the problems but after a few IOC restarts I got the PVs stuck again.

asynReport 10 PortName gives the following:

c3-tsrv1-p16 multiDevice:No canBlock:Yes autoConnect:Yes
    enabled:Yes connected:Yes numberConnects 1
    nDevices 0 nQueued 5 blocked:No
    asynManagerLock:No synchronousLock:Yes
    exceptionActive:No exceptionUsers 1 exceptionNotifys 0
    traceMask:0x1 traceIOMask:0x0 traceInfoMask:0x1
    interposeInterfaceList
        asynOctet pinterface 0x7f3dff3a9a40 drvPvt 0xe99ad0
    interfaceList
        asynCommon pinterface 0x7f3dff3a6d20 drvPvt 0xe98bc0
        asynOctet pinterface 0xe98ca8 drvPvt 0xe98bc0
    Port 10.17.2.60:4016: Connected
                    fd: 5
    Characters written: 117
       Characters read: 227

After running this, a few seconds later I got the message:

"2020/04/06 18:45:23.072801 c3-tsrv1-p16 XF:17ID-CT{RG:C3}T:1-SP: Timeout after reading 176 bytes "...<ff><ff><ff><ff><ff><ff><ff><ff><ff><ff><ff><ff><ff><ff><ff><ff><ff><ff><ff><ff>""

And then the PV started reading data again. I am not sure if the asynReport command is somehow fixing this or if it is just a coincidence.

Alex

-----Original Message-----
From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Monday, April 6, 2020 6:10 PM
To: Sobhani, Bayan <bsobhani at bnl.gov>
Cc: tech-talk <tech-talk at aps.anl.gov>
Subject: Re: StreamDevice, prevent records from getting "stuck"

What asyn driver are you using (drvAsynIPPort, drvAsynSerialPort, etc.)?


You should run the following to see if the problem is in the asyn driver


asynReport 10 PortName


where PortName is the name of the asyn port.  That will tell you if the port is connected, etc.


Mark



________________________________
From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Sobhani, Bayan via Tech-talk <tech-talk at aps.anl.gov>
Sent: Monday, April 6, 2020 4:57 PM
To: tech-talk at aps.anl.gov
Subject: StreamDevice, prevent records from getting "stuck"


For StreamDevice I notice that sometimes when the signal is noisy, the PV gets "stuck" if it makes one too many failed requests to the device. Getting "stuck" means that no matter how many times I process the PV with a StreamDevice OUT field, it does not seem to attempt to send anything to the device until I restart the IOC.



I see no technical reason why this must happen so I think this is probably something intentional in StreamDevice. Can I turn this off? I want the PV to always try to send a signal to the device, and if it can't I want to see the red letters saying there was a mismatch.



Is there any way to prevent the PVs from getting "stuck"?



Alex


Replies:
Re: StreamDevice, prevent records from getting "stuck" Mark Rivers via Tech-talk
RE: StreamDevice, prevent records from getting "stuck" Sobhani, Bayan via Tech-talk
References:
StreamDevice, prevent records from getting "stuck" Sobhani, Bayan via Tech-talk
Re: StreamDevice, prevent records from getting "stuck" Mark Rivers via Tech-talk
RE: StreamDevice, prevent records from getting "stuck" Sobhani, Bayan via Tech-talk
RE: StreamDevice, prevent records from getting "stuck" Mark Rivers via Tech-talk
Re: StreamDevice, prevent records from getting "stuck" Sobhani, Bayan via Tech-talk

Navigate by Date:
Prev: Re: Cross compiling for non-$EPICS_HOST_ARCH targets Timo Korhonen via Tech-talk
Next: Re: StreamDevice, prevent records from getting "stuck" Mark Rivers via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  <20202021  2022  2023  2024 
Navigate by Thread:
Prev: RE: StreamDevice, prevent records from getting "stuck" Sobhani, Bayan via Tech-talk
Next: Re: StreamDevice, prevent records from getting "stuck" Mark Rivers via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  <20202021  2022  2023  2024 
ANJ, 14 Apr 2020 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·