Re: Connection Timeout Using Asyn and StreamDevice with Ethernet Device

[Kyle Hesse via Tech-talk <tech-talk@aps.anl.gov>]

Hello Mark,

It is the former, I left the connection open for 10 days and made requests to the meter every five seconds. I have posted the script below.

Yes, a restart of the IOC does not reconnect. I have to cycle power. AutoConnect does try its best though.

2019/11/07 09:56:19.197 Can't connect to 129.57.230.28:23: No route to host ETH1 -1 autoConnect could not connect

2019/11/07 09:56:42.193 Can't connect to 129.57.230.28:23: No route to host ETH1 -1 autoConnect could not connect

2019/11/07 09:57:05.189 Can't connect to 129.57.230.28:23: No route to host ETH1 -1 autoConnect could not connect

2019/11/07 09:57:28.186 Can't connect to 129.57.230.28:23: No route to host ETH1 -1 autoConnect could not connect

2019/11/07 09:57:51.182 Can't connect to 129.57.230.28:23: No route to host ETH1 -1 autoConnect could not connect

2019/11/07 09:58:14.178 Can't connect to 129.57.230.28:23: No route to host ETH1 -1 autoConnect could not connect

2019/11/07 09:58:37.174 Can't connect to 129.57.230.28:23: No route to host ETH1 -1 autoConnect could not connect

Script:

#!/usr/csite/pubtools/bin/expect -f

set IP "129.57.230.28"

spawn telnet $IP

expect "Escape character is '^]'."

for {set i 0} {$i < 2000000000} {incr i} {

set cmd ":POWER?\r"

 send "$cmd"

 expect "${cmd}\n\r\n*\r" {}

 sleep 5

}

---

From: Mark Rivers <rivers@cars.uchicago.edu>
Sent: Thursday, November 7, 2019 3:16 PM
To: Kyle Hesse <hesse@jlab.org>
Cc: tech-talk@aps.anl.gov <tech-talk@aps.anl.gov>
Subject: [EXTERNAL] RE: Connection Timeout Using Asyn and StreamDevice with Ethernet Device

 

Hi Kyle,

 

Ø  As a test, I decided to remove EPICS from the equation and run a Tcl/tk expect script that opens the telnet connection and sends a request every five seconds. This ran for ten days without issue.

 

Does your Tcl/tk script leave the telnet connection open for 10 days, or does it close and re-open the connection every 5 seconds?  If it is the latter then it is not a good comparison to the EPICS IOC.

 

Restarting the IOC will not allow it to reconnect, you have to power-cycle the meter?

 

Mark

 

From: Tech-talk <tech-talk-bounces@aps.anl.gov> On Behalf Of Kyle Hesse via Tech-talk
Sent: Thursday, November 7, 2019 10:16 AM
To: tech-talk@aps.anl.gov
Subject: Connection Timeout Using Asyn and StreamDevice with Ethernet Device

 

Hello,

Thanks in advance for any help you may provide.

Problem:

 

Over the past few weeks we have noticed that our telnet connection on our Mini-Circuits power meter will go unresponsive when using asynDriver after several days of successful use. We can still ping it, the HTML interface over ethernet is responsive, but the telnet port can no longer connect without a device power cycle. As a test, I decided to remove EPICS from the equation and run a Tcl/tk expect script that opens the telnet connection and sends a request every five seconds. This ran for ten days without issue. Any other diagnostic recommendations or theories as to what might be causing this behavior?      

 

Setup:

 

We use a Mini-Circuits PWR-8P-RC Power Meter which is network configured. The control protocol is SCPI ASCII. We communicate through Telnet (port 23) over a TCP/IP connection. ASYN (4-31) with StreamDevice (2-7) is used for the EPICS (3.14.12.3) implementation.

 

PV Example:

 

record(ai,"$(P)$(R)POWER_PEAK_RAW") {

field(SCAN,"5 second")

field(DTYP,"stream")

field(FLNK,"$(P)$(R)POWER_PEAK")

field(INP,"@PWR-xP.proto getPwrPeak $(PORT)")

field(PREC,"3")

field(EGU,"dBm")

}

 

Protocol Example:

 

OutTerminator = CR LF;

InTerminator = LF CR;

writetimeout = 300;

readtimeout = 1000;

replytimeout = 2000;

ExtraInput = Ignore;

 

getPwrPeak {

    out ":POWER?";

    in "%f dBm %*f dBm";

}

 

Port Configuration:

 

drvAsynIPPortConfigure("ETH1",129.57.230.28:23)

 

ERROR Messages:

 

scan5 HOM:PWR1:POWER_PEAK_RAW lockRequest: pasynManager->queueRequest: port ETH1 not connected

 

"Can't connect to 129.57.230.28:23: Connection timed out ETH1 -1 autoConnect could not connect"

 

Diagnostics:

 

This was the response after the device had disconnected and I restarted the softIoc.

 

epics>asynReport(1,ETH1)

    ETH1 multiDevice:No canBlock:Yes autoConnect:Yes

    enabled:Yes connected:No numberConnects 0

    nDevices 0 nQueued 1 blocked:No

    asynManagerLock:No synchronousLock:Yes

    exceptionActive:No exceptionUsers 1 exceptionNotifys 0

    traceMask:0x1 traceIOMask:0x0 traceInfoMask:0x1

    Port 129.57.230.28:23: Disconnected

 

This was a response after the power meter was power cycled.

 

epics> asynReport(1,ETH1)

ETH1 multiDevice:No canBlock:Yes autoConnect:Yes

    enabled:Yes connected:Yes numberConnects 1

    nDevices 0 nQueued 5 blocked:No

    asynManagerLock:No synchronousLock:Yes

    exceptionActive:No exceptionUsers 1 exceptionNotifys 0

    traceMask:0x1 traceIOMask:0x0 traceInfoMask:0x1

    Port 129.57.230.28:23: Connected