EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: RE: How to detect AsynIPPort disconnect?
From: Mark Rivers via Tech-talk <[email protected]>
To: 'Dirk Zimoch' <[email protected]>, 'Torsten Bögershausen' <[email protected]>
Cc: "'[email protected]'" <[email protected]>
Date: Fri, 15 Feb 2019 17:45:10 +0000
Hi Dirk,

Indeed what I suspected about disconnectOnReadTimeout incorrectly disconnecting if pasynUser->timeout is 0 was the problem.

That is not logical behavior, and it breaks StreamDevice.  I have fixed that problem in asyn and pushed to Github.

Now StreamDevice works correctly with 3 records each scanned at 1 second.

This is what I see if disconnectOnReadTimeout is not set and I disconnect the Ethernet cable to my device.

******************************
2019/02/15 11:33:50.870 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:33:50.872 gse-tetramm2:10001 read 7
CHN:4\r\n
2019/02/15 11:33:50.873 gse-tetramm2:10001 write 6
HVV:?\r
2019/02/15 11:33:50.874 gse-tetramm2:10001 read 10
HVV:0.00\r\n
2019/02/15 11:33:51.868 gse-tetramm2:10001 write 7  DISCONNECTED CABLE HERE
TEMP:?\r
2019/02/15 11:33:52.870 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:33:53.872 gse-tetramm2:10001 write 6
HVV:?\r
2019/02/15 11:33:54.874 gse-tetramm2:10001 write 7
TEMP:?\r
2019/02/15 11:33:55.876 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:33:56.878 gse-tetramm2:10001 write 6
HVV:?\r
2019/02/15 11:33:57.880 gse-tetramm2:10001 write 7
TEMP:?\r
2019/02/15 11:33:58.883 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:33:59.885 gse-tetramm2:10001 write 6
HVV:?\r
2019/02/15 11:34:00.887 gse-tetramm2:10001 write 7  RECONNECTED CABLE HERE
TEMP:?\r
2019/02/15 11:34:01.889 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:34:02.891 gse-tetramm2:10001 write 6
HVV:?\r
2019/02/15 11:34:03.892 gse-tetramm2:10001 write 7
TEMP:?\r
2019/02/15 11:34:04.894 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:34:05.896 gse-tetramm2:10001 write 6
HVV:?\r
2019/02/15 11:34:06.898 gse-tetramm2:10001 write 7
TEMP:?\r
2019/02/15 11:34:07.900 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:34:08.902 gse-tetramm2:10001 write 6
HVV:?\r
2019/02/15 11:34:09.904 gse-tetramm2:10001 write 7
TEMP:?\r
2019/02/15 11:34:10.906 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:34:11.908 gse-tetramm2:10001 write 6
HVV:?\r
2019/02/15 11:34:12.910 gse-tetramm2:10001 write 7
TEMP:?\r
2019/02/15 11:34:13.911 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:34:14.913 gse-tetramm2:10001 write 6
HVV:?\r
2019/02/15 11:34:15.915 gse-tetramm2:10001 write 7
TEMP:?\r
2019/02/15 11:34:16.917 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:34:17.601 gse-tetramm2:10001 write 6  COMMUNICATION RE-ESTABLISHED HERE
HVV:?\r
2019/02/15 11:34:17.602 gse-tetramm2:10001 read 10 
HVV:0.00\r\n
2019/02/15 11:34:17.868 gse-tetramm2:10001 write 7
TEMP:?\r
2019/02/15 11:34:17.869 gse-tetramm2:10001 read 9
TEMP:34\r\n
2019/02/15 11:34:17.870 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:34:17.871 gse-tetramm2:10001 read 7
******************************

So with disconnectOnReadTimeout not set it kept trying to write, and it took 17 seconds to recover after plugging the cable back in.


This is what I see with disconnectOnReadTimeout set to "Y".

******************************
2019/02/15 11:39:00.115 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:39:00.116 gse-tetramm2:10001 read 7
CHN:4\r\n
2019/02/15 11:39:00.118 gse-tetramm2:10001 write 6
HVV:?\r
2019/02/15 11:39:00.119 gse-tetramm2:10001 read 10
HVV:0.00\r\n
2019/02/15 11:39:01.113 gse-tetramm2:10001 write 7   CABLE DISCONNECTED HERE
TEMP:?\r

CABLED WAS RECONNECTED AT 11:39:11

2019/02/15 11:39:18.113 gse-tetramm2:10001 write 7
TEMP:?\r
2019/02/15 11:39:18.114 gse-tetramm2:10001 read 9
TEMP:34\r\n
2019/02/15 11:39:18.115 gse-tetramm2:10001 write 6
CHN:?\r
2019/02/15 11:39:18.116 gse-tetramm2:10001 read 7
CHN:4\r\n
******************************

So with disconnectOnReadTimeout set there is not the constant stream of write attempts once the cable is disconnected.  After reconnecting the cable it only took 7 seconds for communication to recover, compared to 17 seconds without disconnectOnReadTimeout.

These observations are consistent with the ones I made yesterday without StreamDevice.

Mark

-----Original Message-----
From: Mark Rivers 
Sent: Friday, February 15, 2019 10:59 AM
To: 'Dirk Zimoch' <[email protected]>; 'Torsten Bögershausen' <[email protected]>
Cc: '[email protected]' <[email protected]>
Subject: RE: How to detect AsynIPPort disconnect?

Hi Dirk,

> I tried it but I have a problem when two or more StreamDevice records try to access the port concurrently (e.g. both scanned "1 second". 
> Normally they are queued by queueRequest(), but with disconnectOnReadTimeout enabled, the second queueRequest fails with timeout and the ports disconnects.

I have reproduced this issue.  However, for me it fails even with a single record scanned at "1 second".  

I suspect this is because StreamDevice flushes the input using a series of read commands until a read times out, rather than calling flush.  It does this to support I/O Intr scanning I believe.  Perhaps drvAsynIPPort should treat the case of a timeout of 0 specially and not disconnect the port if a read with a timeout of 0 times out.

This is what I see with a single input record if I use asynSetOption(PORT, 0, "disconnectOnReadTimeout", "Y").  If I don't use that it works fine with 3 records.

drvAsynIPPortConfigure("PORT","gse-tetramm2:10001",0,0,0)
asynOctetSetInputEos(PORT, 0, "\r\n")
asynOctetSetOutputEos(PORT, 0, "\r")
asynSetTraceIOMask(PORT, 0, 2)
asynSetTraceMask(PORT, 0, 9)
asynSetOption(PORT, 0, "disconnectOnReadTimeout", "Y")

dbLoadRecords("stream2.db", "P=13TEST:, R=R:, PORT=PORT") #dbLoadRecords("$(ASYN)/db/asynRecord.db", "P=13TEST:, R=asyn1, PORT=PORT, ADDR=0, IMAX=80, OMAX=80")

iocInit
Starting iocInit
############################################################################
## EPICS R7.0.2
## EPICS Base built Dec 21 2018
############################################################################
drvStreamInit: Warning! STREAM_PROTOCOL_PATH not set. Defaults to "."
iocRun: All initialization complete
2019-02-15T10:54:27.549 Using dynamically assigned TCP port 33371.
asynSetTraceMask(PORT, 0, 9)
epics> 2019/02/15 10:54:28.049 gse-tetramm2:10001 write 7
TEMP:?\r

epics> dbl
13TEST:R:Temp
epics>
epics> asynReport 10
PORT multiDevice:No canBlock:Yes autoConnect:Yes
    enabled:Yes connected:No numberConnects 1
    nDevices 0 nQueued 0 blocked:No
    asynManagerLock:No synchronousLock:No
    exceptionActive:No exceptionUsers 2 exceptionNotifys 0
    traceMask:0x9 traceIOMask:0x2 traceInfoMask:0x1
    interposeInterfaceList
        asynOctet pinterface 0x1da89e0 drvPvt 0x26517d0
    interfaceList
        asynCommon pinterface 0x1ce6770 drvPvt 0x264db70
        asynOption pinterface 0x1ce6790 drvPvt 0x264db70
        asynOctet pinterface 0x264dc90 drvPvt 0x264db70
    Port gse-tetramm2:10001: Disconnected
                    fd: -1
    Characters written: 0
       Characters read: 0
epics> 2019/02/15 10:54:49.049 gse-tetramm2:10001 write 7
TEMP:?\r



-----Original Message-----
From: Dirk Zimoch <[email protected]>
Sent: Friday, February 15, 2019 3:15 AM
To: Mark Rivers <[email protected]>; 'Torsten Bögershausen' <[email protected]>
Cc: '[email protected]' <[email protected]>
Subject: Re: How to detect AsynIPPort disconnect?

Hi Mark,

I tried it but I have a problem when two or more StreamDevice records try to access the port concurrently (e.g. both scanned "1 second". 
Normally they are queued by queueRequest(), but with disconnectOnReadTimeout enabled, the second queueRequest fails with timeout and the ports disconnects.

Need to investigate details...

Dirk


On 14.02.19 20:00, Mark Rivers wrote:
> Hi Dirk,
> 
> I just did some simple tests.  I have a TCP device, and an IOC with a single asyn record.  The asyn record is reading the device temperature at 1 Hz.
> 
> I then disconnect the Ethernet cable from the device, and after some time reconnect it.
> 
> This is a test on Linux with the following conditions:
> 
> - autoConnect is true
> - disconnectOnReadTimeout is not set
> - ASYN_TRACE_ERROR and ASYN_TRACEIO_DRIVER are set
> - Cable was disconnected for 10 seconds and then reconnected
> - It took 16 seconds after reconnecting cable for communication to be 
> restored
> 
> ************************************
> 2019/02/14 11:54:58.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:54:58.080 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 11:54:59.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:54:59.080 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 11:55:00.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:00.080 164.54.160.165:10001 read 9             DISCONNECTED HERE
> TEMP:30\r\n
> 2019/02/14 11:55:01.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:03.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:05.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:07.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:09.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:11.079 164.54.160.165:10001 write 7      RECONNECTED HERE
> TEMP:?\r
> 2019/02/14 11:55:13.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:15.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:17.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:19.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:21.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:23.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:25.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:27.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:27.079 quadEMTest:asyn1: Write error, nout=0,
> 164.54.160.165:10001 write error: Broken pipe
> 2019/02/14 11:55:27.079 quadEMTest:asyn1: Error 164.54.160.165:10001 disconnected:
> 2019/02/14 11:55:27.079 quadEMTest:asyn1: error  nread 0 164.54.160.165:10001 disconnected:
> 2019/02/14 11:55:28.079 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 11:55:28.080 164.54.160.165:10001 read 9 TEMP:30\r\n
> ************************************
> 
> This time I set disconnectOnReadTimeout, all other settings are the same.
> drvAsynIPPort disconnected on the first read timeout.
> It only took 6 seconds for communications to restart after reconnecting the cable.
> 
> ************************************
> 2019/02/14 12:12:28.717 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:12:28.718 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:12:29.717 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:12:29.718 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:12:30.717 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:12:30.718 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:12:31.717 164.54.160.165:10001 write 7  DISCONNECTED 
> HERE TEMP:?\r
> 2019/02/14 12:12:32.718 quadEMTest:asyn1: Error 164.54.160.165:10001 
> read error: Resource temporarily unavailable
> 2019/02/14 12:12:32.718 quadEMTest:asyn1: error  nread 0
> 164.54.160.165:10001 read error: Resource temporarily unavailable
> 
> RECONNECTED 10 seconds later, at 12:12:42
> 
> 2019/02/14 12:12:48.717 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:12:48.718 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:12:49.717 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:12:49.718 164.54.160.165:10001 read 9 TEMP:30\r\n
> ************************************
> 
> This time I disabled disconnectOnReadTimeout, and left the cable disconnected until recv() returned 0, i.e. disconnect.
> It took 16 minutes for the OS to disconnect the socket, and thus for asyn to report that the port was disconnected.
> I then reconnected the cable and in about 10 seconds communication resumed.
> 
> ************************************
> 2019/02/14 12:17:49.511 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:17:50.510 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:17:50.511 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:17:51.510 164.54.160.165:10001 write 7   DISCONNECTED HERE
> TEMP:?\r
> 2019/02/14 12:17:53.510 164.54.160.165:10001 write 7 TEMP:?\r ...
> 2019/02/14 12:33:21.510 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:33:23.510 164.54.160.165:10001 write 7
> TEMP:?\r                                                                                        THIS IS WHEN THE OS CLOSED THE SOCKET IT SHOWS AS DISCONNECTED
> 2019/02/14 12:33:24.442 quadEMTest:asyn1: Error 164.54.160.165:10001 
> read error: No route to host
> 2019/02/14 12:33:24.442 quadEMTest:asyn1: error  nread 0
> 164.54.160.165:10001 read error: No route to host
> 2019/02/14 12:33:30.460 IP_TetrAMM -1 autoConnect could not connect
> 
> Plugged cabled back in about 12:33:40.
> 
> 2019/02/14 12:33:50.455 IP_TetrAMM -1 port is now connected
> 2019/02/14 12:33:50.510 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:33:50.511 164.54.160.165:10001 read 9 TEMP:27\r\n
> 2019/02/14 12:33:51.510 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:33:51.511 164.54.160.165:10001 read 9 TEMP:27\r\n
> ************************************
> 
> I then repeated the tests on Windows
> 
> - autoConnect is true
> - disconnectOnReadTimeout is not set
> - ASYN_TRACE_ERROR and ASYN_TRACEIO_DRIVER are set
> - Cable was disconnected for 10 seconds and then reconnected
> - It took 7 seconds after reconnecting cable for communication to be 
> restored
> 
> ************************************
> 2019/02/14 12:45:59.944 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:45:59.947 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:46:00.944 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:46:00.947 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:46:01.944 164.54.160.165:10001 write 7   DISCONNECTED HERE
> TEMP:?\r
> 2019/02/14 12:46:03.946 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:46:05.946 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:46:07.945 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:46:09.944 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:46:11.944 164.54.160.165:10001 write 7  RECONNECTED HERE 
> TEMP:?\r
> 2019/02/14 12:46:13.946 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:46:15.946 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:46:17.945 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:46:17.950 quadEMTest:asyn1: Error 164.54.160.165:10001 
> read error: Unknown error
> 2019/02/14 12:46:17.950 quadEMTest:asyn1: error  nread 0
> 164.54.160.165:10001 read error: Unknown error
> 2019/02/14 12:46:18.944 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:46:18.949 164.54.160.165:10001 read 9 TEMP:30\r\n
> ************************************
> 
> This time I set disconnectOnReadTimeout, all other settings are the same.
> drvAsynIPPort disconnected on the first read timeout.
> It took about 7 seconds for communications to restart after reconnecting the cable.
> 
> ************************************
> 2019/02/14 12:49:25.316 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:49:25.320 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:49:26.316 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:49:26.319 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:49:27.315 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:49:27.319 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:49:28.314 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:49:28.318 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:49:29.314 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:49:29.318 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:49:30.313 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:49:30.316 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:49:31.314 164.54.160.165:10001 write 7  DISCONNECTED 
> HERE TEMP:?\r
> 2019/02/14 12:49:32.317 quadEMTest:asyn1: Error 164.54.160.165:10001 
> read error: Unknown error
> 2019/02/14 12:49:32.317 quadEMTest:asyn1: error  nread 0
> 164.54.160.165:10001 read error: Unknown error
> 
> Reconnected cable at 12:39:42
> 
> 2019/02/14 12:49:54.317 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:49:54.321 164.54.160.165:10001 read 9 TEMP:30\r\n
> ************************************
> 
> This time I disabled disconnectOnReadTimeout, and left the cable disconnected until recv() returned 0, i.e. disconnect.
> It only took 20 seconds for the OS to disconnect the socket, and thus for asyn to report that the port was disconnected.
> I then reconnected the cable and in about 11 seconds communication resumed.
> 
> ************************************
> 2019/02/14 12:53:29.724 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:53:29.728 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:53:30.725 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:53:30.728 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:53:31.724 164.54.160.165:10001 write 7  DISCONNECTED 
> HERE TEMP:?\r
> 2019/02/14 12:53:33.725 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:53:35.723 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:53:37.723 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:53:39.723 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:53:41.725 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:53:43.725 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:53:45.723 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:53:47.723 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:53:49.723 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:53:50.629 quadEMTest:asyn1: Error 164.54.160.165:10001 
> read error: Unknown error
> 2019/02/14 12:53:50.630 quadEMTest:asyn1: error  nread 0
> 164.54.160.165:10001 read error: Unknown error
> 
> Reconnected the cable at ~12:54:00
> 
> 2019/02/14 12:54:11.725 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:54:11.729 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:54:12.725 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:54:12.730 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:54:13.725 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:54:13.730 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:54:14.725 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:54:14.729 164.54.160.165:10001 read 9 TEMP:30\r\n
> 2019/02/14 12:54:15.723 164.54.160.165:10001 write 7 TEMP:?\r
> 2019/02/14 12:54:15.727 164.54.160.165:10001 read 9 TEMP:30\r\n
> ************************************
> 
> So the bottom line is that Linux takes about 16 minutes to disconnect the socket when the cable is disconnected.  Windows only takes 20 seconds.
> 
> If you set disconnectOnReadTimeout it will immediately disconnect the socket on the first read timeout.  But if autoConnect is true it will keep trying to reconnect on each read or write operation, so it will quickly reconnect when the device is available again.
> 
> Mark
> 
> 
> 
> -----Original Message-----
> From: Mark Rivers
> Sent: Thursday, February 14, 2019 11:42 AM
> To: 'Dirk Zimoch' <[email protected]>; Torsten Bögershausen 
> <[email protected]>
> Cc: [email protected]
> Subject: RE: How to detect AsynIPPort disconnect?
> 
>> Well, not quite what I am looking for. A single read timeout may be cause by a device that is busy or just slow.
> 
> But if you set autoConnect=true then even though it disconnected on the read timeout, it will automatically try to reconnect on the next asyn write or read operation, so do you care that it disconnected?
> 
> I am going to set up a simple TCP device and do some experiments with disconnectOnReadTimeout, both on Linux and Windows.
> 
> Mark
> 
> 
> -----Original Message-----
> From: Dirk Zimoch <[email protected]>
> Sent: Thursday, February 14, 2019 10:11 AM
> To: Mark Rivers <[email protected]>; Torsten Bögershausen 
> <[email protected]>
> Cc: [email protected]
> Subject: Re: How to detect AsynIPPort disconnect?
> 
> Hi Mark, Torsten,
> 
> On 14.02.19 12:38, Torsten Bögershausen wrote:
> 
>> What happens then ?
> 
> Then the port goes to disconnect state (as shown by dbior) and the exception callback triggers. Also all further read/write commands fail.
> 
>> Later asyn versions (including 4.34) have an option 
>> "disconnectOnReadTimeout"
> 
> Well, not quite what I am looking for. A single read timeout may be cause by a device that is busy or just slow.
> 
> 
> On 14.02.19 14:14, Mark Rivers wrote:
>> Hi Dirk,
>>
>>
>> The problem you are seeing is due to the behavior of the behavior of recv() on your particular OS.  This is from the man page for recv():
> 
> My particular OS is Scientific Linux 6.10.
> 
>>
>>
>> ***************
>>
>> RETURN VALUE
>>          These calls return the number of bytes received, or -1 if an error occurred.  In the event of an error, errno is set to indicate the error.  The return value will be 0 when the peer  has  performed an orderly shutdown.
>>
>> ERRORS
>>          These are some standard errors generated by the socket layer.  Additional errors may be generated and returned from the underlying protocol modules; see their manual pages.
>>
>>          EAGAIN or EWOULDBLOCK
>>                 The  socket  is marked nonblocking and the receive operation would block, or a receive timeout had been set and the timeout expired before data was received.  POSIX.1-2001 allows either error to be returned for this case, and does not require these constants to have the same value, so a portable application should check for both possibilities.
>>
>> ***************
>>
>>
>> For 15 minutes recv() is returning -1 with SOCKERRNO=EAGAIN, EINTR, or EWOULDBLOCK. After 15 minutes recv() finally returns 0, which according to man recv means the peer has performed an orderly shutdown.  Is this on Linux?  The last time I remember testing on Windows the time before recv() returns 0 was much shorter, 30 seconds I think.
> 
> It may as well be that for 15 minutes the SendQ of the socket fills up and only when it overflows I get an error.
> 
> Also the device does not perform an orderly shutdown.
> 
> With devices that shut down their socket this problem does not exist. At least not at this extent. The next send or recv will fail on an orderly shut down connection. (Unfortunately when you don't communicate, you don't know that the connection is closed. E.g. HTTP messages where the server closes the connection after sending the page are only detected when accessing them the next time. Hence the need for the "http"
> protocol option in drvAsynIPPortConfigure.)
> 
> In my case I simply power off the device. So the socket becomes unresponsive.
> 
> Would the SO_KEEPALIVE socket option be a solution (accessible via asynSetOption)?
> 
> Dirk
> 
>>
>>
>> As Torsten suggested, you can use asynSetOption to set the option DisconnectOnReadTimeout=Y.  That way any time a read operation times out it will disconnect the port.  This was added in asyn R4-29.
>>
>>
>>
>> The following table summarizes the drvAsynIPPort driver asynSetOption keys and values.
>>
>> Key     Value   Description
>> disconnectOnReadTimeout N Y     Default=N. If Y then if a read operation times out the driver automatically disconnect the IP port.
>> hostInfo        <host>:<port>[:localPort] [protocol]    The IP port hostInfo specification using the same syntax as drvAsynIPPortConfigure. This option allows changing at run time the Internet host and port to which this asyn port is connected. The only restriction is that the setting of the COM (TELNET RFC 2217) protocol cannot be changed from that specified with drvAsynIPPortConfigure. This is because if COM is specified in the drvAsynIPPortConfigure command then asynOctet and asynOption interpose interfaces are used, and asynManager does not support removing interpose interfaces.
>>
>> In addition to these key/value pairs if the COM protocol is used then the drvAsynIPPort driver uses the same key/value pairs as the drvAsynSerialPort driver for specifying the serial parameters, i.e. "baud", "bits", etc.
>>
>>
>>
>> Mark
>>
>>
>>
>> ________________________________
>> From: [email protected] <[email protected]> 
>> on behalf of Torsten Bögershausen via Tech-talk 
>> <[email protected]>
>> Sent: Thursday, February 14, 2019 5:38 AM
>> To: Dirk Zimoch
>> Cc: EPICS
>> Subject: Re: How to detect AsynIPPort disconnect?
>>
>>
>>> Am 14.02.2019 um 11:51 schrieb Dirk Zimoch via Tech-talk <[email protected]>:
>>>
>>> Hello,
>>>
>>> I have a network device connected with drvAsynIPPort. When I switch off the device, the asyn port stays in "connected" state for about 15 minutes.
>>
>> What happens then ?
>>>
>>> Writes to the device report no error, reads time out of course. But the port is still shown as connected. I have an exceptionCallback installed, but that is not triggered.
>>>
>>
>> Later asyn versions (including 4.34) have an option 
>> "disconnectOnReadTimeout"
>>
>>> How can I make asyn find out within a reasonable time if the device is still connected?
>>>
>>
>> Stolen from asynRecord.c:
>>
>>       case asynRecordDRTO:
>>           status = pasynRecPvt->pasynOption->setOption(pasynRecPvt->asynOptionPvt,
>>              pasynUser, "disconnectOnReadTimeout", drto_choices[pasynRec->drto]);
>>           break;
>>
>>
>>> Using asyn 4.34 and EPICS 7.0.1.
>>>
>>> Dirk
>>
>> HTH
>>

Replies:
Re: How to detect AsynIPPort disconnect? Dirk Zimoch via Tech-talk
References:
How to detect AsynIPPort disconnect? Dirk Zimoch via Tech-talk
Re: How to detect AsynIPPort disconnect? Torsten Bögershausen via Tech-talk
Re: How to detect AsynIPPort disconnect? Mark Rivers via Tech-talk
Re: How to detect AsynIPPort disconnect? Dirk Zimoch via Tech-talk
RE: How to detect AsynIPPort disconnect? Mark Rivers via Tech-talk
RE: How to detect AsynIPPort disconnect? Mark Rivers via Tech-talk
Re: How to detect AsynIPPort disconnect? Dirk Zimoch via Tech-talk
RE: How to detect AsynIPPort disconnect? Mark Rivers via Tech-talk

Navigate by Date:
Prev: Re: How to detect AsynIPPort disconnect? Torsten Bögershausen via Tech-talk
Next: Problems with StreamDevice Mark Rivers via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020  2021  2022  2023  2024 
Navigate by Thread:
Prev: Re: How to detect AsynIPPort disconnect? Torsten Bögershausen via Tech-talk
Next: Re: How to detect AsynIPPort disconnect? Dirk Zimoch via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020  2021  2022  2023  2024 
ANJ, 18 Feb 2019 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·