RE: asynManager isConnected

[Mark Rivers <rivers@cars.uchicago.edu>]

Hi Matt,

I've added a new test application and test IOC to asyn to test the connection problems you reported.  This should be a useful test application to have in the future as well, since I think connection management is one of the under-utilized and under-tested features of asyn.

asyn/testConnectApp
asyn/iocBoot/ioctestConnect

This application is based on asynPortDriver.  It is very simple.

- It is invoked with testConnectConfigure("Port", "IPPort", "outputString")

where Port is the name of the asyn port for the test driver, IPPort is the name of the asyn port for an IP port previously created with drvAsynIPPortConfigure().  outputString is a string that will be sent to the device in the poller thread, followed by reading the response.

- In the constructor pasynOctetSyncIO->connect() is called.  It reports an error if it fails, but it should never fail, even if the IP port is not available.

- The driver creates a poller thread that does the following in a loop with a 1 second sleep time:
  - Prints out the connection state from pasynManager->isConnected().
  - Calls pasynOctetSyncIO->writeRead().  Prints an error message if this fails, or the result of the
    writeRead() if it succeeds.

I have used this new test application to test 3 cases:
1) IPPort port never connects because the IP device does not exist
2) IPPort port immediately connects and succeeds
3) IPPort port initially fails to connect, but then the IP device becomes available,
   and it does connect and communicate.

Here is the output of case 1).  The IP server address 164.54.160.220:20 does not exist, so it never connects.

**********************************
corvette:asyn/iocBoot/ioctestConnect>../../bin/linux-x86/testConnect st.cmd
dbLoadDatabase("../../dbd/testConnect.dbd")
testConnect_registerRecordDeviceDriver(pdbbase)
# This is a bogus IP address that will never connect
drvAsynIPPortConfigure("IPPort", "164.54.160.220:20", 0, 0, 1);
# This is a real IP address that will connect
#drvAsynIPPortConfigure("IPPort", "newport-xps5:5001", 0, 0, 1);
testConnectConfigure("PORT1", "IPPort", "FirmwareVersionGet(char *)")
asynSetTraceMask("PORT1",0,9)
asynSetTraceIOMask("PORT1",0,0x2)
dbLoadRecords("../../db/asynRecord.db","P=testConnect:,R=asyn1,PORT=PORT1,ADDR=0,OMAX=80,IMAX=80")
iocInit()
Starting iocInit
############################################################################
## EPICS R3.14.12.2 $Date: Mon 2011-12-12 14:09:32 -0600$
## EPICS Base built May 31 2012
2012/06/02 11:15:00.461 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
############################################################################
cas warning: Configured TCP port was unavailable.
cas warning: Using dynamically assigned TCP port 33326,
cas warning: but now two or more servers share the same UDP port.
cas warning: Depending on your IP kernel this server may not be
cas warning: reachable with UDP unicast (a host's IP in EPICS_CA_ADDR_LIST)
iocRun: All initialization complete
epics> 2012/06/02 11:15:01.462 testConnect:pollerTask: isConnected = 0
2012/06/02 11:15:01.462 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
2012/06/02 11:15:02.462 testConnect:pollerTask: isConnected = 0
2012/06/02 11:15:02.462 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
2012/06/02 11:15:03.462 testConnect:pollerTask: isConnected = 0
2012/06/02 11:15:03.462 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
2012/06/02 11:15:04.462 testConnect:pollerTask: isConnected = 0
2012/06/02 11:15:04.462 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
2012/06/02 11:15:05.463 testConnect:pollerTask: isConnected = 0
2012/06/02 11:15:05.463 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
2012/06/02 11:15:05.973 Can't connect to 164.54.160.220:20: No route to host IPPort -1 autoConnect could not connect
2012/06/02 11:15:06.463 testConnect:pollerTask: isConnected = 0
2012/06/02 11:15:06.463 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
2012/06/02 11:15:07.463 testConnect:pollerTask: isConnected = 0
**********************************

Note that is always reports isConnected=0, and pasynOctetSyncIO->writeRead always fails with status=3, because the port is not connected.


Here is the output of case 2.  In this case we are connecting to a Newport XPS and sending a valid command and receiving a response:

**********************************
corvette:asyn/iocBoot/ioctestConnect>../../bin/linux-x86/testConnect st.cmd
dbLoadDatabase("../../dbd/testConnect.dbd")
testConnect_registerRecordDeviceDriver(pdbbase)
# This is a bogus IP address that will never connect
#drvAsynIPPortConfigure("IPPort", "164.54.160.220:20", 0, 0, 1);
# This is a real IP address that will connect
drvAsynIPPortConfigure("IPPort", "newport-xps5:5001", 0, 0, 1);
testConnectConfigure("PORT1", "IPPort", "FirmwareVersionGet(char *)")
asynSetTraceMask("PORT1",0,9)
asynSetTraceIOMask("PORT1",0,0x2)
dbLoadRecords("../../db/asynRecord.db","P=testConnect:,R=asyn1,PORT=PORT1,ADDR=0,OMAX=80,IMAX=80")
iocInit()
Starting iocInit
############################################################################
## EPICS R3.14.12.2 $Date: Mon 2011-12-12 14:09:32 -0600$
## EPICS Base built May 31 2012
############################################################################
2012/06/02 11:16:25.532 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
cas warning: Configured TCP port was unavailable.
cas warning: Using dynamically assigned TCP port 60016,
cas warning: but now two or more servers share the same UDP port.
cas warning: Depending on your IP kernel this server may not be
cas warning: reachable with UDP unicast (a host's IP in EPICS_CA_ADDR_LIST)
iocRun: All initialization complete
epics> 2012/06/02 11:16:26.532 testConnect:pollerTask: isConnected = 1
2012/06/02 11:16:26.533 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
2012/06/02 11:16:27.533 testConnect:pollerTask: isConnected = 1
2012/06/02 11:16:27.534 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
2012/06/02 11:16:28.534 testConnect:pollerTask: isConnected = 1
2012/06/02 11:16:28.535 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
2012/06/02 11:16:29.535 testConnect:pollerTask: isConnected = 1
2012/06/02 11:16:29.536 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
2012/06/02 11:16:30.536 testConnect:pollerTask: isConnected = 1
2012/06/02 11:16:30.537 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
**********************************

So this case behaves as expected as well, isConnected is always 1, and the I/O succeeds.

Here is the output of case 3, where the Newport XPS is initially unreachable because it is rebooting, but then becomes reachable.

**********************************
corvette:asyn/iocBoot/ioctestConnect>../../bin/linux-x86/testConnect st.cmd
dbLoadDatabase("../../dbd/testConnect.dbd")
testConnect_registerRecordDeviceDriver(pdbbase)
# This is a bogus IP address that will never connect
#drvAsynIPPortConfigure("IPPort", "164.54.160.220:20", 0, 0, 1);
# This is a real IP address that will connect
drvAsynIPPortConfigure("IPPort", "newport-xps5:5001", 0, 0, 1);
testConnectConfigure("PORT1", "IPPort", "FirmwareVersionGet(char *)")
asynSetTraceMask("PORT1",0,9)
asynSetTraceIOMask("PORT1",0,0x2)
dbLoadRecords("../../db/asynRecord.db","P=testConnect:,R=asyn1,PORT=PORT1,ADDR=0,OMAX=80,IMAX=80")
iocInit()
2012/06/02 11:19:26.664 testConnect:pollerTask: isConnected = 0
2012/06/02 11:19:26.665 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
Starting iocInit
############################################################################
## EPICS R3.14.12.2 $Date: Mon 2011-12-12 14:09:32 -0600$
## EPICS Base built May 31 2012
############################################################################
cas warning: Configured TCP port was unavailable.
cas warning: Using dynamically assigned TCP port 51713,
cas warning: but now two or more servers share the same UDP port.
cas warning: Depending on your IP kernel this server may not be
cas warning: reachable with UDP unicast (a host's IP in EPICS_CA_ADDR_LIST)
iocRun: All initialization complete
epics> 2012/06/02 11:19:27.665 testConnect:pollerTask: isConnected = 0
2012/06/02 11:19:27.665 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
2012/06/02 11:19:28.665 testConnect:pollerTask: isConnected = 0
2012/06/02 11:19:28.665 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
2012/06/02 11:19:29.665 testConnect:pollerTask: isConnected = 0
2012/06/02 11:19:29.665 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
...
2012/06/02 11:19:45.669 testConnect:pollerTask: isConnected = 0
2012/06/02 11:19:45.669 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
2012/06/02 11:19:46.669 testConnect:pollerTask: isConnected = 0
2012/06/02 11:19:46.669 testConnect:pollerTask: error calling pasynOctetSyncIO->writeRead, status=3, error=asynManager::lockPort queueRequest failed: port IPPort not connected
2012/06/02 11:19:47.669 testConnect:pollerTask: isConnected = 1
2012/06/02 11:19:47.670 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
2012/06/02 11:19:48.670 testConnect:pollerTask: isConnected = 1
2012/06/02 11:19:48.671 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
2012/06/02 11:19:49.671 testConnect:pollerTask: isConnected = 1
2012/06/02 11:19:49.672 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
2012/06/02 11:19:50.672 testConnect:pollerTask: isConnected = 1
2012/06/02 11:19:50.673 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
2012/06/02 11:19:51.673 testConnect:pollerTask: isConnected = 1
2012/06/02 11:19:51.674 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
2012/06/02 11:19:52.674 testConnect:pollerTask: isConnected = 1
2012/06/02 11:19:52.675 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
2012/06/02 11:19:53.675 testConnect:pollerTask: isConnected = 1
2012/06/02 11:19:53.676 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
2012/06/02 11:19:54.676 testConnect:pollerTask: isConnected = 1
2012/06/02 11:19:54.677 testConnect:pollerTask: numWrite=26, wrote: FirmwareVersionGet(char *), numRead=60, response=0,XPS-Q8 Firmware Precision Platform V1.0.0 Beta 27,EndOfAPI
**********************************

So in this case isConnected is initially 0 and the pasynOctetSyncIO->writeRead() fails, but then the device becomes reachable, and isConnected() becomes 1 and the I/O succeeds.

So my testing indicates that things are working as expected.  pasynManager->isConnect() reports the connection state of the underlying device, not the connection state of the pasynUser to the asyn port.

Can you see if the new test program works for you?  Perhaps you'll find something you are doing wrong, or perhaps there is a bug in asyn which is showing up for you because you are doing things differently from this test program.

The new test program and test IOC have been committed to Subversion:

https://svn.aps.anl.gov/epics/asyn/trunk

Cheers,
Mark






________________________________________
From: tech-talk-bounces@aps.anl.gov [tech-talk-bounces@aps.anl.gov] on behalf of matthew.pearson@diamond.ac.uk [matthew.pearson@diamond.ac.uk]
Sent: Friday, June 01, 2012 10:45 AM
To: jlmuir@imca-cat.org
Cc: tech-talk@aps.anl.gov
Subject: RE: asynManager isConnected

Thanks Lewis, Mark and Eric,

>
> I don't know of a function in Asyn that allows one to determine
> whether the IP socket is connected. I always just determine
> this based on the asynStatus returned from a read or write. But
> maybe there's a way that I don't know about?
>
>

That's what I was hoping to do too, but I'm not getting a return from the writeRead function. It blocks forever (at least, more than 15 mins) if I attempt a writeRead using the un-connected IP port. Eric suggested this might be due to the operating system timeout on TCP connections (which I gather can be fixed to infinite).

I'm probably doing something wrong, because I have the asynManager isConnected problem too.

The reason I'm interested in understanding this, rather than just working around it, is because I want to handle connections to ethernet PMAC motor controllers automatically if they come up *after* the IOC has already started. Currently we have to restart an IOC if any of our PMACs come up that were turned off when the IOC was started.

Cheers,
Matt

--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom