Re: About "Identical process variable names on multiple servers" with invalid address

[Andrew Johnson via Tech-talk <tech-talk at aps.anl.gov>]

Hi Emilio,

This sounds like an interesting story! Before your report gets lost in the archives, could you turn it into either an GitHub issue or even better a pull-request against the 7.0 branch of EPICS Base please. That will ensure the core developers will remember to discuss it.

Thanks,

- Andrew

On 4/18/23 4:48 AM, Perez Juarez, Emilio (DLSLtd, RAL, LSCI) via Tech-talk wrote:

Hi all, I hope you are well

Today, I would like to mention an interesting issue that manifest itself in camonitor (with any PV, no matter the server) as follows:

CA.Client.Exception...............................................

    Warning: "Identical process variable names on multiple servers"

    Context: "Channel: "SR-DI-DCCT-01:SIGNAL.NAME", Connecting to: cs03r-cs-gate-01.cs.diamond.ac.uk:5064, Ignored: 0.0.0.1:5"

    Source File: ../cac.cpp line 1308

The problem has been identified and it is caused by a series of unfortunate events described below:

1. Some IOC was mistakenly started with EPICS_CA_SERVER_PORT=5065
2. That IOC contains CA links that will try and retry (forever) to send CA_PROTO_SEARCH request messages (using port 5065)
3. Meanwhile, in the same network, there is one or many CA clients to random PVs, the caRepeater will forward the CA_PROTO_SEARCH request message to every CA client (mistakenly thinking that it is a beacon message)
4. The CA client mistakenly parses the CA_PROTO_SEARCH request thinking that is a CA_PROTO_SEARCH response (given that channel access doesn't have any flag to indicate if it's a request or a response)
5. The CA client will check the last field of the header of CA_PROTO_SEARCH message which in both cases (reply and response) correspond to the Search ID... this means the message is most of the times discarded because of a non-matching search ID, but because the faulty IOC is sending CA_PROTO_SEARCH requests continuously (starting with low values of search ID), it eventually clashes with the Search ID that that client used.
6. The CA client will mistakenly parse the fields server port and IP which results in the message with invalid IP shown above

Now, I would like to suggest a solution (which I think is simpler than the alternatives)... according to the CA spec, the data type field of the CA_PROTO_SEARCH response should be 0, while in the response, the same offset is the version and shouldn't be 0, therefore, we can use that for filtering the CA_PROTO_SEARCH  request. My proposal is reflected in the following patch (which hasn't been tested but should work):

diff --git a/modules/ca/src/client/udpiiu.cpp b/modules/ca/src/client/udpiiu.cpp

index c6ffb6def..27f91f5ad 100644

--- a/modules/ca/src/client/udpiiu.cpp

+++ b/modules/ca/src/client/udpiiu.cpp

@@ -680,6 +680,13 @@ bool udpiiu :: searchRespAction (

         return true;

     }

 

+    /*

+     * CA requires this field to be 0 in the search response

+     */

+    if (msg.m_dataType != 0) {

+        return true;

+    }

+

     /*

      * Starting with CA V4.1 the minor version number

      * is appended to the end of each UDP search reply.

I hope you also find this issue interesting,

Kind Regards,

Emilio

 

-- 

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 

-- 
Complexity comes for free, Simplicity you have to work for.