Re: Binding an IOC to a single network interface

[Till Straumann <strauman@slac.stanford.edu>]

Andrew.

I actually don't think it's the concurrency problems that are hard to 
tackle.
Freddie's patch modifies the cast_server() and req_server().

The former is already designed to run as a separate task. cast_server() 
could
easily be parametrized cast_server(bind_addr) and a separate task be spawned
for each address.

Multiple sockets bound to different addresses can be handled by req_server()
via good-ole 'select'.

There are other problems, though - which AFAIK haven't been mentioned yet:

The identity of the TCP server is stored in global variables 
'ca_server_addr', 'ca_server_port'
which are used when answering search replies:
  ca_server_addr is set from the EPICS_CAS_ADDR env-var (0 if unset)
  ca_server_port is set from EPICS_CAS_SERVER_PORT or 
EPICS_CA_SERVER_PORT or  CA_SERVER_PORT.
If binding of the TCP socket to ca_server_port fails (EADDRINUSE) then 
this socket is bound to an
arbitrary port (0) and ca_server_port read back from the system.

The problem now is that if there are multiple TCP sockets bound to 
different  ports
then there is no easy way to know which one to use. 'ca_server_port' 
would have to
be expanded into a list (one for each configured socket address) and 
corresponding lookup
logic would have to be implemented and built into the handling of search 
replies.
It is not clear what the semantics of EPICS_CAS_ADDR should be once multiple
addresses are set in EPICS_CAS_INTF_ADDR_LIST.

Finally, there is another issue to resolve if we want to be compatible 
with CAS:
When you list an interface address in EPICS_CAS_INTF_ADDR_LIST and bind 
the UDP socket
to this address then this socket will not receive broadcasts (you could 
list the IF's broadcast
address instead but then the socket wouldn't receive unicasts).
CAS handles this by automatically creating a second socket bound to
the IF's broadcast address. This is used to receive UDP messages. 
However, a comment
mentions an additional problem: apparently under solaris (no idea if 
this is still true) when
you send from the socket bound to the broadcast address then the 
broadcast address
will be used as the sender's address (rather than the IF address through 
which the datagram
leaves). In order to work around this problem, CAS uses two sockets (for 
each IF):
one bound to the broadcast address (for reception) and another one bound 
to the unicast
address (for reception on the unicast address and for sending replies).

So yeah, I agree, handling this in the most general way is not trivial.

Given that, I would argue that the approach taken by the existing patch
gives a considerable gain for a small effort whereas the effort required
for the 'clean' solution is considerable yielding only a small 
additional gain
(I don't think that there are that many use cases where you would really 
need
a interface list with multiple entries) and thus restricting the list to
one entry is reasonable.

The remaining cases (restricting CA to a subset of IFs with multiple 
members)
can always be addressed with administrative means (e.g., firewall rules).

- Till

PS:
Another approach (albeit deviating from what CAS does) would be just 
limiting
the address(es) the UDP server binds to. The TCP server would still use 
the wildcard
address. However, CA searches would only be satisfied when targeting one
of the addresses where the UDP server listens. This would be a little 
easier to
implement but still face some of the problems mentioned above (listen on 
uni-
+ bcst- sockets, solaris hack...)

On 02/10/2014 11:46 PM, Andrew Johnson wrote:
> On 1/22/14, 8:30 AM, freddie.akeroyd@stfc.ac.uk wrote:
> > > > we are looking to run a set of IOCs that bind channel access to just
> > > > the loopback interface on the computer; however I saw from the 3.14
> > > > docs that EPICS_CAS_INTF_ADDR_LIST is not currently implemented in the
> > > > iocCore CA Server. I had a look at the code in base/src/rsrv and the
> > > > attached diff (against 3.14.12.2) seems to work for us with:
> > > >
> > > > EPICS_CAS_INTF_ADDR_LIST=127.0.0.1
> > > > EPICS_CAS_BEACON_ADDR_LIST=127.255.255.255
> > > >
> > > > I was just wondering if these additions are sufficient, or have I
> > > > missed something elsewhere that either needs changing too or might
> > > > cause issues later on?
> I agree with Ralph's assessment, this patch looks like it should do what
> is necessary to bind the server to one interface/IP address.
>
> On 28.01.2014 23:28, J. Lewis Muir asked:
> > > Have you had a chance to consider the patch sent by Freddie?  You seem
> > > open to reviewing and accepting such a patch in the following Tech-Talk
> > > post:
> > >
> > >     http://www.aps.anl.gov/epics/tech-talk/2013/msg01493.php
> > >
> > > I haven't looked at the patch, but the ability to specify the addresses
> > > on which the IOC CA server listens has been a feature I've wished for
> > > for a long time.
> I am indeed happy to review and accept patches that will implement this
> functionality. The main limitation of Freddie's patch is that it only
> accepts a single hostname or IP address in the EPICS_CAS_INTF_ADDR_LIST
> variable (as comments in his code acknowledge). It won't help you if you
> need to configure an IOC to bind to more than one interface. I would be
> reluctant to incorporate these changes into Base without the full
> multi-interface functionality, which also seems to be Ralph's preference:
>
> On 01/29/2014 03:15 AM, Ralph Lange added:
> > At first glance, this patch looks fine and should cover almost all use
> > cases. That feature has been on the wish list for a long time, I know.
> > However, I would prefer a slightly more elaborate implementation that
> > makes the server side environment variables work the same on both CAS
> > and rsrv. (This will also minimize the necessary changes in the
> > documentation.)
> The work needed to support multiple independently-configured interfaces
> is unfortunately not trivial, the current code in src/rsrv uses the
> INADDR_ANY wild-card to open a socket on all the currently configured
> interfaces at once. As a result it only needs one thread to handle the
> UDP socket and one thread for the TCP socket.
>
> To allow configuring of multiple interfaces implies creating multiple
> UDP and TCP sockets, so we have to add a list to hold the parsed
> interface addresses and split the existing code up to create a separate
> task for each socket. Adding those new tasks implies checking for
> sufficient mutexes and possible deadlocks, and adding information about
> them to the CA server-report 'casr' output.
>
> This isn't impossible to do, but as I said it's not trivial; I could see
> it taking a few weeks for someone to get it right.
>
> However this doesn't take away from the usefulness of Freddie's patch;
> it if meets the needs of some sites they can still apply it locally.
>
> - Andrew