Experimental Physics and
Industrial Control System

Michael Ritzert via Core-talk <core-talk at aps.anl.gov> · Mon, 04 Jan 2021 20:08:47 -0000

Public bug reported:

I noticed that I have a Linux system where broadcasts are not sent to
all configured broadcast addresses. The reason is easily identified:
EPICS base still uses the SIOCGIFBRDADDR way to identify the addresses
in osiSockDiscoverBroadcastAddresses. This fails in recent systems, when
several IPs are added to one interface e.g. via "ip addr add" as
NetworkManager likes to do:

2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 44:39:d4:9d:25:b6 brd ff:ff:ff:ff:ff:ff
    inet 10.49.126.61/24 brd 10.49.126.255 scope global dynamic noprefixroute eno1
       valid_lft 834648sec preferred_lft 834648sec
    inet 10.49.125.2/24 brd 10.49.125.255 scope global eno1
       valid_lft forever preferred_lft forever

In this case, only 10.49.126.255 is detected by EPICS (twice):

ifreqNext() pifreq 0x55ccbc0cc5f0, size 0x28, ifr 0x0x55ccbc0cc618
osiSockDiscoverBroadcastAddresses(): found IFACE: lo len: 0x10 current_ifreqsize: 0x28 
osiSockDiscoverBroadcastAddresses(): net intf "lo" flags: 49
osiSockDiscoverBroadcastAddresses(): ignoring loopback interface: "lo"
ifreqNext() pifreq 0x55ccbc0cc618, size 0x28, ifr 0x0x55ccbc0cc640
osiSockDiscoverBroadcastAddresses(): found IFACE: eno1 len: 0x10 current_ifreqsize: 0x28 
osiSockDiscoverBroadcastAddresses(): net intf "eno1" flags: 1043
found broadcast addr = a317eff
osiSockDiscoverBroadcastAddresses(): net intf "eno1" found
ifreqNext() pifreq 0x55ccbc0cc640, size 0x28, ifr 0x0x55ccbc0cc668
osiSockDiscoverBroadcastAddresses(): found IFACE: eno1 len: 0x10 current_ifreqsize: 0x28 
osiSockDiscoverBroadcastAddresses(): net intf "eno1" flags: 1043
found broadcast addr = a317eff
osiSockDiscoverBroadcastAddresses(): net intf "eno1" found

The workaround here would be to put the second IP on another interface,
i.e. eno1:1, but I believe the software shouldn't dictate that.

The more modern way to identify all broadcast addresses is via
AF_NETLINK, and the code is not overly complex. I actually do have a
first patch ready, see the attachment. Obviously, this code should not
be in the generic directory, and it should probably only be selected
when #ifdef AF_NETLINK to also consider older versions of Linux (I think
kernel ≥ 2.2 (from 1999) should work, but the oldest kernel I actually
tested with the netlink code was 3.2.102). So this is just an RFC for
now. Please excuse the bad formatting. I wasn't sure if there is a style
file available for clang-format or the likes and didn't bother to do it
by hand for this first demo.

With the patch, the behavior is changed in one subtle way: Interfaces
that are down are not ignored, because this information is not provided
for RTM_GETADDR but for RTM_GETLINK. So I'd have to keep track of which
interfaces are down in between these two calls. For other corner cases
(loopback, PTP), the behavior should be identical as explained in the
code.

Obviously, osiLocalAddrOnce also still uses the old interface, but since
it should only find any address, this should be OK.

** Affects: epics-base
     Importance: Undecided
         Status: New

** Patch added: "demo patch to introduce RT_NETLINK to osiSockDiscoverBroadcastAddresses"
   https://bugs.launchpad.net/bugs/1910148/+attachment/5449266/+files/rtnetlink.diff

-- 
You received this bug notification because you are a member of EPICS
Core Developers, which is subscribed to EPICS Base.
Matching subscriptions: epics-core-list-subscription
https://bugs.launchpad.net/bugs/1910148

Title:
  Broadcast address detection is unreliable on Linux

Status in EPICS Base:
  New

Bug description:
  I noticed that I have a Linux system where broadcasts are not sent to
  all configured broadcast addresses. The reason is easily identified:
  EPICS base still uses the SIOCGIFBRDADDR way to identify the addresses
  in osiSockDiscoverBroadcastAddresses. This fails in recent systems,
  when several IPs are added to one interface e.g. via "ip addr add" as
  NetworkManager likes to do:

  2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
      link/ether 44:39:d4:9d:25:b6 brd ff:ff:ff:ff:ff:ff
      inet 10.49.126.61/24 brd 10.49.126.255 scope global dynamic noprefixroute eno1
         valid_lft 834648sec preferred_lft 834648sec
      inet 10.49.125.2/24 brd 10.49.125.255 scope global eno1
         valid_lft forever preferred_lft forever

  In this case, only 10.49.126.255 is detected by EPICS (twice):

  ifreqNext() pifreq 0x55ccbc0cc5f0, size 0x28, ifr 0x0x55ccbc0cc618
  osiSockDiscoverBroadcastAddresses(): found IFACE: lo len: 0x10 current_ifreqsize: 0x28 
  osiSockDiscoverBroadcastAddresses(): net intf "lo" flags: 49
  osiSockDiscoverBroadcastAddresses(): ignoring loopback interface: "lo"
  ifreqNext() pifreq 0x55ccbc0cc618, size 0x28, ifr 0x0x55ccbc0cc640
  osiSockDiscoverBroadcastAddresses(): found IFACE: eno1 len: 0x10 current_ifreqsize: 0x28 
  osiSockDiscoverBroadcastAddresses(): net intf "eno1" flags: 1043
  found broadcast addr = a317eff
  osiSockDiscoverBroadcastAddresses(): net intf "eno1" found
  ifreqNext() pifreq 0x55ccbc0cc640, size 0x28, ifr 0x0x55ccbc0cc668
  osiSockDiscoverBroadcastAddresses(): found IFACE: eno1 len: 0x10 current_ifreqsize: 0x28 
  osiSockDiscoverBroadcastAddresses(): net intf "eno1" flags: 1043
  found broadcast addr = a317eff
  osiSockDiscoverBroadcastAddresses(): net intf "eno1" found

  The workaround here would be to put the second IP on another
  interface, i.e. eno1:1, but I believe the software shouldn't dictate
  that.

  The more modern way to identify all broadcast addresses is via
  AF_NETLINK, and the code is not overly complex. I actually do have a
  first patch ready, see the attachment. Obviously, this code should not
  be in the generic directory, and it should probably only be selected
  when #ifdef AF_NETLINK to also consider older versions of Linux (I
  think kernel ≥ 2.2 (from 1999) should work, but the oldest kernel I
  actually tested with the netlink code was 3.2.102). So this is just an
  RFC for now. Please excuse the bad formatting. I wasn't sure if there
  is a style file available for clang-format or the likes and didn't
  bother to do it by hand for this first demo.

  With the patch, the behavior is changed in one subtle way: Interfaces
  that are down are not ignored, because this information is not
  provided for RTM_GETADDR but for RTM_GETLINK. So I'd have to keep
  track of which interfaces are down in between these two calls. For
  other corner cases (loopback, PTP), the behavior should be identical
  as explained in the code.

  Obviously, osiLocalAddrOnce also still uses the old interface, but
  since it should only find any address, this should be OK.

To manage notifications about this bug go to:
https://bugs.launchpad.net/epics-base/+bug/1910148/+subscriptions

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System