Re: EPICS Archiver routing issue

["Hu, Yong via Tech-talk" <tech-talk at aps.anl.gov>]

Hi Adam,

 

Different subnets. I remember I have to use double quotes “” (and spaces inside “”) for the IP list if you pay close attention to the lines below. Also, do not forget to restart the Archiver Appliance (AA) after changed are made on archappl.conf.

 

Other suggestions you may try:

1. On the Epics Archiver Computer (192.168.10.20), type “cainfo PVName”. You should see something like this: Host: 192.168.11.21:5064;
2. On the Epics Archiver Computer (192.168.10.20), type “traceroute 192.168.11.21”, “sudo netstat -nr” to see the gateway/router settings;
3. Check AA log files, i.e. search PVName in log files;

 

The last thing you could try is to use Wireshark (with Channel Access filter/plugin) on both 192.168.10.20 and 192.168.11.21 to see what is going on when you click the “Archive” button to archive a new PV from AA web interface.

 

Good luck!

Yong

 

From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
Reply-To: "Manoussakis, Adamandios" <manoussakis1 at llnl.gov>
Date: Tuesday, January 26, 2021 at 1:05 PM
To: "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
Subject: RE: EPICS Archiver routing issue

 

Hi Yong,

 

Are you going across different subnets in your setup? Or is the archiver on the same subnet as the IOCs you are reading PVs from?  I did have success when they were on the same subnet.

 

From: Hu, Yong <yhu at bnl.gov>
Sent: Tuesday, January 26, 2021 7:25 AM
To: Manoussakis, Adamandios <manoussakis1 at llnl.gov>; Abdalla Ahmad <Abdalla.Ahmad at sesame.org.jo>; tech-talk at aps.anl.gov
Subject: Re: EPICS Archiver routing issue

 

I also used Han’s scripts for installation of Archiver Appliance. And it works for me after adding these lines in archappl.conf and restarting the AA after changes are made on archappl.conf:

 

EPICS_CA_AUTO_ADDR_LIST=YES

EPICS_CA_ADDR_LIST="localhost 10.0.142.197"  

 

 

From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
Reply-To: "Manoussakis, Adamandios" <manoussakis1 at llnl.gov>
Date: Tuesday, January 26, 2021 at 2:19 AM
To: Abdalla Ahmad <Abdalla.Ahmad at sesame.org.jo>, "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
Subject: RE: EPICS Archiver routing issue

 

At least according to the documentation it made it sound like setting AUTO=YES just created a list of local ips then anything in the ADDR_LIST would just be appended on.  But will have to see if maybe the archiver is doing something weird.

 

From: Abdalla Ahmad <Abdalla.Ahmad at sesame.org.jo>
Sent: Monday, January 25, 2021 11:17 PM
To: Manoussakis, Adamandios <manoussakis1 at llnl.gov>; tech-talk at aps.anl.gov
Subject: RE: EPICS Archiver routing issue

 

In our case, setting it to YES will default CA traffic to the local subnet and would not go across VLANs (ignoring the address list) rendering the archiver not seeing any PV. But something I am not sure of is that which one overrides the other.

 

Best Regards,

Abdalla.

 

From: Manoussakis, Adamandios <manoussakis1 at llnl.gov>
Sent: Tuesday, January 26, 2021 9:12 AM
To: Abdalla Ahmad <Abdalla.Ahmad at sesame.org.jo>; tech-talk at aps.anl.gov
Subject: RE: EPICS Archiver routing issue

 

That seems odd, I thought CSStudio and the catools (caget,camonitor etc) were the same as what the archiver was using.  I will try setting the AUTO_ADDR_LIST=NO, I assumed this was just helpful in general since it would do an introspection of network and didn’t think it would do any harm (What might be the issue of running this with YES?).  Currently I have the EPICS_CA_ADDR_LIST set in ~/.profile as well to the ip of the host running the IOC. 

 

From: Abdalla Ahmad <Abdalla.Ahmad at sesame.org.jo>
Sent: Monday, January 25, 2021 11:05 PM
To: Manoussakis, Adamandios <manoussakis1 at llnl.gov>; tech-talk at aps.anl.gov
Subject: RE: EPICS Archiver routing issue

 

Hello Adam

 

I think CSS is using different CA implementation other than the archiver, maybe that’s why some configs won’t work. Try set the address list in the archiver (and in ~/.bashrc) to the IP of the host running the IOC and if there are multiple IOCs configure the network manager as in the article I sent. Also you can try setting EPICS_CA_AUTO_ADDR_LIST=NO in the archiver and see if it helps.

 

Best Regards,

Abdalla

 

From: Manoussakis, Adamandios <manoussakis1 at llnl.gov>
Sent: Tuesday, January 26, 2021 9:00 AM
To: Abdalla Ahmad <Abdalla.Ahmad at sesame.org.jo>; tech-talk at aps.anl.gov
Subject: RE: EPICS Archiver routing issue

 

Hey Abdalla,

 

Currently I set the EPICS_CA_ADDR_LIST in the archappl.conf (using Hans site specific install currently) and wondering if I need to set it to the gateway or to the ip of the host running the IOC (Usually I just set it to the IP of the host running the IOC)?  But at the moment I only have one IOC running on that host anyways.

 

I just don’t understand why the CS Studio (which is on one VLAN) and CAGET are able to grab the PV data from the IOC that is on the other VLAN but the Archiver is not getting through.  I am starting to wonder if the archappl.conf is not reading the EPICS_CA_ADDR_LIST properly?

 

Thanks,
Adam

 

From: Abdalla Ahmad <Abdalla.Ahmad at sesame.org.jo>
Sent: Monday, January 25, 2021 10:32 PM
To: Manoussakis, Adamandios <manoussakis1 at llnl.gov>; tech-talk at aps.anl.gov
Subject: RE: EPICS Archiver routing issue

 

Hello Adamandios

 

It depends on how the VLANs and subnets are configured. We have the archiver on a separate VLAN from the IOCs VLAN (both VLANs are not isolated from each other), but the archiver can see and archive all PVs from all the IOCs through:

1. Setting EPICS_CA_ADDR_LIST variable set in the sampleStartup.sh script.
2. Configuring the NetworkManager dispatcher if you have multiple IOCs on the same host: https://wiki-ext.aps.anl.gov/epics/index.php/How_to_Make_Channel_Access_Reach_Multiple_Soft_IOCs_on_a_Linux_Host

 

For CSS, since the IOCs and the clients are on the same subnet, we don’t set any EPICS variables, except if we want some PVs from other VLANs like beamlines which are requested through EPICS CA gateway, here we set the EPICS_CA_ADDR_LIST to the IP of the gateway.

 

Best Regards,

Abdalla.

 

From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Manoussakis, Adamandios via Tech-talk
Sent: Tuesday, January 26, 2021 2:56 AM
To: EPICS tech-talk <tech-talk at aps.anl.gov>
Subject: EPICS Archiver routing issue

 

Hello All,

 

I think I am having problems with routing with the epics archiver.  When I have the archiver on a LAN and is on the same subnet as the IOCs being archived it seems to work fine but when I have archiver on a different subnet it doesn’t seem to want to archive those PVs anymore. For example

 

Same Subnet, CS Studio Picks up PVs, caget works, archiver works

192.168.10.20 Epics Archiver Computer

192.168.10.21 IOC running

 

 

Different Subnet, CS Studio picks up PVs, caget works, Archiver will not archive PVs

192.168.10.20 Epics Archiver Computer

192.168.11.21 IOC running

 

 

I have both the epics variables set in my env variables, in archappl.conf (archiver file), in settings.ini (cs studio file).  It seems odd that caget, cs studio work fine but the archiver is having an issue seeing the IOC that is on the other subnet.  Is there another environment variable I am missing that needs to be set when services are on different subnets like this?  I checked firewalls/ports/traffic it seems like the same requests are being made by cs studio/caget as the archiver but the archiver ones don’t seem to go anywhere (just keep trying to retransmit).

 

EPICS_CA_ADDR_LIST=192.168.11.21

EPICS_CA_AUTO_ADDR_LIST=YES