Re: CA client (on IOC) question

[Andrew Johnson <anj@aps.anl.gov>]

Ralph Lange wrote:
> looking at bad things happening and logs from our network switches it 
> seems that the CA client that runs on the IOC does a name resolve 
> request whenever any record with a link pointing into nirwana (aka an 
> unconnected link) is being processed.
> Example: on IOC1, there are 100 records (scanned at 10 Hz) pointing to 
> 100 other records sitting on IOC2. As soon as IOC2 is down, IOC1 
> broadcasts a name resolution request for those 100 channels 10 times a 
> second.
>
> Is that true? Is that smart?

I believe that is the case, we've had similar issues here.  I suspect 
this stuff could be rewritten to be completely event driven, but I don't 
claim to fully understand how it works or why it behaves like this at 
the moment.  dbGetLinkValue() calls dbCaGetLink() which looks at the 
connection state of the link: since pca->isConnected should be false, 
link_action must be zero and there should be no further action by the 
dbCaTask().  However since we're seeing some effect there must be 
something else going on, which I don't have time to study right now; 
volunteers welcome...

As a workaround, you should be able to connect the SDIS links of those 
records to some local record scanned every 10 seconds that provides the 
connection state to that IOC and prevents them scanning if it's offline 
- maybe use SDIS=connrec.STAT and DISV=14 (LINK_ALARM).

We really shouldn't force people to have to adopt these kind of measures 
though, so if anyone comes up with a fix for this I'd love to include it 
in a future version of Base.

> (trying to find out why a single IOC going halfway down drives _all_ our 
> IOCs into 95+ percent of cpu usage)

Ouch.

- Andrew
--
Not everything that can be counted counts,
and not everything that counts can be counted.
  -- Albert Einstein