Re: Channel Access Gateway Bugs

[Matthias Clausen <Matthias.Clausen@desy.de>]

Dear all,

Ken's mail is showing to me clearly that we are all not only technical 
people but also human beings with personal feelings - which have been 
tempted (if this is the right term) in this case.
I am sure that those mentioned by Ken are about to reply directly - so 
should I.
But I think we should at first think it over and see what went wrong. 
There are always more than one party involved if such a mail gets written.
I want to propose that first of all Ken and Dirk get in contact directly 
(leaving out cc-mail to tech-talk) and present their technical findings 
afterwards - here.
Second I want to invite Ken to a glass of wine to understand what went 
wrong.

Last not least I was glad to see many of you in Archamps.
And - by the way - the presentations - yes all of them - are already on 
the web.
Maybe there's a chance to add one or two pages with recent results/ 
solutions to one of the presentations?

Regards
Matthias

evans@aps.anl.gov wrote:

> This note is in reponse to the talk presented at the EPICS 
> Collaboration Meeting yesterday at Archamps and entitled "Channel 
> Access Gateway Bugs".
>
> This talk broadsided me.  I was not aware that it was to be given, 
> since neither the author nor my colleagues informed me; there was no 
> printed aganda available; and I had no particular reason nor convenient 
> means of viewing the program online.  To say that it ruined an 
> otherwise pleasant meeting for me is an understatement.
>
> I am very conscientious and concerned about my programs working well, 
> and I spend much of my life insuring that they do and also in helping 
> people with problems with EPICS software of all kinds.  The same may be 
> said of most of the people with whom I work.  I think a paper on the 
> [alleged] bugs in someone else's program is an inappropriate subject 
> for a presentation at a meeting where the proceedings are made 
> available to the entire world.
>
> That, in addition, one of the conference organizers stated that this 
> was the best paper of the meeting is completely inappropriate, in my 
> opinion.  If this collaboration feels this type of paper is the best 
> sort, then it is a collaboration of which I do not wish to be a part.
>
> I do not object to criticism, rather I feel strongly that it needs to 
> be done professionally, in an atmosphere of respect and consideration, 
> the same atmosphere in which people should be treated in general.
>
> To respond to the presentation itself:  The automobile is one of the 
> most useful inventions of our time.  However, when misused it kills 
> people, more even than the wars we wage.  The same can be said of the 
> Gateway, though it is neither as valuable nor as destructive.  Thus, I 
> will respond to the conclusion of the paper first and agree 
> wholeheartedly that Gateway, as with any other software that is 
> critical to your control system, must be used with caution until you 
> are sure it will not have an adverse impact.  It is clear that the 
> Gateway is not working that way for Dirk.
>
> On the other hand it works very well under heavy and extensive use on 
> both Linux and Solaris at the APS, the only situation over which I have 
> any control.  This experience is described in a paper available on the 
> Gateway web page, along with the manual and other relevant 
> information.  Apart from Dirk's problems and known problems with the 
> Alarm Handler, which were stated in this forum recently, I am not aware 
> that other sites are having any problems that have not been fixed by 
> using 3.14.7 plus the latest patches from Jeff.  It would not hurt, but 
> is not as critical, to use the latest version of the Gateway.  
> Otherwise I would not have released it this week.  The point is that 
> many sites are using the Gateway successfully and not finding 
> the "bugs" that Dirk does.
>
> The fact is that the Gateway is arguably the most powerful Channel 
> Access application and the one that stresses Channel Access the most.  
> In our case it is serving all of the process variables from several 
> hundred IOCs, requiring that it contact those IOCs as a Channel Access 
> client, such as MEDM, and also act as a server, such as an IOC, to the 
> huge numbers of MEDMs and other clients that are attached to it.  It is 
> typically used in an extended network environment spanning several 
> subnets.  To manage the Gateway properly requires significantly more 
> knowledge, especially in the areas of networking in general and in the 
> internal operation of Channel Access, than running a program such as 
> MEDM does.  It is for this reason that much of this informaion is 
> included in the Gateway Reference Manual and in other presentations I 
> have made and which are available on the Gateway web site.  It is up to 
> the Gateway administrator to understand it.
>
> Dirk states that the Gateway does not forward beacon anomalies 
> correctly.  The fact is that it does not forward them at all.  It does 
> generate a beacon anomaly when it starts, as do all CA servers, and it 
> also generates a beacon anomaly when any IOC to which it has been 
> connected reconnects after being disconnected.  Whether these are 
> forwarded depends on the network.  Beacon anomalies are UDP broadcasts, 
> and many routers do not propagate UDP broadcasts across subnets for 
> security reasons.  In any event, it is easy to tell.  You just run CASW 
> (as described in the manual and in many other links from the Gateway 
> web page).  Similarly, you run CaSnooper to see if search requests from 
> clients connected to the Gateway are happening as expected.  It was not 
> stated if Dirk has done that or not, but it should be done before 
> presenting papers such as this.
>
> The problem Dirk mentions that if any client connects a monitor, then 
> all clients only get the latest monitor value is hard for me to 
> understand.  What the Gateway does is monitor the value itself from the 
> IOC.  It caches this value whenever it receives it and then post 
> monitors to any MEDMs attached to it that have requested monitors and 
> also gives this value to any clients that do gets.  There is only one 
> value in the Gateway, and it is the most current it has received.  This 
> is not a bug, this is what it does.  If it does not have the latest 
> value from the IOC, there is more likely a problem with the IOC or the 
> network, not the Gateway, and those need to be investigated.  The same 
> could be said of MEDM or any other clients.  The client side of the 
> Gateway is no different than the client side of MEDM.
>
> In regard to large arrays:  We have been running our whole control 
> system with EPICS_MAX_ARRAY_BYTES set for more than a year.  There was 
> a bug in Channel Access related to doing that that was tracked down by 
> me and fixed by Jeff some time ago.  Other than that, there have been 
> no problems, and there are no problems now that I am aware of.  Nobody 
> is complaining, here.  In addition, you do need to set it large enough 
> to hold the maximum number of bytes being sent plus some extra for the 
> protocol header.  However, if you request more bytes than that, you 
> should just get a message saying it won't be allowed.  It does not 
> crash the Gateway nor the clients for us.
>
> In any event, I cannot duplicate Dirk's problems, and our more than 50 
> Gateways work quite well.  To proceed further he needs to do some 
> investigating himself.  A site the size of PSI should have someone with 
> the ability to get the Gateway in a debugger and determine what, if 
> anything it is doing wrong.  That is what I would do if I were there.  
> I cannot fix his problem with the information given, and I cannot be 
> there to do it myself.  The benefits of the collaboration is that it 
> did not cost him anything.  The disadvantage is that he may have to do 
> some of the work himself.
>  


--
------------------------------------------------------------------------
Matthias Clausen                         Cryogenic Controls Group(MKS-2)
phone:  +49-40-8998-3256                Deutsches Elektronen Synchrotron
fax:    +49-40-8994-3256                                    Notkestr. 85
e-mail: Matthias.Clausen@desy.de                           22607 Hamburg
WWW-MKS2.desy.de                                                 Germany
------------------------------------------------------------------------