RE: AlarmHandler as a daemon or service

["Liyu, Andrei" <liyua@ornl.gov>]

Hi,

I suppose I also missed point about message server :( 
If I am not mistaken that message server (from Matthias Clausen) is
Alarm server from me. Definition, definition and definition! Also I
don't understand what is JMS?

Now to point
================
The message server will forward the messages to the JMS message server.
- Point (3) on the to do list
================
- Alarm server gets Alarm messages from IOCs
- Alarm server hold some latest messages in memory (time 1, 24 hours or
message quantity 100, 1000, 10 000 messages ?) that decreased timeout
for Alarm clients
- Alarm server saves messages to Alarm log file (or another) in time
order (!). (That shows again that distributed control system needs to
have a DB. (I don't write about DB in IOC!) Maybe time comes Epics
community to look on this moment too?)
- Alarm server is responsible to provide Alarm clients information from
buffers and from log files (?) with filtering + ???

--------- Also ----------- 
- If Alarm server uses 50% CPU it can drop clients excluding
chief-operator OPI

Thanks, Andrei.




-----Original Message-----
From: Matthias Clausen [mailto:Matthias.Clausen@desy.de] 
Sent: Wednesday, April 05, 2006 5:56 AM
To: Williams Jr, Ernest L.
Cc: EPICS tech-talk
Subject: Re: AlarmHandler as a daemon or service

Back from my trip to eclipsecon and my visit at SLAC - thanks to the 
organizers of the little eclispe/XAL workshop I want to summarize where 
we are with the alarm message approach.


>   
>> For the future we want to change the alh message stream from opt-in
(as 
>> it is now) to opt-out - any alarm will be sent from the IOC to an
alarm 
>> queue. 
>>     
> Following the idea of iocLogServer?  This is a really good idea; what
is
> your time frame?
>   
To implement this feature we will need the right hook into IOC-core to 
make sure that we will catch all alarms generated on an IOC. There has 
been a discussion on tech-talk already about this subject:

http://www.aps.anl.gov/epics/tech-talk/2005/msg00809.php

The discussion going on there is a good starting point to find out which

implementation would be the 'right' one.
There's one implementation (or is it really a hack?) for the D0 IOC's
There's the proposal from Andrei and the comments from Kay.
Did someone else already think about this and what are the results?
In any case we will need an implementation which is EPICS-core 
compatible and will in the end make it into one of the next EPICS
versions.
- Point (1) on the to do list.

Second is the implementation on the iocAlarmMessage task on the IOC.
These are the mandatory functions:
- Receive triggers from record processing that an alarm has been raised
- Retrieve alarm information from record and prepare the alarm message
- (We are currently working on the necessary properties for alarm 
messages and messages in general. This list will be close to the cmlog 
approach)
- Write alarm message to message queue (fixed size - no malloc/free here

;-) )
- Write messages to message servers (one at a time)
- Implement redundant paths to redundant message servers
- Point (2) on the to do list
(This might a quick one because we have implemented a lot of these 
features already for the caPutLog tasks on the IOC.)

The message server will forward the messages to the JMS message server.
- Point (3) on the to do list

The messages will get written to the Oracle database by a message client

receiving the messages from the alarm message queue.
- Point (4) on the to do list
(This will be implemented in due time)


Message display and acknowledge
-----------------------------------------------

Displaying messages from the (Oracle) archive
Implemented in eclipse-BIRT - is one of the results of the eclipsecon!

Displaying online messages:
Open questions:
- What shall the hierarchy look like?
- How to configure the hierarchy? (using the existing alhConfig files?
- How to implement acknowledge alarms?
- What kind of filters are currently used/ necessary - or missing?
- Point (5) on the to do list

Alarm actions:
What kind of actions are necessary?
- Sending mail
- Sending SMS messages
- Send speech messages
- Changing values on the same - or other - records
- Point (6) on the to do list

So in the end we have - at least - 6 action points.
We plan to work on them during this year to get at least the basic 
features running.
We can speed up the process if others step in and we work on it 
collaboratively ;-))

Does this answer the question?
> what is
> your time frame?


>> This way you will run a system process in the background which 
>> makes sure that any alarm from an IOC (independent of any
configuration 
>> file) will be written to e.g. Oracle or a file.
>>
>>     
> I saw an option in the current alarmhandler to write to ORACLE. hmmm?
>   
Yes it's being used here for some of the alarm handlers.
But also this approach relies on running alarm handlers (with the same 
dependencies) writing through a message queue to an Oracle service 
writing the alarm messages to the database.
>
> By the way have you thought about having the AlarmHandler speak
> messages?
>   
For now we just send SMS messages.
Since Nokia joined the eclipse foundation we might get an easy way in 
the future to send text messages to GSM phones. Also the text to speech 
package in eclipse is really interesting to implement something in this 
regime.
But - it's at least not our list for now.
In the overall design it would 'just' be another message client 
converting message strings into speech messages and calling the on call 
shift...
> We are playing with text annunciation (on miniMAC) here at the SNS.
>
>   
We might discuss future alarm message designs during the upcoming EPICS 
workshop...
Maybe we can form an interest-group before?

-Matthias

-- 
------------------------------------------------------------------------
Matthias Clausen                         Cryogenic Controls Group(MKS-2)
phone:  +49-40-8998-3256                Deutsches Elektronen Synchrotron
fax:    +49-40-8994-3256                                    Notkestr. 85
e-mail: Matthias.Clausen@desy.de                           22607 Hamburg
WWW-MKS2.desy.de                                                 Germany
------------------------------------------------------------------------