Errors detected by an IOC can be divided into classes: Errors related to a particular client and errors not attributable to a particular client. An example of the ﬁrst type of error is an illegal Channel Access request. For this type of error, a status value should be passed back to the client. An example of the second type of error is a device driver detecting a hardware error. This type of error should be reported to a system wide error handler.
Dividing errors into these two classes is complicated by a number of factors.
If used properly, the error handling facilities described in this chapter can process both types of errors.
This chapter describes the following:
NOTE: Many sites use CMLOG instead of iocLog.
NOTE: recGbl error routines are also provided. They in turn call one of the error message routines.
errlogPrintf and errlogVprintf are like printf and vprintf provided by the standard C library, except that their output is sent to the errlog task; unless conﬁgured not to, the output will appear on the console as well. Consult any book that describes the standard C library such as “The C Programming Language ANSI C Edition” by Kernighan and Ritchie.
errlogMessage sends message to the errlog task.
errlogFlush wakes up the errlog task and then waits until all messages are ﬂushed from the queue.
errlogSevPrintf and errlogSevVprintf are like errlogPrintf and errlogVprintf except that they add the severity to the beginning of the message in the form “sevr=<value>” where value is one of “info, minor, major, fatal”. Also the message is suppressed if severity is less than the current severity to suppress. If epicsThreadIsOkToBlock is true, which is true during iocInit, errlogSevVprintf does NOT send output to the errlog task.
errlogGetSevEnumString gets the string value of severity.
errlogSetSevToLog sets the severity to log. errlogGetSevToLog gets the current severity to log.
Routine errMessage (actually a macro that calls errPrintf) has the following format:
Where status is deﬁned as:
errMessage, via a call to errPrintf, prints the message, the status symbol and string values, and the name of the task which invoked errMessage. It also prints the name of the source ﬁle and the line number from which the call was issued.
The calling routine is expected to pass a descriptive message to this routine. Many subsystems provide routines built on top of errMessage which generate descriptive messages.
An IOC global variable errVerbose, deﬁned as an external in errMdef.h, speciﬁes verbose messages. If errVerbose is TRUE then errMessage should be called whenever an error is detected even if it is known that the error belongs to a speciﬁc client. If errVerbose is FALSE then errMessage should be called only for errors that are not caused by a speciﬁc client.
Routine errPrintf is normally called as follows:
Where status is deﬁned as:
FILE and LINE are deﬁned as:
The remaining arguments are just like the arguments to the C printf routine. errVerbose determines if the ﬁlename and line number are shown.
An EPICS status code can also be converted to a string. If the supplied status code isn’t registered in the status code database then the raw status code number is converted into a string in the destination buﬀer.
These are macros that call errlogPrintf and errlogVprintf. They are provided for compatibility.
Any code can receive errlog message. The following are the calls to add and remove a listener.
These routines add/remove a callback that receives each error message. These routines are the interface to the actual system wide error handlers.
The error message routines can be called by any non-interrupt level code. These routines pass the message to the errlog Thread. If any of the error message routines are called at interrupt level, epicsInterruptContextMessage is called with the message “errlogPrintf called from interrupt level”.
errlogThread manages the messages. Messages are placed in a message queue, which is read by errlogThread. The message queue uses a ﬁxed block of memory to hold all messages. When the message queue is full additional messages are rejected but a count of missed messages is kept. The next time the message queue empties an extra message about the missed messages is generated.
The maximum message size is by default 256 characters. If a message is longer, the message is truncated and a message explaining that it was truncated is appended. There is a chance that long messages corrupt memory. This only happens if client code is defective. Long messages most likely result from “%s” formats with a bad string argument.
errlogThread passes each message to any registered listener.
The errlog system can also display messages on the ioc console. It calls epicsThreadIsOkToBlock to decide when to display the message. If it is OK to block, the message is displayed by the same thread that calls one of the errlog print routines. If it is not OK to block, errlogThread displays the messages.
Normally the errlog system displays all messages on the console. eltc can be used to suppress these messages.
eltc determines if errlog task writes message to the console. During error message storms this command can be used to suppress console messages. A argument of 0 suppresses the messages, any other value lets messages go to the console.
errlogInit or errlogInit2 can be used to initialize the error logging system with a larger buﬀer and maximum message size. The default buﬀer size is 1280 bytes, and the default maximum message size is 256.
EPICS deﬁned status values provide the following features:
IOC routines often return a long integer status value, encoded similar to the vxWorks error status encoding. On some modern architectures a long integer is more than 32 bits wide, but in order to keep the API compatible the status values are still passed as long integers, even though only 32 bits are ever used. The most signiﬁcant 16 bits indicate the subsystem or module where the error occurred. The least signiﬁcant 16 bits contain a subsystem-speciﬁc status value. In order that status values do not conﬂict with the vxWorks error status values, all subsystem numbers are greater than 500.
A header ﬁle errMdef.h deﬁnes macros for all the subsystem numbers. For example the database access routines use this module number:
There are header ﬁles for every IOC subsystem that returns standard status values. The status values are encoded with lines of the following format:
For example, when dbGetField detects a bad database request type, it executes the statement:
The calling routine checks the return status as follows:
NOTE: Many sites use CMLOG instead of iocLog. See the CMLOG documentation for details.
This consists of two modules: iocLogServer and iocLogClient. The client code runs on each ioc and listens for the messages generated locally by the errlog system. It also reports the messages from the vxWorks logMsg facility.
This runs on a host. It receives messages for all enabled iocLogClients in the local area network. The messages are written to a ﬁle. Epics base provides a startup ﬁle “base/src/libCom/log/rc2.logServer”, which is a SystemV init script to start the server. Consult this script for details.
To start a log server on a UNIX or PC workstation you must ﬁrst set the following environment variables and then run the executable “iocLogServer” on your PC or UNIX workstation.
The name and path to the log ﬁle.
The maximum size in characters for the log ﬁle. If the ﬁle grows larger than this limit the server will seek back to the beginning of the ﬁle and write new messages over the old messages starting from the beginning. If the value is zero then there is no limit on the size of the log ﬁle.
A shell command string used to obtain the log ﬁle path name during initialization and in response to SIGHUP. The new path name will replace any path name supplied in EPICS_IOC_LOG_FILE_NAME.
Thus, if EPICS_IOC_LOG_FILE_NAME is “a/b/c.log” and EPICS_IOC_LOG_FILE_COMMAND returns “A/B” or “A/B/” the log server will be stored at “A/B/c.log”
If EPICS_IOC_LOG_FILE_COMMAND is empty then this behavior is disabled. This feature is used at some sites for switching the server to a new directory at a ﬁxed time each day. This variable is currently used only by the UNIX version of the log server.
THE TCP/IP port used by the log server.
To conﬁgure an IOC to log its messages it must have an environment variable EPICS_IOC_LOG_INET set to the IP address of the host that is running the log server, and EPICS_IOC_LOG_PORT to the TCP/IP port used by the log server.
Defaults for all of the above parameters are speciﬁed in the ﬁles $(EPICS_BASE)/conﬁg/CONFIG_SITE_ENV and $(EPICS_BASE)/conﬁg/CONFIG_ENV.
This runs on each ioc. It is started by calling:
The global variable iocLogDisable can be used to enable/disable the messages from being sent to the server. Setting this variable to (0,1) (enables, disables) the messages generation. If iocLogDisable is set to 1 before calling iocLogInit then iocLogClient will not even initialize itself. iocLogDisable can also be changed to turn logging on or oﬀ.
iocLogClient calls errlogAddListener and sends each message to the iocLogServer.
In a testing environment it is desirable to use a private log server. This can be done as follows:
Many sites run multiple soft IOCs on the same machine. With some log viewers like cmlogviewer it is not possible to distinguish between the log messages from these IOCs since their hostnames are all the same. One solution to this is to add a unique preﬁx to every log message.
The iocLogPrefix command can be run from the startup ﬁle during IOC initialization to establish such a preﬁx that will be prepended to every log message when it is sent to the iocLogServer.
For example, adding the following lines to your st.cmd ﬁle
will categorize all log messages from this IOC as belonging to the facility LI21 and to the process sioc-b34-mc10.
Note that log messages echoed to the IOC’s standard output will not show the preﬁx, it only appears in the version sent to the log server. iocLogPrefix should appear fairly early in the startup script so the IOC doesn’t try to send any log messages without the preﬁx. Once the preﬁx has been set, it cannot be changed without rebooting the IOC. One can determine if a log preﬁx has been set using iocLogShow.