realTimePerform: Real Time Performance Measurements of EPICS IOCcore

 

 


License Agreement

This product is available via the open source license described at the end of this document.


Description

EPICS Input/Output Controller core (IOCcore) software has been ported to several different operating systems (OSs) and many hardware platforms. This module compares the EPICS IOCcore runtime performance on one hardware platform (MVME2100 PowerPC) and three popular Operating Systems: vxWorks, RTEMS, and Linux.  For Linux the following versions were tested: Linux 2.4.2 hard hat 2.0, standard Linux 2.4.30, and Linux 2.6.13. For Linux 2.6.13, the kernel was built both preemptive and non-preemptive.

 

Three real-time parameters are measured: interrupt, context switch, and total response latency. On Linux, more detailed interrupt latencies are measured: interrupt top half to bottom half, and interrupt bottom half to user space interrupt service routine.

 

To implement the tests, several software components were developed. In order to port to other operating systems or hardware platforms, only one component has to be implemented.

Parameters measured

Interrupt Latency

Interrupt latency is the time from interrupt generation until the interrupt service routine starts executing. Linux has an additional concept of interrupt “top half” and “bottom half”. The top half interrupt handler, which executes with interrupts disabled, should complete quickly. If more work is required it can schedule a bottom half handler. On both vxWorks and RTEMS, the bottom half is replaced by a high priority thread. The VME support on Linux provides an additional feature, a user-level interrupt handler. A user-level handler is registered to receive a VME interrupt. POSIX signals are used to call the user-level handler. On Linux all of the following latencies are measured: top half, top half to bottom half, and bottom half to user-level Interrupt Service Routine (ISR).

Context Switch Latency

Context switch latency is the time from one context switching to another. The context switch time from a kernel space interrupt to user space-thread context is measured in this paper.

Total Response Latency

Total response latency is the time from an interrupt to when a waiting user thread begins execution.

Measurement Results comparison

The test IOC is heavily loaded, as described below, in all the tests. Four different values of each parameter are collected: minimum, median, maximum, and percentage of samples over some value. To make the results comparable, most tests were conducted on a private network for one hour. To look for network interference, some tests were run for 16 hours on a public network. Another test was run to measure user-level interrupt latency. Tables 1–5 show the results.

 

All latencies are in microseconds.  For vxWorks, two results are shown. One has the task priority lower than the vxWorks network task. The other has the task priority higher than the vxWorks network task.

Results analysis and conclusion

The results show that vxWorks and RTEMS have excellent real-time performance. The average performance of Linux 2.6.13 is similar to Linux 2.4.30, but the maximum total response latency is greatly reduced.

 

The recent Linux 2.6 kernel looks promising for use in real-time applications. It might have better performance if the real-time preemptive patch is applied. No big differences have been seen between the low-latency preemptive kernel and the non-preemptive kernel of Linux 2.6 at this time.

 

Switching context involved saving the CPU’s registers and loading a new state, flushing the caches, and changing the virtual memory mapping. Context switch latency is highly architecture dependent and different hardware may get different results.

 

If these tests were repeated or if the tests lasted for a longer time, the maximum value of each parameter could be higher.

 

On Linux all of the latency measurements are of interest. The application real-time requirements determine how interrupts should be handled. At one extreme, user-level interrupt handling will be sufficient. At the other extreme, the top-level interrupt handler will have to process most interrupts and only involve the bottom half and user threads at appropriate times.

Run Test Application

Reconfigure the envPaths or cdCommands and st.cmd file in iocBoot/iocsampler/ and start the IOC. Run EPICS client medm GUI:  runmedm

Set "samples per set".

Click the "startOnce" button to collect one set of samples

Click the menu botton to select the period time to collect samples.

To look at the details of each real time parameter configure the corresponding "X cutoff" or "Y cutoff".

 

Run performCaget and performCaput, on the host, issue the command:

<top>/bin/<host>/performCaget 5000 1

<top>/bin/<host>/performCaput 5000 5

Porting to Other Platform

There are three platform dependent files rtemsSampler.c, linuxSampler.c and vxworksSampler.c in sampler/samplerApp/src/, which are for RTEMS, Linux and vxWorks respectively.

 

To measure the performance on other platform, a platform dependent file must be developed. It must be able to generate an interrupt and provide a microsecond resolution timestamp.

 

Reference

"REAL-TIME PERFORMANCE MEASUREMENTS OF EPICS IOCCORE", ICALEPCS 2005, Geneva.


License Agreement

Copyright (c) 2002 University of Chicago All rights reserved.
realTimePerform is distributed subject to the following license conditions:
 
SOFTWARE LICENSE AGREEMENT
Software: realTimePerform
 
 1. The "Software", below, refers to realTimePerform (in either source code, or
    binary form and accompanying documentation). Each licensee is
    addressed as "you" or "Licensee."
 
 2. The copyright holders shown above and their third-party licensors
    hereby grant Licensee a royalty-free nonexclusive license, subject to
    the limitations stated herein and U.S. Government license rights.
 
 3. You may modify and make a copy or copies of the Software for use
    within your organization, if you meet the following conditions:
      a. Copies in source code must include the copyright notice and this
         Software License Agreement.
      b. Copies in binary form must include the copyright notice and this
         Software License Agreement in the documentation and/or other
         materials provided with the copy.
 
 4. You may modify a copy or copies of the Software or any portion of it,
    thus forming a work based on the Software, and distribute copies of
    such work outside your organization, if you meet all of the following
    conditions:
      a. Copies in source code must include the copyright notice and this
         Software License Agreement;
      b. Copies in binary form must include the copyright notice and this
         Software License Agreement in the documentation and/or other
         materials provided with the copy;
      c. Modified copies and works based on the Software must carry
         prominent notices stating that you changed specified portions of
         the Software.
 
 5. Portions of the Software resulted from work developed under a U.S.
    Government contract and are subject to the following license: the
    Government is granted for itself and others acting on its behalf a
    paid-up, nonexclusive, irrevocable worldwide license in this computer
    software to reproduce, prepare derivative works, and perform publicly
    and display publicly.
 
 6. WARRANTY DISCLAIMER. THE SOFTWARE IS SUPPLIED "AS IS" WITHOUT WARRANTY
    OF ANY KIND. THE COPYRIGHT HOLDERS, THEIR THIRD PARTY LICENSORS, THE
    UNITED STATES, THE UNITED STATES DEPARTMENT OF ENERGY, AND THEIR
    EMPLOYEES: (1) DISCLAIM ANY WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
    BUT NOT LIMITED TO ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS
    FOR A PARTICULAR PURPOSE, TITLE OR NON-INFRINGEMENT, (2) DO NOT ASSUME
    ANY LEGAL LIABILITY OR RESPONSIBILITY FOR THE ACCURACY, COMPLETENESS,
    OR USEFULNESS OF THE SOFTWARE, (3) DO NOT REPRESENT THAT USE OF THE
    SOFTWARE WOULD NOT INFRINGE PRIVATELY OWNED RIGHTS, (4) DO NOT WARRANT
    THAT THE SOFTWARE WILL FUNCTION UNINTERRUPTED, THAT IT IS ERROR-FREE
    OR THAT ANY ERRORS WILL BE CORRECTED.
 
 7. LIMITATION OF LIABILITY. IN NO EVENT WILL THE COPYRIGHT HOLDERS, THEIR
    THIRD PARTY LICENSORS, THE UNITED STATES, THE UNITED STATES DEPARTMENT
    OF ENERGY, OR THEIR EMPLOYEES: BE LIABLE FOR ANY INDIRECT, INCIDENTAL,
    CONSEQUENTIAL, SPECIAL OR PUNITIVE DAMAGES OF ANY KIND OR NATURE,
    INCLUDING BUT NOT LIMITED TO LOSS OF PROFITS OR LOSS OF DATA, FOR ANY
    REASON WHATSOEVER, WHETHER SUCH LIABILITY IS ASSERTED ON THE BASIS OF
    CONTRACT, TORT (INCLUDING NEGLIGENCE OR STRICT LIABILITY), OR
    OTHERWISE, EVEN IF ANY OF SAID PARTIES HAS BEEN WARNED OF THE
    POSSIBILITY OF SUCH LOSS OR DAMAGES.