EPICS sequencer Unix port

This document contains notes on the port of the EPICS sequencer to Unix. Comments are welcome.

William Lupton, wlupton@keck.hawaii.edu, 01-Mar-2000


Requirements

  1. The sequencer will run under both Unix and VxWorks (or other RTOS)
  2. Performance under VxWorks will not be significantly worse (say not more than 5% worse) than that of the existing sequencer
  3. Access to the control system will be via either CDEV or CA (maybe implemented using #ifdefs but preferably via a thin message system-independent layer, which would permit other sufficiently functional message systems to be interfaced)
  4. Operating system access will be via Marty Kraimer's new "OSI" layer, developed (as a collaboration) as necessary
  5. Overall file structure will not change unnecessarily (in parallel with this effort will be an effort, spear-headed by Greg White, to add various new syntax to SNL)
  6. The SNL compiler should build without access to any EPICS include files or libraries
  7. The SNL run-time support should build with access only to the OSI library and to the control system access layer (CDEV, CA, or system-independent layer)


Portability of SNL compiler and generated C code

  1. The SNL compiler is nearly EPICS-independent; whilst one file includes cadef.h (gen_ss_code.c) and two files include dbDefs.h (gen_ss_code.c and gen_tables.c), no definitions in these include files are used
  2. The sole dependence comes from the use of the EPICS time-stamp format; one file (gen_tables.c) includes tsDefs.h via seqCom.h; this can be avoided by defining a SEQ_TIME_STAMP structure which is (for now at least) identical to the EPICS one
  3. The generated C code is completely independent of EPICS (TS_STAMP is only used when the pvTimeStamp() routine is used) and the only VxWorks dependence is an unused OPT_VXWORKS option
  4. I have made the changes to avoid the above problems


Portability of run-time environment

  1. The string "ca_" occurs in 4 files:
  2. The string "task" occurs in 8 files:
  3. The string "sem" occurs in 6 files:
  4. The first thing to do is simply to replace task and sem calls with the appropriate ones from Marty's newBase
  5. Any missing routines or problems can be addressed as they arise
  6. The CA calls can be left more-or-less unchanged for now, since they exist under Unix; however I believe that Unix CA is not thread-safe and also that ca_import() and friends are not supported under Unix, so some changes will be necessary; initially, I suggest minimal changes to make things compile
  7. We can add a rule to Makefile.Host to build a Unix libSeq.The Unix C output from snc should now link OK against libSeq, libOsi and libca (maybe libCom too?)
  8. Now we need to ensure that the CA  / CDEV interface works properly; I don't know whether CDEV is thread-safe but I don't think we can assume that CDEV services are all thread-safe, so I think we will need to restrict message system i/o to a single thread in any case
  9. From a quick look at the code, CA / CDEV seems to be handled more nicely in alh than in medm; the alh approach looks close to having a message-system independent layer and may be the approach to use/develop for the sequencer


Notes on port to Solaris and Linux

  1. Remaining sections were written after the initial port to Solaris and Linux; they document the experience of that port


Missing features

  1. task delete hooks and task suspension are used for sequence deletion (although only because the VxWorks excTask has no floating point support)
  2. threads register with the EPICS task watchdog; this is probably useful functionality to have in OSI (c.f. ring.cc)
  3. uses ca_import() and ca_import_cancel()
  4. uses ioGlobalStdGet() to get fd for standard output; this is necessary is it? can't just use "1"? also uses STD_IN (#define'd as 0)
  5. uses errnoGet() and printErrno()
  6. uses taskIdFigure()
  7. uses ioctl() for FIOSYNC and FIOGETNAME; portable?


EPICS dependencies

  1. The portable sequencer still uses the following EPICS include files:
  2. Unix sequencers must be linked against the following EPICS libraries:
  3. I think that OSI could/should provide an OSItypes.h and a list.h / list.cc (c.f. ring.h / ring.cc)
  4. I also think that OSI could/should contain a reporting / logging API in place of epicsPrintf()
  5. This is a non-exhaustive list of the libCom facilities used by CA, together with comments as to whether they could reasonably be in OSI:
  6. Should all these be in OSI? is there a problem with CA retaining references to things in libCom? maybe not


OSIsem

  1. should have semaphore initializer; don't assume zero?
  2. used a base class (needed so could use base class pointer in the C interface)
  3. added OSIsem:: static definitions (so it can work with the Solaris implementation)


OSIthread

  1. under VxWorks (NB, addressed if there were an abstract base class):
  2. should have an OSI type for priority (and avoid application knowledge of actual priority values, range, whether 0 is low or high etc.; currently I am ignoring priority with Posix threads)
  3. similarly, how are portable programs to set stack sizes (currently I am ignoring stack sizes with Posix threads)?
  4. Posix implementation needs to review key usage (c.f. task variables) and to protect key creation via a semaphore
  5. need to re-work the logic on thread creation; thread needs to execute an OSI-supplied wrapper before invoking the application routine; the wrapper should wait for creation completion in a secure way
  6. made setDestroySafe() and Unsafe() static (this makes sense at least for VxWorks and Posix)
  7. OSIthreadName() shouldn't return pointer
  8. need taskIdFigure() or equivalent?
  9. would like known "null" thread id (is int under VxWorks?)


OSItime

  1. under VxWorks:
  2. is dependent on tsDefs.h and TS_STAMP; do we want this?
  3. fixed 1970->1990 correction (was 19 years) under both VxWorks and Solaris


C++ issues

  1. shouldn't the interface seen by the OS-independent program be defined in an abstract base class and shouldn't the OS-specific implementations inherit from it? that way, adherence to the interface can be checked by the compiler
  2. why implement the C++ and C versions independently? is this due to efficiency concerns? for Posix, I implemented in C++ and made the C wrappers call the C++


CA

  1. hacked in OSI locks for Solaris (should base on "Unix" rather than Solaris)
  2. changed osiSleep() call to use OSIthreadsleep() (may have been stack overflow problem...)
  3. really want to remove special VxWorks code as far as possible and rely on OSI; would give ca_import() etc., which the sequencer uses/needs


Misc OSI and sequencer

  1. error handling: want consistent way of knowing whether a feature is missing (maybe a set of enquiry routines and applications or library init routines should verify that the features that they require are present; don't always want to return an error at run-time; sometimes prefer just to do nothing...)
  2. need to check that timeouts are being handled consistently as FP seconds everywhere (were VxWorks ticks)
  3. why do I need ::thr_suspend() and not :: for the Posix ones?
  4. strictly you should compile with _POSIX_C_SOURCE set to 199506L under Solaris to use Posix threads; however, I just defined _REENTRANT (convenient because EPICS is already built that way) and, touch wood, everything is OK
  5. want to move libCom OSI stuff to src/osi (confusing at present)
  6. mustn't forget to go through the thread and sem implementations and ensure that all available options are being used correctly, errors being checked for etc.
  7. note that can't delete sequencers under VxWorks; have commented out the ca_import and delete stuff; must get to that
  8. fixed errlogInit problem under VxWorks (was never setting the "initialized" flag to TRUE); still get two "errlog" tasks?!
  9. want to support local variables and "long i, j;" declarations
  10. should detect bad pvGetQ (etc) usage (i.e. initialize queue index to -1 and test before use... at compile time if possible)


Message system

  1. want to be able to use the sequencer with CA, CDEV and KTL (although KTL only via CDEV might be OK)
  2. will not use "#ifdef CDEV" approach but will instead go via generic API, "PV"
  3. need to replace all CA calls and also all DBR usage; the API should define its own simple type scheme (sequencer uses only DBR_xxx and DBR_TIME_xxx, which is status, severity, time-stamp and value; xxx values used are CHAR, SHORT, LONG, FLOAT, DOUBLE, STRING)
  4. CA routines and constants used are:
  5. so will define pvInit(), pvGet() etc. interface and will implement pvCa.cc, pvCdev.cc, pvKtl.cc etc. plus add Makefile support (will stop short of run-time shareable library activation; use CDEV if you want that)
  6. have to worry about ca_import(), ca_import_cancel(); from looking at CA code, it's OK to ignore ca_import() and ca_import_cancel() if all threads in an application are to share a CA context; they are needed if some do and some don't
  7. have to worry about locking; can't assume that the library under the pv layer is thread-safe, so the pv layer will implement its own locking (using OSI locks); a single mutex will ensure that only one thread at a time is within the pv library


PV (see also the list in this PV-specific document)

  1. error handling is poor; could implement a "maximize severity" and/or error stack scheme; could use C++ exceptions? also could maintain on a per-variable rather than a per-system basis? it also ignores multi-threaded issues (maybe use thread-specific data?)
  2. useful to add "valid" flags to classes? then destructors can set them to FALSE (would protect against use of destroyed objects)
  3. really sort out use of const wherever possible (will need to retrofit somewhat); get into the habit of using it!
  4. not queueing callbacks when called from within initiating routine (OK, but need to document)
  5. TS_STAMP... what to do? OSI uses it
  6. KTL type conversions not fully implemented
  7. KTL problem: need to perform operation on keyword in order to trigger search and therefore connection hander
  8. idea of supporting KTL_ID ioctl() and read/write (backwards compatible but would allow channel info to be cached)
  9. now working nicely with KTL; major problem is the fact that have to monitor everything so that a connection remains open (related to KTL_ID notion)
  10. not working under VxWorks; compiles but channels don't connect
  11. demo sequence doesn't behave well on channel disconnect under KTL (probably CAKE problem)


14-Feb-00

  1. After a long gap, have returned...
  2. Greg White's changes were merged in in Oct-99
  3. Have packaged up a Linux / Solaris demo and tagged as FEB00
  4. other issues:


01-Mar-00

  1. Have made good progress with the new OSI and with cleaning up sequencer code so that it should run unchanged (and without any of those nasty "###" comments) under at least Solaris, Linux and VxWorks
  2. Now is the time to sit back and make those final punch-lists

Sequencer

  1. support building of a shareable library
  2. complete the local variable and "long i, j;" support
  3. avoid warnings re use of NULL and ASYNC (use different names, e.g. pvPutCallback())
  4. in PV layer, setting error status each time, regardless of new error (should clear message on success?)
  5. in PV layer, bootstrap error problem: how to report pvSysCreate() failure?)
  6. NB, in-built assumption that only one state-set reads/writes a given PV
  7. get rid of all (possible) compilation warnings
  8. stop using doubles for intervals?
  9. use same DEBUG macro approach everywhere

KTL

  1. finalize CAKE parallel completion support (allow on "wait", not just "notify"; requires new keywords to be backwards-compatible)
  2. add type conversion (currently fails quietly); adjust the module written for CDEV/CORBA?
  3. add time-stamp support
  4. define COKE parallel completion support
  5. check for leaks with string/array callbacks
  6. need lock in pvKtl.cc for error strings
  7. need to maintain list to permit more than one keyword object per underlying keyword (KTL name lookup issue)

Testing

  1. extract wfccas2 code to make general standalone CA server that will configure itself from .db and .dbd files (but have no scanning or device support)
  2. still some memory leak issues on sequencer deletion

Documentation

  1. re-write manual (wants a major overhaul; current one is very hard to use as a reference work)
  2. include full description of PV layer (enough info to write a new service, e.g. CDEV)

General

  1. need README (or README.html) files in all directories
  2. need packaging up per the Andrew Johnson recommendations
  3. make default build KTL-independent but make it easy to build KTL-specific test directories
  4. Solaris Workshop 5.0 problems (-Xc broke KTL signal code; had to disable exceptions; some un-demangeable names; switched to g++; can't run purify!)
  5. would like error log messages to be identified like syslog messages (hostname is not enough)