EPICS sequencer Unix port
This document contains notes on the port of the EPICS sequencer to Unix.
Comments are welcome.
William Lupton, wlupton@keck.hawaii.edu,
01-Mar-2000
Requirements
-
The sequencer will run under both Unix and VxWorks (or other RTOS)
-
Performance under VxWorks will not be significantly worse (say not more
than 5% worse) than that of the existing sequencer
-
Access to the control system will be via either CDEV or CA (maybe implemented
using #ifdefs but preferably via a thin message system-independent layer,
which would permit other sufficiently functional message systems to be
interfaced)
-
Operating system access will be via Marty Kraimer's new "OSI" layer, developed
(as a collaboration) as necessary
-
Overall file structure will not change unnecessarily (in parallel with
this effort will be an effort, spear-headed by Greg White, to add various
new syntax to SNL)
-
The SNL compiler should build without access to any EPICS include files
or libraries
-
The SNL run-time support should build with access only to the OSI library
and to the control system access layer (CDEV, CA, or system-independent
layer)
Portability of SNL compiler and generated C code
-
The SNL compiler is nearly EPICS-independent; whilst one file includes
cadef.h (gen_ss_code.c) and two files include dbDefs.h (gen_ss_code.c and
gen_tables.c), no definitions in these include files are used
-
The sole dependence comes from the use of the EPICS time-stamp format;
one file (gen_tables.c) includes tsDefs.h via seqCom.h; this can be avoided
by defining a SEQ_TIME_STAMP structure which is (for now at least) identical
to the EPICS one
-
The generated C code is completely independent of EPICS (TS_STAMP is only
used when the pvTimeStamp() routine is used) and the only VxWorks dependence
is an unused OPT_VXWORKS option
-
I have made the changes to avoid the above problems
Portability of run-time environment
-
The string "ca_" occurs in 4 files:
-
seq.h:1, seq_ca.c:19, seq_if.c:13, seq_task.c:18
-
The string "task" occurs in 8 files:
-
seq.h:6, seq_ca.c:11, seq_if.c:3, seq_mac.c:6, seq_main.c:42, seq_prog.c:9,
seq_qry.c:29, seq_task.c:81
-
The string "sem" occurs in 6 files:
-
seq.h:7, seq_ca.c:2, seq_if.c:12, seq_main.c:13, seq_prog.c:17, seq_task.c:26
-
The first thing to do is simply to replace task and sem calls with the
appropriate ones from Marty's newBase
-
Any missing routines or problems can be addressed as they arise
-
The CA calls can be left more-or-less unchanged for now, since they exist
under Unix; however I believe that Unix CA is not thread-safe and also
that ca_import() and friends are not supported under Unix, so some changes
will be necessary; initially, I suggest minimal changes to make things
compile
-
We can add a rule to Makefile.Host to build a Unix libSeq.The Unix C output
from snc should now link OK against libSeq, libOsi and libca (maybe libCom
too?)
-
Now we need to ensure that the CA / CDEV interface works properly;
I don't know whether CDEV is thread-safe but I don't think we can assume
that CDEV services are all thread-safe, so I think we will need to restrict
message system i/o to a single thread in any case
-
From a quick look at the code, CA / CDEV seems to be handled more nicely
in alh than in medm; the alh approach looks close to having a message-system
independent layer and may be the approach to use/develop for the sequencer
Notes on port to Solaris and Linux
-
Remaining sections were written after the initial port to Solaris and Linux;
they document the experience of that port
Missing features
-
task delete hooks and task suspension are used for sequence deletion (although
only because the VxWorks excTask has no floating point support)
-
should try to avoid this in the sequencer via a higher-level OSI mechanism?
-
I don't think we can support full VxWorks task hook functionality under
Posix
-
threads register with the EPICS task watchdog; this is probably useful
functionality to have in OSI (c.f. ring.cc)
-
uses ca_import() and ca_import_cancel()
-
uses ioGlobalStdGet() to get fd for standard output; this is necessary
is it? can't just use "1"? also uses STD_IN (#define'd as 0)
-
uses errnoGet() and printErrno()
-
uses taskIdFigure()
-
uses ioctl() for FIOSYNC and FIOGETNAME; portable?
EPICS dependencies
-
The portable sequencer still uses the following EPICS include files:
-
cadef.h (CA)
-
epicsTypes.h (epicsBoolean, epicsIndex and epicsStatus)
-
ellLib.h (linked-list prototypes)
-
Unix sequencers must be linked against the following EPICS libraries:
-
libca.a (CA)
-
libCom.a (epicsPrintf() but also lots of stuff used by CA)
-
I think that OSI could/should provide an OSItypes.h and a list.h / list.cc
(c.f. ring.h / ring.cc)
-
I also think that OSI could/should contain a reporting / logging API in
place of epicsPrintf()
-
This is a non-exhaustive list of the libCom facilities used by CA, together
with comments as to whether they could reasonably be in OSI:
-
tsStampToText() (OSI)
-
bucketLib (OSI)
-
epicsAssert() (OSI)
-
envLib, EPICS_CA_SERVER_PORT etc. (?)
-
freeListLib (OSI)
-
ellLib (OSI; see above)
-
errlogLib (OSI; see above)
-
aToIPAddr() (?)
-
bsdSockLib (OSI)
-
installSignPipeIgnore() (?)
-
Should all these be in OSI? is there a problem with CA retaining references
to things in libCom? maybe not
OSIsem
-
should have semaphore initializer; don't assume zero?
-
used a base class (needed so could use base class pointer in the C interface)
-
added OSIsem:: static definitions (so it can work with the Solaris implementation)
OSIthread
-
under VxWorks (NB, addressed if there were an abstract base class):
-
fixed GetPriority -> getPriority and SetPriority -> setPriority
-
added getStackSize
-
corrected C OSIthreadNameToId return type
-
should have an OSI type for priority (and avoid application knowledge of
actual priority values, range, whether 0 is low or high etc.; currently
I am ignoring priority with Posix threads)
-
similarly, how are portable programs to set stack sizes (currently I am
ignoring stack sizes with Posix threads)?
-
Posix implementation needs to review key usage (c.f. task variables) and
to protect key creation via a semaphore
-
need to re-work the logic on thread creation; thread needs to execute an
OSI-supplied wrapper before invoking the application routine; the wrapper
should wait for creation completion in a secure way
-
made setDestroySafe() and Unsafe() static (this makes sense at least for
VxWorks and Posix)
-
OSIthreadName() shouldn't return pointer
-
need taskIdFigure() or equivalent?
-
would like known "null" thread id (is int under VxWorks?)
OSItime
-
under VxWorks:
-
added getCurrentTime returning double
-
is dependent on tsDefs.h and TS_STAMP; do we want this?
-
fixed 1970->1990 correction (was 19 years) under both VxWorks and Solaris
C++ issues
-
shouldn't the interface seen by the OS-independent program be defined in
an abstract base class and shouldn't the OS-specific implementations inherit
from it? that way, adherence to the interface can be checked by the compiler
-
why implement the C++ and C versions independently? is this due to efficiency
concerns? for Posix, I implemented in C++ and made the C wrappers call
the C++
CA
-
hacked in OSI locks for Solaris (should base on "Unix" rather than Solaris)
-
changed osiSleep() call to use OSIthreadsleep() (may have been stack overflow
problem...)
-
really want to remove special VxWorks code as far as possible and rely
on OSI; would give ca_import() etc., which the sequencer uses/needs
Misc OSI and sequencer
-
error handling: want consistent way of knowing whether a feature is missing
(maybe a set of enquiry routines and applications or library init routines
should verify that the features that they require are present; don't always
want to return an error at run-time; sometimes prefer just to do nothing...)
-
need to check that timeouts are being handled consistently as FP seconds
everywhere (were VxWorks ticks)
-
why do I need ::thr_suspend() and not :: for the Posix ones?
-
strictly you should compile with _POSIX_C_SOURCE set to 199506L under Solaris
to use Posix threads; however, I just defined _REENTRANT (convenient because
EPICS is already built that way) and, touch wood, everything is OK
-
want to move libCom OSI stuff to src/osi (confusing at present)
-
mustn't forget to go through the thread and sem implementations and ensure
that all available options are being used correctly, errors being checked
for etc.
-
note that can't delete sequencers under VxWorks; have commented out the
ca_import and delete stuff; must get to that
-
fixed errlogInit problem under VxWorks (was never setting the "initialized"
flag to TRUE); still get two "errlog" tasks?!
-
want to support local variables and "long i, j;" declarations
-
should detect bad pvGetQ (etc) usage (i.e. initialize queue index to -1
and test before use... at compile time if possible)
Message system
-
want to be able to use the sequencer with CA, CDEV and KTL (although KTL
only via CDEV might be OK)
-
will not use "#ifdef CDEV" approach but will instead go via generic API,
"PV"
-
need to replace all CA calls and also all DBR usage; the API should define
its own simple type scheme (sequencer uses only DBR_xxx and DBR_TIME_xxx,
which is status, severity, time-stamp and value; xxx values used are CHAR,
SHORT, LONG, FLOAT, DOUBLE, STRING)
-
CA routines and constants used are:
-
ca_task_initialize()
-
ca_build_and_connect()
-
ca_import()
-
ca_field_type()
-
ca_element_count()
-
ca_puser()
-
ca_add_array_event()
-
ca_array_get_callback()
-
ca_array_put()
-
ca_flush_io()
-
ca_pend_event()
-
ca_clear_event()
-
ca_clear_channel()
-
ca_import_cancel()
-
ca_task_exit()
-
ECA_DISCONN
-
ECA_NORMAL
-
ECA_TIMEOUT
-
so will define pvInit(), pvGet() etc. interface and will implement pvCa.cc,
pvCdev.cc, pvKtl.cc etc. plus add Makefile support (will stop short of
run-time shareable library activation; use CDEV if you want that)
-
have to worry about ca_import(), ca_import_cancel(); from looking at CA
code, it's OK to ignore ca_import() and ca_import_cancel() if all
threads in an application are to share a CA context; they are needed if
some do and some don't
-
have to worry about locking; can't assume that the library under the pv
layer is thread-safe, so the pv layer will implement its own locking (using
OSI locks); a single mutex will ensure that only one thread at a time is
within the pv library
-
error handling is poor; could implement a "maximize
severity" and/or error stack scheme; could use C++ exceptions? also could
maintain on a per-variable rather than a per-system basis? it
also
ignores multi-threaded issues (maybe use thread-specific data?)
-
useful to add "valid" flags to classes? then destructors can set them to
FALSE (would protect against use of destroyed objects)
-
really sort out use of const wherever possible (will need to retrofit somewhat);
get into the habit of using it!
-
not queueing callbacks when called from within initiating routine (OK,
but need to document)
-
TS_STAMP... what to do? OSI uses it
-
KTL type conversions not fully implemented
-
KTL problem: need to perform operation on keyword in order to trigger search
and therefore connection hander
-
idea of supporting KTL_ID ioctl() and read/write (backwards compatible
but would allow channel info to be cached)
-
now working nicely with KTL; major problem is the fact that have to monitor
everything so that a connection remains open (related to KTL_ID notion)
-
not working under VxWorks; compiles but channels don't connect
-
demo sequence doesn't behave well on channel disconnect under KTL (probably
CAKE problem)
14-Feb-00
-
After a long gap, have returned...
-
Greg White's changes were merged in in Oct-99
-
Have packaged up a Linux / Solaris demo and tagged as FEB00
-
other issues:
-
while building KTL with Workshop 5.0, used "-Xc" and this broke KTL signal
code (avoided using it...)
01-Mar-00
-
Have made good progress with the new OSI and with cleaning up sequencer
code so that it should run unchanged (and without any of those nasty "###"
comments) under at least Solaris, Linux and VxWorks
-
Now is the time to sit back and make those final punch-lists
Sequencer
-
support building of a shareable library
-
complete the local variable and "long i, j;" support
-
avoid warnings re use of NULL and ASYNC (use different names, e.g. pvPutCallback())
-
in PV layer, setting error status each time, regardless of new error
(should clear message on success?)
-
in PV layer, bootstrap error problem: how to report pvSysCreate() failure?)
-
NB, in-built assumption that only one state-set reads/writes a given PV
-
get rid of all (possible) compilation warnings
-
stop using doubles for intervals?
-
use same DEBUG macro approach everywhere
KTL
-
finalize CAKE parallel completion support (allow on "wait", not just "notify";
requires new keywords to be backwards-compatible)
-
add type conversion (currently fails quietly); adjust the module written
for CDEV/CORBA?
-
add time-stamp support
-
define COKE parallel completion support
-
check for leaks with string/array callbacks
-
need lock in pvKtl.cc for error strings
-
need to maintain list to permit more than one keyword object per underlying
keyword (KTL name lookup issue)
Testing
-
extract wfccas2 code to make general standalone CA server that will configure
itself from .db and .dbd files (but have no scanning or device support)
-
still some memory leak issues on sequencer deletion
Documentation
-
re-write manual (wants a major overhaul; current one is very hard to use
as a reference work)
-
include full description of PV layer (enough info to write a new service,
e.g. CDEV)
General
-
need README (or README.html) files in all directories
-
need packaging up per the Andrew Johnson recommendations
-
make default build KTL-independent but make it easy to build KTL-specific
test directories
-
Solaris Workshop 5.0 problems (-Xc broke KTL signal code; had to disable
exceptions; some un-demangeable names; switched to g++; can't run purify!)
-
would like error log messages to be identified like syslog messages (hostname
is not enough)