Re: SNL sequencer apparently losing connection to underlying network

[Matt Newville <newville@cars.uchicago.edu>]

Hi Kevin,

What I meant to suggest was to consider taking the python code you are using to turn PV events into electronic-logbook entries out of the SNL code and run that code in a separate process.  You might have to change some logic about when logging events get triggered, but that might be as simple as setting a PV flag to mean "log now", and then have the python process look for changes in that PV and act accordingly.  Having a long/infinitely-running python process looking for a PV state change and then doing some combination of CA and logging/database interaction is really very easy.  I do this all the time.  

But, to summarize the earlier suggestions: if you are embedding pyepics in SNL,  you might need to create a new CA context in the python script with

    import epics

    epics.ca.context_create(0)

I don't know for sure that this is needed, but from the problems you describe it seems worth trying.

--Matt
 

On Fri, Dec 2, 2016 at 1:38 AM, Kevin Meyer <kevin.meyer@cosylab.com> wrote:

Hi Matt and Ben,

----- On 1 Dec, 2016, at 21:41, Matt Newville <newville@cars.uchicago.edu> wrote:

On Thu, Dec 1, 2016 at 6:21 AM, Benjamin Franksen <benjamin.franksen@helmholtz-berlin.de> wrote:

On 01.12.2016 10:15, Kevin Meyer wrote:
>
> Yes, the Python program uses pyEpics's epics.caget, but there are
> some complications here:
> 1. The Python context is created in the
> sequencer's global entry block and shut down in the global exit block
> (the PyObject's are initialised there and re-used in subsequent State
> entry calls).

Which method are you using to call Python functions from C?

In the SNL global entry block, which is called once, when the sequencer starts up, I do all the Python bindings:

For example, I initialise my "handle" structure members (such as "PyObject * pFuncText "):

    // pFuncText is also a borrowed reference
    elogbook->pFuncText = PyDict_GetItemString(elogbook->pDict, (char*)"createOlogEntry");
    //pClient would be null if the Python syntax is wrong, for example
    if (elogbook->pFuncText == NULL) {
        if (PyErr_Occurred()) {
            PyErr_Print();
            elogbook->initialised = 0;
            log_error("Failed to get logbook createOlogEntry function");
            log_python_error();
            return -3;
        }
    }

Then, in the API that is re-used by the SNL (i.e. in a particular state "entry" block), I use the structures created:

/*
* Write a text message to the logbook handle.
*
* Parameters:
*   log_handle = handle from call to elogbook_open.
*
* Returns 0 on success.
*  -1 = input parameter error
*  -2 = invalid handle
*  -3 = handle not initialised.
*  -4 = Other Python error
*/
int elogbook_log_text (int log_handle, const char *text){
    // sanity checking - allow to be called with invalid handle.
.. snip ..

    PyObject *pArgs, *pValue, *pText;

    //Build a tuple to hold my arguments
    pText = PyString_FromString(text);
    pArgs = PyTuple_New(2);
    PyTuple_SetItem(pArgs, 0, elogbook->pObjLogbook);
    PyTuple_SetItem(pArgs, 1, pText);

    //Call createOlogEntry function, passing it the input text
    pValue = PyObject_CallObject(elogbook->pFuncText, pArgs);
    if (pValue == NULL) {
        if (PyErr_Occurred()) {
            PyErr_Print();
            log_error("Call to createOlogEntry from elogbook_log_text failed");
            log_python_error();
            return -4;
        }
        log_error("Call to createOlogEntry returned NULL instead of return code");
        result = -4;
    } else {
        result = (int)PyLong_AsLong(pValue);
        Py_DECREF(pValue);
    }
    return result;
}

What happens if you do _not_ shut down the client context? (It may be
necessary to call ca_detach_context to avoid that being done
automatically by some finalizer on the Python side.)

> 2. How would the Python process EPICS get ahold of the
> calling-sequencer's C EPICS context?

I don't know how the Python binding works, but from C you would call
ca_current_context[1] to get the client context and ca_detach_context[2]
when you are done -- the last is so that the context is NOT destroyed
when the Python program exits.

Pyepics does provide 'ca.current_context()', 'ca.attach_context()', 'ca.detach_context()' and so on.   These are very thin wrappings of those provided from the C CA library.

By default, the initial context (that is, the one automatically created on the first access to the CA library) is used for all CA connections.  You have to explicitly tell pyepics to not do this.

I don't use any such context calls. The Python in question just does things like:

import epics
... snip ...

''' Create a new entry in the logbook from SUP POS status'''
def createOlogPOSPulseEndEntry(logbook, status, write_elog=1):
    Id          = epics.caget('CTRL:PULSE-ID', as_string=True)
    StartTime   = epics.caget('CTRL-SUP-TIME:T0-HR', as_string=True)
    EndTime     = epics.caget('CTRL-SUP-TIME:TEND-HR', as_string=True)

..snip...

Should I be doing anything different with ca contexts in the Python code??

By default, pyepics uses preemptive callbacks for nearly all connections.  To disable this, do

    import epics

    epics.ca.PREEMPTIVE_CALLBACK = False

prior to creating any connections.  Or, if you're creating contexts yourself, you can also use
    import epics

    epics.ca.context_create(0)

to create a context that does not use preemptive callbacks.

For Kevin's observation that some 'epics.caget(xxx)' calls work:  Are these channels connecting *only* from Python, or are they also connected by CA in the sequence program?

Good question: I will look into that...

I don't know of anyone using pyepics from an embedded C program, including an SNL program.  If that process is also doing CA calls, I can believe there could be trouble.  I don't think it *has* to fail, but I think you might be the first person to try this.

As an alternative way of looking at the problem: Is more work being done in the SNL code or in the Python code?  If most of the work is in Python, would it be plausible to move all the logic from the SNL code to a python process?  Pyepics doesn't provide a built-in finite-state-machine, but these are not too hard to write in python.  There are some advantages to having  PVs defined loaded and defined in an IOC and all the logic that interacts with those PVs in a separate process.   That's not saying you shouldn't use the amazing sequencer, but sometimes what you really want is a small program that operates on a set of PVs.

Well, my SNL is generated from Enterprise Architect XMI via an XSLT .. I *could* craft an XMI to Python XSLT, too!! :D

Regards,

Kevin

--
Kevin Meyer
Cosylab, Ljubljana, Slovenia
Telephone: +386-(0)1-320-47-82   Mobile: +386-(0)70-260-321