Re: EPICS Python client application survey

[Matt Newville <newville@cars.uchicago.edu>]

Hi Michael,

> Did you mean to cc this to the list?  It would probably be of interest to the list (and your "all" makes me think maybe you meant to).

I did mean to cc my first message to tech talk and forgot.

> I think I've managed to make the cothread documentation
>  reasonable.  I've never been a huge fan of in-code
>  documentation, as it often ends up with the wrong
>  structure (code structure and documentation structure are
>  generally quite different things!), even when it's done well,
>  which I think yours is.

I agree that in-code documentation (or inter-leaved code and doc) is
sometimes not helpful.  I have more difficulty with the PythonCA
interface, which seems to have close to zero documentation (last I
looked).  I would definitely assume that if someone is using Python,
they may not know C or even Channel Access very well, and are probably
looking for something very simple (say, like caput and caget) and
pythonic.

> I am wondering if it would be possible to unify some of the
> interface at least.  I guess we'd need to understand each
> other's code!

I'm willing to try to do this (understand and unify).  I know my C /
swig interface could definitely be improved or replaced.

For threads, I have no strong opinion and very little experience. This
wasn't available when I started using CA with Python, and I've never
actually needed it.  I am a little surprised when I read that not
supporting threads and having to call poll() are seen as significant
drawbacks, but I'm willing to use something that supports some aspect
of threading that works reasonably well.

>> Personally, I don't see SWIG as a major drawback (it is not required
>> for installation, just to help generate the interface code), but I'm
>> certainly willing to *not* use it.
>
> Certainly I found moving to ctypes to be very liberating.  Take
> a peek at cadef.py and dbr.py in cothread.  Looking at your
> swig/epics.i I'm not persuaded that SWIG is helping us: it's
> looking awfully like a mildly obfuscated C Python
> implementation!

I agree.  I started out using SWIG before ctypes was available.  The
SWIG-dependent code I have (mostly type mapping and exposure of C
functions to Python) could easily be replaced with straight C/Python
and ctypes.   That would definitely improve the code.

>> I do feel there are several
>> features that might seem optional to a C library, but are more than
>> just niceties for a Python interface.  These include:
>>   - automatic conversion of character waveforms to/from Python strings
>> (so as to automatically handle strings >40 characters).
>

> I think *automatic* would be acutely unfriendly, but on demand
>  would be nice.  But then wouldn't that really be down to how
>  numpy character arrays work?  Certainly I've found that
>  delivering waveform as numpy waveforms works very nicely.

Ah, I mean that I often make character waveform records so that I can
have strings longer than 40 characters.  I want to be able to do a
put() and get() to these with normal Python strings.  I'm not thinking
of them as numpy character arrays.

I suppose that having an option for how to handle character waveforms
would be best.  Similarly, options for how to map other array values
would be helpful.  Arrays of integers might be re returned as a list,
numpy array (possibly inferring its shape?), or perhaps even a PIL
Image depending on the content.

I think that fits with the idea of not needing all the DBR_*** types.
The values for PVs in Python should be sensible Python objects, and
may not necessarily be the same as the C values.

>>   - automatic support of "raw" and "string" representation of values.
>>   - support for DBR_CTRL_*** types: I admit I was lazy and implemented
>> only the DBR_*** and DBR_CTRL_*** types, not the
>>        DBR_STS, DBR_GR,  ...  variations.
>

> For cothread I looked long and hard at this, and decided that
>  there were too many DBR_ variations: really I think it's enough
>  to just provide DBR_*, DBR_TIME_* and DBR_CTRL_*, which is what
>  I've done.

I agree.  I did not implement the TIME type, but it should be
available.  STS and GR are subsets of CTRL, and trying to save a
little memory with these cannot be justified with Python.

>>   - automatic fetching of enum strings for enum types.
>>   - a working "wait" field on put() and/or an easy to use polling
>> mechanism for put-is-complete.
>
> These work nicely in cothread, and were quite straightforward to implement.
>
>> Some of these need (or are better off) being in the C level, but I
>> think they are pretty orthogonal to any of the other aspects of the
>> code.
>>
>> Perhaps I should look more closely at Cothreads or John's new code.  I
>> was not very impressed with the KEK or PSI codes, and they didn't seem
>> very impressed by my needs or desires either.....  As you can probably
>> tell from the above, I do have some strong opinions, but I'd certainly
>> be willing to collaborate and make a better overall implementation and
>> package.
>>
>> Where should we start?
>
> I guess a closer read of each other's code and some more
>  e-mail, to begin with.  We're all going to be wedded to our own
>  code, I guess, but let's see what we can steal from each other.

Sounds good.  I need to study your code.  I'm also interested in what
John Hammonds's working on. Again, I am not at all wedded to my code
(especially my C/swig code!).  I am more committed to having a PV (or
Channel) object that is a fairly high-level object that supports
user-defined callbacks and but hides most of the C implementation.
I'm sure that this could be put on top of nearly any reasonable C
wrapper.

--Matt Newville <newville at cars.uchicago.edu>