Re: pyepics get_with_metadata inconstant behaviour

[Matt Newville via Tech-talk <tech-talk at aps.anl.gov>]

Hi Marco, 

On Fri, Sep 16, 2022 at 1:38 PM Marco A. Barra Montevechi Filho via Tech-talk <tech-talk at aps.anl.gov> wrote:

Good evening!

We are perceiving a strange behaviour with get_with_metadata method in pyepics here.
We have a motor record PV with defined PREC and EGU fields. We have a script that goes:

my_pv = epics.PV("MGN:B:PB01:CS7:m4")
my_dict = my_pv.get_with_metadata(use_monitor=True)

First weird behaviour: between the many values returned in my_dict, the "precision" key is associated with "None" value and the same goes for "unit" key:

{'pvname': 'MGN:B:PB01:CS7:m4', 'value': 3.56963, 'char_value': '3.56963', 'status': 0, 'ftype': 20, 'chid': 28543096, 'host': '10.10.10.38:46317', 'count': 1, 'access': 'read/write', 'write_access': True, 'read_access': True, 'severity': 0, 'timestamp': 1663352086.023605, 'posixseconds': 1663352086.0, 'nanoseconds': 23605915, 'precision': None, 'units': None, 'enum_strs': None, 'upper_disp_limit': None, 'lower_disp_limit': None, 'upper_alarm_limit': None, 'lower_alarm_limit': None, 'lower_warning_limit': None, 'upper_warning_limit': None, 'upper_ctrl_limit': None, 'lower_ctrl_limit': None, 'nelm': 1, 'type': 'time_double', 'typefull': 'time_double'}

Second weird behaviour: when executed several times, dictionaries with different number of keys are returned. Changing use_monitor=False limited the dictionary size to

{'status': 0, 'severity': 0, 'timestamp': 1663353141.426072, 'posixseconds': 1663353141.0, 'nanoseconds': 426072330, 'value': 3.56963}

and the number of keys does not vary, but now we dont have precision and unit, which would be useful. Are we doing something wrong? Where does get_with_metadata gets its information from? Shouldnt "unit" come from EGU field?

I couldnt find answers for that in the documentation. Thanks for any help 🙂

Sorry for the confusion!

For each PV in Channel Access, there are a few choices in "record type":  "plain", "severity", "time", "control", and "graphics" (I might be missing one or two).  These will change which set of metadata can be retrieved with the data.   

If I recall correctly, "severity" is a subset of "time" and "graphics" is a subset of "control".   "plain" gives no extra metadata, "time" gives timestamp, alarm, and severity, and "control" giving lots of metadata, but importantly not the timestamp.   

In pyepics, there is a choice of this record type (called "form" in the pyepics code and docs, as "type" is a builtin in Python), with "time" being the default, "plain" and "ctrl" being available.   That is

    mypv = epics.PV(pvname, form='ctrl') 

will create a PV with all the control metadata available but by default

    mypv = epics.PV(pvname, form='ctrl') 

will create a PV with time metadata.   Even for a "time" PV, doing `mypv.get_ctrlvars()` will return the "control" metadata (precision, limits, etc).

`mypv.get_with_metadata()` will try to give all the available metadata, but it might not explicitly look those up - it might rely on cached values from the last monitor event.   The intention for the `use_monitor` option for  `mypv.get_with_metadata()` is to control whether those cached values are used, but it does seem to be doing something incorrectly: I'll investigate.  

I think that the best way to get an accurate and complete set of all the metadata, use

   mypv.get_with_metadata(with_ctrlvars=True)

That will really do a fetch of the current values of the control metadata, or to get the control metadata with 

   mypv.get_ctrlvars()

Again, sorry for the confusion. 

--Matt