Re: Mapping customised pva structures to epics v4 Normative Types and v3 PVs (channel access).

[Gabriel Fedel via Tech-talk <tech-talk@aps.anl.gov>]

Hello Heinz,

so, I'm not much familiar of using p4p to implement a IOC serving PVA PVs, but as far I can tell the issue is probably that you may need to implement a unwrap function that is able to handle the case where only one column is changed... It seems it expect that the change always send all the columns... I may be wrong. Sorry for not being able to help here.

What I can tell is that if you implement your table using a database, and qsrv , qsrv will handle it properly. I have implemented a table as yours (attached) and it worked fine.

Cheers

Gabriel

---

From: Heinz Junkes
Sent: Friday, August 15, 2025 5:18 PM
To: Gabriel Fedel
Cc: Anders Lindh Olsson; Mark Rivers via Tech-talk
Subject: Re: Mapping customised pva structures to epics v4 Normative Types and v3 PVs (channel access).

Hallo Gabriel,
Thank you very much for taking the time, and I apologise for only replying now.
I had to finish an application that is due tonight.

When calling pvput TEST:LLPG "{'value' : {'Ch':[1,2]}}”
Unfortunately, I get this error message:
Error: Unable to unwrap Value(id:epics:nt/NTTable:1.0, Value(array([1, 2], dtype=uint16))) with <function NTTable.unwrap at 0x7f2b7b5bc680>

Auf der ’Server-Seite’ sieht es so aus:

(p4ptest) epics@epics-spielwiese:~/FHI/TEST_PVA$ python pvaTest.py
Unexpected
Traceback (most recent call last):
  File "/home/epics/FHI/TEST_PVA/p4ptest/lib/python3.11/site-packages/p4p/server/raw.py", line 21, in value
    return self._unwrap(V)
           ^^^^^^^^^^^^^^^
  File "/home/epics/FHI/TEST_PVA/p4ptest/lib/python3.11/site-packages/p4p/nt/__init__.py", line 231, in unwrap
    for rval in izip(*cols):
                ^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/epics/FHI/TEST_PVA/p4ptest/lib/python3.11/site-packages/p4p/server/thread.py", line 19, in _on_queue
    M(*args)
  File "/home/epics/FHI/TEST_PVA/pvaTest.py", line 14, in put
    pv.post(op.value(), timestamp=time.time())
            ^^^^^^^^^^
  File "/home/epics/FHI/TEST_PVA/p4ptest/lib/python3.11/site-packages/p4p/server/raw.py", line 23, in value
    raise ValueError("Unable to unwrap %r with %r"%(V, self._unwrap))
ValueError: Unable to unwrap Value(id:epics:nt/NTTable:1.0, Value(array([1, 2], dtype=uint16))) with <function NTTable.unwrap at 0x7f2b7b5bc680>

Das Testfile ist angelehnt an ein Beispiel welches ich beim SLAC gefunden hatte:

(p4ptest) epics@epics-spielwiese:~/FHI/TEST_PVA$ cat pvaTest.py

# Python script to generate NTTable
from p4p.nt import NTTable, NTScalar
from p4p.server import Server, ServerOperation
from p4p.server.thread import SharedPV
import time

import numpy as np

class Handler(object):
    """ A handler for dealing with put requests to our test PVs """

    def put(self, pv: SharedPV, op: ServerOperation) -> None:
        """ Called each time a client issues a put operation on the channel using this handler """
        pv.post(op.value(), timestamp=time.time())
        op.done()

table_type = NTTable([('Ch', 'H'),('Width', 'H'), ('Delay', 'H'), ('ExtraDly', 'H'), ('Atten', 'H')])
table_rows = []
table_rows.append({"Ch": 1, "Width": 1000, "Delay": 100, "ExtraDly": 10, "Atten": 1})
table_rows.append({"Ch": 2, "Width": 2000, "Delay": 200, "ExtraDly": 20, "Atten": 2})
initial_table = table_type.wrap(table_rows)

table_pv = SharedPV(nt=table_type, initial=initial_table, handler=Handler(), timestamp=time.time())
server = Server.forever(providers=[{"TEST:LLPG": table_pv}])


Danke Heinz


> On 15. Aug 2025, at 13:03, Gabriel Fedel <gabriel.fedel@ess.eu> wrote:
>
> Hi Heinz,
>
> On phoebus you can access specific fields from NT PVs using /, for example you could access the labels as:
> TEST:LLPG/labels
>
> As far I know phoebus still does not implement a way to edit the table itself (but it may exist an interest for it).Using a NTTable you could also use Save and Restore (we are using it here at ESS), but there is some limitations (e.g. not possible to change labels).
>
> if you want to change only one column of the table, you could do something like this:
>
> pvput TEST:LLPG "{'value' : {'Ch':[1,2]}}"
>
> Cheers
> Gabriel
> From: Tech-talk on behalf of Heinz Junkes via Tech-talk
> Sent: Thursday, August 14, 2025 4:25 PM
> To: Anders Lindh Olsson
> Cc: Mark Rivers via Tech-talk
> Subject: Re: Mapping customised pva structures to epics v4 Normative Types and v3 PVs (channel access).
>
> Hi,
>
> I have now chosen NTTables instead of a custom structure in order to at least make ArchiverApplianc happy.
>
> (p4ptest) epics@epics-spielwiese:~/FHI/TEST_PVA$ pvinfo TEST:LLPG
> TEST:LLPG
> Server: 10.0.0.40:5075
> Type:
>     epics:nt/NTTable:1.0
>         string[] labels
>         structure value
>             ushort[] Ch
>             ushort[] Width
>             ushort[] Delay
>             ushort[] ExtraDly
>             ushort[] Atten
>         string descriptor
>         alarm_t alarm
>             int severity
>             int status
>             string message
>         time_t timeStamp
>             long secondsPastEpoch
>             int nanoseconds
>             int userTag
>
> (p4ptest) epics@epics-spielwiese:~/FHI/TEST_PVA$ pvget  TEST:LLPG
> TEST:LLPG <undefined>
> Ch Width Delay ExtraDly Atten
>  1  1000   100       10     1
>  2  2000   200       20     2
>
> Unfortunately, I am unable to set individual values in the table using pvput. I can only address everything:
>
> epics@epics-spielwiese:~/FHI/TEST_PVA$ pvput TEST:LLPG value.Ch="[3,4]" value.Width="[111, 222]" value.Delay="[11, 223]"  value.ExtraDly="[11, 22]"  value.Atten="[1331, 2232]”
>
> Old : 2025-08-14 16:22:29.599
> Ch Width Delay ExtraDly Atten
>  3   111    11       11  1111
>  4   222   223       22  2232
> New : 2025-08-14 16:22:35.699
> Ch Width Delay ExtraDly Atten
>  3   111    11       11  1331
>  4   222   223       22  2232
>
> I also have no idea how I can then address the individual fields in Phoebus.
>
> Heinz
>
> > On 13. Aug 2025, at 11:24, Anders Lindh Olsson via Tech-talk <tech-talk@aps.anl.gov> wrote:
> >
> > Hello,
> >  On your question:
> > >> Now the question to the group: Is there a better way? And how would you implement my idea?
> >  I would map the data structure of the system to EPICS rather than the other way around. So I would compose PVs like FHIFEL:llpg_1:Ch1-Width, FHIFEL-llpg_1:Ch1-Delay, etc.
> >   Cheers
> > Anders
> >  On 2025-08-13, 07:27, "Tech-talk" <tech-talk-bounces@aps.anl.gov> wrote:
> > Guten Morgen,
> >  One thing that has been on my mind for a long time:
> >  We have developed our own devices such as llrf generators, pulse generators, etc., which run on a Raspberry Pi or BeagleBone Black as a management interface.
> > We then use p4p on that devices and offer the data points in a pva structure tailored to the respective device.
> >  E.g.
> >  (base) hactar:~ junkes$ pvinfo  FHIFEL:llpg_1
> > FHIFEL:llpg_1
> > Server: 141.14.134.173:5075
> > Type:
> >     structure
> >         ushort Ch1_Width
> >         ushort Ch2_Width
> >         ushort Ch1_Delay
> >         ushort Ch2_Delay
> >         ushort Ch1_ExtraDly
> >         ushort Ch2_ExtraDly
> >         ushort Ch1_Atten
> >         ushort Ch2_Atten
> >   epics@felpvagate:~$ pvinfo FHIFEL:bbamp_1
> > FHIFEL:bbamp_1
> > Server: 10.0.0.110:5075
> > Type:
> >     structure
> >         boolean ITLK
> >         boolean TRIG
> >         boolean PWR
> >         boolean STAT
> >         double SWR
> >         double FWD
> >         double REV
> >         byte RFIN
> >         double TSET
> >         double TMEAS
> >         int PIDI
> >         int PIDV
> >         boolean TTHI
> >         boolean PIDLU
> >         double PID_PWR
> >         boolean CLRER
> >  Now we are experiencing some issues with it.
> >  1.) The ArchiverAppliance cannot archive these PVs.
> > 2.) Our 'old' Save&Restore supports channel access only.
> >  How should we best deal with this?
> >  My current idea is to implement a service (Python?) that reads this user-defined structure with pva, breaks down the structure into individual NT-pvs and offers these 'individually' via pva (for the Archiver Appliance, for example) and, at the same time, also offers these PVs as ca-PVs for channel access (for our Save&Restore).
> >  Now the question to the group: Is there a better way? And how would you implement my idea?
> >  Danke Heinz
> >  ------------------------------------------------------------------------------
> > Fritz-Haber-Institut    | Phone:         (+49 30) 8413-4270
> > Heinz Junkes             | Fax (G3+G4):   (+49 30) 8413-5900
> > Faradayweg 4-6        | VC: https://zoom.fhi.berlin/junkes
> > D - 14195 Berlin        | E-Mail:        junkes@fhi-berlin.mpg.de
> > ------------------------------------------------------------------------------
> > “Sorry I’m a bit late, had a terrible time…
> > All sort of things cropping up at the last moment. Uh, how are we for time?”
> > —Zarquon's address to Milliways

Attachment: test_heinz.db
Description: test_heinz.db