Re: Reading data with StreamDevice - order of input records processing for scalcout not respected

[Zimoch Dirk via Tech-talk <tech-talk at aps.anl.gov>]

Exactly. To ensure in order execution, only use output PP links and forward links. Never use input PP links.

Dirk

> On 28 Feb 2025, at 09:30, Andrea Celentano via Tech-talk <tech-talk at aps.anl.gov> wrote:
> 
> Hi Dirk,
> I think I figured out the answer myself. Quoting from the StreamDevice documentation: 
> "StreamDevice is an asynchronous device support (see IOC Application Developer's Guide chapter 12: Device Support). Whenever the record is processed, the protocol is scheduled to start and the record is left active (PACT=1). The protocol itself runs in another thread. That means that any waiting in the protocol does not delay any other part of the IOC."
> So, in the chain below of FLNK: 
>  "$(P):C1:BurstWord” —>  "$(P):C1:Burst” —(X)--->  "$(P):C1:BurstCalc” —> "$(P):C1:BurstCalcDouble” and $(P):C1:BurstString”.
> the arrow marked as "X" will be executed asynchronously, at later time, and that's why my other record $(P):C1:BurstStatus grabs the "old" value of $(P):C1:BurstString..
> 
> I'll find another way to proceed, keeping this clearly in mind!
> 
> On 2/27/25 22:31, Andrea Celentano via Tech-talk wrote:
>> Hi Dirk, 
>> thanks for your reply. I fully agree on the difficulty of using input links with PP together with asynchronous device support, as in StreamDevice.
>> 
>> That’s why I used FLNK in my design, and this works properly: when I process the record "$(P):C1:BurstWord”, the processing sequence is  "$(P):C1:BurstWord” —>  "$(P):C1:Burst” —>  "$(P):C1:BurstCalc” —> "$(P):C1:BurstCalcDouble” and $(P):C1:BurstString”.
>> 
>> What I do not understand is why, for the following two records:
>> 
>> record(scalcout, "$(P):C1:BurstStatus")
>> {
>>     field(CALC, "BB")
>>     field(INAA, "$(P):C1:BurstStatusS PP")
>>     field(INBB, "$(P):C1:BurstCalcString")
>>     field(SCAN, "Passive")
>> }
>> 
>> record(stringout, "$(P):C1:BurstStatusS")
>> {
>>     field(FLNK, "$(P):C1:BurstWord")
>>     field(OMSL, "closed_loop")
>>     field(OUT,  "$(P):C1:BurstWord")
>>     field(VAL,  "STATE")
>> }
>> 
>> when I process the first record, the order of the inputs is not respected. 
>> 
>> In particular, INBB retrieves immediately its value from "$(P):C1:BurstCalcString”, without waiting for the sequence triggered by INAA="$(P):C1:BurstStatusS PP” to be completed.
>> According to manual (https://urldefense.us/v3/__https://docs.epics-controls.org/en/latest/process-database/EPICS_Process_Database_Concepts.html*closing-an-analog-control-loop__;Iw!!G_uCfscf7eWS!cqUV0b8kgyQgyz6_RFXTSrlBtgC3sN51kVqi1k_xRfLU5gTvVo8Cr1VXBZDDvUnZir-GkNJs8VigB_1Y_naiNjy9HA$ , description of Fig. 3), the CALC record, “it fetches the inputs for the calc record in order”. But here this does not seem to be the case..
>> 
>> Thanks,
>> Andrea
>> 
>> 
>> 
>> 
>>> On Feb 27, 2025, at 22:12, Zimoch Dirk <dirk.zimoch at psi.ch> wrote:
>>> 
>>> Hi Andrea 
>>> 
>>> Input links with PP do not work well with asynchronous device supports like StreamDevice.
>>> The link starts processing the target record and returns immediately. Before anything actually happens. In particular, it returns the old value.
>>> 
>>> IStreamDevice assumes that the same request always results in the same reply format. If that is not the case, it becomes difficult. If the difference only is that a partikular piece of data is missing bin some cases, it may be possible to use „%?“ which returns 0 if parsing fails. But for more detailed suggestions I need to see some examples of the different possible formats.
>>> 
>>> Dirk
>>> 
>>>> Am 27.02.2025 um 18:49 schrieb Andrea Celentano via Tech-talk <tech-talk at aps.anl.gov>:
>>>> 
>>>>  Dear colleagues,
>>>> I am struggling with a problem related to a waveform StreamDevice record.
>>>> Specifically, the device I want to interface with (Lecroy T3AFG200) has a remote interface based on a text protocol. When I send the string "C1:WaveBurst?", it replies with a string such as:
>>>> C1:BTWV STATE,ON,PRD,0.01S,STPS,0,TRSR,INT,TRMD,OFF,TIME,1,DLAY,1.38052e-06S
>>>> where the value of multiple parameters is reported. I want to read these values - in the above example, e.g., PRD is "0.01S", TRMD is "OFF", DLAY is "1.38052e-06S". 
>>>> What makes this task not trivial is that the number of reported parameters, and their order, depends on the status of the device. Thus, the output string is not always the same. 
>>>> I tried to work as follows.
>>>>     • I read the data in a waveform record, where each element is a string, splitting the string by comma (and ignoring the initial C1:BTWV part)
>>>>     • I process this data through an aSub record. The associated sub-routine takes as input the waveform record (INAA), and as also another string record reporting the field to be extracted (INBB). The two outputs are a double value and a string value containing, respectively, the double representation (OUTA) and the string representation (OUTB) of the element following INBB, if any.
>>>>         • For example, if the waveform record is the one above, and the string identifier record is "TRSR", the routine will return "INT" in the string representation. If the string identifier record is "DLAY", the routine will return "1.38052e-06" in the double representation (I trim out the last "S" character)
>>>> I developed the following database records.
>>>> To read the data from the instrument and parse it, I have four records. The order of processing is dictated by the fact that, for a StreamDevice record, I cannot use it trivially as an input link with "Process Passive", since this will result in the OLD value to be reported before the protocol is executed, so I must go "the other way around".
>>>> record(waveform, "$(P):C1:Burst")
>>>> {
>>>>     field(DTYP, "stream")
>>>>     field(FLNK, "$(P):C1:BurstCalc")
>>>>     field(FTVL, "STRING")
>>>>     field(INP,  "@LecroyT3AFG.proto ChannelBurst($(P),1) T3AFG")
>>>>     field(NELM, "100")
>>>>     field(SCAN, "Passive")
>>>> }
>>>> record(stringin, "$(P):C1:BurstWord")
>>>> {
>>>>     field(FLNK, "$(P):C1:Burst")
>>>>     field(SCAN, "Passive")
>>>> }
>>>> record(aSub, "$(P):C1:BurstCalc")
>>>> {
>>>>     field(FTA,  "STRING")
>>>>     field(FTB,  "STRING")
>>>>     field(FTVA, "DOUBLE")
>>>>     field(FTVB, "STRING")
>>>>     field(INPA, "$(P):C1:Burst")
>>>>     field(INPB, "$(P):C1:BurstWord")
>>>>     field(LFLG, "IGNORE")
>>>>     field(NOA,  "100")
>>>>     field(OUTA, "$(P):C1:BurstCalcDouble")
>>>>     field(OUTB, "$(P):C1:BurstCalcString")
>>>>     field(SNAM, "LecroyT3AFG_asub")
>>>> }
>>>> 
>>>> record(ai, "$(P):C1:BurstCalcDouble")
>>>> {
>>>> }
>>>> 
>>>> record(stringin, "$(P):C1:BurstCalcString")
>>>> {
>>>>     field(VAL,  "INITIAL_VALUE")
>>>> }
>>>> 
>>>> To read the status, I have two specific records
>>>> record(scalcout, "$(P):C1:BurstStatus")
>>>> {
>>>>     field(CALC, "BB")
>>>>     field(INAA, "$(P):C1:BurstStatusS PP")
>>>>     field(INBB, "$(P):C1:BurstCalcString")
>>>>     field(SCAN, "Passive")
>>>> }
>>>> 
>>>> record(stringout, "$(P):C1:BurstStatusS")
>>>> {
>>>>     field(FLNK, "$(P):C1:BurstWord")
>>>>     field(OMSL, "closed_loop")
>>>>     field(OUT,  "$(P):C1:BurstWord")
>>>>     field(VAL,  "STATE")
>>>> }
>>>> My idea was the following: when I process "$(P):C1:BurstStatus", the following sequence should take place:
>>>>     • INAA is processed: $(P):C1:BurstStatusS -> $(P):C1:BurstWord -> $(P):C1:Burst ->$(P):C1:BurstCalc. The two records "$(P):C1:BurstCalcDouble" and "$(P):C1:BurstString" are populated.
>>>>     • INBB is processed $(P):C1:BurstCalcString
>>>> I thus assumed that the full chain triggered by INAA was processed before the INBB record was read. 
>>>> However, this is not the case. After triggering the process of $(P):C1:BurstStatus, I have $(P):C1:BurstStatus.SVAL equal to "INITIAL_VALUE", as if INBB is processed before (or during) INAA processing, and the order is not respected. But the value of $(P):C1:BurstCalcString is the correct one..
>>>> I am currently using epics version 7.0
>>>> Thanks,
>>>> Bests,
>>>> Andrea Celentano
>>>> 
>>>> 
>>>> 
>>