Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020 
<== Date ==> <== Thread ==>

Subject: Re: StreamDevice processing issue
From: "Davis, Mark via Tech-talk" <>
To: Dirk Zimoch <>, EPICS <>
Date: Fri, 15 Mar 2019 18:26:56 +0000
Hi Dirk,

Actually, the device does send ETX to indicate the end of the data portion of a message, but it also adds a checksum byte after that.  I could, of course, just tell StreamDevice to ignore the extra byte, but I discovered that the Moxa has a setting that lets you tell it to include the char AFTER the delimiter char, so I enabled the option to send a packet when it got the delimiter char (+1 more).  But this made no difference.

I have also been experimenting with the StreamDevice timeouts (which I already had to set fairly high to eliminate various timeouts) and the various timeouts and packet-related settings in the Moxa.  So far, nothing I have tried has made any difference.

Here are my current settings for the StreamDevice protocols file:## times are in milliseconds
LockTimeout = 20000;  # max wait for exclusive access to the device
WriteTimeout =  500;  # max wait to be able to write to the device
ReplyTimeout = 1000;  # max wait until the start of a reply
ReadTimeout =  1000;  # max idle time after start of reply
PollPeriod =    200;  # How often to poll for new input for "I/O Intr" mode records

ExtraInput = Ignore;
MaxInput = 0;

I decided to test my hypothesis regarding the limitation in the StreamDevice, and sure enough, it works.  If I change the protocols that need to grab data that comes in the 2nd packet of a response that is split across multiple packets, using 2 "in" commands (one to recognize and absorb the entirety of the 1st packet and one to grab the desired value from the 2nd packet), it works just fine.

Unless I can find another solution (or StreamDevice is improved to handle matching "in" patterns that span multiple packets), the question is just how consistent is the packet size used by the Moxa's?  As long as it is consistent, then it SHOULDN'T be a problem.  Even with large values for the setting that tells it how long to wait for more characters before sending the packet, it still sends what it has (always using the same packet size) in the middle of a stream of characters coming from the device.  So it probably IS consistent.  But it still seems like a major kludge as workarounds go.

There is one possible saving grace:  Suppose you have a protocol with a single "in" command that works just fine.  If you break the contents of the "in" command into multiple consecutive "in" commands, will it still process and produce the same result given the input that worked with the single "in" command version (ignoring the possibility that the pattern is now multiple smaller patterns that could match things the single larger pattern would not have)?

I think I experimented with this earlier and found that the answer is NO:  That StreamDevice assumes a 1-to-1 relationship between the current "in" command waiting for a match and the "responses" from the device.  Or said another way, once an "in" command finds a match in the current "response", it is done with that response and the next "in" command must match something in the next (pr possibly a later) response.  If each response is one "line", then this means multiple/consecutive "in" commands in one protocol cannot grab data from the same "line".

Assuming this is true, then the issue is really how StreamDevice defines what constitutes a "response".  And what I found today suggests that, at least when the data from the device is delivered as packets, that each packet is treated as a separate "response", at least when it comes to matching "in" command patterns in the protocols linked to "I/O Intr" records.

A number of assumptions here, and there may of course be other possibilities I haven't thought of, but so far this is the only model I have thought of that fits the behavior I have seen.

Mark Davis
NSCL/FRIB Control Systems Software Engineer

On 3/15/2019 12:00 PM, Dirk Zimoch wrote:
Hi Mark,

Have you tried to increase ReadTimeout? It should be longer that the gap between two packages.

As your device neither has separators (because it is binary, not ASCII) nor fixed input length, timeout is about the only thing to define "end of input". So you need to fine-tune ReadTimeout somewhere between the delay between packages and the delay between messages.

I do not have such a device, but I can try to write a simulation. But that will take a while.

Best regards,

On 15.03.19 16:38, Davis, Mark via Tech-talk wrote:
HI all,

I am working on a StreamDevice interface for an Edwards STP turbomolecular pump and am having a problem with processing of one of the longer replies.

The connection to the pump consists of a TCP connection from the IOC to a 4-port Moxa serial/Ethernet server and an RS-232 connection between the Moxa and the pump running at 9600 baud.

So far, everything works fine this way except for the one case where the Moxa is breaking the response up in to multiple packets.

Below are the protocol definitions for the problem case.  Using the data capture ability of the Moxa (or the asyn trace commands, although that appears to be limited to showing a max of 80 of the bytes received for each packet, even though the counts are correct) , I can see that the device is sending the expected response, and that the ReadEventsWT protocol (linked to a record that scans periodically) is processing the 2nd and 3rd "out" commands at the appropriate time, which tells me it must be getting all the bytes expected for each "in" command despite that fact that they are split across 2 or 3 packets.  This is further supported by that fact that, before I got the character counts correct, I was getting a CALC error for the record linked to the ReadEventsWT protocol.

The problem appears to be that the ability to match a sequence of bytes for the "in" command when the bytes span multiple packets does not work for protocols that do not start with an "out" command (i.e. those linked to "I/O Intr" records).

Is this a known problem?  Has anyone run across this before?

I understand that this would not be a trivial issue to fix.  To make this work, StreamDevice would have to determine the maximum length of bytes expected by all the "in" commands in protocols that had no "out" commands, and each time new bytes were received, retesting for a match for each of those protocols would have to include not just the most recently received bytes but enough of the ones that came before it to guarantee that you did not miss a match.

Looking at the examples portion of the StreamDevice doco pages, I expect there is a good chance that if I treated every packet like a separate "line" of input, with one "in" command for each packet, that would allow it to work.  But even if it did, it would require that the Moxa always broke up the responses in to the same size packets, and that the protocol knew what that size was.  Not exactly a recipe for reliable operation.

There are some other possibilities I will be looking in to that might force the Moxa to send the entire response as one packet, but this seems like an issue that could apply to other devices, especially when a Moxa or similar device is part of the connection.

NOTE:  I am currently using StreamDevice version 2.5.  Even before this issue came up I tried 2.6 but it didn't work (I don't remember exactly what the problem was, and I didn't have time then to look in to it) so I went back to 2.5.

##-----[ReadEventsWithTime]---------------------------------------------------- # Record for this protocol must process periodically ReadEventsWT { out "\x02" "001?}" "\x03%<xor8ff>"; in "\x06\x02" "001 }%253c" "\x17%*1c"; out "\x06"; in "\x02" "002%*151c" "\x03%*1c"; out "\x06"; } # records for these protocols must use "I/O Intr" SCAN # These assume err time flag format = "01" curNumErrs { in "\x02" "001 }%2x"; } # current # of errors maxNumErrs { in "\x02" "001 }%*2c%2x"; } # max # of errors err1Type { in "\x02" "001 }%*4c%2x"; } # Error 1 # err1Date { in "\x02" "001 }%*8c%6c"; } # Error 1 date (YYMMDD) err1Time { in "\x02" "001 }%*14c%4c"; } # Error 1 time (HHMM) err2Type { in "\x02" "001 }%*24c%2x"; } # Error 2 # err2Date { in "\x02" "001 }%*28c%6c"; } # Error 2 date (YYMMDD) err2Time { in "\x02" "001 }%*34c%4c"; } # Error 2 time (HHMM) err3Type { in "\x02" "001 }%*44c%2x"; } # Error 3 # err3Date { in "\x02" "001 }%*48c%6c"; } # Error 3 date (YYMMDD) err3Time { in "\x02" "001 }%*54c%4c"; } # Error 3 time (HHMM) err4Type { in "\x02" "001 }%*64c%2x"; } # Error 4 # err4Date { in "\x02" "001 }%*68c%6c"; } # Error 4 date (YYMMDD) err4Time { in "\x02" "001 }%*74c%4c"; } # Error 4 time (HHMM) err5Type { in "\x02" "001 }%*84c%2x"; } # Error 5 # err5Date { in "\x02" "001 }%*88c%6c"; } # Error 5 date (YYMMDD) err5Time { in "\x02" "001 }%*94c%4c"; } # Error 5 time (HHMM) err6Type { in "\x02" "001 }%*104c%2x"; } # Error 6 # err6Date { in "\x02" "001 }%*108c%6c"; } # Error 6 date (YYMMDD) err6Time { in "\x02" "001 }%*114c%4c"; } # Error 6 time (HHMM)

Mark Davis
NSCL/FRIB Control Systems Software Engineer

Re: StreamDevice processing issue Davis, Mark via Tech-talk
StreamDevice processing issue Davis, Mark via Tech-talk
Re: StreamDevice processing issue Dirk Zimoch via Tech-talk

Navigate by Date:
Prev: Re: StreamDevice processing issue Dirk Zimoch via Tech-talk
Next: Are there EPICS users in Max Planck Institutes? Heinz Junkes via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020 
Navigate by Thread:
Prev: Re: StreamDevice processing issue Dirk Zimoch via Tech-talk
Next: Re: StreamDevice processing issue Davis, Mark via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020 
ANJ, 18 Mar 2019 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·