Experimental Physics and Industrial Control System
Hi Jens,
Thanks for looking at this.
> Currently timeout is set to: timeout = epicsThreadSleepQuantum() + 0.001;
>
> Here this sums up to 21 msecs (epicsThreadSleepQuantum() = 20 msecs) and I
> never got a timeout here.
In our case (mvme 6100 and RTEMS) epicsThreadSleepQuantum() = 0.001, so we were only waiting 2 ms before timeout. I have increased it to 21ms.
> Could you send me your modifications?
Here is the patch - it does a wait after clearing the message semaphore and then rechecks if any more messages have been sent. This also has the hardcoded timeout of 0.021s.
--- motorApp/OmsAsynSrc/omsMAXv.cpp 2014-01-20 04:58:11.000000002 -0800
+++ motorApp/OmsAsynSrc/omsMAXv.cpp 2014-03-11 08:24:22.000000002 -0700
@@ -296,7 +296,8 @@
}
/* see if junk at input port - should not be any data available */
- if ((epicsUInt16) pmotor->inGetIndex != (epicsUInt16) pmotor->inPutIndex)
+ int flushTime = 0;
+ while (((epicsUInt16) pmotor->inGetIndex != (epicsUInt16) pmotor->inPutIndex) && (flushTime < 5))
{
// flush cards response Buffer
#ifdef DEBUG
@@ -311,7 +312,8 @@
flag1.All = pmotor->status1_flag.All;
pmotor->status1_flag.All = flag1.All;
pmotor->msg_semaphore=0;
-
+ epicsThreadSleep(epicsThreadSleepQuantum());
+ flushTime++;
}
putIndex = (epicsUInt16) pmotor->outPutIndex;
@@ -383,7 +385,7 @@
itera = 0;
double time = 0.0;
- double timeout = epicsThreadSleepQuantum() + 0.001;
+ double timeout = 0.021;
while ((pmotor->status1_flag.Bits.text_response == 0) && (time < timeout)){
Debug(32, "%s:%s:%s: Waiting for reponse, itera:%d\n",
driverName, functionName, portName, itera);
________________________________________
From: Jens Eden [[email protected]]
Sent: Monday, March 10, 2014 8:04 AM
To: D'Ewart, J. Mitch
Cc: [email protected]
Subject: Re: MAXv model-3 - error reading axis position after 5 attempts
Mitch,
> It looks like the timeout may not be long enough so our messages are getting out of sync.
Do you get timeout error messages, if setting "motorOMSBASEdebug = 1"?
> 1. increase the timeout time ...
Currently timeout is set to: timeout = epicsThreadSleepQuantum() + 0.001;
Here this sums up to 21 msecs (epicsThreadSleepQuantum() = 20 msecs) and I
never got a timeout here.
Since epicsThreadSleepQuantum may vary, it shouldn't be used to set the timeout.
Could you try to set timeout to 21 msecs (timeout = 0.021) and see if that
is sufficient?
2. multiple responses in sendOnly().
I did this quicj test to reproduce this:
1. send "AY LT?", read response, send "WY" read response
caput OMSMv:io3401omsSendRecv "AY LT?" ; caput OMSMv:io3401omsSendRecv "WY"
omsMAXv::send: sending: AY LT?
omsMAXv::sendReceive: received =l
omsMAXv::send: sending: WY
omsMAXv::sendReceive: received MAXv-8000 ver:1.45 ...
2. send "AY LT?", don't read response, send "WY" read response
caput OMSMv:io3401omsSend "AY LT?" ; caput OMSMv:io3401omsSendRecv "WY"
omsMAXv::send: sending: AY LT?
omsMAXv::send: sending: WY
omsMAXvAsyn:sendOnly:MAXv1: flushing 3 characters <= flush unread response
omsMAXv::sendReceive: received MAXv-8000 ver:1.45 ...
3. send "AY LT? AX LT?", don't read 2 responses, send "WY" read response
caput OMSMv:io3401omsSend "AY LT? AX LT?" ; caput OMSMv:io3401omsSendRecv "WY"
omsMAXv::send: sending: AY LT? AX LT?
omsMAXv::send: sending: WY
omsMAXvAsyn:sendOnly:MAXv1: flushing 3 characters <= flush first unread response
omsMAXv::sendReceive: received =l <= 2nd unread response, should be firmware string
4. send more than 2 requests: MAXv remains unresponsive until power cycle:
caput OMSMv:io3401omsSend "AY LT?;AX LT?;AZ LT?;" ; caput OMSMv:io3401omsSendRecv "WY"
> 2) Mitch, changed the function sendOnly() to flush multiple responses.
Could you send me your modifications?
Thanks
Jens
Am 07.03.2014 18:57, schrieb D'Ewart, J. Mitch:
> The problem we are seeing
>
> omsBaseDriver:getAxesArray:MAXv-01: array string conversion error, count: 0, axes: 8, input: >MNNN,PNNN,MNNN,PNLN,PNLN,PNLN,PNLN,PNLN<^M
>
> This seems to be a request and response not matching up. In this case the response to getAxesArray() should be numerical, for example
>
> "AM RP?"
> responds with
> "0,0,0,0,0,0,0"
> but is getting
> "MNNN,PNNN,MNNN,PNLN,PNLN,PNLN,PNLN,PNLN"
>
> In this case sendOnly() is called from sendReceive(). sendOnly() flushes the response queue
>
> /* see if junk at input port - should not be any data available */
> if ((epicsUInt16) pmotor->inGetIndex != (epicsUInt16) pmotor->inPutIndex)
> {
> // flush cards response Buffer
> #ifdef DEBUG
> int deltaIndex = ((epicsUInt16)pmotor->inPutIndex) - ((epicsUInt16)pmotor->inGetIndex);
> Debug(32, "%s:%s:%s: flushing %d characters\n",
> driverName, functionName, portName, (((deltaIndex < 0) ? BUFFER_SIZE +
> deltaIndex : deltaIndex)));
> #endif
> putIndex = (epicsUInt16) pmotor->inPutIndex;
> pmotor->inGetIndex = putIndex;
> pmotor->status1_flag.Bits.text_response = 0;
> flag1.All = pmotor->status1_flag.All;
> pmotor->status1_flag.All = flag1.All;
> pmotor->msg_semaphore=0;
>
> }
>
> and then sends the new request. sendReceive() then waits for a response flag to be set or timeout.
>
> Debug(64, "omsMAXv::sendReceive: receiving\n");
> itera = 0;
> double time = 0.0;
> double timeout = epicsThreadSleepQuantum() + 0.001;
> while ((pmotor->status1_flag.Bits.text_response == 0) && (time < timeout)){
> Debug(32, "%s:%s:%s: Waiting for reponse, itera:%d\n",
> driverName, functionName, portName, itera);
> // busy-waiting but not more than 2000 times
> if (itera > 2000){
> time += epicsThreadSleepQuantum();
> epicsThreadSleep(epicsThreadSleepQuantum());
> }
> itera++;
> }
>
> It looks like the timeout may not be long enough so our messages are getting out of sync. The two proposed solutions are to 1. increase the timeout time and 2. flush multiple responses in sendOnly().
>
> Mitch D'Ewart
>
>
>
> ________________________________________
> From: [email protected] [[email protected]] On Behalf Of Mark Rivers [[email protected]]
> Sent: Thursday, March 06, 2014 3:33 PM
> To: Luchini, Kristi L.; [email protected]
> Subject: RE: MAXv model-3 - error reading axis position after 5 attempts
>
> When sendOnly() is called does the MaxV actually send a response to that message? If so then sendOnly should wait for it, even if it will discard the response. If it does not then the next time a thread does a write/read it cannot reliably know if a response it got was from its query or a previous sendOnly response.
>
> Mark
>
>
> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On Behalf Of Luchini, Kristi L.
> Sent: Thursday, March 06, 2014 5:29 PM
> To: [email protected]
> Subject: MAXv model-3 - error reading axis position after 5 attempts
>
> Hello,
>
> Has anyone seen this error on the new model-3 version of the MAXv driver (omsAsyn) ?
>
> 8544 omsBaseDriver:BasePoller:MAXv-01: error reading axis position after 5 attempts^M
> 8545 omsBaseDriver:BasePoller:MAXv-01: error reading axis position after 5 attempts^M
> 8546 omsBaseDriver:getAxesArray:MAXv-01: array string conversion error, count: 0, axes: 8, input: >MNNN,PNNN,MNNN,PNLN,PNLN,PNLN,PNLN,PNLN<^M
> 8547 2014/02/05 12:03:30.540 omsBaseDriver:readInt32:MAXv-01 wrong AUX state N^M
> 8548 2014/02/05 12:03:30.670omsBaseDriver:getAxesArray:MAXv-01: array string conversion error, count: 0, axes: 8, input: >MNNN,PNNN,MNNN,PNLN,PNLN,PNLN,PNLN,PNLN<^M
>
> We have seen this error in a few motion control applications with MAXv controllers. All of the IOCs were
> running motor-R6-8 asyn MAXv driver. The problem seems to occur when we're using a number of axis, and in
> some cases using virtual motors where multiple axis are moved at once.
>
> 1) MAXv controller, with 2 steppers and 1 servo motors. This MAXv has FW 1.45
> 2) MAXv controller , with 8 steppers. This MAXv has FW 1.44 (vitual motors moving 4-axis at once, and 2-axis at once using transform record)
> 3) MAXv controller, with 7 steppers. This MAXv has FW 1.31 (virtual motors, moves 4 axis at once, and 2-axis using transform record)
>
>
> Mitch D'Ewart found that "The omsMAXv::sendOnly function checks the input buffer and flushes one response if there is anything there but we seem to have come on a case where there are multiple responses backed up. These responses are controlled by a message semaphore. It seems that once sendOnly flushes the input buffer and clears the semaphore then the MAXv sends another backed up message response. Finally the receive part of the calling function gets the wrong message.
>
> There are two solutions that have been implemented recently at SLAC. Both will be booted on Monday 3/10/2014, in production.
> So we don't know if the problem is fixed yet.
> 1) Ziga added eieio in omsMAXv.cpp by including basicIoOps.h from Till Strauman.
>
> Example:
> static void writeReg8(volatile epicsUInt8 *a8, epicsUInt8 value)
> {
> #ifdef HAS_IOOPS_H
> out_8(a8, value);
> #else
> *a8 = value;
> #endif
> }
>
> 2) Mitch, changed the function sendOnly() to flush multiple responses.
>
> Details:
> Motor-R6-8
> Asyn-R4-12
> EPICS R3.14.12
> RTEMS-4.9.4
> MVME6100 CPU.
> MAXV-8000
>
> QUESTION:
> How shold asyn support implementation respond in this sort of case, where the response messages get out of sync with the queries?
> Do you simply clear you the response buffer, buffer and start again, or is there another solution? Has anyone else implemented a fix to this
> Particular MAxv but or to another asyn support driver?
>
> Regards,
> Kristi
>
>
>
>
>
- Replies:
- RE: MAXv model-3 - error reading axis position after 5 attempts Jens . Eden
- References:
- MAXv model-3 - error reading axis position after 5 attempts Luchini, Kristi L.
- RE: MAXv model-3 - error reading axis position after 5 attempts Mark Rivers
- RE: MAXv model-3 - error reading axis position after 5 attempts D'Ewart, J. Mitch
- Re: MAXv model-3 - error reading axis position after 5 attempts Jens Eden
- Navigate by Date:
- Prev:
PCaPAC Abstract Deadline Elder Matias
- Next:
RE: MAXv model-3 driver - omsMAXvConfig problem D'Ewart, J. Mitch
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
<2014>
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: MAXv model-3 - error reading axis position after 5 attempts Jens Eden
- Next:
RE: MAXv model-3 - error reading axis position after 5 attempts Jens . Eden
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
<2014>
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024