Re: more setpoint oddities with pmac driver and CS axes

["Davis, Mark via Tech-talk" <tech-talk@aps.anl.gov>]

Hi Giles,

Thanks for the info.  The simple case of a two-drive slit is my first 
foray in to coordinated motion, so I don't pretend to understand how all 
this might affect other, more complex scenarios.

As for having the driver generate prog 10 itself, I expect that could 
make some things easier and others more difficult.  At the very least it 
would limit individual variations on its implementation to what the 
driver supported (e.g. my current version of this program uses the 
underlying motor speeds rather than setting the feedrate mode for the 
axes).

In any case, as I was thinking about your latest response and the 
problems with the unwanted changes in the setpoints, it occurred to me 
that there is an alternative to depending on the driver logic to behave 
as I want.

Both of my issues (unwanted changes to setpoints by the driver and the 
motor record reporting current setpoint values that do not match the 
actual in-use ones) would not occur if the readbacks matched the 
setpoints.  This means that if I implement the deadband logic in the PLC 
code on the controller, I can effectively override this unwanted and 
confusing behavior by having the controller report a readback value 
equal to the value it wants the setpoint to have.

The only side effect of doing it this way that I can think of is that 
the readback value in the control system will not include any changes in 
the actual readback whenever the set/read difference is less than the 
deadband threshold.  In comparison, if the deadband was handled in the 
driver, it could still report all changes in the readback but otherwise 
ignore them when the set/read difference is less than the deadband.

I just tried this and it appears to work quite well.  I still see all 
the changes in the readback while it is moving, and if the difference 
does exceed the threshold even after it has stopped, then the change 
shows up in the control system and the normal existing logic still has 
the intended effect (i.e. if the amplifier was disabled after the last 
move, the driver changes the corresponding setpoint to match the 
readback at the start of the next move, so it includes any error that 
exceeded the deadband).

Because of the one side effect (the suspicious stability of the readback 
between movements), it would still be preferable to have the driver 
manage the deadband logic, but implementing it in the controller appears 
to be sufficient for our needs.


SIDE NOTE:  Regarding the motivation for disabling the motors between 
movements:  I was recently told that, in addition to wanting to make 
sure there isn't even a TINY bit of jitter in the position of "idle" 
devices, it is necessary to avoid minor disturbances in the beam that 
can be caused by active closed-loop control of motors next to the beam 
line.  In cases where disabling the motors COULD result in the position 
drifting, we include brakes in the design.

Mark




Mark Davis NSCL/FRIB Control Systems Software Engineer davism50@msu.edu
On 6/5/2019 6:09 AM, giles.knap@diamond.ac.uk wrote:
> Hi Mark,
>
>
> I have spent some time reviewing options for makeCsConsistent behaviour in the pmac.
>
> What I would really like to do is modify the code so that it generates prog 10 and only does a move on the axes you have commanded. This would mean we could remove the makeCsConsistent code and much of the nasty complexity.
>
>
> However, this would be a major change and we would have to make some similar changes to our trajectory scan motion program. It is not immediately obvious how to do this (because this program is large and downloading it for each move is not viable).
>
> Worse, I am not convinced that this approach would resolve your specific issue. Because you are killing the motors between moves, they would be prone to drifting.
>
> I added the check for kill into makeCsConsistent for safety because a kill could mean the previous move did not complete and the next move could result in axes continuing on to the previous demand. This happened with dangerous effect on one of our beamlines. Prior to this safety feature, the code would have performed perfectly for you because all motors were always moved to there most recent demand and thus drift would not occur.
>
> The only solution that I can guarantee will work is applying a deadband rule to killed motor behaviour so if all real motors in the CS are within a tolerance of their demanded position then we leave the demands as they (instead of resetting them to the current readback which).
>
> I'm not particularly happy with the solution because it is adding complexity to an already over-complex function.
>
> Perhaps we need to discuss this directly. If you would like to give me a call in UK hours I am on +44 1235 77 8909.
>
> Regards,
> giles.
>
>
>
> ________________________________
> From: tech-talk-bounces@aps.anl.gov <tech-talk-bounces@aps.anl.gov> on behalf of giles.knap--- via Tech-talk <tech-talk@aps.anl.gov>
> Sent: 03 June 2019 08:28
> To: tech-talk@aps.anl.gov; Davis, Mark; Mercado, Ronaldo (DLSLtd,RAL,LSCI)
> Subject: Re: more setpoint oddities with pmac driver and CS axes
>
> Ronaldo,
>
>
> I'm copying you in on this because it is another issue that we find with using an intelligent motor record with an intelligent controller and it might be worth discussing at the EPICS Meeting motor TWG this week.
>
>
> Mark,
>
>
> My first thought as to the cause is that your motors are jittering a little after a move and perhaps especially when you kill them.
>
>
> The motor record will always copy the current readback into the current demand when the DMOV value goes from 0 to 1 (i.e. it perceives the motor has finished moving).
>
>
> In the pmac driver, we report DMOV up to the motor record based primarily on the InPos status for the axis.
>
>
> If your motors are jittering for some reason and it is enough for the controller to decide that InPos should be 0, then the motor record will briefly see the axis is moving and then copy its new position to the demand field.
>
>
> One possible solution is to simply increase the controller's in position band for these axes. This would be acceptable if the amount of jitter is smaller than the smallest move you need to make.
>
>
> -------------------------------------------------------------------------------------------
>
>
> I have been looking at your suggestion re changes to the 'makeCsConsistent' function. I don't think we can use a tolerance because it will affect the ability to switch between using real and virtual motors. However, I am investigating a different approach that modifies PROG10 depending on which axes you have commanded. If this works then it will no longer require 'makeCsConsistent' and will allow you to kill motors between moves without affecting demands.
>
>
> Regards,
>
> giles.
>
>
>
> ________________________________
> From: Davis, Mark <davism50@msu.edu>
> Sent: 31 May 2019 21:01:45
> To: tech-talk@aps.anl.gov; Knap, Giles (DLSLtd,RAL,LSCI)
> Subject: more setpoint oddities with pmac driver and CS axes
>
> Hi Giles,
>
> I have run into yet another odd behavior with coordinated motion using
> the pmac driver with a Power Brick controller.
>
> As you know, I already discovered that disabling the motors between move
> commands causes the driver to change the current setpoints for all the
> CS axes to the current reading just before triggering the next move.
>
> But now I have found that, even if I don't disable the motors, the
> setpoint values in the motor records still tend to change for a short
> time after I write a new value to the motor record for one of the CS axes.
>
> But this is different than that problem because the change is not real.
> What happens is that, I write a new value to the motor record for one of
> the CA axes, the motors move, and then frequently (but not always), the
> setpoint values (VAL, RVAL, etc) for one or both the CS axes motor
> records change by a little bit (sometimes multiple times).  But the Q7x
> values in the controller do not change, the motors stay put, and the
> setpoints for the motor records for the actual motors don't change.
>
> Example:
>
> The output of camonitor after I changed the setpoint for my SLHCEN motor
> record for the center axes (with the motor amplifiers enabled the whole
> time, before, during, and after the changes):
>
> SCR_BTS35_:SLHCEN_D1541.RVAL   2019-05-31 13:11:22.513686 50000
> SCR_BTS35_:SLHGAP_D1541.RVAL   2019-05-31 13:11:25.708162 30006
> SCR_BTS35_:SLHCEN_D1541.RVAL   2019-05-31 13:11:26.268574 49992
> SCR_BTS35_:SLHGAP_D1541.RVAL   2019-05-31 13:11:26.268644 30018
> SCR_BTS35_:SLHCEN_D1541.RVAL   2019-05-31 13:11:26.508426 49998
> SCR_BTS35_:SLHGAP_D1541.RVAL   2019-05-31 13:11:26.508506 30006
> SCR_BTS35_:SLHCEN_D1541.RVAL   2019-05-31 13:11:26.960236 49998
> SCR_BTS35_:SLHGAP_D1541.RVAL   2019-05-31 13:11:26.960282 30006
> SCR_BTS35_:SLHCEN_D1541.RVAL   2019-05-31 13:11:27.584892 49986
> SCR_BTS35_:SLHGAP_D1541.RVAL   2019-05-31 13:11:27.584968 30006
> SCR_BTS35_:SLHCEN_D1541.RVAL   2019-05-31 13:11:28.257009 49992
> SCR_BTS35_:SLHGAP_D1541.RVAL   2019-05-31 13:11:28.257094 30018
> SCR_BTS35_:SLHCEN_D1541.RVAL   2019-05-31 13:11:28.896481 49992
> SCR_BTS35_:SLHGAP_D1541.RVAL   2019-05-31 13:11:28.896567 30018
> SCR_BTS35_:SLHCEN_D1541.RVAL   2019-05-31 13:11:29.641618 49986
> SCR_BTS35_:SLHGAP_D1541.RVAL   2019-05-31 13:11:29.641702 30006
> SCR_BTS35_:SLHCEN_D1541.RVAL   2019-05-31 13:11:29.979076 49986
> SCR_BTS35_:SLHGAP_D1541.RVAL   2019-05-31 13:11:29.979135 30006
> SCR_BTS35_:SLHCEN_D1541.RVAL   2019-05-31 13:11:30.554313 49998
> SCR_BTS35_:SLHGAP_D1541.RVAL   2019-05-31 13:11:30.554364 30006
>
> The values of the Q7x and Q8x values on the controller after the values
> stopped changing show that the setpoint ones (Q7x) still have the values
> most recently written to the SLHCEN and SLHGAP CS axes records, even
> though the motor records show slightly different values:
>
> Q71=5
> Q72=3
> Q81=4.99921767845749798
> Q82=2.99932463437318786
>
> I have looked over the doco for the motor record multiple times and the
> only thing I found that seemed even possibly related was the LOCK value,
> which I tried setting to YES for both axes but that made no difference.
> So for the moment, I am assuming this is due to something in the pmac
> driver.
>
> Any ideas what could cause this?  That fact that the setpoint value for
> the motor record changes when the driver DOES change the setpoints after
> the motors were disabled at least meant that the operators would be able
> to tell when the actual in-use setpoint "creeped" a bit too far, and
> they could "fix" it by simply reasserting the original setpoint.  But at
> the moment the value being shown is often NOT the real in-use setpoint,
> so even that option is not a practical workaround (they REALLY want the
> motors disabled between moves).
>
> --------------------
> Mark Davis
> NSCL/FRIB
> Control Systems Software Engineer
> davism50@msu.edu
>
>
> --
> This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
> Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
> Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
>
>