Tong,
I think about this a lot when working with our ML models -- and the best I can some up with are guidelines similar to those in any safety-critical design:
- Don't let your controller even have the ability to output something that might damage anything
I am not a fan of relying on access controls as any sort of primary safeguard, as those are outside the purview of the controller itself. If a controller has a capability to damage something (PPS or MPS), then it feels like that is just a huge risk. Seeing PID loops go wrong in RF really highlights that. Now on the flip side, I love access controls for mitigating possible configuration errors -- and having something pop up if you write the wrong PV by mistake is critical. But where that is controlled and who configures that is an interesting question -- I'd rather a pva/ca proxy running on the same machine as the controller and build the access controls right into it.
I'd love to hear other people's thoughts -- this would be a great topic at a workshop.
Josh EC
On Tue, Aug 29, 2023 at 9:52 AM Zhang, Tong via Tech-talk <tech-talk at aps.anl.gov <mailto:tech-talk at aps.anl.gov>> wrote:
Dear Colleguages,____
__ __
Machine learning applications in accelerator controls are indeed gaining popularity, and there are exciting developments in progress. However, concerns persist regarding equipment protection, particularly when dealing with black-box ML models that may make risky decisions, especially during optimization iterations.____
__ __
When it comes to ML model generation, utilizing archived data is a viable approach. However, during the application phase, these models may still generate audacious decisions. Even when trained with live data, the risk remains.____
__ __
As far as I know, leveraging Channel Access security configuration is a sound strategy to manage PV write permissions at a granular level, covering individuals, groups, and workstations. This level of control ensures that the ML code's write permissions can be finely tuned. I’m still wondering is this way totally secure?____
__ __
Absolutely, incorporating the machine protection system as the primary safeguard on the device side is crucial. Your valuable insights/experience on this subject are greatly appreciated.____
__ __
Thanks,____
Tong____
__ __
--____
Tong Zhang, Ph.D. (he/him)____
Controls Physicist____
Facility for Rare Isotope Beams,____
Michigan State University____
__ __