Re: MAXnet Alarm State Under High Motor Command Load

[Jiajun Tian via Tech-talk <tech-talk at aps.anl.gov>]

Hi Kevin

No MAXnet communication errors on the IOC shell. 

When the error occurred, "Controller Error" was displayed on the MEDM ( from .STAT PV)

Here is the full Bluesky error message:
[I 2026-01-27 20:51:15,009 bluesky_queueserver.manager.plan_monitoring] Run was closed: 'b6ba8da7-91a8-4b70-b7b2-7175759a9c57'

ERROR:bluesky:Run aborted

ophyd.utils.errors.UnknownStatusFailure: The status MoveStatus(done=False, pos=sy, elapsed=15.9, success=False, settle_time=0.0) has failed. To obtain more specific, helpful errors in the future, update the Device to use set_exception(...) instead of _finished(success=False).

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/home/chem_epics/anaconda3/envs/pydm/lib/python3.10/site-packages/bluesky/run_engine.py", line 1609, in _run

    msg = self._plan_stack[-1].send(resp)

  File "/home/chem_epics/anaconda3/envs/pydm/lib/python3.10/site-packages/bluesky/preprocessors.py", line 413, in subs_wrapper

    return (yield from finalize_wrapper(_inner_plan(), _unsubscribe()))

  File "/home/chem_epics/anaconda3/envs/pydm/lib/python3.10/site-packages/bluesky/preprocessors.py", line 536, in finalize_wrapper

    ret = yield from plan

  File "/home/chem_epics/anaconda3/envs/pydm/lib/python3.10/site-packages/bluesky/preprocessors.py", line 411, in _inner_plan

    return (yield from plan)

  File "<string>", line 201, in uvvis_channel_scan

  File "<string>", line 166, in inner_plan

  File "<string>", line 131, in uvvis_csv_scan

  File "/home/chem_epics/anaconda3/envs/pydm/lib/python3.10/site-packages/bluesky/utils/__init__.py", line 1258, in dec_inner

    return (yield from plan)

  File "/home/chem_epics/anaconda3/envs/pydm/lib/python3.10/site-packages/bluesky/preprocessors.py", line 358, in run_wrapper

    yield from contingency_wrapper(plan, except_plan=except_plan, else_plan=close_run)

  File "/home/chem_epics/anaconda3/envs/pydm/lib/python3.10/site-packages/bluesky/preprocessors.py", line 604, in contingency_wrapper

    ret = yield from plan

  File "<string>", line 102, in inner_plan

  File "/home/chem_epics/anaconda3/envs/pydm/lib/python3.10/site-packages/bluesky/utils/__init__.py", line 1952, in __iter__

    return (yield from self._iter)

  File "/home/chem_epics/anaconda3/envs/pydm/lib/python3.10/site-packages/bluesky/plan_stubs.py", line 392, in mv

    yield Msg("wait", None, group=group)

  File "/home/chem_epics/anaconda3/envs/pydm/lib/python3.10/site-packages/bluesky/run_engine.py", line 2399, in _status_object_completed

    raise FailedStatus(ret) from exc

bluesky.utils.FailedStatus: MoveStatus(done=True, pos=sy, elapsed=15.9, success=False, settle_time=0.0)

[E 2026-01-27 20:51:15,010 bluesky_queueserver.manager.worker] The plan failed: Plan failed: MoveStatus(done=True, pos=sy, elapsed=15.9, success=False, settle_time=0.0)

ERROR:ophyd.objects:Motion failed: sx is in an alarm state status=AlarmStatus.STATE severity=AlarmSeverity.MAJOR

Jiajun

Jiajun Tian, Ph.D.

Postdoctoral Researcher

NSF’s ChemMatCARS, Pritzker School of Molecular Engineering

University of Chicago

9700 S Cass Ave Bldg 434D

Lemont, IL-60439

United States

---

From: Peterson, Kevin M. <kmpeters at anl.gov>
Sent: Wednesday, January 28, 2026 2:01 PM
To: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>; Jiajun Tian <jiajtian at uchicago.edu>
Subject: Re: MAXnet Alarm State Under High Motor Command Load

 

Jiajun,

Are there any MAXnet communication errors on the IOC's shell when the bluesky/ophyd errors occur?

Kevin


________________________________________
From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Jiajun Tian via Tech-talk <tech-talk at aps.anl.gov>
Sent: Wednesday, January 28, 2026 10:32 AM
To: tech-talk at aps.anl.gov
Subject: MAXnet Alarm State Under High Motor Command Load

Hi All,

We are using an OMS MAXnet motor controller via an EPICS IOC, with motion driven by Bluesky plans (Ophyd). Under a high rate of consecutive motor move commands, the motor PVs enter an EPICS alarm state after several moves. When this happens, Ophyd reports a failed MoveStatus, and the Bluesky RunEngine aborts.

The MAXnet is primarily connected via Ethernet. We have also observed similar behavior when using a serial connection, where the issue occurs more frequently.

Has anyone experienced similar MAXnet issues with EPICS, or can advise on relevant settings or configuration considerations?

Related error observed in Bluesky/Ophyd:
ERROR:ophyd.objects: Motion failed: sx is in an alarm state status=AlarmStatus.STATE severity=AlarmSeverity.MAJOR

Thanks in advance,
Jiajun




Jiajun Tian, Ph.D.

Postdoctoral Researcher
NSF’s ChemMatCARS, Pritzker School of Molecular Engineering
University of Chicago
9700 S Cass Ave Bldg 434D
Lemont, IL-60439
United States