RE: Scan overflow issue

["Zhang, Tong via Tech-talk" <tech-talk at aps.anl.gov>]

Hi Ralph,

It works great, thanks for the help!

 

The issue was gone after I set it as 5000, though I don’t know what is the optimal setting.

The version of the deployed EPICS base is 3.15.9. Does it support checking the queue size through some command in the IOC shell? And How?

 

Thanks,

Tong

 

From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Ralph Lange via Tech-talk
Sent: Thursday, August 11, 2022 09:54
To: EPICS Tech Talk <tech-talk at aps.anl.gov>
Subject: Re: Scan overflow issue

 

[EXTERNAL] This email originated from outside of FRIB

On Thu, 11 Aug 2022 at 15:39, Zhang, Tong via Tech-talk <tech-talk at aps.anl.gov> wrote:

One of my IOC hosts ~20k records, when restarting it, sometimes it may throw "scanOnce: Ring buffer overflow" messages, which leads to some incorrect statistical calculations.

I guess this might come from:

1. autosave, which will trig a bunch of crowded restoring, is that possible to slow down the process?
2. It's too many records in one IOC, if so, what would you recommend? Dose splitting it into multiple ones help?

Please correct me if my understanding is wrong, and I appreciate any sharing on resolving this issue.

 

My guess: as all records that have PINI=YES are put in the internal queue for records that need to be processed once when the IOC boots, the default queue size of 1000 is too small for your IOC.

Use the command scanOnceSetQueueSize from your startup script (needs to be done before calling iocInit) to set the queue size to a larger value. (See "SetQueueSize" in the AppDevGuide.)

Recent versions of EPICS also have a high water mark mechanism for the internal queues. Looking at those values (from the iocShell or via iocStats) can give you an idea of how large the queues need to be.

 

Splitting the IOC would also help, obviously. But 20k is probably fine. We have been running IOCs with >100k records without issues. (But the queue sizes are certainly increased accordingly.)

The number is only one of the parameters... what kind of records, how much driver code is executing, how much IO is being done, how many calculations... the load depends on the combination.

 

Cheers,
~Ralph