EPICS 3.15 mlockall()/dbThreadRealtimeLock high memory usage

[Lucas Russo via Tech-talk <tech-talk@aps.anl.gov>]

Hello everyone,

Recently, I noticed one of my IOCs consuming a lot more memory right after IOC startup (from linux "top" command, RES field, which I know it's not a good estimate sometimes, but still...) after upgrading it from EPICS base 3.14.12.6 to 3.15.6. All of the other EPICS modules was kept in the same version:

asyn: R4-35

areaDetector: R3-6

Even with other IOCs, areaDetector ADSimDetector, for instance, the memory consumption after IOC startup was way higher than before. For example:

EPICS 3.14 ADSimDetector: RES = 356502 MiB

EPICS 3.15 ADSimDetector: RES = 48613 MiB

After doing some debugging and memory profiling I saw that the ADSimDetector IOC running with EPICS 3.15 had the "VMLck" field from /proc/<pid>/status with:

VmLck: 4586544 kB

And the IOC running EPICS 3.14 had:

VmLck: 0  kB

Taking a look at the Release Notes there was a mention of "mlockall" being moved to some other place ("Moved mlockall() into its own epicsThread routine", "Changes made between 3.15.3 and 3.15.4")

I changed the "dbThreadRealtimeLock" variable from 1 (default) to 0 and the memory consumption dropped back to ~48613 MiB. Even after running the IOC for some time, doing some large acquisitions, the memory consumption was not close to the 356502 MiB.

Can anyone help me understand if this is somewhat expected?

And wouldn't it be better to have the variable dbThreadRealtimeLock with the default value of 0, instead of 1?

Cheers and Regards,

Lucas