Experimental Physics and
Industrial Control System



Hi All.

It seems my original answer did never make it to tech-talk. It can still be found
below but here are some more data I gathered:

Even with the suggested work-arounds I could not get labca-3.7.2 and
epics-7.0.4.1 or 7.05 to work with matlab 2020b. It would either hang or crash when
quitting matlab.

I cut a new [labca 3.8.0 release](https://github.com/till-s/epics-labca/releases/tag/labca_3_8_0)

which addresses this problem but you still need one (no need for both) of the following
two work-arounds:

a) use a build of epics base with posix priority scheduling disabled. In configure/CONFIG_SITE set
       USE_POSIX_THREAD_PRIORITY_SCHEDULING=NO
    then 'make clean' and 'make'. Obviously, make sure no real-time systems are using this new build.
b) use LD_PRELOAD to load and initialize libCom before starting matlab

    LD_PRELOAD=<path_to_base>/lib/<arch>/libCom.so  matlab <options>

If someone has good connections to MathWorks (or a MW engineer is reading this)
they could use the attached simple (and standalone) 'mex' file to reproduce the problem
(w/o any epics).

Cheers
- Till


On 3/1/21 5:56 PM, Miroslaw Dach wrote:

Hi Till,

Thank you very much for the in depth study of the problem. It looks like Mathworks has changed something in the code and even worse - they have introduced an "unwanted feature" which affects the Matlan2020b and LabCa users on RHEL7.

We will try one of your suggestions and let you know how things are.

Many Thanks

Mirek

On Mon, Mar 1, 2021 at 2:01 AM Till Straumann <till.straumann at psi.ch> wrote:

Hi Mirek.

I have investigated this deadlock and come to the conclusion that it is a problem
with matlab, probably under RHEL7 (I have no way to test under other systemes, in particular: windows,
though).

When I debugged the deadlock I found that some matlab threads deadlock in a library called

glibc-2.17_shim.so

This (mathworks proprietary) library is LD_PRELOADed from the 'matlab' driver script where we find a comment:

    # Preload glibc_shim in case of RHLE7 variants
    test -e /usr/bin/ldd &&  ldd --version |  grep -q "(GNU libc) 2\.17"  \
            && export LD_PRELOAD="$LD_PRELOAD:$MATLAB/bin/glnxa64/glibc-2.17_shim.so" \
            && export MW_GLIBC_SHIM="$MATLAB/bin/glnxa64/glibc-2.17_shim.so"

which leads to the hypothesis that RHEL7 only may be affected.

The deadlock happens when matlab
 - loads a shared object (or library)
 - AND the shared object executes some initialization code (e.g., constructors of global objects defined in the library)
 - AND the initialization code calls 'pthread_join()'. 'pthread_join()' then never returns.

Note that if 'ordinary' code in the shared object (i.e., as opposed to initialization code) uses 'pthread_join()' then
that works fine.

A simple example mex file (attached) which is not using labca or epics and reproduces the described behaviour.

EPICS' libCom does use 'pthread_join()' during initialization and is therefore affected.

At this point I can suggest two possible work-arounds (using one of them is sufficient):

1.) Use an EPICS-base build with posix priority scheduling disabled. This avoids a section of initialization
     code which calls 'pthread_join()'

     E.g., in configure/CONFIG_SITE:

    USE_POSIX_THREAD_PRIORITY_SCHEDULING = NO

2.) LD_PRELOAD EPICS' libCom.so *before* starting matlab

    LD_PRELOAD=<path_to_my_epics_lib>/libCom.so  matlab

HTH
- Till



On 2/19/21 4:20 AM, Miroslaw Dach wrote:

Hi Till,

We have crossed each other. You came to PSI from the US and I did the opposite. I moved to work in LBL.

Are you still maintaining the LabCa?

We are facing a problem with Matlab 2020b crashes when using labCa 3.7.2.

It looks like the incompatibility between the Matlab 2020b and labCa latest official version.

The labCa 3.7.2 seems to be the latest version unless you have the newer one?

Best Regards

Mirek

<mexJoin.cc>