Experimental Physics and
Industrial Control System

Gerry Swislow <[email protected]> · Thu, 28 Jan 2016 09:46:58 -0500

Hi Mark,

You're right.  That mlockall() call is exactly the problem.  I found and confirmed that even later last night after my submission.  I searched the tech-talk archives and found that the code was added to support certain users of real-time systems:

  https://code.launchpad.net/~mshankar/epics-base/memlock_linux/+merge/232466
  http://www.aps.anl.gov/epics/tech-talk/2015/msg01822.php

Unfortunately, the test for whether one is running on a realtime system:

   if (pcommonAttr->maxPriority > pcommonAttr->minPriority) 

is not robust, since it appears to be also true when running a set-user-id-root process on a non-realtime system.  That is a big problem for spec users that want to use EPICS and controller cards in their PCs at the same time.  (I got the spec + EPICS 3.15 bug report from a user at Bessy yesterday.)

I hope that the EPICS developers can rethink the implementation of that feature, either by coming up with a more robust run-time test, an environment option, a function call to turn on memory locking, or a build-time option.  Also, I think the default should be for the feature to be turned off, as that would be appropriate for the more casual users.  If one has a real-time system, one might be a more technically minded user who is comfortable in tuning the EPICS build or run-time environment.

Thanks for the reply.

- Gerry

Begin forwarded message:

> From: Mark Rivers <[email protected]>
> Subject: RE: set-user-id root and EPICS 3.15
> Date: January 28, 2016 at 9:24:31 AM EST
> To: Gerry Swislow <[email protected]>, "[email protected]" <[email protected]>
> 
> Hi Gerry,
> 
> I don't know what the problem is, but my first guess is to look at the difference between the posix code in EPICS 3.14 and 3.15.  In particular osdProcess.c and osdThread.c.
> 
> diff -U3 /usr/local/epics/base-3.14.12.5/src/libCom/osi/os/posix/osdThread.c /usr/local/epics/base-3.15.2/src/libCom/osi/os/posix/
> 
> Here is a difference that looks looks like it could be relevant, for example:
> 
> +
> +#if defined(_POSIX_MEMLOCK) && _POSIX_MEMLOCK > 0
> +    if(errVerbose)  { 
> +        fprintf(stderr, "LRT: min priority: %d max priority %d\n", 
> +            pcommonAttr->minPriority, pcommonAttr->maxPriority);
> +    }
> +    if (pcommonAttr->maxPriority > pcommonAttr->minPriority) {
> +        status = mlockall(MCL_CURRENT | MCL_FUTURE);
> +        if(status) { 
> +            fprintf(stderr, "Unable to lock the virtual address space using mlockall\n");
> +        } else { 
> +            fprintf(stderr,"Successfully locked memory using mlockAll\n");
> +        }
> +    }
> +#endif
> +
> 
> Mark
> 
> 
> ________________________________________
> From: [email protected] [[email protected]] on behalf of Gerry Swislow [[email protected]]
> Sent: Wednesday, January 27, 2016 8:52 PM
> To: [email protected]
> Subject: set-user-id root and EPICS 3.15
> 
> Hi Tech-Talkers,
> 
> I'm having a problem with EPICS 3.15 in the "spec" set-user-id-root executable.
> 
> Why do I need a set-user-id-root binary?  Many sites use spec on Linux platforms with PC board motor controllers, counter/timers, GPIB interfaces, etc.  For many years now, spec has eschewed kernel drivers in favor of user-level support.  The user-level support works on all Linux distributions over all iterations of the Linux kernel with no effort needed by the users. (Try that with kernel drivers!)
> 
> User level support requires access to I/O ports or on-board memory, which requires super-user privilege.  Thus, in most installations, spec is installed as a set-user-id root program.  The privilege escalation is immediately reset to that of the real user first thing when spec starts, and is only reactivated for the needed system calls, along these lines:
> 
>        if (setreuid(-1, 0) == 0) {
>                iopl(3);
>                setreuid(-1, getuid());
>        }
> 
> There is no issue with users having root access while running spec with respect to permissions to do anything else that would not normally be permitted.
> 
> The problem is that in such an environment, certain library or system calls fail in a strange way, but only when spec is linked with EPICS 3.15.  There is no issue with EPICS 3.14.  When run as a normal user, the following set-user-id-root test program fails with the messages:
> 
> Successfully locked memory using mlockAll
> pthread_create error Resource temporarily unavailable
> 
> Of course, if run as root, there is no error.  Using the command "sudo strace -u gerry ./a.out", I see that the error is associated with a mmap() call:
> 
> mmap(NULL, 2101248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = -1 EAGAIN (Resource temporarily unavailable)
> 
> In spec, the errors occur with shmat(), getpwuid(), which uses mmap(), and malloc().  spec fails for good when malloc() fails.
> 
> I should also point out that the EPICS call isn't even needed.  Just the act of linking with the EPICS 3.15 libraries breaks the code.
> 
> To be specific, I used EPICS 3.15.3 and EPICS 3.14.12.  I tested on CentOS 7.2 and Linux Mint 17.2, both 64 bits.  And I've spent all day trying to figure this one out.
> 
> Test code and a build script follow.  Does anybody have a clue?
> 
> Thanks,
> 
> Gerry
> 
> -------------------------------------
> /* file foo.c */#include <cadef.h>
> 
> int
> main() {
>        char    *p;
> 
>        setreuid(-1, getuid());
>        ca_context_create(ca_disable_preemptive_callback);
> }
> 
> -------------------------------------
> #!/bin/bash
> 
> BASE=/usr/local/epics/base-3.14.12
> BASE=/usr/local/epics/base-3.15.3
> 
> cc -I$BASE/include/os/Linux -I$BASE/include  -I$BASE/include/compiler/gcc \
>   foo.c -L$BASE/lib/linux-x86_64 -lca -lCom
> 
> sudo chown root a.out
> sudo chmod u+s a.out
> ./a.out
> 
> -------------------------------------
> 
> 
> -------------------------------------
>    Gerry Swislow
>    Certified Scientific Software
>    PO Box 390640
>    Cambridge, MA  02139-0007
> 
>  phone:  (617) 576-1610
>     fax:  (617) 497-4242
>    email:  [email protected]
>       web:  http://www.certif.com
> 
> 

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System