Re: epicsThreadSetPriority called by non epics thread

[Andrew Johnson <anj@aps.anl.gov>]

Jeff Hill wrote:
> One has to configure the build setting "USE_POSIX_THREAD_PRIORITY_SCHEDULING
> = YES" on Linux in order to see these messages occurring whenever a client
> connects.

Interesting - with that setting I can replicate this on our 2-processor 
linux server (saturn) but not on my 1-processor linux workstation 
(apsajnt).  I noticed that Jeff was running on an SMP kernel, and I 
assume his machine has more than one CPU:

> Linux version 2.4.21-20.ELsmp (bhcompile@tweety.build.redhat.com) (gcc
> version 3.2.3 20030502 (Red Hat Linux 3.2.3-42)) #1 SMP Wed Aug 18 20:46:40
> EDT 2004

My boxes look like this:

saturn% cat /proc/version
Linux version 2.4.20-28.8smp (bhcompile@daffy.perf.redhat.com) (gcc 
version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)) #1 SMP Thu Dec 18 
12:25:21 EST 2003

apsajnt% cat /proc/version
Linux version 2.4.20-19.9 (bhcompile@stripples.devel.redhat.com) (gcc 
version 3.2.2 20030222 (Red Hat Linux 3.2.2-5)) #1 Tue Jul 15 17:18:13 
EDT 2003


Looking at the code in libCom/osi/os/posix/osdThread.c I can now see the 
problem and explain why it's only happening on SMP systems.

This code is from epicsThreadCreate():

    pthreadInfo = init_threadInfo(name,priority,stackSize,funptr,parm);
    if(pthreadInfo==0) return 0;
    pthreadInfo->isEpicsThread = 1;
    setSchedulingPolicy(pthreadInfo,SCHED_FIFO);
    status = pthread_create(&pthreadInfo->tid,&pthreadInfo->attr,
                start_routine,pthreadInfo);
    if(status==EPERM){
        pthreadInfo = init_threadInfo(name,priority,stackSize,funptr, 
parm);
        if(pthreadInfo==0) return 0;
        status = pthread_create(&pthreadInfo->tid,&pthreadInfo->attr,
                start_routine,pthreadInfo);
    }

The code creates a threadInfo object and calls setSchedulingPolicy(). If 
the subsequent pthread_create() call fails with an EPERM, it tries again 
without the setSchedulingPolicy().  There are two problems with this: a 
memory leak, since it never frees the original threadInfo object, and it 
doesn't set the isEpicsThread flag on the retry.  My conjecture is that 
on an SMP linux kernel pthread_create() does actually return EPERM if 
the user isn't root, whereas on my single processor (UP) kernel it just 
accepts (and probably ignores) the thread priority request without 
error.  I have tried running the IOC as root on my workstation, and 
epicsThreadShowAll lists all the threads with a zero for OSSPRI so it 
seems that the UP kernel completely ignores the thread priority. 
Unfortunately I don't have the ability to run the IOC as root on the SMP 
system, so I can't confirm my guess that way.

I have committed a fix to the second problem (forgetting to set the 
isEpicsThread flag) which removes the messages Jeff was seeing, but the 
memory leak is harder to fix since the free_threadInfo() routine tries 
to remove the threadInfo object from a list which it hasn't been added 
to yet.  From the CVS history I see that Marty has already reworked this 
section of code a bit; I'm going to leave it for him to decide how to 
fix (there's a FIXME: comment at the relevent line).

- Andrew
--
* * Matt Santos / / Leo McGarry * * For a Brighter America * *