Experimental Physics and
Currently EPICS base 3.14 only supports thread priorities on vxWorks, RTEMS, and WIN32. What David is doing is certainly of interest.
END OF SUMMARY
First a few comments about the existing implementation of threads and mutexes in <base3_14>/src/libCom/osi/os/posix.
The existing implementation works on solaris (at least 6,7,8,9), linux (at least several releases of redhat), and darwin.
The following statements appear:
/* Until these can be demonstrated to work leave them undefined*/ #undef _POSIX_THREAD_ATTR_STACKSIZE #undef _POSIX_THREAD_PRIORITY_SCHEDULING
The big one is _POSIX_THREAD_PRIORITY_SCHEDULING. This means that priority scheduling is NOT being used on solaris, linux, or darwin.
The following statements appear /* Until these can be demonstrated to work leave them undefined*/ /* On solaris 8 _POSIX_THREAD_PRIO_INHERIT fails*/ #undef _POSIX_THREAD_PROCESS_SHARED #undef _POSIX_THREAD_PRIO_INHERIT
The big one is _POSIX_THREAD_PRIO_INHERIT. But, of course, it will only be of use when _POSIX_THREAD_PRIORITY_SCHEDULING is supported.
Also osdThread has THREE!! implementations of a mutex:
1) uses spin locks. I am sending a separate message about this to Jeff and David because I am confused about the implications.
2) Uses pthread_mutex_xxx IF the pthreads implemention support recursive mutexes.
3) Uses a combination of a pthread_mutex and a pthread_cond if the pthreads implementation does not gurantee recursive mutexes.
It is certainly desirable to support at least _POSIX_THREAD_PRIORITY_SCHEDULING and _POSIX_THREAD_PRIO_INHERIT wherever possible. Who should we do this? I see two major problems: 1) Priority Scheduling, and 2) different platforms implement different subsets of pthreads.
Priority Scheduling, at least implementing real-time priorities, gets us into the problem of privileges. Some kind of privilege (often root) is required. How will an EPICS IOC do this? Running everything as root is NOT a solution.
Different Platforms: Not only different platforms but different releases. This is a MAJOR problem. I have seen some strange behavior on redhat 8 that went away with redhat 9. I had big problems getting things to work on both solaris 6 and solaris 8. If we turn on _POSIX_THREAD_PRIORITY_SCHEDULING for all platforms, I bet that most (All?) will fail.
What should we do?
For Priority Scheduling work must be done to develop a good environment. Eric Norum's message could be a start.
With regards to threads and mutexes I see two approaches:
1) Keep all pthreads code in src/libCom/osi/os/posix and use #ifdefs to take care of differences between platforms and releases of platforms.
2) Keep src/libCom/osi/os/posix but leave osdThread and osdMutex "dumbed down" just like they are now and provide separate implementations for platforms that are carefully tested.
I say do 2). I suspect that 1) will lead to a set of code that will be extremely difficult to understand and maintain.
Even if we do 2) the problems of releases still arises.
Now I have a question for David.
When the 3.6 Linux kernel is available won't it provide most of what redhawk linux provides?
Now I will answer some specifics:
David Kelly wrote:
From what I answered above, I agree with the idea of creating a new subdirectory libCom/osi/os/redhawk
Eric and I also mentioned the priviledge problem.
3) Compile the ioc base with all the changes and run the "epicsXXXXTest" programs.
I answered this above. I agree with your approach.
b) The posix/osdTime.cpp file appears to contain current references to "gmtime_r()" and "localtime_r()" functions, which I gather are specific to VxWorks. These do not appear to be resolved anywhere in a build under Linux (or RedHawk). Is this correct, or am I missing something ?
These are defined in the Single Unix Specification Version 3 (The successor to POSIX standards). See
getOssPriorityValue()c) I don't yet understand the way the thread priorities are calculated in posix/osdThread.c, when _POSIX_THREAD_PRIORITY_SCHEDULING is enabled. Can someone please explain to me the effect that the getOssPriorityValue() function is intended to have ?
Given a epicsThread priority return a OS specific priority for it.
epicsThread priorities are from 0 to 99 with a higher number meaning higher priority.
On vxWorks priorities are from 0 to 255 with a lower number meaning higher priority. For the vxWorks implemenation we decided to map epics priorities 0,...,99 to vxWorks priorities 199,...100.
d) Under RedHawk, if a process desires to run with a real-time priority (i.e. use a real-time scheduling policy of SCHED_RR or SCHED_FIFO, where the real-time priorities range from 99 down to 1 and are all therefore higher than the standard Linux SCHED_OTHER timesharing priority of zero), then the process must be owned by, or run in the context of, the root user. I could do this by logging on as root and MAKEing the code as root. However, I prefer to MAKE the project in the context of my normal user login, which means that I then need to "chown root XXXX" and "chmod a+s XXXX" all the created binary executables so that they run as the root user. I'd like to be able to automate the "chown" and "chmod" as part of the build (MAKE) process, but I haven't yet figured out how to modify the build environment to do this. Can anyone suggest how to do this, or at least where I should be looking ?
e) What other areas of the code should I be looking at, in order to improve the real-time performance (in a Linux-type environment) ?
Now for the "trees"-type question....
Perhaps Odagiri-San (email@example.com) could comment. He is the one who did the whitepaper.
Thanks for any and all advice.
|ANJ, 10 Aug 2010||
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·