Deadlock when using asyn with StreamDevice. (Was: Deadlock when using asyn?)

["Rees, NP \(Nick\)" <nick.rees@diamond.ac.uk>]

Eric,

The short answer is I don't know. That is in StreamDevice, and the
reason why I sent out the email is that I thought I had simplified the
system down to the stage where I had eliminated StreamDevice.  However,
in looking at it I ended up using the wrong startup script and
StreamDevice activity was still included.

Now I see this the problem seems to be that streamDevice doesn't seem to
work well with tasks using asynOctetSyncIO on the same asyn port. I am
using StreamDevice-2-1-patch20060717 off the SLS website.
intrCallbackOctet looks like:

void intrCallbackOctet(void* /*pvt*/, asynUser *pasynUser,
    char *data, size_t numchars, int /*eomReason*/)
{
    int wasQueued;
    AsynDriverInterface* interface =
        static_cast<AsynDriverInterface*>(pasynUser->userPvt);
    if (interface->ioAction == AsyncRead ||
        interface->ioAction == AsyncReadMore)
    {
    // cancel possible readTimeout or poll timer
        interface->cancelTimer();
    // cancel possible read poll
        pasynManager->cancelRequest(interface->pasynUser, &wasQueued);
        interface->asynReadHandler(data, numchars);
    }
}

I suppose I am now waiting for Dirk to help :-).

Cheers,

Nick Rees
Principal Software Engineer           Phone: +44 (0)1235-778430
Diamond Light Source                  Fax:   +44 (0)1235-446713 
-----Original Message-----
From: Eric Norum [mailto:norume@aps.anl.gov] 
Sent: 03 August 2006 18:27
To: Rees, NP (Nick)
Cc: EPICS tech-talk
Subject: Re: Deadlock when using asyn?


Why is your intrCallbackOctet routine calling cancelRequest?


On Aug 3, 2006, at 11:58 AM, Rees, NP (Nick) wrote:


Hi,


I have an asynSerial port that on VxWorks that is accessed by two tasks.
The first calls asynOctetSyncIO->writeRead in a private driver
background thread and the second access is in the asyn port thread using
asynRecord. The two tasks seem to deadlock over access to the two
semaphores: mutex pport->synchronousLock, and event
puserPvt->callbackDone.


Has anyone seen anything similar? What are the rules between mixing
asynOctetSyncIO and normal calls in the asynPort thread? I thought that
the lockPort in asynOctetSyncIO->writeRead was all that was needed.


Further information:


1. Asyn version: 4-6


2. Driver background thread stack trace:


#0  0x2276dc in semBTake ()
#1  0x2283ac in semTake ()
#2  0x1e74eb78 in cancelRequest (pasynUser=0x1ecf4168,
wasQueued=0x1ec9a18c) at ../../asyn/asynDriver/asynManager.c:1458
#3  0x1e72f0f8 in intrCallbackOctet (pasynUser=0x1ed179d8,
data=0x1ed4c130 "0\r", numchars=2) at ../AsynDriverInterface.cc:780
#4  0x1e75b3f0 in callInterruptUsers (pasynUser=0x1ed4c498,
pasynPvt=0x1ed4c5f0, data=0x1ed4c130 "0\r", nbytesTransfered=0x1ec9a294,
eomReason=0x1ec9a370) at ../../asyn/interfaces/asynOctetBase.c:218
#5  0x1e75b770 in readRaw (drvPvt=0x1ed4c7e0, pasynUser=0x1ed4c498,
data=0x1ed4c130 "0\r", maxchars=600, nbytesTransfered=0x1ec9a294,
eomReason=0x1ec9a370) at ../../asyn/interfaces/asynOctetBase.c:282
#6  0x1e762bec in readIt (ppvt=0x1ed4c580, pasynUser=0x1ed4c498,
data=0x1ed460ec "", maxchars=80, nbytesTransfered=0x1ec9a36c,
eomReason=0x1ec9a370) at ../../asyn/miscellaneous/asynInterposeEos.c:241
#7  0x1e75d300 in writeRead (pasynUser=0x1ed4c498,
write_buffer=0x1ed4609c "M161", write_buffer_len=4,
read_buffer=0x1ed460ec "", read_buffer_len=80, timeout=1,
nbytesOut=0x1ec9a368, nbytesIn=0x1ec9a36c, eomReason=0x1ec9a370) at
../../asyn/interfaces/asynOctetSyncIO.c:342
#8  0x1e6f4b84 in drvPmacMbxWriteRead (card=0, writebuf=0x1ed4609c
"M161", readbuf=0x1ed460ec "", errmsg=0x1ed4613c "") at
../drvPmac.c:1580
#9  0x1e6f52ec in drvPmacMbxTask (card=0) at ../drvPmac.c:1746
#10 0x231f7c in vxTaskEntry ()


3. Line 1458 in asynManager.c:
epicsEventMustWait(puserPvt->callbackDone);


4. Information on puserPvt->callbackDone:


ioc135 -> semShow 0x1ed113e0,5


Semaphore Id        : 0x1ed113e0
Semaphore Type      : BINARY
Task Queuing        : FIFO
Pended Tasks        : 1
State               : EMPTY
Options             : 0x0       SEM_Q_FIFO


VxWorks Events
--------------
Registered Task     : NONE
Event(s) to Send    : N/A
Options             : N/A


Pended Tasks
------------
   NAME      TID    PRI TIMEOUT
---------- -------- --- -------
pmacMbx0   1ec9a440  45       0


5. Stack trace for asynPort thread:


#0  0x229348 in semMTake ()
#1  0x2283ac in semTake ()
#2  0x1e8799ac in epicsMutexLock ()
#3  0x1e74b900 in portThread (pport=0x1ed4fcc0) at
../../asyn/asynDriver/asynManager.c:769
#4  0x1e882c68 in createFunction ()
#5  0x231f7c in vxTaskEntry ()


6. Line 769 in asynManager.c is:
epicsMutexMustLock(pport->synchronousLock);


7. Information on the pport->synchronousLock mutex is:


ioc135 -> epicsMutexShow 0x1ed4fbe0,5
epicsMutexId 0x1ed4fbe0 source ../../asyn/asynDriver/asynManager.c line
1714


Semaphore Id        : 0x1ed4fc10
Semaphore Type      : MUTEX
Task Queuing        : PRIORITY
Pended Tasks        : 1
Owner               : 0x1ec9a440   (pmacMbx0)
Options             : 0xd       SEM_Q_PRIORITY
                                SEM_DELETE_SAFE
                                SEM_INVERSION_SAFE


VxWorks Events
--------------
Registered Task     : NONE
Event(s) to Send    : N/A
Options             : N/A


Pended Tasks
------------
   NAME      TID    PRI TIMEOUT
---------- -------- --- -------
PMAC_ASC_0 1ed4f7b0 149       0




Any help would be greatefully received.


Cheers,


Nick Rees
Principal Software Engineer           Phone: +44 (0)1235-778430
Diamond Light Source                  Fax:   +44 (0)1235-446713


-- 
Eric Norum <norume@aps.anl.gov>
Advanced Photon Source
Argonne National Laboratory
(630) 252-4793