Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <20142015  2016  2017  2018  2019  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <20142015  2016  2017  2018  2019 
<== Date ==> <== Thread ==>

Subject: RE: [CSS] freeze and lost running PV in Boy Screens
From: "Hill, Jeff" <johill@lanl.gov>
To: Maurizio Montis <Maurizio.Montis@lnl.infn.it>, "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>
Date: Wed, 30 Apr 2014 22:48:55 +0000
Hello Maurizio,

> A call to 'assert(ev_que->evque[ev_que->putix] == EVENTQEMPTY)'
>     by thread 'CAS-client' failed in ../dbEvent.c line 701.
> EPICS Release EPICS R3.14.12.4

This presumably shouldn't be caused by external circumstances such as the order of starting CA clients, or the type of CA clients.

The code involved in the IOC has been quite robust for some time now, and so my first guess is that there is some type of corruption of the data structures in the IOC. Are there some new drivers installed in this system? Does the IOC fail in other ways sometimes or is it always with this exact same assert fail? If you change the order in which driver memory is allocated during IOC startup, and or the size of memory that is allocated by drivers, does it change the outcome? Changing the order might cause the corruption to hit a different data structure in the IOC.

I also had a quick look at the logic surrounding this particular assert fail and it appears to be sound. Nevertheless, there could be something new that is occurring. If you can reproduce it in a smaller system that might help with isolating the cause (the mantra from the support department of every software company DYN).

We are quite busy now bringing the upgraded LANSCE RF systems on line. If you are feeling fairly certain that this isn't caused by layered code (probably device drivers) then let me know, and I will stare harder at the source code involved. 

Jeff

> -----Original Message-----
> From: tech-talk-bounces@aps.anl.gov [mailto:tech-talk-bounces@aps.anl.gov]
> On Behalf Of Maurizio Montis
> Sent: Wednesday, April 30, 2014 3:34 PM
> To: tech-talk@aps.anl.gov
> Subject: [CSS] freeze and lost running PV in Boy Screens
> 
> Hello to everyone,
> 
> using the CSS GUI to control EPICS variables provided by a VME system, i
> saw "PV Disconnection" on the screen. On the VxWorks EPICS shell i found
> this message:
> 
> ##################################################################
> #####
> A call to 'assert(ev_que->evque[ev_que->putix] == EVENTQEMPTY)'
>     by thread 'CAS-client' failed in ../dbEvent.c line 701.
> EPICS Release EPICS R3.14.12.4 $Date: Mon 2013-12-16 15:51:45 -0600$.
> Local time is 2014-04-29 10:05:52.897742610 MET
> Please E-mail this message to the author or to tech-talk@aps.anl.gov
> Calling epicsThreadSuspendSelf()
> Thread CAS-client (0x6a2910) suspended
> ##################################################################
> #####
> 
> The VME System has 2 different kind of acquisition boards, one for "slow
> acquisition" and one for "fast acquisition". On the CSS side, i have a
> main panel for the sub-system controlled and 2 popup windows, one for
> each acquisition.
> 
> I tried some tests with this system (VME+CSS), and after some
> investigation i've understood and observed the following points:
> 
> * when i open the popup related to the slow acquisition OR to the fast
> acquisition, everything works correctly
> 
> * when i open one popup, i close it and i open the other one, it seem to
> work correctly
> 
> * when i open the slow popup and after open the fast without closing the
> previous one, in the slow acquisition i lost the connection with all the
> PVs (freezing values AND/OR loosing connection). If i close the "fast"
> popup, i return in the normal situation
> 
> * during this tests i checked through common epics command line tools
> the PVs status and they continuously running without problems (no
> problems with the VME IOC)
> 
> * i also tried to change the CSS version, using all the versions
> available from SNS site (BASIC versions from 3.2.1 to 3.2.16) but i had
> always the same situation.
> 
> 
> Has someone seen this kind of behavior or this kind of messages before?
> An indication for a point to start a deep investigation will be
> appreciated.
> 
> Thanks in advance for any help
> 
> Regards,
> 
> - Maurizio
> 
> 
> 
> 
> ~~ Maurizio Montis - Control System Engineer ~~
>    mobile: +39 3408428089
>    mail: maurizio.montis@lnl.infn.it
>    skype: maurizio_montis
> 
> Istituto Nazionale di Fisica Nucleare - Laboratori Nazionali di Legnaro
>    V.le dell'Universita', 2
>    35020 LEGNARO (PD) - ITALY



References:
[CSS] freeze and lost running PV in Boy Screens Maurizio Montis

Navigate by Date:
Prev: [CSS] freeze and lost running PV in Boy Screens Maurizio Montis
Next: Re: [CSS] freeze and lost running PV in Boy Screens Kasemir, Kay
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <20142015  2016  2017  2018  2019 
Navigate by Thread:
Prev: [CSS] freeze and lost running PV in Boy Screens Maurizio Montis
Next: Re: [CSS] freeze and lost running PV in Boy Screens Kasemir, Kay
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <20142015  2016  2017  2018  2019 
ANJ, 17 Dec 2015 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·