EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024  Index 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
<== Date ==> <== Thread ==>

Subject: Re: VxWorks network problem
From: Andrew Johnson via Core-talk <core-talk at aps.anl.gov>
To: "Zimoch Dirk (PSI)" <dirk.zimoch at psi.ch>, "core-talk at aps.anl.gov" <core-talk at aps.anl.gov>
Date: Fri, 26 Aug 2022 11:30:19 -0500
Hi Dirk,

On 8/26/22 8:05 AM, Zimoch Dirk (PSI) via Core-talk wrote:
Since I migrated most of our IOCs from VxWorks 5.5 to VxWorks 6.9, I notices a severe network driver problem.
After running fine for weeks or months, suddenly the tNet0 tasks consumed 100% CPU and stays so, which of course makes
the IOC unusable. Only reboot helps. This happens on multiple IOCs but not synchronously.

It looks like the tNet0 task is constantly timing out and trying to drop some connection. But I cannot find out which
one. Any network diagnostics function freezes in ipcom_block_wait at a semaphore.

Has anyone seen this behavior before?
It hasn't happened here at APS to my knowledge. Is it unique to a particular board type, or a specific network interface driver? What sub-version of VxWorks 6.9 are you running, and have you checked with the Wind River knowledge forums and/or support?

- Andrew

tt tNet0
0x0012c68c vxTaskEntry  +0x48 : ipcomNetTask ()
0x00113c40 ipcomNetTask +0x34 : jobQueueProcess ()
0x0029d9d8 jobQueueProcess+0xe8 : 0x00241584 ()
0x002415b4 ipcom_atomic_sub_and_return+0x9c : 0x001f9208 ()
0x001f9274 ipnet_timeout_cancel+0xe4 : 0x00226910 ()
0x0022699c iptcp_drop_connection+0x8a4: ipcom_list_remove ()
value = 0 = 0x0
tt tNet0
0x0012c68c vxTaskEntry  +0x48 : ipcomNetTask ()
0x00113c40 ipcomNetTask +0x34 : jobQueueProcess ()
0x0029d9d8 jobQueueProcess+0xe8 : 0x00241584 ()
0x002415b4 ipcom_atomic_sub_and_return+0x9c : 0x001f9208 ()
0x001f9274 ipnet_timeout_cancel+0xe4 : 0x00226910 ()
0x00226a48 iptcp_drop_connection+0x950: 0x0022b380 ()
0x0022c4e0 sockInfo     +0x2ca4: ipcom_list_insert_first (0x1f, 0x1f)
value = 0 = 0x0
tt tNet0
0x0012c68c vxTaskEntry  +0x48 : ipcomNetTask ()
0x00113c40 ipcomNetTask +0x34 : jobQueueProcess ()
0x0029d9d8 jobQueueProcess+0xe8 : 0x00241584 ()
0x002415b4 ipcom_atomic_sub_and_return+0x9c : 0x001f9208 ()
0x001f9274 ipnet_timeout_cancel+0xe4 : 0x00226910 ()
0x00226a48 iptcp_drop_connection+0x950: 0x0022b380 ()
0x0022c4bc sockInfo     +0x2c80: 0x0022c58c ()
value = 0 = 0x0
tt tNet0
0x0012c68c vxTaskEntry  +0x48 : ipcomNetTask ()
0x00113c40 ipcomNetTask +0x34 : jobQueueProcess ()
0x0029d9d8 jobQueueProcess+0xe8 : 0x00241584 ()
0x002415b4 ipcom_atomic_sub_and_return+0x9c : 0x001f9208 ()
0x001f9274 ipnet_timeout_cancel+0xe4 : 0x00226910 ()
0x00226a48 iptcp_drop_connection+0x950: 0x0022b380 ()
value = 0 = 0x0
tt tNet0
0x0012c68c vxTaskEntry  +0x48 : ipcomNetTask ()
0x00113c40 ipcomNetTask +0x34 : jobQueueProcess ()
0x0029d9d8 jobQueueProcess+0xe8 : 0x00241584 ()
0x002415b4 ipcom_atomic_sub_and_return+0x9c : 0x001f9208 ()
0x001f9274 ipnet_timeout_cancel+0xe4 : 0x00226910 ()
value = 0 = 0x0

Dirk


--
Complexity comes for free, Simplicity you have to work for.


References:
VxWorks network problem Zimoch Dirk (PSI) via Core-talk

Navigate by Date:
Prev: Build failed: epics-base base-7.0-57 AppVeyor via Core-talk
Next: epics-7.0 » linux32 - Build # 426 - Still Unstable! APS Jenkins via Core-talk
Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
Navigate by Thread:
Prev: VxWorks network problem Zimoch Dirk (PSI) via Core-talk
Next: Re: VxWorks network problem Michael Davidsaver via Core-talk
Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
ANJ, 14 Sep 2022 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·