Experimental Physics and
Industrial Control System

Hi fellow VxWorks users,

Since I migrated most of our IOCs from VxWorks 5.5 to VxWorks 6.9, I notices a severe network driver problem.
After running fine for weeks or months, suddenly the tNet0 tasks consumed 100% CPU and stays so, which of course makes
the IOC unusable. Only reboot helps. This happens on multiple IOCs but not synchronously.

It looks like the tNet0 task is constantly timing out and trying to drop some connection. But I cannot find out which
one. Any network diagnostics function freezes in ipcom_block_wait at a semaphore.

Has anyone seen this behavior before?

> tt tNet0 
0x0012c68c vxTaskEntry  +0x48 : ipcomNetTask ()
0x00113c40 ipcomNetTask +0x34 : jobQueueProcess ()
0x0029d9d8 jobQueueProcess+0xe8 : 0x00241584 ()
0x002415b4 ipcom_atomic_sub_and_return+0x9c : 0x001f9208 ()
0x001f9274 ipnet_timeout_cancel+0xe4 : 0x00226910 ()
0x0022699c iptcp_drop_connection+0x8a4: ipcom_list_remove ()
value = 0 = 0x0
> tt tNet0             
0x0012c68c vxTaskEntry  +0x48 : ipcomNetTask ()
0x00113c40 ipcomNetTask +0x34 : jobQueueProcess ()
0x0029d9d8 jobQueueProcess+0xe8 : 0x00241584 ()
0x002415b4 ipcom_atomic_sub_and_return+0x9c : 0x001f9208 ()
0x001f9274 ipnet_timeout_cancel+0xe4 : 0x00226910 ()
0x00226a48 iptcp_drop_connection+0x950: 0x0022b380 ()
0x0022c4e0 sockInfo     +0x2ca4: ipcom_list_insert_first (0x1f, 0x1f)
value = 0 = 0x0
> tt tNet0
0x0012c68c vxTaskEntry  +0x48 : ipcomNetTask ()
0x00113c40 ipcomNetTask +0x34 : jobQueueProcess ()
0x0029d9d8 jobQueueProcess+0xe8 : 0x00241584 ()
0x002415b4 ipcom_atomic_sub_and_return+0x9c : 0x001f9208 ()
0x001f9274 ipnet_timeout_cancel+0xe4 : 0x00226910 ()
0x00226a48 iptcp_drop_connection+0x950: 0x0022b380 ()
0x0022c4bc sockInfo     +0x2c80: 0x0022c58c ()
value = 0 = 0x0
> tt tNet0
0x0012c68c vxTaskEntry  +0x48 : ipcomNetTask ()
0x00113c40 ipcomNetTask +0x34 : jobQueueProcess ()
0x0029d9d8 jobQueueProcess+0xe8 : 0x00241584 ()
0x002415b4 ipcom_atomic_sub_and_return+0x9c : 0x001f9208 ()
0x001f9274 ipnet_timeout_cancel+0xe4 : 0x00226910 ()
0x00226a48 iptcp_drop_connection+0x950: 0x0022b380 ()
value = 0 = 0x0
> tt tNet0
0x0012c68c vxTaskEntry  +0x48 : ipcomNetTask ()
0x00113c40 ipcomNetTask +0x34 : jobQueueProcess ()
0x0029d9d8 jobQueueProcess+0xe8 : 0x00241584 ()
0x002415b4 ipcom_atomic_sub_and_return+0x9c : 0x001f9208 ()
0x001f9274 ipnet_timeout_cancel+0xe4 : 0x00226910 ()
value = 0 = 0x0

Dirk

Replies:

Re: VxWorks network problem Andrew Johnson via Core-talk

Re: VxWorks network problem Michael Davidsaver via Core-talk

Navigate by Date:

Prev: epics-7.0 » linux32 - Build # 425 - Still unstable! APS Jenkins via Core-talk

Next: Build failed: epics-base base-7.0-57 AppVeyor via Core-talk

Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <2022>  2023  2024 

Navigate by Thread:

Prev: Build failed: EPICS Base 7 base-7.0-657 AppVeyor via Core-talk

Next: Re: VxWorks network problem Andrew Johnson via Core-talk

Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <2022>  2023  2024