Experimental Physics and Industrial Control System
Hi All,
I am using Asyn and Mark's modbus driver to talk a number of utility meters (primarily Eaton electricity meters). My soft ioc is frequently crashing with a seg fault. Connectivity of the meters is patchy in some cases and I cannot guarantee their availability at all times, resulting in a lot of communication errors. In the instances I am having trouble, an intermediate gateway device hosts the modbus server and supports multiple downstream meters. I can mitigate the situation by dedicating a soft ioc per IP address (gateway), but I'd like to identify the source of the problem if possible. I have other iocs using the same software to talk to other models of meter without any issues.
I've tried to detail some of my configuration and the errors I'm seeing below. Thanks for any feedback.
Gary
Versions:
=========
EPICS: 3.14.12.3
ASYN: 4-31
MODBUS: 2-8
Asyn & modbus driver initialization:
====================================
drvAsynIPPortConfigure("pxg27", "129.57.15.27:502", 0, 1, 1)
modbusInterposeConfig("pxg27", 0, 5000)
drvModbusAsynConfigure("SA1-SB_1000", "pxg27", 1, 3, 999, 28, 8, 100, "iq150")
drvModbusAsynConfigure("SA1-SB_1100", "pxg27", 1, 3, 1099, 18, 6, 100, "iq150")
drvModbusAsynConfigure("SA1-SB_3118", "pxg27", 1, 3, 3117, 4, 8, 100, "iq150")
drvModbusAsynConfigure("SA2-SB_1000", "pxg27", 2, 3, 999, 28, 8, 100, "iq150")
drvModbusAsynConfigure("SA2-SB_1100", "pxg27", 2, 3, 1099, 18, 6, 100, "iq150")
drvModbusAsynConfigure("SA2-SB_3118", "pxg27", 2, 3, 3117, 4, 8, 100, "iq150")
... several other asyn ports, etc. ...
dbLoadRecords("powerMetersV/db/iq150.db", "meter=SA1-SB")
dbLoadRecords("powerMetersV/db/iq150.db", "meter=SA2-SB")
Example Record:
===============
record(ai,"$(meter):rcvWh") {
field(SCAN,"10 second")
field(DTYP,"asynInt32")
field(FLNK,"$(meter):rcvkWh")
field(INP,"@asyn($(meter)_1100, 0)")
field(EGU,"Wh")
}
Error trace #1:
===============
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/asynDriver/asynManager.c:1807] [Compton_1000Read,0x8679f20,50] pxg12 queueUnlockPort
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/asynDriver/asynManager.c:1838] [Compton_1000Read,0x8679f20,50] pxg12 asynManager::queueUnlockPort waiting for event
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/asynDriver/asynManager.c:860] [pxg12,0x8660100,90] asynManager::portThread port=pxg12 callback
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/asynDriver/asynManager.c:914] [pxg12,0x8660100,90] pxg12 asynManager::queueLockPortCallback signaling begin event
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/asynDriver/asynManager.c:919] [pxg12,0x8660100,90] pxg12 asynManager::queueLockPortCallback waiting for mutex from queueUnlockPort
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/asynDriver/asynManager.c:1841] [Compton_1000Read,0x8679f20,50] pxg12 queueUnlockPort unlock mutex 0xb4f006e0 complete.
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/asynDriver/asynManager.c:1786] [HNSS_Moeller_1000Read,0x8673ee8,50] pxg12 asynManager::queueLockPort got event from callback
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/drvAsynSerial/drvAsynIPPort.c:795] [HNSS_Moeller_1000Read,0x8673ee8,50] 129.57.15.12:502 flush
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/drvAsynSerial/drvAsynIPPort.c:556] [HNSS_Moeller_1000Read,0x8673ee8,50] 129.57.15.12:502 write.
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/drvAsynSerial/drvAsynIPPort.c:558] [HNSS_Moeller_1000Read,0x8673ee8,50] 129.57.15.12:502 write 12
\000\001\000\000\000\006\002\003\003ç\000\034
00 01 00 00 00 06 02 03 03 e7 00 1c
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/drvAsynSerial/drvAsynIPPort.c:654] [HNSS_Moeller_1000Read,0x8673ee8,50] wrote 12 to 129.57.15.12:502, return asynSuccess.
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/interfaces/asynOctetSyncIO.c:262] [HNSS_Moeller_1000Read,0x8673ee8,50] asynOctetSyncIO wrote:
ç
\002\003\003ç\000\034
02 03 03 e7 00 1c
2017/07/05 15:58:06.942 [pxg12,-1,0] [../../asyn/drvAsynSerial/drvAsynIPPort.c:674] [HNSS_Moeller_1000Read,0x8673ee8,50] 129.57.15.12:502 read.
2017/07/05 15:58:06.942 drvModbusAsyn::doModbusIO port Compton_1000 error calling writeRead, error=129.57.15.12:502 timeout: Resource temporarily unavailable, nwrite=6/6, nread=0
2017/07/05 15:58:06.942 Compton:aAmps devAsynFloat64 process read error
2017/07/05 15:58:06.942 Compton:abVolt devAsynFloat64 process read error
2017/07/05 15:58:06.942 Compton:anVolt devAsynFloat64 process read error
2017/07/05 15:58:06.942 Compton:bAmps devAsynFloat64 process read error
2017/07/05 15:58:06.942 Compton:bcVolt devAsynFloat64 process read error
2017/07/05 15:58:06.942 Compton:bnVolt devAsynFloat64 process read error
2017/07/05 15:58:06.943 Compton:cAmps devAsynFloat64 process read error
2017/07/05 15:58:06.943 Compton:caVolt devAsynFloat64 process read error
2017/07/05 15:58:06.943 Compton:cnVolt devAsynFloat64 process read error
2017/07/05 15:58:06.943 Compton:frequency devAsynFloat64 process read error
2017/07/05 15:58:06.943 Compton:totVA devAsynFloat64 process read error
2017/07/05 15:58:06.943 Compton:totVAR devAsynFloat64 process read error
2017/07/05 15:58:06.943 Compton:totW devAsynFloat64 process read error
2017/07/05 15:58:07.042 drvModbusAsyn::doModbusIO port Compton_1000 has I/O error
2017/07/05 15:58:07.042 [pxg12,-1,0] [../../asyn/asynDriver/asynManager.c:1518] [Compton_1000Read,0x8679f20,50] pxg12 addr -1 queueRequest priority 3 not lockHolder
Segmentation fault (core dumped)
Error trace #2:
===============
2017/07/11 10:03:48.585 pxg13 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
2017/07/11 10:03:48.664 pxg12 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
2017/07/11 10:03:48.664 drvModbusAsyn::doModbusIO port C_UPH5_3118 error calling writeRead, error=129.57.15.12:502 read error: Connection reset by peer, nwrite=6/6, nread=0
2017/07/11 10:03:48.664 C_UPH5:maxDemPosVAR devAsynFloat64 process read error
2017/07/11 10:03:48.664 C_UPH5:maxDemPosW devAsynFloat64 process read error
2017/07/11 10:03:48.686 pxg30 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
2017/07/11 10:03:48.756 pxg11 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
2017/07/11 10:03:48.764 drvModbusAsyn::doModbusIO port C_UPH5_3118 has I/O error
2017/07/11 10:03:48.780 pxg13 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
2017/07/11 10:03:48.838 pxg11 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
...
2017/07/11 10:03:52.104 pxg13 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
2017/07/11 10:03:52.104 pxg27 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
2017/07/11 10:03:52.104 drvModbusAsyn::doModbusIO port SA1-SB_1100 error calling writeRead, error=129.57.15.27:502 timeout: Resource temporarily unavailable, nwrite=6/6, nread=0
2017/07/11 10:03:52.104 SA1-SB:dlvWh devAsynInt32 process read error
2017/07/11 10:03:52.104 SA1-SB:negVARh devAsynInt32 process read error
2017/07/11 10:03:52.104 SA1-SB:netVARh devAsynInt32 process read error
2017/07/11 10:03:52.104 SA1-SB:netWh devAsynInt32 process read error
2017/07/11 10:03:52.104 SA1-SB:posVARh devAsynInt32 process read error
2017/07/11 10:03:52.104 SA1-SB:rcvWh devAsynInt32 process read error
2017/07/11 10:03:52.104 SA1-SB:totVARh devAsynInt32 process read error
2017/07/11 10:03:52.104 SA1-SB:totVAh devAsynInt32 process read error
2017/07/11 10:03:52.104 SA1-SB:totWh devAsynInt32 process read error
2017/07/11 10:03:52.145 pxg30 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
2017/07/11 10:03:52.153 pxg13 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
2017/07/11 10:03:52.189 pxg11 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
2017/07/11 10:03:52.204 drvModbusAsyn::doModbusIO port SA1-SB_1100 has I/O error
2017/07/11 10:03:52.225 pxg30 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
2017/07/11 10:03:52.278 pxg11 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
...
2017/07/11 10:03:53.463 pxg13 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
2017/07/11 10:03:53.467 pxg27 asynManager::queueLockPortCallback got mutex from queueUnlockPort, signaling end event
Segmentation fault (core dumped)
Also seen once or twice:
========================
A call to 'assert(status == epicsMutexLockOK)'
by thread 'A/MP/HAD_9018Read' failed in ../../asyn/asynDriver/asynManager.c line 2342.
EPICS Release EPICS R3.14.12.3 $Date: Mon 2012-12-17 14:11:47 -0600$.
Local time is 2017-06-29 14:43:11.601380355 EDT
Please E-mail this message to the author or to [email protected]
Calling epicsThreadSuspendSelf()
- Replies:
- Re: ioc crashing while using asyn & modbus Mark Rivers
- Re: ioc crashing while using asyn & modbus Mark Rivers
- Navigate by Date:
- Prev:
CA gateway with systemd Benjamin Franksen
- Next:
Re: help - writing a table column from OPI BOY to PV Tomasz Brys
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
<2017>
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: CA gateway with systemd Wesley Moore
- Next:
Re: ioc crashing while using asyn & modbus Mark Rivers
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
<2017>
2018
2019
2020
2021
2022
2023
2024