Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019 
<== Date ==> <== Thread ==>

Subject: Re: segmentation error pvdata -> libc
From: Michael Davidsaver <mdavidsaver@gmail.com>
To: Patrik Marschalik <marschalik@fhi-berlin.mpg.de>
Cc: tech-talk@aps.anl.gov
Date: Thu, 25 Oct 2018 09:09:28 -0700
On 10/25/18 7:45 AM, Patrik Marschalik wrote:
> Hi Michael,
> 
> the issue is even stranger.
> 
> We have a host with one network interface, starting a testServer in one
> terminal and calling pvlist in another one shows nothing. pvlist
> localhost works as expected. Do you have an idea what might be the
> problem?

At least in part #132 which is now fixed.

https://github.com/epics-base/pvAccessCPP/issues/132

Which revisions of the various modules are you building?

Also, could you extract line numbers from the stack trace?
0x7fe7638179c5 isn't very meaningful :)  Please run the following,
replacing the addresses with those from a newer run if you have
rebuilt the executables/libraries.

> addr2line -e ./path/to/pvget 0x7fe7638179c5 0x7fe76234bfc6 0x7fe762345bfb


> Thanks
> 
> Patrik
> 
> On Wed, Oct 24, 2018 at 09:37:29PM +0200, Heinz Junkes wrote:
>> Hi Michael,
>>
>> this I got got today from Patrik once:
>>
>> *** Error in `pvget': munmap_chunk(): invalid pointer:
>> 0x00007fe74e0a1010 ***
>> ======= Backtrace: =========
>> /lib/x86_64-linux-gnu/libc.so.6(+0x70bfb)[0x7fe762345bfb]
>> /lib/x86_64-linux-gnu/libc.so.6(+0x76fc6)[0x7fe76234bfc6]
>> /home/patrik/GITHUB/epics-base/epics-base/lib/linux-x86_64/libpvAccess.so.6.0.1(_ZN5epics8pvAccess6detail25BlockingTCPTransportCodecD2Ev+0x425)[0x7fe7638179c5]
>>
>> But there's a more general problem on our side.
>>
>> It has something to do with multiple networks? And I still haven't understood the impact of all the environments.
>>
>> Starting testserver:
>>
>> epics@kannsein:~$ testServer
>> VERSION : pvAccess Server v6.0.1-SNAPSHOT
>> PROVIDER_NAMES : local,
>> BEACON_ADDR_LIST :
>> AUTO_BEACON_ADDR_LIST : 1
>> BEACON_PERIOD : 15
>> BROADCAST_PORT : 5076
>> SERVER_PORT : 5075
>> RCV_BUFFER_SIZE : 16384
>> IGNORE_ADDR_LIST:
>> INTF_ADDR_LIST : 0.0.0.0
>>
>>
>> On an other Terminal
>>
>> epics@kannsein:~/BASE/epics-base$ pvlist
>> (no output)
>> epics@kannsein:~/BASE/epics-base$ pvlist 141.14.133.59
>> testADC
>> testAggregate
>> testBinaryCounter
>> testCounter
>> testHistogram
>> testMP
>> testNameValue
>> testRandom
>> testTable
>>
>> epics@kannsein:~/BASE/epics-base$ pvlist 10.20.30.1
>> testADC
>> testAggregate
>> testBinaryCounter
>> testCounter
>> testHistogram
>> testMP
>> testNameValue
>> testRandom
>> testTable
>>
>> Seems to works on both networks. But why no output on pvlist (without arguments) ?
>>
>> epics@kannsein:~$ export EPICS_PVAS_AUTO_BEACON_ADDR_LIST=NO
>> epics@kannsein:~$ export EPICS_PVAS_BEACON_ADDR_LIST=10.20.30.1
>> epics@kannsein:~$ export EPICS_PVAS_INTF_ADDR_LIST=10.20.30.1
>> epics@kannsein:~$ testServer
>> VERSION : pvAccess Server v6.0.1-SNAPSHOT
>> PROVIDER_NAMES : local,
>> BEACON_ADDR_LIST : 10.20.30.1
>> AUTO_BEACON_ADDR_LIST : 0
>> BEACON_PERIOD : 15
>> BROADCAST_PORT : 5076
>> SERVER_PORT : 5075
>> RCV_BUFFER_SIZE : 16384
>> IGNORE_ADDR_LIST:
>> INTF_ADDR_LIST : 10.20.30.1
>>
>>
>> After 15 about 15 seconds the other terminal shows
>>
>> epics@kannsein:~/BASE/epics-base$ pvlist
>> GUID 0x0CC9D05B00000000422E1420 version 1: tcp@[10.20.30.1:5075]
>> epics@kannsein:~/BASE/epics-base$ pvlist 0x0CC9D05B00000000422E1420
>> Failed to resolve GUID '0x0CC9D05B00000000422E1420’!
>>
>> After that no more output on pvlist without arguments????
>> epics@kannsein:~/BASE/epics-base$ pvlist
>> epics@kannsein:~/BASE/epics-base$
>>
>> this still works
>>
>> epics@kannsein:~/BASE/epics-base$ pvlist 10.20.30.1
>> testADC
>> testAggregate
>> testBinaryCounter
>> testCounter
>> testHistogram
>> testMP
>> testNameValue
>> testRandom
>> testTable
>>
>> What is the right way to define the environments in our situation?
>>
>> Thank you Heinz
>>
>> ------------------------------------------------------------------------------
>> Fritz-Haber-Institut    | Phone:         (+49 30) 8413-4270
>> Heinz Junkes             | Fax (G3+G4):   (+49 30) 8413-5900
>> Faradayweg 4-6        | 
>> D - 14195 Berlin        | E-Mail:        junkes@fhi-berlin.mpg.de
>> ------------------------------------------------------------------------------
>>
>>> On 24. Oct 2018, at 16:15, Michael Davidsaver <mdavidsaver@gmail.com> wrote:
>>>
>>> What modules versions/revisions are involved?  Can you use gdb
>>> to extract a stack trace from the core file?
>>>
>>>
>>> On 10/24/18 5:54 AM, Heinz Junkes wrote:
>>>> Using epics-base master on Ubuntu 16.4 (same on debian 9)
>>>>
>>>> pvget -> segmentation fault
>>>>
>>>> epics@kannsein:/var/log$ pvget CRYAFM:struckNTTable
>>>> CRYAFM:struckNTTable
>>>> epics:nt/NTTable:1.0 
>>>>    time_t timeStamp 2018-10-24T15:27:48.075 0
>>>>    string[] labels [The SIS TS,The X,The Y,The Z]
>>>>    structure value
>>>>        double[] ts [0,0,0,0,0,0,0,0,100,0,0,0,0,0,0,0,200,0,0,0,0,0,0,0,300,0,0,0,0,0,0,0,400,0,0,0,0,0,0,0,500,0,0,0,0,0,0,0,600,0,0,0,0,0,0,0,700,0,0,0,0,0,0,0,800,0,0,0,0,0,0,0,900,0,0,0,0,0,0,0,1000,0,0,0,0,0,0,0,1100,0,0,0,0,0,0,0,1200,0,0,0]
>>>>        double[] X []
>>>>        double[] Y []
>>>>        double[] Z []
>>>>
>>>>
>>>> Segmentation fault (core dumped)
>>>>
>>>>
>>>> In kern.log we can find
>>>> Oct 24 14:42:11 kannsein kernel: [788447.325497] traps: pvget[55565] general protection ip:7f6e9577c512 sp:7fffe1ea90d0 error:0 in libc-2.23.so[7f6e956f8000+1c0000]
>>>> Oct 24 14:42:19 kannsein kernel: [788456.015443] traps: pvget[55587] general protection ip:7f00b8c3a512 sp:7ffdbf0c6150 error:0 in libc-2.23.so[7f00b8bb6000+1c0000]
>>>> Oct 24 14:43:04 kannsein kernel: [788500.715803] traps: pvget[55616] general protection ip:7f88c21b8512 sp:7ffcf2fdba60 error:0 in libc-2.23.so[7f88c2134000+1c0000]
>>>> Oct 24 14:45:14 kannsein kernel: [788630.968783] traps: pvget[55636] general protection ip:7f4d4fa00512 sp:7ffc280a7b40 error:0 in libc-2.23.so[7f4d4f97c000+1c0000]
>>>> Oct 24 14:45:27 kannsein kernel: [788643.257952] traps: pvget[55648] general protection ip:7f088b665512 sp:7ffcc5d6ed40 error:0 in libc-2.23.so[7f088b5e1000+1c0000]
>>>> Oct 24 14:46:09 kannsein kernel: [788685.910016] traps: pvget[55661] general protection ip:7f750da52512 sp:7ffd13bdf0b0 error:0 in libc-2.23.so[7f750d9ce000+1c0000]
>>>> Oct 24 14:46:21 kannsein kernel: [788697.759313] traps: pvget[55673] general protection ip:7f859f104512 sp:7fff5629ff80 error:0 in libc-2.23.so[7f859f080000+1c0000]
>>>> Oct 24 14:47:10 kannsein kernel: [788746.457913] traps: pvget[55685] general protection ip:7f9d4348f512 sp:7fffd170dba0 error:0 in libc-2.23.so[7f9d4340b000+1c0000]
>>>> Oct 24 14:47:27 kannsein kernel: [788763.257920] traps: pvget[55697] general protection ip:7fc04a7b2512 sp:7ffc9e722280 error:0 in libc-2.23.so[7fc04a72e000+1c0000]
>>>> Oct 24 14:47:40 kannsein kernel: [788776.330929] traps: pvget[55709] general protection ip:7f0774726512 sp:7ffd41eb8d90 error:0 in libc-2.23.so[7f07746a2000+1c0000]
>>>>
>>>> earlier this morning:
>>>>
>>>> Oct 24 11:13:14 kannsein kernel: [775910.021560] TCP-rx[53453]: segfault at 7f57bb5fd028 ip 00007f57be8fcad7 sp 00007f57bb9fd890 error 6 in libpvData.so.7.0.1[7f57be826000+178000]
>>>> Oct 24 11:14:06 kannsein kernel: [775961.525926] TCP-rx[53465]: segfault at 7fcbd9a80028 ip 00007fcbdcd7fad7 sp 00007fcbd9e80890 error 6 in libpvData.so.7.0.1[7fcbdcca9000+178000]
>>>> Oct 24 11:27:12 kannsein kernel: [776747.380789] TCP-rx[53569]: segfault at 7efe437dd020 ip 00007efe46adcad7 sp 00007efe43bdd890 error 6 in libpvData.so.7.0.1[7efe46a06000+178000]
>>>> Oct 24 11:27:54 kannsein kernel: [776790.105153] TCP-rx[53586]: segfault at 7fdb33afc038 ip 00007fdb3ae14ad7 sp 00007fdb33efc890 error 6 in libpvData.so.7.0.1[7fdb3ad3e000+178000]
>>>> Oct 24 13:24:55 kannsein kernel: [783811.567841] TCP-rx[53954]: segfault at 7f9e25019038 ip 00007f9e28318ad7 sp 00007f9e25419890 error 6 in libpvData.so.7.0.1[7f9e28242000+178000]
>>>> Oct 24 13:25:18 kannsein kernel: [783834.055359] TCP-rx[53983]: segfault at 7f04138fc000 ip 00007f0416bfbad7 sp 00007f0413cfc890 error 6 in libpvData.so.7.0.1[7f0416b25000+178000]
>>>> Oct 24 13:26:06 kannsein kernel: [783882.134154] TCP-rx[53998]: segfault at 7f9094ad6038 ip 00007f9097dd5ad7 sp 00007f9094ed6890 error 6 in libpvData.so.7.0.1[7f9097cff000+178000]
>>>> Oct 24 13:26:31 kannsein kernel: [783907.132452] TCP-rx[54010]: segfault at 7f16139fa010 ip 00007f1616cf9ad7 sp 00007f1613dfa890 error 6 in libpvData.so.7.0.1[7f1616c23000+178000]
>>>> Oct 24 13:26:45 kannsein kernel: [783921.247824] TCP-rx[54022]: segfault at 7f7cfebe6010 ip 00007f7d01ee5ad7 sp 00007f7cfefe6890 error 6 in libpvData.so.7.0.1[7f7d01e0f000+178000]
>>>> Oct 24 13:31:08 kannsein kernel: [784183.770344] TCP-rx[54084]: segfault at 7fcce9f67020 ip 00007fcced266ad7 sp 00007fccea367890 error 6 in libpvData.so.7.0.1[7fcced190000+178000]
>>>>
>>>>
>>>> Sometimes it works well … ;-(
>>>>
>>>> System runs with two network-domains (141.14.128.0, 10.0.30.0)
>>>>
>>>> Any ideas?
>>>> Heinz
>>>>
>>>>
>>>> ----------------------------------------------------------------------
>>>> Das FHI verarbeitet, speichert und loescht Daten im Rahmen seiner
>>>> Geschaeftstaetigkeit gemaess der Datenschutz-Grundverordnung (DSGVO)
>>>> [General Data Protection Regulation (GDPR)] der Europaeischen Union.
>>>>
>>>
>>
> 
> 
> 


Replies:
Re: segmentation error pvdata -> libc Heinz Junkes
References:
segmentation error pvdata -> libc Heinz Junkes
Re: segmentation error pvdata -> libc Michael Davidsaver
Re: segmentation error pvdata -> libc Patrik Marschalik

Navigate by Date:
Prev: Re: DM recommedation. J. Lewis Muir
Next: RE: DM recommedation. Kline, David
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019 
Navigate by Thread:
Prev: Re: segmentation error pvdata -> libc Patrik Marschalik
Next: Re: segmentation error pvdata -> libc Heinz Junkes
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019 
ANJ, 26 Oct 2018 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·