EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
<== Date ==> <== Thread ==>

Subject: Re: asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES
From: "Marco A. Barra Montevechi Filho via Tech-talk" <tech-talk at aps.anl.gov>
To: Mark Rivers <rivers at cars.uchicago.edu>
Cc: EPICS tech-talk <tech-talk at aps.anl.gov>
Date: Wed, 16 Mar 2022 18:18:23 +0000
The output is the following:

epics> epicsPrtEnvParams
EPICS_CA_ADDR_LIST: 10.0.38.46:60000 10.0.38.59:60000 10.10.10.76 s-mgn-vtx01-l 10.20.46.17 10.20.46.18 10.32.4.27 s-mgn-a-rio01 s-mgn-a-pb02 mgn-fanless1  s-mgn-a-rio02
EPICS_CA_CONN_TMO: 30.0
EPICS_CA_AUTO_ADDR_LIST: YES
EPICS_CA_REPEATER_PORT: 5065
EPICS_CA_SERVER_PORT: 5064
EPICS_CA_MAX_ARRAY_BYTES: 15728645
EPICS_CA_MAX_SEARCH_PERIOD: 300.0
EPICS_CA_NAME_SERVERS is undefined
EPICS_CAS_INTF_ADDR_LIST is undefined
EPICS_CAS_IGNORE_ADDR_LIST is undefined
EPICS_CAS_AUTO_BEACON_ADDR_LIST is undefined
EPICS_CAS_BEACON_ADDR_LIST is undefined
EPICS_CAS_SERVER_PORT is undefined
EPICS_CA_BEACON_PERIOD: 15.0
EPICS_CAS_BEACON_PERIOD is undefined
EPICS_CAS_BEACON_PORT is undefined
EPICS_BUILD_COMPILER_CLASS: gcc
EPICS_BUILD_OS_CLASS: Linux
EPICS_BUILD_TARGET_ARCH: linux-x86_64
EPICS_TIMEZONE: CUS::360:031102:110402
EPICS_TS_NTP_INET is undefined
EPICS_IOC_LOG_PORT: 7004
EPICS_IOC_LOG_INET is undefined
EPICS_IOC_LOG_FILE_LIMIT: 1000000
EPICS_IOC_LOG_FILE_NAME is undefined
EPICS_IOC_LOG_FILE_COMMAND is undefined
EPICS_CMD_PROTO_PORT is undefined
EPICS_AR_PORT: 7002
IOCSH_PS1: epics>
IOCSH_HISTSIZE: 50

So i suppose the environment variable is set correctly(?)
I will investigate it with gdb then and post the results when found. Thanks for the suggestion!

Marco

From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: 16 March 2022 15:11
To: Marco A. Barra Montevechi Filho <marco.filho at lnls.br>
Cc: EPICS tech-talk <tech-talk at aps.anl.gov>
Subject: Re: asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES
 
Please send the output of the following command at the IOC prompt:

epicsPrtEnvParams

That will show if the environment variable is set correctly in the IOC.

The seg fault is definitely happening in the IOC, not the equipment controller. You can use gdb to get the traceback from the seg fault to determine what line of code it is failing on.

Mark


Sent from my iPhone

On Mar 16, 2022, at 8:06 AM, Marco A. Barra Montevechi Filho <marco.filho at lnls.br> wrote:


Hello, Mark! Good morning.

Unfortunately it didnt. Im not sure if i understood the question correctly, but if i did, the answer is the following:

The IOC runs in a virtual machine (host) and is accessed by another virtual machine (client) which is configured via EPICS_CA_ADDR_LIST to search in the host's IP. This design works well here in LNLS, so it should not be the problem. EPICS_CA_MAX_ARRAY_BYTES is defined both in the host and in the client and set to the same values.
We also tried running caget in a jupyter notebook which also had EPICS_CA_MAX_ARRAY_BYTES defined on it as an environment variable. Neither strategy (runnning caget on the machine or in the notebook) worked.

For some reason, the error which i sent here earlier ("data transfer bigger than available memory or EPICS_CA_MAX_ARRAY_BYTES") is not showing up again. Instead, we get a direct segmentation fault that kills the IOC. Thus, i raise the following doubts:

-Does it makes sense to think of an IOC segmentation fault? Or is it just reporting a segmentation fault that actually happened inside the equipment controller and terminating for some reason? Does it makes sense to think that the data transfer is bigger than whats set in IMAX or in EPICS_CA_MAX_ARRAY_BYTES and so the IOC sigfaults and dies?
-If the answer to the first question is no, then does it makes sense to think of the IOC just propagating the segmentation fault message that comes from the equipment controller?
- if the answer to both previous questions is no, then should i only believe that sigfault comes from machine memory error?
-Is asyn prepared to deal with IMAX as big as 524288? (im not sure if this last question matters, though, since requesting large arrays of data with IMAX set to 256 also resulted in error sometimes).

If it helps, the field in which we are trying to caget is .BINP. .IFMT is configured to hybrid.

Thanks for your attention!

From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: 16 March 2022 14:11
To: Siddons, David <siddons at bnl.gov>
Cc: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>; Marco A. Barra Montevechi Filho <marco.filho at lnls.br>
Subject: Re: asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES
 
Did Pete’s suggestion fix the problem?


Sent from my iPhone

On Mar 15, 2022, at 9:42 AM, Siddons, David via Tech-talk <tech-talk at aps.anl.gov> wrote:


Hi Marco,
   Is EPICS_CA_MAX_ARRAY_BYTES set on both ends of the link?
Pete.


From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Marco A. Barra Montevechi Filho via Tech-talk <tech-talk at aps.anl.gov>
Sent: Tuesday, March 15, 2022 3:39 PM
To: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES
 
Hello all

Im debugging an EPICS asyn IOC which was working fine until now. The st.cmd of the IOC has a line which loaded asynRecord.db:

dbLoadRecords("$(ASYN)/db/asynRecord.db","P=$(DEV),R=asynEth,PORT=E873_ETH,ADDR=0,OMAX=256,IMAX=256")

and caput caget BL:H:EQ:asynEth.IFMT 1 (Hybrid)
caget BL:H:EQ:asynEth.BINP

in the IOC machine returned an array of numbers without any problem. However, it is recovering an array from a file much alrger than 256, so i suppose it just threw parts of the file away.
We wanted to get more of the file, so we raised IMAX to 350000 and defined the env variable EPICS_CA_MAX_ARRAY_BYTES=15728645. SO we got the error

caget MGN:A:E873:DIAG:A:asynEth.BINP
CA.Client.Exception...............................................
    Warning: "The requested data transfer is greater than available memory or EPICS_CA_MAX_ARRAY_BYTES"
    Context: "op=0, channel=MGN:A:E873:DIAG:A:asynEth.BINP, type=DBR_TIME_CHAR, count=16406, ctx="server unable to load read (or subscription update) response into protocol buffer PV="MGN:A:E873:DIAG:A:asynEth" max bytes=16408""
    Source File: ../getCopy.cpp line 92
    Current Time: Tue Mar 15 2022 16:32:11.936961493
..................................................................
Read operation timed out: some PV data was not read.
MGN:A:E873:DIAG:A:asynEth.BINP 350000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  (the zeroes continue)

So we tried lowering IMAX to 16406 but still the error persisted:

CA.Client.Exception...............................................
    Warning: "The requested data transfer is greater than available memory or EPICS_CA_MAX_ARRAY_BYTES"
    Context: "op=0, channel=MGN:A:E873:DIAG:A:asynEth.BINP, type=DBR_TIME_CHAR, count=16406, ctx="server unable to load read (or subscription update) response into protocol buffer PV="MGN:A:E873:DIAG:A:asynEth" max bytes=16408""
    Source File: ../getCopy.cpp line 92
    Current Time: Tue Mar 15 2022 16:32:11.936961493
..................................................................
Read operation timed out: some PV data was not read.
MGN:A:E873:DIAG:A:asynEth.BINP 16406 0 0 0 0 0 0

does anyone know if we are doing something wrong or forgetting something? I dont think memory should be the problem here... Where do these "max bytes=16408" come from?


Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente.

Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately.


Replies:
Re: asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES Mark Rivers via Tech-talk
References:
asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES Marco A. Barra Montevechi Filho via Tech-talk
Re: asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES Siddons, David via Tech-talk
Re: asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES Mark Rivers via Tech-talk
Re: asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES Mark Rivers via Tech-talk

Navigate by Date:
Prev: Re: asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES Mark Rivers via Tech-talk
Next: Re: asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES Mark Rivers via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
Navigate by Thread:
Prev: Re: asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES Mark Rivers via Tech-talk
Next: Re: asyn IMAX data transfer exceeding EPICS_CA_MAX_ARRAY_BYTES Mark Rivers via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
ANJ, 14 Sep 2022 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·