Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020 
<== Date ==> <== Thread ==>

Subject: RE: connection timeout between ioc and logstash
From: "Kivel Niko \(PSI\) via Tech-talk" <tech-talk@aps.anl.gov>
To: "J. Lewis Muir" <jlmuir@imca-cat.org>, 吴煊 <wux@ihep.ac.cn>
Cc: tech-talk <tech-talk@aps.anl.gov>
Date: Fri, 16 Aug 2019 05:51:04 +0000
Hi all

while it's true that logstash (or whatever other logging facilities) could artificially keep the connection alive, it's only indirectly related to the actual problem. Which is, that the logger - while obviously noticing the broken connection - doesn't buffer and resend the failed message. It only sends the next message after the reconnect. Which might happen after another timeout.

best
Niko

--
Paul Scherrer Institute
Dr. Niko Kivel
Forschungsstrasse 111
WBGB/014
5232 Villigen PSI
Switzerland

phone: +41 56 310 2226
email: niko.kivel@psi.ch

________________________________________
From: tech-talk-bounces@aps.anl.gov [tech-talk-bounces@aps.anl.gov] on behalf of J. Lewis Muir via Tech-talk [tech-talk@aps.anl.gov]
Sent: Thursday, August 15, 2019 16:48
To: 吴煊
Cc: tech-talk
Subject: Re: connection timeout between ioc and logstash

On 08/15, 吴煊 via Tech-talk wrote:
>   We are testing for using logstash to replace iocLogServer. We found that it will lose connection between the ioc and logstash if none of PV has been written more than an hour(not very accuracy). If writting a PV again, it will establish the connection again, but the caPutLog message of this time will not be sent to logstash. It prompted that in IOC terminal:
>
> log client: lost contact with log server at "10.1.236.30:7011" because "Connection timed out"
>
> log client: connected to log server at "10.1.236.30:7011"
>
> from then on, it will work fine if the interval time of writting PV was not very long.So how can keep the connection all the time without timeout. It seems that using iocLogServer will not lose connection.

How are you logging to Logstash?  A custom Logstash client?  How does it
work?

Can you determine what is closing the connection?  Is it the Logstash
server?  Is it a firewall?  Is it a NAT proxy?

Can you look at the configuration of your Logstash server and see if it
has a timeout or keep-alive option and ensure it is set correctly?

For example, if you're using the TCP input plugin of Logstash

  https://www.elastic.co/guide/en/logstash/current/plugins-inputs-tcp.html

it has a tcp_keep_alive option that defaults to false; you could try
setting that to true.

Regards,

Lewis

Replies:
Re: connection timeout between ioc and logstash J. Lewis Muir via Tech-talk
References:
connection timeout between ioc and logstash 吴煊 via Tech-talk
Re: connection timeout between ioc and logstash J. Lewis Muir via Tech-talk

Navigate by Date:
Prev: Re: Undefined Timestamp on Motor Record William Layne via Tech-talk
Next: Re: Undefined Timestamp on Motor Record Torsten Bögershausen via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020 
Navigate by Thread:
Prev: Re: connection timeout between ioc and logstash J. Lewis Muir via Tech-talk
Next: Re: connection timeout between ioc and logstash J. Lewis Muir via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020 
ANJ, 16 Aug 2019 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·