Experimental Physics and
Industrial Control System

Mark Rivers <[email protected]> · Thu, 14 Jul 2016 20:39:39 +0000

Hi Eric,

Yes, I found the same article.

I am able to fix the problem with procServ on RHEL6 with the following patch:

************************************************
det@dec1248:/usr/local/procServ-2.6.0> diff -U3 acceptFactory.cc.orig acceptFactory.cc

--- acceptFactory.cc.orig    2016-07-15 00:18:54.757105145 +0200
+++ acceptFactory.cc    2016-07-15 00:26:55.669105117 +0200
@@ -61,7 +61,7 @@
     _fd = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
     assert(_fd>0);

-    setsockopt(_fd, SOL_SOCKET, SO_REUSEPORT, &optval, sizeof(optval));
+//    setsockopt(_fd, SOL_SOCKET, SO_REUSEPORT, &optval, sizeof(optval));

     memset(&addr, 0, sizeof(addr));
     addr.sin_family = AF_INET;
************************************************

So acceptFactory.cc is setting the SO_REUSEPORT option on the socket before calling bind().

Interestingly at the beginning of acceptFactory.cc are these lines.

#ifndef SO_REUSEPORT               // Linux doesn't know SO_REUSEPORT
#define SO_REUSEPORT SO_REUSEADDR
#endif

So apparently at the time procServ was written Linux did not support SO_REUSEPORT.

I have 2 questions:

- Was there a good reason for setting this option in procServ?  In my very limited testing with my patch above that does not set that option procServ seems to work correctly.

- Why does setting SO_REUSEPORT not cause the undesired behavior on Centos7?  Has the behavior in the article you found been reverted in later versions of Linux?

Thanks,
Mark


________________________________
From: Eric Norum [[email protected]]
Sent: Thursday, July 14, 2016 3:22 PM
To: Mark Rivers
Cc: Ralph Lange; [email protected]
Subject: Re: Problem with procServ on RHEL6

The SO_REUSEPORT option provides this capability — see https://lwn.net/Articles/542629/<redir.aspx?REF=AJThvDFy4pIkHbEhZ1izoYIGXq0XmSaOzCbK-hXSkQ6d_RVLJazTCAFodHRwczovL2x3bi5uZXQvQXJ0aWNsZXMvNTQyNjI5Lw..> for the full article.  Here’s a summary:

The basic concept of SO_REUSEPORT is simple enough. Multiple servers (processes or threads) can bind to the same port if they each set the option as follows:

   . . .

So long as the first server sets this option before binding its socket, then any number of other servers can also bind to the same port if they also set the option beforehand. The requirement that the first server must specify this option prevents port hijacking—the possibility that a rogue application binds to a port already used by an existing server in order to capture (some of) its incoming connections or datagrams. To prevent unwanted processes from hijacking a port that has already been bound by a server using SO_REUSEPORT, all of the servers that later bind to that port must have an effective user ID that matches the effective user ID used to perform the first bind on the socket.

SO_REUSEPORT can be used with both TCP and UDP sockets. With TCP sockets, it allows multiple listening sockets—normally each in a different thread—to be bound to the same port. Each thread can then accept incoming connections on the port by calling accept(). This presents an alternative to the traditional approaches used by multithreaded servers that accept incoming connections on a single socket.

But this seems like a fairly esoteric thing to do.

On Jul 14, 2016, at 1:02 PM, Mark Rivers <[email protected]<redir.aspx?REF=bS68F70J6f6M9woyfC8C6fOd6pAgpHQJcZPoi7TBT66d_RVLJazTCAFtYWlsdG86cml2ZXJzQGNhcnMudWNoaWNhZ28uZWR1>> wrote:

Why is bind() not returning EADDRINUSE the second time it is called on the same port on Centos6?  Is there some socket option that allows multiple processes to bind to the same port?

--
Eric Norum
[email protected]<redir.aspx?REF=mCosJSCPxhoIBXB7IS4dhxpS2AYG2D4uP8hVHJ3OhMqd_RVLJazTCAFtYWlsdG86ZXJpY0Bub3J1bS5jYQ..>







Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System