Subject: SO_REUSEADDR setsockopt() and SCTP

From: "Anatoly Khusid" <[email-masked]>
Date: Fri, 11 Feb 2005 16:30:02 -0500

This question deals more with "SCTP Sockets API" draft.  I have not seen any
explanation in this document regarding the usage of SO_REUSEADDR on SCTP
sockets.
The following code fails on Solaris 10:

ADDR = Some valid address to bind
fd1 = socket(PF_INET, SOCK_STREAM, IPPROTO_SCTP);
setsockopt(fd1, SOL_SOCKET, SO_REUSEADDR)
bind(fd1, ADDR)
fd2 = socket(PF_INET, SOCK_STREAM, IPPROTO_SCTP);
setsockopt(fd2, SOL_SOCKET, SO_REUSEADDR)
bind(fd2, ADDR)   <=== Fails with errno="Address in use"

However, the above code works fine on Linux SLES9.  Does it mean that Linux
has a bug, or this is something implementation dependent?
What is the right behavior for SO_REUSEADDR on SCTP sockets?
Do you think this should be explained/documented in "SCTP Sockets API"?

Thanks,

Anatoly Khusid

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 11 Feb 2005 15:03:38 -0700

Anatoly,

SO_REUSEADDR is described by POSIX and is protocol independent.

--brian

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦

---

From: "Anatoly Khusid" <[email-masked]>
Date: Fri, 11 Feb 2005 17:28:30 -0500

>SO_REUSEADDR is described by POSIX and is protocol independent.

It works different on Linux SLES9 vs. Solaris when using SCTP sockets.
I provided an example which illustrates the differences.
This web site only explains how SO_REUSEADDR work with TCP.
Is TCP TIME_WAIT equivalent to SCTP_SHUTDOWN_PENDING ?

>You can use setsockopt() to set the SO_REUSEADDR socket option, which
explicitly
>allows a process to bind to a port which remains in TIME_WAIT (it still
only
>allows a single process to be bound to that port). This is the both the
simplest
>and the most effective option for reducing the "address already in use"
error.

http://hea-www.harvard.edu/~fine/Tech/addrinuse.html

-----Original Message-----
From: Brian F. G. Bidulock [mailto:[email-masked]]
Sent: Friday, February 11, 2005 5:04 PM
To: Anatoly Khusid
Cc: [email-masked]
Subject: Re: SO_REUSEADDR setsockopt() and SCTP

Anatoly,

--brian

---

From: Randall Stewart <[email-masked]>
Date: Fri, 11 Feb 2005 17:50:50 -0500

Anatoly:

I know for KAME-BSD you would also get a failure. We don't
support the concept of SO_REUSADDR. Namely because:

a) There is no timed wait with an SCTP endpoint. Thus
    you don't necessarily need this to reopen a socket.

and

b) The one-to-many model allows you to peel things off
    if you want to push associations out to other threads.

The sockets API draft does not cover it.. and so it is
really implementation dependant.. I don't know why one
would want it...

R

-- 
Randall Stewart
ITD
803-345-0369 <or> 815-342-5222

---

From: Randall Stewart <[email-masked]>
Date: Fri, 11 Feb 2005 17:58:39 -0500

Exactly.. so-reusaddr is to help escape time-wait.

SCTP does not have a time-wait.. except maybe on V-tags.. not
on ports or connections...

> Is TCP TIME_WAIT equivalent to SCTP_SHUTDOWN_PENDING ?

No, shutdown-pending is when you the side shutting down
does something like:

send(sd,...)
close(sd)

What happens is the data is put out (assuming you have
not tweaked the so_linger options) and is still waiting
for a SACK.. and you close.. so the association gets
a shutdown-pending flagged on it. When the close completes
the assoc frees...

Of course you don't have quite the same scenario in a
1-2-many model.. you can do

sendxx(sd,addr,...data)
sendxx(sd,addr, MSG_EOF) (if I remember the right flag here)

Which gracefully shuts down the association.. and again
since the MSG_EOF comes right after the send.. the data
is probably not ack'd.. the associaiton goes into
SHUTDOWN-PENDING....

Timed wait happens after all is closed in TCP and you are
waiting the 2 MSL's before you can safely reuse the
port again.... SCTP does not have this issue due to the
v-tags

R
>>This question deals more with "SCTP Sockets API" draft.  I have not seen
> 
> any
> 
>>However, the above code works fine on Linux SLES9.  Does it mean that
> 
> Linux
>

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 11 Feb 2005 16:48:10 -0700

Anatoly,

POSIX:

 SO_REUSEADDR
     Specifies that the rules used in validating addresses supplied to bind()
     should allow reuse of local addresses, if this is supported by the protocol.
     This option takes an int value.  This is a boolean option.

SO_REUSEADDR is necessary to bind to a protocol address that is already
listening or connected (or in the case of TCP in timed wait as well).

The former two are perfectly applicable to SCTP.

--brian

---

From: Kacheong Poon <[email-masked]>
Date: Fri, 11 Feb 2005 17:29:40 -0800

> bind(fd2, ADDR)   <=== Fails with errno="Address in use"

The second bind() fails because the first socket is still in bound
state.  If the first socket is in, say, established state, the
second bind() will succeed.  This is the same behavior as a TCP socket.

-- 

						K. Poon.
						[email-masked]

---

From: "Anatoly Khusid" <[email-masked]>
Date: Mon, 14 Feb 2005 07:29:00 -0500

Brian,

>SO_REUSEADDR is described by POSIX and is protocol independent.

But then you quote POSIX, which says:

"if this is supported by the protocol."

So how can it be protocol independent?  I agree with Randall that
SO_REUSEADDR does not make sense for SCTP.

Anatoly

Sent: Friday, February 11, 2005 6:48 PM
To: Anatoly Khusid
Cc: [email-masked]
Subject: Re: SO_REUSEADDR setsockopt() and SCTP

Anatoly,

POSIX:

 SO_REUSEADDR
     Specifies that the rules used in validating addresses supplied to
bind()
     should allow reuse of local addresses, if this is supported by the
protocol.
     This option takes an int value.  This is a boolean option.

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 14 Feb 2005 06:54:14 -0700

Anatoly,

The socket option is protocol independent, but reuse of local
addresses is not supported by all protocols.  (e.g. under SCCP
or X.25, a local address describes a complete connection unlike
TCP or SCTP where it merely names a socket.)

On Linux:

  ADDR = Some valid address to bind
  fd1 = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
  bind(fd1, ADDR)
  fd2 = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
  bind(fd2, ADDR)   <=== Fails EADDRINUSE

  ADDR = Some valid address to bind
  fd1 = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
  setsockopt(fd1, SOL_SOCKET, SO_REUSEADDR)
  bind(fd1, ADDR)
  fd2 = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
  setsockopt(fd2, SOL_SOCKET, SO_REUSEADDR)
  bind(fd2, ADDR)   <=== Succeeds

For OpenSS7 SCTP (and LKSCTP I suppose):

  ADDR = Some valid address to bind
  fd1 = socket(PF_INET, SOCK_STREAM, IPPROTO_SCTP);
  bind(fd1, ADDR)
  fd2 = socket(PF_INET, SOCK_STREAM, IPPROTO_SCTP);
  bind(fd2, ADDR)   <=== Fails EADDRINUSE

  bind(fd2, ADDR)   <=== Succeeds

Which makes perfect sense and is consistent with POSIX.

You will find the following true for Linux as well:

  ADDR = Some valid address to bind
  fd1 = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
  setsockopt(fd1, SOL_SOCKET, SO_REUSEADDR);
  bind(fd1, ADDR);
  listen(fd1);
  fd2 = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
  setsockopt(fd2, SOL_SOCKET, SO_REUSEADDR);
  bind(fd2, ADDR);   <=== Fails EADDRINUSE

Which makes less sense...  I think this should actually succeed.

Not having such a mechanism will make peer-to-peer protocols
like M2PA quite difficult to implement in user space, and will
go quite against the POSIX sockets model.

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 14 Feb 2005 07:07:30 -0700

Anatoly,

See also UNIX'98 XTI/TLI IP option T_IP_REUSEADDR (where it came from).

 T_IP_REUSEADDR
      Many TCP implementations do not allow the user to bind more than
      one transport endpoint to addresses with identical port numbers.
      If T_IP_REUSEADDR is set to T_YES this restriction is relaxed in
      the sense that it is now allowed to bind a transport endpoint to
      an address with a port number and an underspecified internet
      address ("wild card" address) and further endpoints to addresses
      with the same port number and (mutually exclusive) fully specified
      internet addresses.

That is more like:

  int val=1;
  ADDR1 = IN_ADDRANY, port 32
  ADDR2 = 192.168.0.1, port 32

  fd1=socket(PF_INET, SOCK_STREAM, IPPROTO_SCTP);
  setsockopt(fd1, SOL_SOCKET, SO_REUSEADDR, &val, sizeof(val));
  bind(fd1, &ADDR1, sizeof(ADDR1))
  fd2=socket(PF_INET, SOCK_STREAM, IPPROTO_SCTP);
  setsockopt(fd2, SOL_SOCKET, SO_REUSEADDR, &val, sizeof(val));
  bind(fd2, &ADDR2, sizeof(ADDR2))  <=== Succeeds

  fd1=socket(PF_INET, SOCK_STREAM, IPPROTO_SCTP);
  bind(fd1, &ADDR1, sizeof(ADDR1))
  fd2=socket(PF_INET, SOCK_STREAM, IPPROTO_SCTP);
  bind(fd2, &ADDR2, sizeof(ADDR2))  <=== Fails EADDRINUSE

Technically, following this older rule, the Solaris behavior may be
correct.  Check the above with a wildcard and specific address on the
same port number on Solaris and see if it exhibits the above behavior.
It should, Solaris being originally SVR4.2 based can be expected to
follow the UNIX'98 behavior.
      
--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 14 Feb 2005 07:30:54 -0700

Anatoly,

See also, all the references in the index under SO_REUSEADDR and
SO_REUSEPORT in Stevens' TCP/IP Illustrated Volume 1.

--brian