Subject: Re: [Tsvwg] Sockets API suggestion concerning sctp_assoc_t

From: Sridhar Samudrala <[email-masked]>
Date: Wed, 11 May 2005 14:38:19 -0700

I think sctp-impl is the right mailing list for SCTP implementation
specific discussions. So i am adding sctp-impl to the CC list.

I would like to clarify how associd is currently defined and 
implemented in linux SCTP(lksctp).
- sctp_assoc_t is defined as signed 32 bit value.
- The valid values of associd are 1-0x7fffffff.
- They don't have any relationship with file descriptors.
- All associations have a associd irrespective of whether they belong
  to 1-1 or 1-many style sockets.
- The associd will not change when the association is peeled-off to
  a different socket.

Thanks
Sridhar

Anders Torger wrote:

>One problem with sctp_assoc_t is that there is no clear definition of 
>what the values can be.
>
>The only thing I can find out from draft-ietf-tsvwg-sctpsocket-10.txt is 
>that zero is not a valid value for an association, but any other value 
>may be.
>
>It would be great if the API draft would specify some facts/rules about 
>sctp_assoc_t and association ID. I *think* this is true, and I'd like 
>that it was mentioned more clearly in the draft:
>
> - sctp_assoc_t is an integer of unspecified size and may be signed or 
>   unsigned
> - a valid association has an id != 0 (that is both positive and
>   negative sctp_assoc_t are valid values).
> - a one-to-one socket has no association id
> - a peeled off socket will lose its association id (the file descriptor
>   should be used for future reference), and the old id may be
>   immediately re-used by a new one-to-many association.
>
>In lksctp, it seems like one-to-one sockets have an association id 
>anyway, which is equal to the file descriptor value, while in Solaris 
>10 a random value is provided as association id in events on one-to-one 
>sockets.
>
>I would prefer if the API draft would specify that the OS must set the 
>association id to zero in returned events for one-to-one sockets 
>instead of allowing it to contain a random value like in Solaris 10, or 
>something that might be usable like in lksctp. This will make the API 
>safer (=bugs in user code is easier detected).
>
>When mixing one-to-many and one-to-one sockets there can often be a need 
>to provide a single space of handles for associations to upper layers 
>within the application. Handles which work for both one-to-many sockets 
>and one-to-one sockets, and does not change at peeloff. This is a bit 
>of a pain, since file descriptor values can collide with association id 
>values, and association id is lost at peeloff. But I don't think there 
>is much one can do about it. Virtual handles unrelated to file 
>descriptors and sctp_assoc_t combined with a lookup table is necessary 
>for that.
>
>I had an idea that one could define rules for sctp_assoc_t such that 
>valid one-to-many association ids would be outside the range of valid 
>file descriptors, and give one-to-one sockets an association id equal 
>to the file descriptor value, and that way provide a single handle 
>space without the need for virtual handles and lookup tables in 
>user-space. However, it breaks with peeloff, since then the association 
>id would have to change. Therefore my suggestion is simply to clarify 
>what the current rules are.
>
>/Anders Torger
>
>_______________________________________________
>tsvwg mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/tsvwg
>
>  
>

---

From: Kacheong Poon <[email-masked]>
Date: Sat, 14 May 2005 23:45:01 +0800

Removed tsvwg from the cc list.  This thread is all about
implementation...

> that zero is not a valid value for an association, but any other value 

I think the original intention is to allow an implementation to
define what sctp_assoc_t should be.  This means that instead of
0, the invalid value should probably be defined as something like
SCTP_ASSOC_INVALID.  So an implementation may use, say, a pointer
as sctp_assoc_t.  And SCTP_ASSOC_INVALID may be defined as NULL.
This should be fixed.

> that it was mentioned more clearly in the draft:

While this is fine, but I think we should not restrict what an
implementation may want to use as sctp_assoc_t unless this is
necessary.

>  - a one-to-one socket has no association id

I believe in all the places where sctp_assoc_t are mentioned, it
should state something like, "For one-to-one style socket, this
field is ignored."  I think it is probably not a good idea to
state that a 1-1 socket has no association ID.  An implementation
may choose to assign an assoc ID to the association controlled
by a 1-1 style socket.  But the callers of the API must not depend
on this.  OTOH, it should also not state the other way round.  A
caller should only use an assoc ID with a 1-N style socket.

>    immediately re-used by a new one-to-many association.

To allow more flexibility, I guess what we should say is that
after an association is peeled off and is under control of a 1-1
style socket, the caller should not use the original assoc ID
and the 1-N socket to control the association.  This allows the
stack the ability to reuse the assoc ID or not.  This means that
before calling sctp_peeloff(), the caller should "forget" about
the assoc ID and then use the "forgotten value" as the parameter
to sctp_peeloff().

> 10 a random value is provided as association id in events on one-to-one 

Yes, in Solaris, the assoc_id field is not initialized for 1-1
style socket, thus the random value.  Could you elaborate on the
problem you are having if the assoc_id field is not initialized
by the stack?

> safer (=bugs in user code is easier detected).

Even though in some implementations a 1-1 style socket also has
an assoc ID, the caller of the API MUST not use it or depend on
it.  As mentioned above, sctp_assoc_t can be a lot of things.

> descriptors and sctp_assoc_t combined with a lookup table is necessary 

The issue here is that the caller of the API cannot depend on the
type of sctp_assoc_t.  This means that if the caller is providing
a service to upper layers, it should have its own handle and should
not use the assoc ID.  And the assoc ID should be thought as a
handle only exists under a 1-N style socket.  Could you elaborate
the problem you are having here?  Even if the app mixes the use of
1-1 and 1-N style socket, I expect that the caller should know the
style of a socket.  Otherwise, a lot of things can be broken.

BTW, I'd recommend not to mix the two styles of SCTP socket in an
app...

> what the current rules are.

This assumes that sctp_assoc_t has the same type as a file descriptor.
But this may not be true in some implementations.  Since sctp_assoc_t
lives only under a 1-N style socket conceptually, it may be a good
idea for an implementation not to define sctp_assoc_t as the same type
as a file descriptor.  Well, it seems that all the existing impls define
sctp_assoc_t as an int...  But this is not necessary as currently
stated in the API draft (-:

-- 

						K. Poon.
						[email-masked]

---

From: Kacheong Poon <[email-masked]>
Date: Sun, 15 May 2005 00:04:55 +0800

Anders Torger wrote:
> Ok, but if I have understood the API draft correctly, it is not possible 
> to get or access a one-to-one socket with an sctp_assoc_t, thus the 
> association id would be hidden for the user.

This is correct.  After all, an implementation may only support 1-1
style socket.  And the type sctp_assoc_t may be defined as anything.

> It seems to me however, that in lksctp, the association id is indeed 
> visible in (for example) notifications for one-to-one sockets, although 
> it says in the API draft that the value should be "ignored". On Solaris 
> 10, a seemingly random value is delivered instead.
> 
> I would prefer the lksctp model, or that the value is set to zero. In 
> any case, it is not clear how it should work in the API draft.

By zero, I presume that you are referring to something like
SCTP_ASSOC_INVALID as suggested in my previous email.  As a
caller should not use or even check the value of assoc_id field
if the socket is 1-1 style, the value should not matter.  I
guess initialization it to a value is a good idea.

> I also like the model where sctp_assoc_t is signed, and that only 
> positive values are valid associations, in that way looking a bit like 
> file descriptors, and making it possible to make functions that return 
> sctp_assoc_t giving either a valid (>0), undefined (=0) or error (<0) 
> as result.
> 
> It is also nice that the association id is kept at peeloff, avoiding 
> that the id would be re-used by another one-to-many association before 
> the original socket has closed. Also this is not specified how it 
> should work in the API draft.

By specifying this, we will unnecessarily restrict how an implementation
handles an assoc ID.  After an association is peeled off from a 1-N
style socket, it will be controlled under a 1-1 style socket.
Logically, the two sockets are separate entity and do not need to share
anything.  Requiring the original assoc ID not be used until the 1-1
style socket is closed means that the two sockets are linked in a
special (SCTP only) way.  I guess what we should specify is that after
sctp_peeloff() is called, the caller should disassociate the association
with the original assoc ID.

-- 

						K. Poon.
						[email-masked]

---

From: Kacheong Poon <[email-masked]>
Date: Sun, 15 May 2005 00:12:08 +0800

Randall Stewart wrote:

> Unfortunately there are notes in the document that specify that
> the assoc-id is ignored for 1-2-1.. but I think this restriction
> should be removed. To me, it would make more sense from an
> application point of view to be able to have an assoc-id
> even on 1-2-1 sockets..

I disagree (-:  The sctp_assoc_t was originally added because of
performance reason for 1-N style socket.  1-1 style socket does not
have this issue.  I don't see why we need to force 1-1 style
socket to assign an assoc_id to the association the socket controls.
In fact, an implementation may not even choose to support 1-N style
socket.  Requiring it to have an extra assoc ID, which does not have
any meaning conceptually, with a 1-1 style socket is not a good
design.

-- 

						K. Poon.
						[email-masked]