From daemon@ns.ietf.org  Tue Apr  2 19:35:11 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA09584
	for <ipoverib-archive@odin.ietf.org>; Tue, 2 Apr 2002 19:35:11 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA29164
	for ipoverib-archive@odin.ietf.org; Tue, 2 Apr 2002 19:35:10 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA28278;
	Tue, 2 Apr 2002 19:25:31 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA28244
	for <ipoverib@ns.ietf.org>; Tue, 2 Apr 2002 19:25:28 -0500 (EST)
Received: from e31.esmtp.ibm.com. (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA09498
	for <ipoverib@ietf.org>; Tue, 2 Apr 2002 19:25:27 -0500 (EST)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e31.esmtp.ibm.com. (8.12.2/8.12.2) with ESMTP id g330M1cY143048;
	Tue, 2 Apr 2002 19:22:01 -0500
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.00) with ESMTP id g330PQi118758;
	Tue, 2 Apr 2002 17:25:26 -0700
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g330PPX29425;
	Tue, 2 Apr 2002 16:25:25 -0800
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g330POJ01520;
	Tue, 2 Apr 2002 16:25:24 -0800 (PST)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204030025.g330POJ01520@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] DHCP over InfiniBand
To: roy.k.larsen@intel.com (Larsen, Roy K)
Date: Tue, 2 Apr 2002 16:25:23 -0800 (PST)
Cc: kashyapv@us.ibm.com ('Vivek Kashyap'), ipoverib@ietf.org
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4F44@ORSMSX108> from "Larsen, Roy K" at Mar 27, 2002 09:56:57 AM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> Back to the client-identifier.  Although I don't believe this was your
> intent, Section 2.1.1 gives the impression that the draft is specifying
> fixed length format.  I suggest changing the language of the first sentence
> from "The format.... is as defined below" to "A suggested format....".
> Perhaps the section title should be changed to "Considerations for using HW
> address as client-identifier".

But isn't a fixed length setup better ? It goes with the type. The
client identifier can be set to anything a host desires but if it is
set to type 32(InfiniBand) the length is 20 bytes. For those nodes
that don't have any use of the 4 bytes of 'interface-id' the field can
be set to zero. Note that a host can also use FQDN, EUI64 or any other
well-defined type suitable for its needs.

Vivek

> 
> Roy
> 
> -----Original Message-----
> From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> Sent: Tuesday, March 26, 2002 6:05 PM
> To: roy.k.larsen@intel.com
> Cc: ipoverib@ietf.org
> Subject: Re: [Ipoverib] DHCP over InfiniBand
> 
> 
> Roy,
>         The DHCP server needs a value by which it can uniquely
>         identify the client. This uniqueness is certainly satisfied by
>         the use of the IPoIB link address. 
> 	
>         The server records this unique identifier in its database. But
>         for the server to be able to utilise this information across
>         client boots the client must present the same identifier to
>         it; hence the suggestion of invariance of the
>         client-identifier. 
> 
>         If the chaddr field is used and the client-identifier is not
>         used, then too the same 'chaddr' field must be presented to
>         the server to ensure the server can recognise the client.
> 
>         Note that this issue is of prominence here only because there
>         maybe multiple IPoIB interfaces associated with the same GID.
>         An EUI-64 value or the GID will do otherwise. If you plan to
>         use only one IPoIB interface per GID then there is really no
>         issue - just use the GID.
> 
>         If the IPoIB link address was equal or less than 16-bytes then
>         we wouldn't have to use the Broadcast bit. 
> 
> Vivek
> 
> > 
> > Vivek,
> > 
> > This looks good.  Thanks.  I do have one question.  I assume that the
> > "invariantness" of the interface-id would also apply to the chaddr if that
> > were available to us.  Does this mean that if the IPoIB hardware address
> > were <= 16 bytes and contained the QPN that it would still be unusable for
> > DHCP?  What I'm trying to get at is, why can't we just say the
> > client-identifier IS the IPoIB hardware address found in an ARP?  Is the
> > "variantness" of the hardware address acceptable in ARP but not in DHCP?
> > The asymmetry concerns me and I just want to make sure there are no
> > stumbling blocks down the road related to the variant nature of the IPoIB
> > hardware address.
> > 
> > Roy
> > 
> > -----Original Message-----
> > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > Sent: Tuesday, March 26, 2002 3:38 PM
> > To: ipoverib@ietf.org
> > Subject: [Ipoverib] DHCP over InfiniBand
> > 
> > 
> > I submitted the following draft based on the discussions we had on the
> > mailing list and at the IETF meeting. It doesn't appear to me that we
> > are moving in the direction of shortening the link-layer address. So
> > the 'broadcast bit' is the solution.
> > 
> > Please comment.
> > 
> > Thanks
> > 	Vivek
> > 
> > <snip>
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Tue Apr  2 20:21:03 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA10786
	for <ipoverib-archive@odin.ietf.org>; Tue, 2 Apr 2002 20:21:03 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id UAA01693
	for ipoverib-archive@odin.ietf.org; Tue, 2 Apr 2002 20:21:03 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id UAA01526;
	Tue, 2 Apr 2002 20:17:15 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id UAA01488
	for <ipoverib@ns.ietf.org>; Tue, 2 Apr 2002 20:17:12 -0500 (EST)
Received: from caduceus.fm.intel.com ([192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA10674
	for <ipoverib@ietf.org>; Tue, 2 Apr 2002 20:17:11 -0500 (EST)
Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.37 2002/04/01 17:50:55 root Exp $) with ESMTP id g331GjJ10479
	for <ipoverib@ietf.org>; Wed, 3 Apr 2002 01:16:45 GMT
Received: from fmsmsxvs042.fm.intel.com (fmsmsxv042-1.fm.intel.com [132.233.48.110])
	by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.15 2002/04/01 17:51:48 root Exp $) with SMTP id g331FrH06600
	for <ipoverib@ietf.org>; Wed, 3 Apr 2002 01:15:53 GMT
Received: from fmsmsx26.fm.intel.com ([132.233.42.26])
 by fmsmsxvs042.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002040217205417540
 ; Tue, 02 Apr 2002 17:20:54 -0800
Received: by fmsmsx26.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <H39Q6ZGK>; Tue, 2 Apr 2002 17:17:10 -0800
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F57@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Vivek Kashyap'" <kashyapv@us.ibm.com>
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] DHCP over InfiniBand
Date: Tue, 2 Apr 2002 17:17:08 -0800 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

/rkl
embedded
rkl/

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
Sent: Tuesday, April 02, 2002 4:25 PM
To: roy.k.larsen@intel.com
Cc: kashyapv@us.ibm.com; ipoverib@ietf.org
Subject: Re: [Ipoverib] DHCP over InfiniBand


> Back to the client-identifier.  Although I don't believe this was your
> intent, Section 2.1.1 gives the impression that the draft is specifying
> fixed length format.  I suggest changing the language of the first
sentence
> from "The format.... is as defined below" to "A suggested format....".
> Perhaps the section title should be changed to "Considerations for using
HW
> address as client-identifier".

But isn't a fixed length setup better ? It goes with the type.
/rkl
NO!  Isn't that how we got here?  If DHCP hadn't defined the 'chaddr' as
fixed length we wouldn't even be having this discussion.  The
client-identifier is an opaque value so why should the length matter?
rkl/
                                                                The
client identifier can be set to anything a host desires but if it is
set to type 32(InfiniBand) the length is 20 bytes. For those nodes
that don't have any use of the 4 bytes of 'interface-id' the field can
be set to zero.
/rkl
Since the 'interface-id' must be invariant when needed, I would like to see
a bigger field so that I can use an ASCII string value.  So, if it must be
fixed length, I would like to see an 'interface-id' of no less than 16
bytes.
rkl/
                 Note that a host can also use FQDN, EUI64 or any other
well-defined type suitable for its needs.
/rkl
This is probably showing my ignorance of DHCP.  If I used FQDN, EUI64, etc.,
would I use a different type value than 32?  RFC2131 implies this should be
a hardware type defined in RFC1340.  In the draft you've posted, how should
I interpret the last paragraph of section 2.1 that says "'client identifier'
option MAY consist of any data.?

Roy
rkl/

Vivek

> 
> Roy
> 
> -----Original Message-----
> From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> Sent: Tuesday, March 26, 2002 6:05 PM
> To: roy.k.larsen@intel.com
> Cc: ipoverib@ietf.org
> Subject: Re: [Ipoverib] DHCP over InfiniBand
> 
> 
> Roy,
>         The DHCP server needs a value by which it can uniquely
>         identify the client. This uniqueness is certainly satisfied by
>         the use of the IPoIB link address. 
> 	
>         The server records this unique identifier in its database. But
>         for the server to be able to utilise this information across
>         client boots the client must present the same identifier to
>         it; hence the suggestion of invariance of the
>         client-identifier. 
> 
>         If the chaddr field is used and the client-identifier is not
>         used, then too the same 'chaddr' field must be presented to
>         the server to ensure the server can recognise the client.
> 
>         Note that this issue is of prominence here only because there
>         maybe multiple IPoIB interfaces associated with the same GID.
>         An EUI-64 value or the GID will do otherwise. If you plan to
>         use only one IPoIB interface per GID then there is really no
>         issue - just use the GID.
> 
>         If the IPoIB link address was equal or less than 16-bytes then
>         we wouldn't have to use the Broadcast bit. 
> 
> Vivek
> 
> > 
> > Vivek,
> > 
> > This looks good.  Thanks.  I do have one question.  I assume that the
> > "invariantness" of the interface-id would also apply to the chaddr if
that
> > were available to us.  Does this mean that if the IPoIB hardware address
> > were <= 16 bytes and contained the QPN that it would still be unusable
for
> > DHCP?  What I'm trying to get at is, why can't we just say the
> > client-identifier IS the IPoIB hardware address found in an ARP?  Is the
> > "variantness" of the hardware address acceptable in ARP but not in DHCP?
> > The asymmetry concerns me and I just want to make sure there are no
> > stumbling blocks down the road related to the variant nature of the
IPoIB
> > hardware address.
> > 
> > Roy
> > 
> > -----Original Message-----
> > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > Sent: Tuesday, March 26, 2002 3:38 PM
> > To: ipoverib@ietf.org
> > Subject: [Ipoverib] DHCP over InfiniBand
> > 
> > 
> > I submitted the following draft based on the discussions we had on the
> > mailing list and at the IETF meeting. It doesn't appear to me that we
> > are moving in the direction of shortening the link-layer address. So
> > the 'broadcast bit' is the solution.
> > 
> > Please comment.
> > 
> > Thanks
> > 	Vivek
> > 
> > <snip>
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Wed Apr  3 15:32:52 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14974
	for <ipoverib-archive@odin.ietf.org>; Wed, 3 Apr 2002 15:32:51 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA25064
	for ipoverib-archive@odin.ietf.org; Wed, 3 Apr 2002 15:32:54 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA24185;
	Wed, 3 Apr 2002 15:27:43 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA24151
	for <ipoverib@ns.ietf.org>; Wed, 3 Apr 2002 15:27:40 -0500 (EST)
Received: from e31.esmtp.ibm.com. (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14748
	for <ipoverib@ietf.org>; Wed, 3 Apr 2002 15:27:36 -0500 (EST)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e31.esmtp.ibm.com. (8.12.2/8.12.2) with ESMTP id g33KO7P1027350;
	Wed, 3 Apr 2002 15:24:09 -0500
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.00) with ESMTP id g33KRUe136468;
	Wed, 3 Apr 2002 13:27:30 -0700
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g33KRTX26876;
	Wed, 3 Apr 2002 12:27:29 -0800
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g33KRSn15868;
	Wed, 3 Apr 2002 12:27:28 -0800 (PST)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204032027.g33KRSn15868@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] DHCP over InfiniBand
To: roy.k.larsen@intel.com (Larsen, Roy K)
Date: Wed, 3 Apr 2002 12:27:27 -0800 (PST)
Cc: kashyapv@us.ibm.com ('Vivek Kashyap'), ipoverib@ietf.org
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4F57@ORSMSX108> from "Larsen, Roy K" at Apr 02, 2002 04:17:08 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> /rkl
> NO!  Isn't that how we got here?  If DHCP hadn't defined the 'chaddr' as
> fixed length we wouldn't even be having this discussion.  The
> client-identifier is an opaque value so why should the length matter?
> rkl/

        chaddr applies to all hardware types. The value 32 applies to
        InfiniBand. The choice is between listing the hardware address
        or just the GID. I prefer the full hardware address -
        including the QPN. Now, for those that don't have a constant
        QPN and whose uniqueness requirement is served by just the GID
        can set the 4 bytes of QPN/reserved flags to 0.

        The length in the option field therefore automatically becomes
        21 since we need to add the 1 byte of hardware type.

>                                                                 The
> client identifier can be set to anything a host desires but if it is
> set to type 32(InfiniBand) the length is 20 bytes. For those nodes
> that don't have any use of the 4 bytes of 'interface-id' the field can
> be set to zero.
> /rkl
> Since the 'interface-id' must be invariant when needed, I would like to see
> a bigger field so that I can use an ASCII string value.  So, if it must be
> fixed length, I would like to see an 'interface-id' of no less than 16
> bytes.
> rkl/

        If you want to use some arbitrary data then use the type value
        of 0. 

>                  Note that a host can also use FQDN, EUI64 or any other
> well-defined type suitable for its needs.
> /rkl
> This is probably showing my ignorance of DHCP.  If I used FQDN, EUI64, etc.,
> would I use a different type value than 32?  RFC2131 implies this should be
> a hardware type defined in RFC1340.  In the draft you've posted, how should
> I interpret the last paragraph of section 2.1 that says "'client identifier'
> option MAY consist of any data.?

        Yes, you need to use a different type. 27 for EUI64, 0 for
        FQDN or other arbitrary data. See RFC2132, section 9.14.

        Also, RFC1340 has long been obsoleted. Look under 'ARP
        parameters' at www.iana.org.

Vivek

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr  3 16:21:45 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA16073
	for <ipoverib-archive@odin.ietf.org>; Wed, 3 Apr 2002 16:21:45 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA27681
	for ipoverib-archive@odin.ietf.org; Wed, 3 Apr 2002 16:21:48 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA27523;
	Wed, 3 Apr 2002 16:17:24 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA27489
	for <ipoverib@optimus.ietf.org>; Wed, 3 Apr 2002 16:17:21 -0500 (EST)
Received: from caduceus.fm.intel.com ([192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA15947
	for <ipoverib@ietf.org>; Wed, 3 Apr 2002 16:17:16 -0500 (EST)
Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.37 2002/04/01 17:50:55 root Exp $) with ESMTP id g33LGlL18251
	for <ipoverib@ietf.org>; Wed, 3 Apr 2002 21:16:48 GMT
Received: from fmsmsxvs042.fm.intel.com (fmsmsxv042-1.fm.intel.com [132.233.48.110])
	by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.15 2002/04/01 17:51:48 root Exp $) with SMTP id g33LIT323674
	for <ipoverib@ietf.org>; Wed, 3 Apr 2002 21:18:29 GMT
Received: from fmsmsx26.fm.intel.com ([132.233.42.26])
 by fmsmsxvs042.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002040313205800676
 ; Wed, 03 Apr 2002 13:20:58 -0800
Received: by fmsmsx26.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2HL98FQS>; Wed, 3 Apr 2002 13:17:13 -0800
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F5A@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Vivek Kashyap'" <kashyapv@us.ibm.com>
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] DHCP over InfiniBand
Date: Wed, 3 Apr 2002 13:17:07 -0800 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

/rkl
embedded
rkl/

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
Sent: Wednesday, April 03, 2002 12:27 PM
To: roy.k.larsen@intel.com
Cc: kashyapv@us.ibm.com; ipoverib@ietf.org
Subject: Re: [Ipoverib] DHCP over InfiniBand


> /rkl
> NO!  Isn't that how we got here?  If DHCP hadn't defined the 'chaddr' as
> fixed length we wouldn't even be having this discussion.  The
> client-identifier is an opaque value so why should the length matter?
> rkl/

        chaddr applies to all hardware types. The value 32 applies to
        InfiniBand. The choice is between listing the hardware address
        or just the GID. I prefer the full hardware address -
        including the QPN. Now, for those that don't have a constant
        QPN and whose uniqueness requirement is served by just the GID
        can set the 4 bytes of QPN/reserved flags to 0.

        The length in the option field therefore automatically becomes
        21 since we need to add the 1 byte of hardware type.
/rkl
Since the QPN is an output parameter of the CREATE QUEUE PAIR verbs
interface, I don't believe anyone should assume an invariant QPN.
Therefore, I would venture to guess that the client-identifier will rarely
reflect the true IPoIB hardware address.  Since it is an opaque value and
there is no requirement that it reflect the IPoIB hardware address, it
doesn't follow that it must be equal in length.  But, given that I can use
type 0 to construct a friendly invariant client-id, there is no reason to
push it any further here.

Thanks for clarifying the type usage in DHCP.
Roy
rkl/

<snip>

Vivek

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr  3 19:41:28 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA20124
	for <ipoverib-archive@odin.ietf.org>; Wed, 3 Apr 2002 19:41:27 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA09726
	for ipoverib-archive@odin.ietf.org; Wed, 3 Apr 2002 19:41:28 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA09446;
	Wed, 3 Apr 2002 19:35:45 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA09418
	for <ipoverib@optimus.ietf.org>; Wed, 3 Apr 2002 19:35:42 -0500 (EST)
Received: from e21.esmtp.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA20017
	for <ipoverib@ietf.org>; Wed, 3 Apr 2002 19:35:40 -0500 (EST)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.esmtp.ibm.com (8.12.2/8.12.2) with ESMTP id g340TFXt042218;
	Wed, 3 Apr 2002 18:29:25 -0600
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.00) with ESMTP id g340Z73160752;
	Wed, 3 Apr 2002 19:35:12 -0500
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g340YtM29029;
	Wed, 3 Apr 2002 16:34:55 -0800 (PST)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g340Yrs29675;
	Wed, 3 Apr 2002 16:34:53 -0800 (PST)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204040034.g340Yrs29675@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] DHCP over InfiniBand
To: roy.k.larsen@intel.com (Larsen, Roy K)
Date: Wed, 3 Apr 2002 16:34:52 -0800 (PST)
Cc: kashyapv@us.ibm.com ('Vivek Kashyap'), ipoverib@ietf.org
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4F5A@ORSMSX108> from "Larsen, Roy K" at Apr 03, 2002 12:17:07 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> > /rkl
> > NO!  Isn't that how we got here?  If DHCP hadn't defined the 'chaddr' as
> > fixed length we wouldn't even be having this discussion.  The
> > client-identifier is an opaque value so why should the length matter?
> > rkl/
> 
>         chaddr applies to all hardware types. The value 32 applies to
>         InfiniBand. The choice is between listing the hardware address
>         or just the GID. I prefer the full hardware address -
>         including the QPN. Now, for those that don't have a constant
>         QPN and whose uniqueness requirement is served by just the GID
>         can set the 4 bytes of QPN/reserved flags to 0.
> 
>         The length in the option field therefore automatically becomes
>         21 since we need to add the 1 byte of hardware type.
> /rkl
> Since the QPN is an output parameter of the CREATE QUEUE PAIR verbs
> interface, I don't believe anyone should assume an invariant QPN.

        It is not 'assume an invariant QPN' but 'know the invariant
        QPN' i.e. if the implementation ensures invariant QPN it can
        use it. Verbs don't restrict anyone from implementing an
        interface that returns a specific queue pair or any other
        solution that allows for invariant QPs for a particular
        function. This is not the issue though.

        I suppose the length can be a different fixed length - but
        then, as you too seem to agree below, one can just use type 0
        for a non-htype client id.

> Therefore, I would venture to guess that the client-identifier will rarely
> reflect the true IPoIB hardware address.  Since it is an opaque value and
> there is no requirement that it reflect the IPoIB hardware address, it
> doesn't follow that it must be equal in length.  But, given that I can use
> type 0 to construct a friendly invariant client-id, there is no reason to
> push it any further here.

        Agreed. Does this mean you are ok with the description in the
        draft ?

Vivek

	

> 
> Thanks for clarifying the type usage in DHCP.
> Roy
> rkl/
> 
> <snip>
> 
> Vivek
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr  3 20:00:41 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA20374
	for <ipoverib-archive@odin.ietf.org>; Wed, 3 Apr 2002 20:00:40 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id UAA10527
	for ipoverib-archive@odin.ietf.org; Wed, 3 Apr 2002 20:00:40 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA10256;
	Wed, 3 Apr 2002 19:55:50 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA10224
	for <ipoverib@optimus.ietf.org>; Wed, 3 Apr 2002 19:55:47 -0500 (EST)
Received: from mail6.microsoft.com (mail6.microsoft.com [131.107.3.126])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA20273
	for <ipoverib@ietf.org>; Wed, 3 Apr 2002 19:55:45 -0500 (EST)
Received: from inet-vrs-06.redmond.corp.microsoft.com ([157.54.6.201]) by mail6.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Wed, 3 Apr 2002 16:55:16 -0800
Received: from 157.54.6.197 by inet-vrs-06.redmond.corp.microsoft.com (InterScan E-Mail VirusWall NT); Wed, 03 Apr 2002 16:55:12 -0800
Received: from red-msg-02.redmond.corp.microsoft.com ([157.54.12.20]) by inet-hub-06.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Wed, 3 Apr 2002 16:55:11 -0800
X-MimeOLE: Produced By Microsoft Exchange V6.0.6177.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Subject: RE: [Ipoverib] DHCP over InfiniBand
Date: Wed, 3 Apr 2002 16:55:07 -0800
Message-ID: <24A715275661C8428C00432EFCA5CB7C055F9C0B@red-msg-02.redmond.corp.microsoft.com>
Thread-Topic: [Ipoverib] DHCP over InfiniBand
Thread-Index: AcHbcL4ynP7RJlW6R3C257ycSJogmQAAgOfw
From: "Jim Pinkerton" <jpink@microsoft.com>
To: "Vivek Kashyap" <kashyapv@us.ibm.com>,
        "Larsen, Roy K" <roy.k.larsen@intel.com>
Cc: <ipoverib@ietf.org>
X-OriginalArrivalTime: 04 Apr 2002 00:55:11.0432 (UTC) FILETIME=[5F93AC80:01C1DB73]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id TAA10225
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 8bit


I'm also confused with why the parameter specified in the client id
should not be opaque. My understanding of the DHCP spec is that the
server is *required* to view the content as opaque.  Thus I don't
understand the value in specifying the content, or its length.


Jim




-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
Sent: Wednesday, April 03, 2002 4:35 PM
To: Larsen, Roy K
Cc: 'Vivek Kashyap'; ipoverib@ietf.org
Subject: Re: [Ipoverib] DHCP over InfiniBand

> 
> > /rkl
> > NO!  Isn't that how we got here?  If DHCP hadn't defined the
'chaddr' as
> > fixed length we wouldn't even be having this discussion.  The
> > client-identifier is an opaque value so why should the length
matter?
> > rkl/
> 
>         chaddr applies to all hardware types. The value 32 applies to
>         InfiniBand. The choice is between listing the hardware address
>         or just the GID. I prefer the full hardware address -
>         including the QPN. Now, for those that don't have a constant
>         QPN and whose uniqueness requirement is served by just the GID
>         can set the 4 bytes of QPN/reserved flags to 0.
> 
>         The length in the option field therefore automatically becomes
>         21 since we need to add the 1 byte of hardware type.
> /rkl
> Since the QPN is an output parameter of the CREATE QUEUE PAIR verbs
> interface, I don't believe anyone should assume an invariant QPN.

        It is not 'assume an invariant QPN' but 'know the invariant
        QPN' i.e. if the implementation ensures invariant QPN it can
        use it. Verbs don't restrict anyone from implementing an
        interface that returns a specific queue pair or any other
        solution that allows for invariant QPs for a particular
        function. This is not the issue though.

        I suppose the length can be a different fixed length - but
        then, as you too seem to agree below, one can just use type 0
        for a non-htype client id.

> Therefore, I would venture to guess that the client-identifier will
rarely
> reflect the true IPoIB hardware address.  Since it is an opaque value
and
> there is no requirement that it reflect the IPoIB hardware address, it
> doesn't follow that it must be equal in length.  But, given that I can
use
> type 0 to construct a friendly invariant client-id, there is no reason
to
> push it any further here.

        Agreed. Does this mean you are ok with the description in the
        draft ?

Vivek

	

> 
> Thanks for clarifying the type usage in DHCP.
> Roy
> rkl/
> 
> <snip>
> 
> Vivek
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr  3 20:24:21 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA20586
	for <ipoverib-archive@odin.ietf.org>; Wed, 3 Apr 2002 20:24:21 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id UAA11540
	for ipoverib-archive@odin.ietf.org; Wed, 3 Apr 2002 20:24:23 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id UAA11382;
	Wed, 3 Apr 2002 20:18:17 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id UAA11346
	for <ipoverib@optimus.ietf.org>; Wed, 3 Apr 2002 20:18:13 -0500 (EST)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA20538
	for <ipoverib@ietf.org>; Wed, 3 Apr 2002 20:18:08 -0500 (EST)
Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.37 2002/04/01 17:50:55 root Exp $) with ESMTP id g341HhG10985
	for <ipoverib@ietf.org>; Thu, 4 Apr 2002 01:17:43 GMT
Received: from fmsmsxvs041.fm.intel.com (fmsmsxv041-1.fm.intel.com [132.233.48.109])
	by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.15 2002/04/01 17:51:48 root Exp $) with SMTP id g341JQ117563
	for <ipoverib@ietf.org>; Thu, 4 Apr 2002 01:19:26 GMT
Received: from fmsmsx019.fm.intel.com ([132.233.42.130])
 by fmsmsxvs041.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002040317184003181
 ; Wed, 03 Apr 2002 17:18:40 -0800
Received: by fmsmsx019.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <H303H913>; Wed, 3 Apr 2002 17:18:10 -0800
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F5B@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Vivek Kashyap'" <kashyapv@us.ibm.com>
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] DHCP over InfiniBand
Date: Wed, 3 Apr 2002 17:17:38 -0800 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
Sent: Wednesday, April 03, 2002 4:35 PM
To: roy.k.larsen@intel.com
Cc: kashyapv@us.ibm.com; ipoverib@ietf.org
Subject: Re: [Ipoverib] DHCP over InfiniBand

<snip>

> Therefore, I would venture to guess that the client-identifier will rarely
> reflect the true IPoIB hardware address.  Since it is an opaque value and
> there is no requirement that it reflect the IPoIB hardware address, it
> doesn't follow that it must be equal in length.  But, given that I can use
> type 0 to construct a friendly invariant client-id, there is no reason to
> push it any further here.

        Agreed. Does this mean you are ok with the description in the
        draft ?
/rkl
I still believe the language is somewhat ambiguous when one considers the
last paragraph of 2.1 followed by the first paragraph of 2.1.1 followed by
the last paragraph of 2.1.1.  It leaves the reader asking what is the
purpose of section 2.1.1. if the actual format of the client-identifier does
not affect IPoIB conformance.  Nevertheless, I can accept it as it is if you
see if differently.

Roy
rkl/

Vivek

> 
> Thanks for clarifying the type usage in DHCP.
> Roy
> rkl/
> 
> <snip>
> 
> Vivek
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr  4 13:44:00 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA03724
	for <ipoverib-archive@odin.ietf.org>; Thu, 4 Apr 2002 13:44:00 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id NAA23510
	for ipoverib-archive@odin.ietf.org; Thu, 4 Apr 2002 13:44:02 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id NAA23343;
	Thu, 4 Apr 2002 13:39:49 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id NAA23311
	for <ipoverib@optimus.ietf.org>; Thu, 4 Apr 2002 13:39:47 -0500 (EST)
Received: from e31.esmtp.ibm.com (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA03590
	for <ipoverib@ietf.org>; Thu, 4 Apr 2002 13:39:44 -0500 (EST)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e31.esmtp.ibm.com (8.12.2/8.12.2) with ESMTP id g34IaJnL143538;
	Thu, 4 Apr 2002 13:36:19 -0500
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.00) with ESMTP id g34IdfV47976;
	Thu, 4 Apr 2002 11:39:41 -0700
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g34IdeX29124;
	Thu, 4 Apr 2002 10:39:40 -0800
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g34IddX20706;
	Thu, 4 Apr 2002 10:39:39 -0800 (PST)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204041839.g34IddX20706@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] DHCP over InfiniBand
To: roy.k.larsen@intel.com (Larsen, Roy K)
Date: Thu, 4 Apr 2002 10:39:38 -0800 (PST)
Cc: ipoverib@ietf.org
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4F5B@ORSMSX108> from "Larsen, Roy K" at Apr 03, 2002 04:17:38 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit


> I still believe the language is somewhat ambiguous when one considers the
> last paragraph of 2.1 followed by the first paragraph of 2.1.1 followed by
> the last paragraph of 2.1.1.  It leaves the reader asking what is the
> purpose of section 2.1.1. if the actual format of the client-identifier does
> not affect IPoIB conformance.  Nevertheless, I can accept it as it is if you
> see if differently.

I see your point. I have added some detail to make things clear. 


Modified excerpt:

----------------------------------

        The DHCP protocol states that the 'client identifier' option
        may be used as the unique identifying value for the client.
        This value must be unique within the subnet the client is a
        member of. 
        
        The client identifier option includes a type and identifier
        pair. The identifier included in the client-identifier option
        may consist of a hardware address or any other unique value
        such as the DNS name of the client. When a hardware address is
        used, the type field should be one of the ARP hardware types
        listed in ARPPARAM. A type of 0 (zero) should be used when the
        included identifier is other than a hardware address.
        [RFC2132] 

        The client-identifier itself SHOULD not be interpreted by the
        server. [RFC2132] 

2.1 IPoIB specific usage of DHCP message fields

        A DHCP client, when working over an IPoIB interface, MUST
        follow the following rules:

	'htype' (hardware address type) MUST be 32 [ARPPARAM]

	'hlen' (hardware address length) MUST be 0.

        The 'chaddr' (client hardware address) field MUST be zeroed.
        The server and the relay agent MUST ignore 'chaddr' on
        receipt. 

        The 'client identifier' option MUST be used in DHCP messages.
        The client-identifier may consist of any data.
	
        IPoIB clients SHOULD use the following format for the
        client-identifier option:


    Code  Len   Type  Client-Identifier
   +-----+-----+-----+-----+-----+-----+-----+-------------------....----+
   |  61 | 21  |  32 |Interface-id (4 bytes) |   GID (16 bytes)          |
   +-----+-----+-----+-----+-----+-----+-----+-------------------....----+

        Every IPoIB interface is associated with an identifier
        referred to as the GID [IPoIB_ARCH]. The GID is unique in the
        InfiniBand fabric. An invariant GID is formed by appending the
        port's EUI-64 identifier to the InfiniBand subnet prefix.

        The GID is associated with a particular hardware port. The GID
        and a QPN define an IPoIB interface at the port. Therefore an
        implementation may associate multiple IPoIB interfaces on the
        same port. It is up to the implementation to ensure a unique
        client-identifier when multiple IPoIB interfaces are defined
        over the same port and same GID. A unique, invariant
        'interface-id' value be included in addition to the GID to
        achieve this.

        Note: a port may be associated with multiple GIDs. Therefore,
        multiple IPoIB interfaces may exist on the same port while
        using a different GID from among the GIDs associated with the
        port. 

        A unique interface-id may be formed by including the QPN
        associated with the relevant IPoIB interface if the
        implementation is designed to keep this association constant
        across boots. A timestamp or some other value unique to the
        implementation may also be used for the same purpose.

        If there is only one IPoIB interface associated with a
        particular GID, then use of the GID with the 'interface-id'
        zeroed is sufficient. By default, an implementation zeroes out
        the interface-id field in the client identifier described
        above. 

        This document does not preclude the use of other 'client
        identifier' type, such as fully qualified domain name(FQDN) or
        the EUI-64 value associated with the interface.


-------------------------------------------------------------------

Vivek

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr  4 15:39:41 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA07952
	for <ipoverib-archive@odin.ietf.org>; Thu, 4 Apr 2002 15:39:41 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA00786
	for ipoverib-archive@odin.ietf.org; Thu, 4 Apr 2002 15:39:44 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA00004;
	Thu, 4 Apr 2002 15:30:22 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA29957
	for <ipoverib@optimus.ietf.org>; Thu, 4 Apr 2002 15:30:19 -0500 (EST)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA07676
	for <ipoverib@ietf.org>; Thu, 4 Apr 2002 15:30:05 -0500 (EST)
Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.37 2002/04/01 17:50:55 root Exp $) with ESMTP id g34KSg329350
	for <ipoverib@ietf.org>; Thu, 4 Apr 2002 20:28:42 GMT
Received: from fmsmsxvs040.fm.intel.com (fmsmsxv040-1.fm.intel.com [132.233.48.108])
	by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.15 2002/04/01 17:51:48 root Exp $) with SMTP id g34KULT28995
	for <ipoverib@ietf.org>; Thu, 4 Apr 2002 20:30:25 GMT
Received: from fmsmsx28.fm.intel.com ([132.233.42.28])
 by fmsmsxvs040.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002040412282223070
 ; Thu, 04 Apr 2002 12:28:22 -0800
Received: by fmsmsx28.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2JN3RZ7F>; Thu, 4 Apr 2002 12:28:57 -0800
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F5C@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Vivek Kashyap'" <kashyapv@us.ibm.com>
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] DHCP over InfiniBand
Date: Thu, 4 Apr 2002 12:20:21 -0800 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Thanks Vivek.  I think the flow and clarity of this section is much better
now.

Roy 

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
Sent: Thursday, April 04, 2002 10:40 AM
To: roy.k.larsen@intel.com
Cc: ipoverib@ietf.org
Subject: Re: [Ipoverib] DHCP over InfiniBand



> I still believe the language is somewhat ambiguous when one considers the
> last paragraph of 2.1 followed by the first paragraph of 2.1.1 followed by
> the last paragraph of 2.1.1.  It leaves the reader asking what is the
> purpose of section 2.1.1. if the actual format of the client-identifier
does
> not affect IPoIB conformance.  Nevertheless, I can accept it as it is if
you
> see if differently.

I see your point. I have added some detail to make things clear. 


Modified excerpt:

----------------------------------

        The DHCP protocol states that the 'client identifier' option
        may be used as the unique identifying value for the client.
        This value must be unique within the subnet the client is a
        member of. 
        
        The client identifier option includes a type and identifier
        pair. The identifier included in the client-identifier option
        may consist of a hardware address or any other unique value
        such as the DNS name of the client. When a hardware address is
        used, the type field should be one of the ARP hardware types
        listed in ARPPARAM. A type of 0 (zero) should be used when the
        included identifier is other than a hardware address.
        [RFC2132] 

        The client-identifier itself SHOULD not be interpreted by the
        server. [RFC2132] 

2.1 IPoIB specific usage of DHCP message fields

        A DHCP client, when working over an IPoIB interface, MUST
        follow the following rules:

	'htype' (hardware address type) MUST be 32 [ARPPARAM]

	'hlen' (hardware address length) MUST be 0.

        The 'chaddr' (client hardware address) field MUST be zeroed.
        The server and the relay agent MUST ignore 'chaddr' on
        receipt. 

        The 'client identifier' option MUST be used in DHCP messages.
        The client-identifier may consist of any data.
	
        IPoIB clients SHOULD use the following format for the
        client-identifier option:


    Code  Len   Type  Client-Identifier
   +-----+-----+-----+-----+-----+-----+-----+-------------------....----+
   |  61 | 21  |  32 |Interface-id (4 bytes) |   GID (16 bytes)          |
   +-----+-----+-----+-----+-----+-----+-----+-------------------....----+

        Every IPoIB interface is associated with an identifier
        referred to as the GID [IPoIB_ARCH]. The GID is unique in the
        InfiniBand fabric. An invariant GID is formed by appending the
        port's EUI-64 identifier to the InfiniBand subnet prefix.

        The GID is associated with a particular hardware port. The GID
        and a QPN define an IPoIB interface at the port. Therefore an
        implementation may associate multiple IPoIB interfaces on the
        same port. It is up to the implementation to ensure a unique
        client-identifier when multiple IPoIB interfaces are defined
        over the same port and same GID. A unique, invariant
        'interface-id' value be included in addition to the GID to
        achieve this.

        Note: a port may be associated with multiple GIDs. Therefore,
        multiple IPoIB interfaces may exist on the same port while
        using a different GID from among the GIDs associated with the
        port. 

        A unique interface-id may be formed by including the QPN
        associated with the relevant IPoIB interface if the
        implementation is designed to keep this association constant
        across boots. A timestamp or some other value unique to the
        implementation may also be used for the same purpose.

        If there is only one IPoIB interface associated with a
        particular GID, then use of the GID with the 'interface-id'
        zeroed is sufficient. By default, an implementation zeroes out
        the interface-id field in the client identifier described
        above. 

        This document does not preclude the use of other 'client
        identifier' type, such as fully qualified domain name(FQDN) or
        the EUI-64 value associated with the interface.


-------------------------------------------------------------------

Vivek

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr  4 16:08:40 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA08842
	for <ipoverib-archive@odin.ietf.org>; Thu, 4 Apr 2002 16:08:40 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA02380
	for ipoverib-archive@odin.ietf.org; Thu, 4 Apr 2002 16:08:43 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA02261;
	Thu, 4 Apr 2002 16:03:55 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA02229
	for <ipoverib@optimus.ietf.org>; Thu, 4 Apr 2002 16:03:53 -0500 (EST)
Received: from e21.esmtp.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA08733;
	Thu, 4 Apr 2002 16:03:47 -0500 (EST)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.esmtp.ibm.com (8.12.2/8.12.2) with ESMTP id g34L0N1H024440;
	Thu, 4 Apr 2002 15:00:23 -0600
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.00) with ESMTP id g34L3mT65036;
	Thu, 4 Apr 2002 16:03:48 -0500
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g34L3mX32218;
	Thu, 4 Apr 2002 13:03:48 -0800
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g34L3l123636;
	Thu, 4 Apr 2002 13:03:47 -0800 (PST)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204042103.g34L3l123636@eng4.beaverton.ibm.com>
To: internet-drafts@ietf.org
Date: Thu, 4 Apr 2002 13:03:46 -0800 (PST)
Cc: ipoverib@ietf.org
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Subject: [Ipoverib] dhcp over infiniband
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit









INTERNET DRAFT                                                    Vivek Kashyap
<draft-ietf-ipoib-dhcp-over-infiniband-01.txt>                    IBM
Expiration Date: October 2002                                     April 2002


                        DHCP over InfiniBand

Status of this memo

        This document is an Internet-Draft and is in full conformance
        with all provisions of Section 10 of RFC 2026.

        Internet-Drafts are working documents of the Internet
        Engineering Task Force (IETF), its areas, and its working
        groups. Note that other groups may also distribute working
        documents as Internet- Drafts.

        Internet-Drafts are draft documents valid for a maximum of six
        months and may be updated, replaced, or obsoleted by other
        documents at any time. It is inappropriate to use
        Internet-Drafts as Reference material or to cite them other
        than as ``work in progress''.

        The list of current Internet-Drafts can be accessed at
        http://www.ietf.org/ietf/1id-abstracts.txt

        The list of Internet-Draft Shadow Directories can be accessed
        at http://www.ietf.org/shadow.html

        This memo provides information for the Internet community.
        This memo does not specify an Internet standard of any kind.
        Distribution of this memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2001).  All Rights Reserved.

Abstract

        An InfiniBand network uses a link-layer addressing scheme that
        is 20-bytes long. This is larger than the 16-bytes reserved for
        the hardware address in DHCP/BOOTP message. The above
        inequality imposes restrictions on the use of the DHCP message
        fields when used over an IP over InfiniBand(IPoIB) network.
        This document describes the use of DHCP message fields when
        implementing DHCP over IPoIB.




Kashyap                                                         [Page 1]





INTERNET-DRAFT            DHCP over InfiniBand                April 2002


1. Introduction

        The Dynamic Host Configuration Protocol(DHCP) provides a
        framework for passing configuration information to hosts on a
        TCP/IP network [RFC2131]. DHCP is based on the Bootstrap
        Protocol (BOOTP) [RFC951] adding the capability of automatic
        allocation of reusable network addresses and additional
        configuration options [RFC2131,RFC2132].

        The DHCP server receives a broadcast request from the DHCP
        client. The DHCP server uses the client interface's
        hardware-address to unicast a reply back when the client
        doesn't yet have an IP address assigned to it. The 'chaddr'
        field in the DHCP message carries the client's hardware
        address.

        The 'chaddr' field is 16-bytes in length. The IPoIB link-layer
        address is 20-bytes in length. Therefore the IPoIB link-layer
        address will not fit in the 'chaddr' field making it
        impossible for the DHCP server to unicast a reply back to the
        client.

        To ensure interoperability the usage of the fields and the
        method for DHCP interaction must be clarified. This document
        describes the IPoIB specific usage of some fields of DHCP. See
        [RFC2131] for the mechanism of DHCP and the explanations of
        each field.

        The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
        NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
        "OPTIONAL" in this document are to be interpreted as described
        in RFC 2119 [RFC2119].

2. The DHCP over IPoIB mechanism

        As noted above, because of the link-layer address length being
        larger than the 'chaddr' field length the link-layer address
        is unavailable to the DHCP server. Therefore, a DHCP client
        MUST request that the server sends a broadcast reply by
        setting the BROADCAST flag when IPoIB ARP is not possible i.e.
        in situations where the client does not know its IP address.

        RFC1542 notes that the use of a broadcast reply is discouraged
        but in the case of IPoIB this is a necessity. There is no
        option but to broadcast back to the client since it is not
        possible to reply the client's unicast address. To
        desynchronise broadcasts at subnet startup, the RFC2131
        suggests that a client wait a random time (1 to 10 seconds)



Kashyap                                                         [Page 2]





INTERNET-DRAFT            DHCP over InfiniBand                April 2002


        before initiating server discovery. The same timeout will
        equally spread out the DHCP server broadcast responses
        generated due to the use of the use of the BROADCAST bit.

        The client hardware address, 'chaddr', is unique in the subnet
        and hence can be used to identify the client interface. But in
        the absence of a unique chaddr the client-identifier must be
        used.

        The DHCP protocol states that the 'client identifier' option
        may be used as the unique identifying value for the client.
        This value must be unique within the subnet the client is a
        member of.

        The client identifier option includes a type and identifier
        pair. The identifier included in the client-identifier option
        may consist of a hardware address or any other unique value
        such as the DNS name of the client. When a hardware address is
        used, the type field should be one of the ARP hardware types
        listed in [ARPPARAM]. A type of 0 (zero) should be used when
        the included identifier is other than a hardware address.
        [RFC2132]

        The client-identifier itself SHOULD not be interpreted by the
        server. [RFC2132]

2.1 IPoIB specific usage of DHCP message fields

        A DHCP client, when working over an IPoIB interface, MUST
        follow the following rules:

        'htype' (hardware address type) MUST be 32 [ARPPARAM]

        'hlen' (hardware address length) MUST be 0.

        The 'chaddr' (client hardware address) field MUST be zeroed.
        The server and the relay agent MUST ignore 'chaddr' on
        receipt.

        The 'client identifier' option MUST be used in DHCP messages.
        'client identifier' option MAY consist of any data.










Kashyap                                                         [Page 3]





INTERNET-DRAFT            DHCP over InfiniBand                April 2002



        IPoIB clients SHOULD use the following format for the
        client-identifier option:

    Code  Len   Type  Client-Identifier
   +-----+-----+-----+-----+-----+-----+-----+-------------------....----+
   |  61 | 21  |  32 |Interface-id (4 bytes) |   GID (16 bytes)          |
   +-----+-----+-----+-----+-----+-----+-----+-------------------....----+

        Every IPoIB interface is associated with an identifier
        referred to as the GID [IPoIB_ARCH]. The GID is unique in the
        InfiniBand fabric. An invariant GID is formed by appending the
        port's EUI-64 identifier to the InfiniBand subnet prefix.

        The GID is associated with a particular hardware port. The GID
        and a QPN define an IPoIB interface at the port. Therefore an
        implementation may associate multiple IPoIB interfaces on the
        same port. It is up to the implementation to ensure a unique
        client-identifier when multiple IPoIB interfaces are defined
        over the same port and same GID. A unique, invariant
        'interface-id' value be included in addition to the GID to
        achieve this.

        Note: a port may be associated with multiple GIDs. Therefore,
        multiple IPoIB interfaces may exist on the same port while
        using a different GID from among the GIDs associated with the
        port.

        A unique interface-id may be formed by including the QPN
        associated with the relevant IPoIB interface if the
        implementation is designed to keep this association constant
        across boots. A timestamp or some other value unique to the
        implementation may also be used for the same purpose.

        If there is only one IPoIB interface associated with a
        particular GID, then use of the GID with the 'interface-id'
        zeroed is sufficient. By default, an implementation zeroes out
        the interface-id field in the client identifier described
        above.

        This document does not preclude the use of other 'client
        identifier' type, such as fully qualified domain name(FQDN) or
        the EUI-64 value associated with the interface.

2.2 Use of the BROADCAST flag

        A DHCP client on IPoIB SHOULD set a BROADCAST flag in
        DHCPDISCOVER and DHCPREQUEST messages (and set 'ciaddr' to



Kashyap                                                         [Page 4]





INTERNET-DRAFT            DHCP over InfiniBand                April 2002


        zero) to ensure that the server (or the relay agent)
        broadcasts its reply to the client.

        Note: As described in [RFC2131], 'ciaddr' MUST be filled in
        with client's IP address during BOUND, RENEWING or REBINDING
        state, therefore, the BROADCAST flag MUST NOT be set. In these
        cases, the DHCP server unicasts DHCPACK message to the address
        in 'ciaddr'. The link address will be resolved by IPoIB ARP.

3. Security Considerations

        DHCP currently provides no authentication or security
        mechanisms. Potential exposures to attack are discussed
        in section 7 of the DHCP protocol specification [RFC2131].

        A malicious client can falsify the client-identifier,
        thus masquerading as another client.

4. Acknowledgement

        This document borrows extensively from [RFC 2855]. Roy Larsen
        pointed out the length discrepancy between the IPoIB link
        address and DHCP's chaddr field.

References

        [RFC2119]       Key words for use in RFCs to Indicate Requirement Levels
                        S, Bradner

        [RFC2131]       Dynamic Host Configuration Protocol, R. Droms

        [RFC2132]       DHCP Options and BOOTP Vendor Extensions,
                        S. Alexander, R. Droms

        [RFC951]        Bootstrap Protocol, B. Croft, J. Gilmore

        [RFC1542]       Clarifications and Extensions for the Bootstrap Protocol
                        W. Wimer

        [ARPPARAM]      http://www.iana.org/numbers.html

        [RFC2855]       DHCP for IEEE 1394, K. Fujisawa

        [IPoIB_ARCH]    draft-ietf-ipoib-architecture-01.txt, V. Kashyap

        [IPoIB_ENCAP]   draft-ietf-ipoib-ip-over-infiniband-00.txt,
                        V. Kashyap, H.K. Jerry Chu




Kashyap                                                         [Page 5]





INTERNET-DRAFT            DHCP over InfiniBand                April 2002


Author's Address

   Vivek Kashyap

   IBM
   15450, SW Koll Parkway
   Beaverton
   OR 97006

   Phone: +1 503 578 3422
   EMail: vivk@us.ibm.com

Full Copyright Statement

   Copyright (C) The Internet Society (2000).  All Rights Reserved.

   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph are
   included on all such copies and derivative works.  However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.












Kashyap                                                         [Page 6]



_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr  4 16:53:23 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA10176
	for <ipoverib-archive@odin.ietf.org>; Thu, 4 Apr 2002 16:53:22 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA04928
	for ipoverib-archive@odin.ietf.org; Thu, 4 Apr 2002 16:53:26 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA04470;
	Thu, 4 Apr 2002 16:40:36 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA04407
	for <ipoverib@optimus.ietf.org>; Thu, 4 Apr 2002 16:40:32 -0500 (EST)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA09918
	for <ipoverib@ietf.org>; Thu, 4 Apr 2002 16:40:27 -0500 (EST)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g34Lb4jH107764
	for <ipoverib@ietf.org>; Thu, 4 Apr 2002 15:37:04 -0600
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.00) with ESMTP id g34LeTT78010
	for <ipoverib@ietf.org>; Thu, 4 Apr 2002 16:40:29 -0500
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g34LeSX00821
	for <ipoverib@ietf.org>; Thu, 4 Apr 2002 13:40:28 -0800
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g34LeRL24125
	for ipoverib@ietf.org; Thu, 4 Apr 2002 13:40:27 -0800 (PST)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204042140.g34LeRL24125@eng4.beaverton.ibm.com>
Subject: [Ipoverib] dhcp over infiniband 
To: ipoverib@ietf.org
Date: Thu, 4 Apr 2002 13:40:26 -0800 (PST)
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

Folks,
        I just realised that I cc'd ipoverib(habit!) when submitting
        the draft to internet-drafts@ietf.org. So if/when replying to
        the earlier mail on the dhcp-over-infiniband-01.txt please
        ensure you don't cc internet-drafts.

Vivek

----------------------------------------------------------------




INTERNET DRAFT                                                    Vivek Kashyap
<draft-ietf-ipoib-dhcp-over-infiniband-01.txt>                    IBM
Expiration Date: October 2002                                     April 2002


                        DHCP over InfiniBand

Status of this memo

        This document is an Internet-Draft and is in full conformance
        with all provisions of Section 10 of RFC 2026.

        Internet-Drafts are working documents of the Internet
        Engineering Task Force (IETF), its areas, and its working
        groups. Note that other groups may also distribute working
        documents as Internet- Drafts.

        Internet-Drafts are draft documents valid for a maximum of six
        months and may be updated, replaced, or obsoleted by other
        documents at any time. It is inappropriate to use
        Internet-Drafts as Reference material or to cite them other
        than as ``work in progress''.

        The list of current Internet-Drafts can be accessed at
        http://www.ietf.org/ietf/1id-abstracts.txt

        The list of Internet-Draft Shadow Directories can be accessed
        at http://www.ietf.org/shadow.html

        This memo provides information for the Internet community.
        This memo does not specify an Internet standard of any kind.
        Distribution of this memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2001).  All Rights Reserved.

Abstract

        An InfiniBand network uses a link-layer addressing scheme that
        is 20-bytes long. This is larger than the 16-bytes reserved for
        the hardware address in DHCP/BOOTP message. The above
        inequality imposes restrictions on the use of the DHCP message
        fields when used over an IP over InfiniBand(IPoIB) network.
        This document describes the use of DHCP message fields when
        implementing DHCP over IPoIB.




Kashyap                                                         [Page 1]





INTERNET-DRAFT            DHCP over InfiniBand                April 2002


1. Introduction

        The Dynamic Host Configuration Protocol(DHCP) provides a
        framework for passing configuration information to hosts on a
        TCP/IP network [RFC2131]. DHCP is based on the Bootstrap
        Protocol (BOOTP) [RFC951] adding the capability of automatic
        allocation of reusable network addresses and additional
        configuration options [RFC2131,RFC2132].

        The DHCP server receives a broadcast request from the DHCP
        client. The DHCP server uses the client interface's
        hardware-address to unicast a reply back when the client
        doesn't yet have an IP address assigned to it. The 'chaddr'
        field in the DHCP message carries the client's hardware
        address.

        The 'chaddr' field is 16-bytes in length. The IPoIB link-layer
        address is 20-bytes in length. Therefore the IPoIB link-layer
        address will not fit in the 'chaddr' field making it
        impossible for the DHCP server to unicast a reply back to the
        client.

        To ensure interoperability the usage of the fields and the
        method for DHCP interaction must be clarified. This document
        describes the IPoIB specific usage of some fields of DHCP. See
        [RFC2131] for the mechanism of DHCP and the explanations of
        each field.

        The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
        NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
        "OPTIONAL" in this document are to be interpreted as described
        in RFC 2119 [RFC2119].

2. The DHCP over IPoIB mechanism

        As noted above, because of the link-layer address length being
        larger than the 'chaddr' field length the link-layer address
        is unavailable to the DHCP server. Therefore, a DHCP client
        MUST request that the server sends a broadcast reply by
        setting the BROADCAST flag when IPoIB ARP is not possible i.e.
        in situations where the client does not know its IP address.

        RFC1542 notes that the use of a broadcast reply is discouraged
        but in the case of IPoIB this is a necessity. There is no
        option but to broadcast back to the client since it is not
        possible to reply the client's unicast address. To
        desynchronise broadcasts at subnet startup, the RFC2131
        suggests that a client wait a random time (1 to 10 seconds)



Kashyap                                                         [Page 2]





INTERNET-DRAFT            DHCP over InfiniBand                April 2002


        before initiating server discovery. The same timeout will
        equally spread out the DHCP server broadcast responses
        generated due to the use of the use of the BROADCAST bit.

        The client hardware address, 'chaddr', is unique in the subnet
        and hence can be used to identify the client interface. But in
        the absence of a unique chaddr the client-identifier must be
        used.

        The DHCP protocol states that the 'client identifier' option
        may be used as the unique identifying value for the client.
        This value must be unique within the subnet the client is a
        member of.

        The client identifier option includes a type and identifier
        pair. The identifier included in the client-identifier option
        may consist of a hardware address or any other unique value
        such as the DNS name of the client. When a hardware address is
        used, the type field should be one of the ARP hardware types
        listed in [ARPPARAM]. A type of 0 (zero) should be used when
        the included identifier is other than a hardware address.
        [RFC2132]

        The client-identifier itself SHOULD not be interpreted by the
        server. [RFC2132]

2.1 IPoIB specific usage of DHCP message fields

        A DHCP client, when working over an IPoIB interface, MUST
        follow the following rules:

        'htype' (hardware address type) MUST be 32 [ARPPARAM]

        'hlen' (hardware address length) MUST be 0.

        The 'chaddr' (client hardware address) field MUST be zeroed.
        The server and the relay agent MUST ignore 'chaddr' on
        receipt.

        The 'client identifier' option MUST be used in DHCP messages.
        'client identifier' option MAY consist of any data.










Kashyap                                                         [Page 3]





INTERNET-DRAFT            DHCP over InfiniBand                April 2002



        IPoIB clients SHOULD use the following format for the
        client-identifier option:

    Code  Len   Type  Client-Identifier
   +-----+-----+-----+-----+-----+-----+-----+-------------------....----+
   |  61 | 21  |  32 |Interface-id (4 bytes) |   GID (16 bytes)          |
   +-----+-----+-----+-----+-----+-----+-----+-------------------....----+

        Every IPoIB interface is associated with an identifier
        referred to as the GID [IPoIB_ARCH]. The GID is unique in the
        InfiniBand fabric. An invariant GID is formed by appending the
        port's EUI-64 identifier to the InfiniBand subnet prefix.

        The GID is associated with a particular hardware port. The GID
        and a QPN define an IPoIB interface at the port. Therefore an
        implementation may associate multiple IPoIB interfaces on the
        same port. It is up to the implementation to ensure a unique
        client-identifier when multiple IPoIB interfaces are defined
        over the same port and same GID. A unique, invariant
        'interface-id' value be included in addition to the GID to
        achieve this.

        Note: a port may be associated with multiple GIDs. Therefore,
        multiple IPoIB interfaces may exist on the same port while
        using a different GID from among the GIDs associated with the
        port.

        A unique interface-id may be formed by including the QPN
        associated with the relevant IPoIB interface if the
        implementation is designed to keep this association constant
        across boots. A timestamp or some other value unique to the
        implementation may also be used for the same purpose.

        If there is only one IPoIB interface associated with a
        particular GID, then use of the GID with the 'interface-id'
        zeroed is sufficient. By default, an implementation zeroes out
        the interface-id field in the client identifier described
        above.

        This document does not preclude the use of other 'client
        identifier' type, such as fully qualified domain name(FQDN) or
        the EUI-64 value associated with the interface.

2.2 Use of the BROADCAST flag

        A DHCP client on IPoIB SHOULD set a BROADCAST flag in
        DHCPDISCOVER and DHCPREQUEST messages (and set 'ciaddr' to



Kashyap                                                         [Page 4]





INTERNET-DRAFT            DHCP over InfiniBand                April 2002


        zero) to ensure that the server (or the relay agent)
        broadcasts its reply to the client.

        Note: As described in [RFC2131], 'ciaddr' MUST be filled in
        with client's IP address during BOUND, RENEWING or REBINDING
        state, therefore, the BROADCAST flag MUST NOT be set. In these
        cases, the DHCP server unicasts DHCPACK message to the address
        in 'ciaddr'. The link address will be resolved by IPoIB ARP.

3. Security Considerations

        DHCP currently provides no authentication or security
        mechanisms. Potential exposures to attack are discussed
        in section 7 of the DHCP protocol specification [RFC2131].

        A malicious client can falsify the client-identifier,
        thus masquerading as another client.

4. Acknowledgement

        This document borrows extensively from [RFC 2855]. Roy Larsen
        pointed out the length discrepancy between the IPoIB link
        address and DHCP's chaddr field.

References

        [RFC2119]       Key words for use in RFCs to Indicate Requirement Levels
                        S, Bradner

        [RFC2131]       Dynamic Host Configuration Protocol, R. Droms

        [RFC2132]       DHCP Options and BOOTP Vendor Extensions,
                        S. Alexander, R. Droms

        [RFC951]        Bootstrap Protocol, B. Croft, J. Gilmore

        [RFC1542]       Clarifications and Extensions for the Bootstrap Protocol
                        W. Wimer

        [ARPPARAM]      http://www.iana.org/numbers.html

        [RFC2855]       DHCP for IEEE 1394, K. Fujisawa

        [IPoIB_ARCH]    draft-ietf-ipoib-architecture-01.txt, V. Kashyap

        [IPoIB_ENCAP]   draft-ietf-ipoib-ip-over-infiniband-00.txt,
                        V. Kashyap, H.K. Jerry Chu




Kashyap                                                         [Page 5]





INTERNET-DRAFT            DHCP over InfiniBand                April 2002


Author's Address

   Vivek Kashyap

   IBM
   15450, SW Koll Parkway
   Beaverton
   OR 97006

   Phone: +1 503 578 3422
   EMail: vivk@us.ibm.com

Full Copyright Statement

   Copyright (C) The Internet Society (2000).  All Rights Reserved.

   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph are
   included on all such copies and derivative works.  However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.












Kashyap                                                         [Page 6]



_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr  4 21:46:40 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA16421
	for <ipoverib-archive@odin.ietf.org>; Thu, 4 Apr 2002 21:46:40 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id VAA20266
	for ipoverib-archive@odin.ietf.org; Thu, 4 Apr 2002 21:46:42 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id VAA20132;
	Thu, 4 Apr 2002 21:42:31 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id HAA24132
	for <ipoverib@optimus.ietf.org>; Thu, 4 Apr 2002 07:00:58 -0500 (EST)
Received: from CNRI.Reston.VA.US (localhost [127.0.0.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA10421;
	Thu, 4 Apr 2002 07:00:55 -0500 (EST)
Message-Id: <200204041200.HAA10421@ietf.org>
Mime-Version: 1.0
Content-Type: Multipart/Mixed; Boundary="NextPart"
To: IETF-Announce: ;
Cc: ipoverib@ietf.org
From: Internet-Drafts@ietf.org
Reply-to: Internet-Drafts@ietf.org
Date: Thu, 04 Apr 2002 07:00:55 -0500
Subject: [Ipoverib] I-D ACTION:draft-ietf-ipoib-channel-adapter-mib-00.txt
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the IP over InfiniBand Working Group of the IETF.

	Title		: Definitions of Managed Objects for Infiniband Channel 
                          Adapters (CA)
	Author(s)	: S. Harnedy
	Filename	: draft-ietf-ipoib-channel-adapter-mib-00.txt
	Pages		: 19
	Date		: 02-Apr-02
	
This memo defines a portion of the Management Information Base (MIB) 
for use with network management protocols in the Internet community.  
In particular, it defines objects for managing InfiniBand Channel 
Adapters (CA).

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-ipoib-channel-adapter-mib-00.txt

To remove yourself from the IETF Announcement list, send a message to 
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-ipoib-channel-adapter-mib-00.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt


Internet-Drafts can also be obtained by e-mail.

Send a message to:
	mailserv@ietf.org.
In the body type:
	"FILE /internet-drafts/draft-ietf-ipoib-channel-adapter-mib-00.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.
		
		
Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="mailserv@ietf.org"

Content-Type: text/plain
Content-ID:	<20020402125250.I-D@ietf.org>

ENCODING mime
FILE /internet-drafts/draft-ietf-ipoib-channel-adapter-mib-00.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-ipoib-channel-adapter-mib-00.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<20020402125250.I-D@ietf.org>

--OtherAccess--

--NextPart--




_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Sat Apr  6 15:24:46 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA15014
	for <ipoverib-archive@odin.ietf.org>; Sat, 6 Apr 2002 15:24:46 -0500 (EST)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA02476
	for ipoverib-archive@odin.ietf.org; Sat, 6 Apr 2002 15:24:49 -0500 (EST)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA02299;
	Sat, 6 Apr 2002 15:14:18 -0500 (EST)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA02269
	for <ipoverib@ns.ietf.org>; Sat, 6 Apr 2002 15:14:16 -0500 (EST)
Received: from kathmandu.sun.com (kathmandu.sun.com [192.18.98.36])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14874
	for <ipoverib@ietf.org>; Sat, 6 Apr 2002 15:14:12 -0500 (EST)
Received: from sunmail1.Sun.COM ([129.145.1.2])
	by kathmandu.sun.com (8.9.3+Sun/8.9.3) with ESMTP id NAA17257;
	Sat, 6 Apr 2002 13:14:14 -0700 (MST)
Received: from jurassic.eng.sun.com (jurassic.Eng.Sun.COM [129.146.87.31])
	by sunmail1.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1-Sun.COM.mod.2) with ESMTP id MAA07904;
	Sat, 6 Apr 2002 12:14:54 -0800 (PST)
Received: from sweethome (dsl-194-138.Eng.Sun.COM [129.146.194.138])
	by jurassic.eng.sun.com (8.12.2+Sun/8.12.2) with SMTP id g36KE8xf159915;
	Sat, 6 Apr 2002 12:14:13 -0800 (PST)
Message-Id: <200204062014.g36KE8xf159915@jurassic.eng.sun.com>
Date: Sat, 6 Apr 2002 12:15:22 -0800 (PST)
From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Reply-To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
To: ipoverib@ietf.org
Cc: bill@strahm.net
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: u8in9Sq4ww3pAVDBKAA13A==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.2 SunOS 5.8 sun4u sparc 
Subject: [Ipoverib] 4/9 deadline for a consensus call
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At the Minneapolis/IETF meeting we had a "room concensus" on the
following:

1. Embedding EtherType in the IPoIB UD payload.
2. Requiring IPv4 over IB to have a minimal MTU of 1500.
(This effectively rules out all the IB MTUs less than 2K for
running IPoIB.)
3. Using DHCP broadcast flag to solve the chaddr length problem
so the IPoIB hardware address stays as QPN+GID.

During the past two weeks since the meeting minutes was published
I have not heard of any objection or different opinion. The DHCPv4
over IB work seems to be going well too. So I'm inclined to declare
the above as WG concensus. But I'm willing to wait til next Tuesday
(4/9) COW for anyone to voice a different opinion. If you do decide
to object, I'd urge you to bring forward a strong case, e.g. new,
convincing evidence that shows the above is really problematic. We
have spent enough time bashing these topics it's time for us to
move on.
 
Your friendly co-chair


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr  8 11:14:20 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA26523
	for <ipoverib-archive@odin.ietf.org>; Mon, 8 Apr 2002 11:14:20 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id LAA13174
	for ipoverib-archive@odin.ietf.org; Mon, 8 Apr 2002 11:14:22 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id LAA12871;
	Mon, 8 Apr 2002 11:08:53 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id LAA12834
	for <ipoverib@ns.ietf.org>; Mon, 8 Apr 2002 11:08:49 -0400 (EDT)
Received: from palrel13.hp.com (palrel13.hp.com [156.153.255.238])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA26404
	for <ipoverib@ietf.org>; Mon, 8 Apr 2002 11:08:46 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel13.hp.com (Postfix) with ESMTP
	id 770034005D2; Mon,  8 Apr 2002 08:07:39 -0700 (PDT)
Received: from mkrause.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id IAA20425;
	Mon, 8 Apr 2002 08:07:38 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020408072042.02b16d90@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Mon, 08 Apr 2002 07:22:01 -0700
To: ipoverib@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
Cc: bill@strahm.net
In-Reply-To: <200204062014.g36KE8xf159915@jurassic.eng.sun.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 12:15 PM 4/6/2002 -0800, H.K. Jerry Chu wrote:
>At the Minneapolis/IETF meeting we had a "room concensus" on the
>following:
>
>1. Embedding EtherType in the IPoIB UD payload.

I've seen strong consensus on the reflector that indicates it should not be 
used.  I don't want to resurface all of the arguments but there were more 
against this from what I could see.

>2. Requiring IPv4 over IB to have a minimal MTU of 1500.
>(This effectively rules out all the IB MTUs less than 2K for
>running IPoIB.)

Clarified to be for IB UD transport service.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr  8 13:34:37 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA00641
	for <ipoverib-archive@odin.ietf.org>; Mon, 8 Apr 2002 13:34:37 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id NAA24107
	for ipoverib-archive@odin.ietf.org; Mon, 8 Apr 2002 13:34:39 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id NAA23179;
	Mon, 8 Apr 2002 13:24:10 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id NAA23148
	for <ipoverib@ns.ietf.org>; Mon, 8 Apr 2002 13:24:03 -0400 (EDT)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA00383
	for <ipoverib@ietf.org>; Mon, 8 Apr 2002 13:23:53 -0400 (EDT)
Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.37 2002/04/01 17:50:55 root Exp $) with ESMTP id g38HNDX18749
	for <ipoverib@ietf.org>; Mon, 8 Apr 2002 17:23:14 GMT
Received: from fmsmsxvs042.fm.intel.com (fmsmsxv042-1.fm.intel.com [132.233.48.110])
	by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.15 2002/04/01 17:51:48 root Exp $) with SMTP id g38HP5N04833
	for <ipoverib@ietf.org>; Mon, 8 Apr 2002 17:25:06 GMT
Received: from fmsmsx26.fm.intel.com ([132.233.42.26])
 by fmsmsxvs042.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002040810272810958
 ; Mon, 08 Apr 2002 10:27:28 -0700
Received: by fmsmsx26.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2HL0BYL2>; Mon, 8 Apr 2002 10:23:37 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F66@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Michael Krause'" <krause@cup.hp.com>, ipoverib@ietf.org
Cc: bill@strahm.net
Subject: RE: [Ipoverib] 4/9 deadline for a consensus call
Date: Mon, 8 Apr 2002 10:23:32 -0700 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

I recently went through the old mails on the EtherType issue and found that
of those folks expressing an opinion, more wanted or were willing to accept
the encapsulation field than were dead set against it.  There certainly
wasn't a strong consensus against.  

Roy

-----Original Message-----
From: Michael Krause [mailto:krause@cup.hp.com]
Sent: Monday, April 08, 2002 7:22 AM
To: ipoverib@ietf.org
Cc: bill@strahm.net
Subject: Re: [Ipoverib] 4/9 deadline for a consensus call


At 12:15 PM 4/6/2002 -0800, H.K. Jerry Chu wrote:
>At the Minneapolis/IETF meeting we had a "room concensus" on the
>following:
>
>1. Embedding EtherType in the IPoIB UD payload.

I've seen strong consensus on the reflector that indicates it should not be 
used.  I don't want to resurface all of the arguments but there were more 
against this from what I could see.

>2. Requiring IPv4 over IB to have a minimal MTU of 1500.
>(This effectively rules out all the IB MTUs less than 2K for
>running IPoIB.)

Clarified to be for IB UD transport service.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr  8 14:14:14 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01713
	for <ipoverib-archive@odin.ietf.org>; Mon, 8 Apr 2002 14:14:14 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA26827
	for ipoverib-archive@odin.ietf.org; Mon, 8 Apr 2002 14:14:16 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA26231;
	Mon, 8 Apr 2002 14:03:30 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA26193
	for <ipoverib@ns.ietf.org>; Mon, 8 Apr 2002 14:03:28 -0400 (EDT)
Received: from devserv.devel.redhat.com (nat-pool-rdu.redhat.com [66.187.233.200])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01399
	for <ipoverib@ietf.org>; Mon, 8 Apr 2002 14:03:25 -0400 (EDT)
Received: (from zaitcev@localhost)
	by devserv.devel.redhat.com (8.11.6/8.11.0) id g38I0YG28578;
	Mon, 8 Apr 2002 14:00:34 -0400
Date: Mon, 8 Apr 2002 14:00:34 -0400
From: Pete Zaitcev <zaitcev@redhat.com>
To: "Larsen, Roy K" <roy.k.larsen@intel.com>
Cc: "'Michael Krause'" <krause@cup.hp.com>, ipoverib@ietf.org, bill@strahm.net
Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
Message-ID: <20020408140034.A24355@devserv.devel.redhat.com>
References: <D9223EB959A5D511A98F00508B68C20C06BC4F66@ORSMSX108>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4F66@ORSMSX108>; from roy.k.larsen@intel.com on Mon, Apr 08, 2002 at 10:23:32AM -0700
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

I do not know if that would count, but I am against that,
because to make a lower level driver to poke into something that
belongs to upper levels seems a little retarded to me. My gut
feeling (e.g. distilled expirience), suggests that such choice
may also result in a performance hit down the road. Or may be not,
we'll see.

Since I am NOT a member of the working group, but a mere implemntor
of what was designed by smarter people (like you), I kept quiet.
I am going to write some code and see if this works well.
Unfortunately, it may be a little too late to change standards
by that time...

-- Pete

> From: "Larsen, Roy K" <roy.k.larsen@intel.com>
> Date: Mon, 8 Apr 2002 10:23:32 -0700 

> I recently went through the old mails on the EtherType issue and found that
> of those folks expressing an opinion, more wanted or were willing to accept
> the encapsulation field than were dead set against it.  There certainly
> wasn't a strong consensus against.  
> 
> Roy
> 
> -----Original Message-----
> From: Michael Krause [mailto:krause@cup.hp.com]
> Sent: Monday, April 08, 2002 7:22 AM
> To: ipoverib@ietf.org
> Cc: bill@strahm.net
> Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
> 
> 
> At 12:15 PM 4/6/2002 -0800, H.K. Jerry Chu wrote:
> >At the Minneapolis/IETF meeting we had a "room concensus" on the
> >following:
> >
> >1. Embedding EtherType in the IPoIB UD payload.
> 
> I've seen strong consensus on the reflector that indicates it should not be 
> used.  I don't want to resurface all of the arguments but there were more 
> against this from what I could see.
> 
> >2. Requiring IPv4 over IB to have a minimal MTU of 1500.
> >(This effectively rules out all the IB MTUs less than 2K for
> >running IPoIB.)
> 
> Clarified to be for IB UD transport service.
> 
> Mike

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr  8 14:35:16 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA02111
	for <ipoverib-archive@odin.ietf.org>; Mon, 8 Apr 2002 14:35:14 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA28509
	for ipoverib-archive@odin.ietf.org; Mon, 8 Apr 2002 14:35:17 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA27201;
	Mon, 8 Apr 2002 14:21:03 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA27167
	for <ipoverib@ns.ietf.org>; Mon, 8 Apr 2002 14:21:00 -0400 (EDT)
Received: from strahm.net (adsl-67-115-29-109.dsl.sndg02.pacbell.net [67.115.29.109])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01824
	for <ipoverib@ietf.org>; Mon, 8 Apr 2002 14:20:57 -0400 (EDT)
Received: (from bill@localhost)
	by strahm.net (8.11.6/8.11.6) id g38IJ1368516;
	Mon, 8 Apr 2002 11:19:01 -0700 (PDT)
	(envelope-from bill)
Date: Mon, 8 Apr 2002 11:19:00 -0700
From: Bill Strahm <bill@strahm.net>
To: Pete Zaitcev <zaitcev@redhat.com>
Cc: "Larsen, Roy K" <roy.k.larsen@intel.com>,
        "'Michael Krause'" <krause@cup.hp.com>, ipoverib@ietf.org,
        bill@strahm.net
Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
Message-ID: <20020408111900.A68455@homegate.strahm.net>
References: <D9223EB959A5D511A98F00508B68C20C06BC4F66@ORSMSX108> <20020408140034.A24355@devserv.devel.redhat.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <20020408140034.A24355@devserv.devel.redhat.com>; from zaitcev@redhat.com on Mon, Apr 08, 2002 at 02:00:34PM -0400
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Just on a note from the WG chair...

By definition you ARE a member of the working group by subscribing
to the mailing list and offering the opinion below...

I am very interested in hearing what implementors have to say, I am
of the opinion that including Ethertype in the payload of the packet
is not the right thing to do, however, based on hearing some 
implementation reports from people who actually ARE doing the work, I
am very close to changing my mind again.  

Can you provide a quick summary on your position (I think I read that
you are against the Ethertype field, but I could be wrong) and especially
any technical reasons for that opinion, such as I can/can't do this
if the field is/isn't there...

Bill
On Mon, Apr 08, 2002 at 02:00:34PM -0400, Pete Zaitcev wrote:
> I do not know if that would count, but I am against that,
> because to make a lower level driver to poke into something that
> belongs to upper levels seems a little retarded to me. My gut
> feeling (e.g. distilled expirience), suggests that such choice
> may also result in a performance hit down the road. Or may be not,
> we'll see.
> 
> Since I am NOT a member of the working group, but a mere implemntor
> of what was designed by smarter people (like you), I kept quiet.
> I am going to write some code and see if this works well.
> Unfortunately, it may be a little too late to change standards
> by that time...
> 
> -- Pete
> 
> > From: "Larsen, Roy K" <roy.k.larsen@intel.com>
> > Date: Mon, 8 Apr 2002 10:23:32 -0700 
> 
> > I recently went through the old mails on the EtherType issue and found that
> > of those folks expressing an opinion, more wanted or were willing to accept
> > the encapsulation field than were dead set against it.  There certainly
> > wasn't a strong consensus against.  
> > 
> > Roy
> > 
> > -----Original Message-----
> > From: Michael Krause [mailto:krause@cup.hp.com]
> > Sent: Monday, April 08, 2002 7:22 AM
> > To: ipoverib@ietf.org
> > Cc: bill@strahm.net
> > Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
> > 
> > 
> > At 12:15 PM 4/6/2002 -0800, H.K. Jerry Chu wrote:
> > >At the Minneapolis/IETF meeting we had a "room concensus" on the
> > >following:
> > >
> > >1. Embedding EtherType in the IPoIB UD payload.
> > 
> > I've seen strong consensus on the reflector that indicates it should not be 
> > used.  I don't want to resurface all of the arguments but there were more 
> > against this from what I could see.
> > 
> > >2. Requiring IPv4 over IB to have a minimal MTU of 1500.
> > >(This effectively rules out all the IB MTUs less than 2K for
> > >running IPoIB.)
> > 
> > Clarified to be for IB UD transport service.
> > 
> > Mike

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr  8 14:42:05 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA02279
	for <ipoverib-archive@odin.ietf.org>; Mon, 8 Apr 2002 14:42:05 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA28881
	for ipoverib-archive@odin.ietf.org; Mon, 8 Apr 2002 14:42:07 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA27790;
	Mon, 8 Apr 2002 14:29:49 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA27745
	for <ipoverib@ns.ietf.org>; Mon, 8 Apr 2002 14:29:46 -0400 (EDT)
Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01995
	for <ipoverib@ietf.org>; Mon, 8 Apr 2002 14:29:43 -0400 (EDT)
Received: from sunmail1.Sun.COM ([129.145.1.2])
	by mercury.Sun.COM (8.9.3+Sun/8.9.3) with ESMTP id LAA27245;
	Mon, 8 Apr 2002 11:29:43 -0700 (PDT)
Received: from jurassic.eng.sun.com (jurassic.Eng.Sun.COM [129.146.81.144])
	by sunmail1.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1-Sun.COM.mod.2) with ESMTP id LAA00228;
	Mon, 8 Apr 2002 11:30:18 -0700 (PDT)
Received: from taipei (taipei.Eng.Sun.COM [129.146.85.178])
	by jurassic.eng.sun.com (8.12.2+Sun/8.12.2) with SMTP id g38ITU35355082;
	Mon, 8 Apr 2002 11:29:31 -0700 (PDT)
Message-Id: <200204081829.g38ITU35355082@jurassic.eng.sun.com>
Date: Mon, 8 Apr 2002 11:29:27 -0700 (PDT)
From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Reply-To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
To: zaitcev@redhat.com
Cc: ipoverib@ietf.org
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: ktkBhFUYzmYRMCM1b0GlrA==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc 
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

>I do not know if that would count, but I am against that,

It's not clear to me what you are against? (Including EtherType?)

>because to make a lower level driver to poke into something that
>belongs to upper levels seems a little retarded to me. My gut

But the above argument seems to favor EtherType.

>feeling (e.g. distilled expirience), suggests that such choice
>may also result in a performance hit down the road. Or may be not,
>we'll see.
>
>Since I am NOT a member of the working group, but a mere implemntor

IETF membership is determined by participation. So you have just
become a member now :-).

Jerry

>of what was designed by smarter people (like you), I kept quiet.
>I am going to write some code and see if this works well.
>Unfortunately, it may be a little too late to change standards
>by that time...
>
>-- Pete
>
>> From: "Larsen, Roy K" <roy.k.larsen@intel.com>
>> Date: Mon, 8 Apr 2002 10:23:32 -0700 
>
>> I recently went through the old mails on the EtherType issue and found that
>> of those folks expressing an opinion, more wanted or were willing to accept
>> the encapsulation field than were dead set against it.  There certainly
>> wasn't a strong consensus against.  
>> 
>> Roy
>> 
>> -----Original Message-----
>> From: Michael Krause [mailto:krause@cup.hp.com]
>> Sent: Monday, April 08, 2002 7:22 AM
>> To: ipoverib@ietf.org
>> Cc: bill@strahm.net
>> Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
>> 
>> 
>> At 12:15 PM 4/6/2002 -0800, H.K. Jerry Chu wrote:
>> >At the Minneapolis/IETF meeting we had a "room concensus" on the
>> >following:
>> >
>> >1. Embedding EtherType in the IPoIB UD payload.
>> 
>> I've seen strong consensus on the reflector that indicates it should not be 
>> used.  I don't want to resurface all of the arguments but there were more 
>> against this from what I could see.
>> 
>> >2. Requiring IPv4 over IB to have a minimal MTU of 1500.
>> >(This effectively rules out all the IB MTUs less than 2K for
>> >running IPoIB.)
>> 
>> Clarified to be for IB UD transport service.
>> 
>> Mike
>
>_______________________________________________
>IPoverIB mailing list
>IPoverIB@ietf.org
>https://www1.ietf.org/mailman/listinfo/ipoverib


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr  8 15:56:20 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04467
	for <ipoverib-archive@odin.ietf.org>; Mon, 8 Apr 2002 15:56:20 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA05122
	for ipoverib-archive@odin.ietf.org; Mon, 8 Apr 2002 15:56:23 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA04053;
	Mon, 8 Apr 2002 15:47:55 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA04017
	for <ipoverib@ns.ietf.org>; Mon, 8 Apr 2002 15:47:53 -0400 (EDT)
Received: from mxout1.netvision.net.il (mxout1.netvision.net.il [194.90.9.20])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04208
	for <ipoverib@ietf.org>; Mon, 8 Apr 2002 15:47:48 -0400 (EDT)
Received: from vaio505he ([139.92.208.202]) by mxout1.netvision.net.il
 (iPlanet Messaging Server 5.1 (built Sep  5 2001))
 with ESMTPA id <0GU9009NWLMNLP@mxout1.netvision.net.il> for ipoverib@ietf.org;
 Mon, 08 Apr 2002 22:47:16 +0300 (IDT)
Date: Mon, 08 Apr 2002 21:48:11 +0200
From: Yaron Haviv <yaronh@voltaire.com>
Subject: RE: [Ipoverib] 4/9 deadline for a consensus call
In-reply-to: <D9223EB959A5D511A98F00508B68C20C06BC4F66@ORSMSX108>
To: "'Larsen, Roy K'" <roy.k.larsen@intel.com>,
        "'Michael Krause'" <krause@cup.hp.com>, ipoverib@ietf.org
Cc: bill@strahm.net
Reply-to: yaronh@voltaire.com
Message-id: <001b01c1df36$50d85f30$cad05c8b@vaio505he>
MIME-version: 1.0
X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4807.1700
X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2910.0)
Content-type: text/plain; charset=iso-8859-1
Content-transfer-encoding: 7BIT
Importance: Normal
X-Priority: 3 (Normal)
X-MSMail-priority: Normal
Content-Transfer-Encoding: 7BIT
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7BIT

Agree with Roy

-----Original Message-----
From: ipoverib-admin@ietf.org [mailto:ipoverib-admin@ietf.org]On Behalf Of
Larsen, Roy K
Sent: Monday, April 08, 2002 7:24 PM
To: 'Michael Krause'; ipoverib@ietf.org
Cc: bill@strahm.net
Subject: RE: [Ipoverib] 4/9 deadline for a consensus call

I recently went through the old mails on the EtherType issue and found that
of those folks expressing an opinion, more wanted or were willing to accept
the encapsulation field than were dead set against it.  There certainly
wasn't a strong consensus against.

Roy

-----Original Message-----
From: Michael Krause [mailto:krause@cup.hp.com]
Sent: Monday, April 08, 2002 7:22 AM
To: ipoverib@ietf.org
Cc: bill@strahm.net
Subject: Re: [Ipoverib] 4/9 deadline for a consensus call


At 12:15 PM 4/6/2002 -0800, H.K. Jerry Chu wrote:
>At the Minneapolis/IETF meeting we had a "room concensus" on the
>following:
>
>1. Embedding EtherType in the IPoIB UD payload.

I've seen strong consensus on the reflector that indicates it should not be
used.  I don't want to resurface all of the arguments but there were more
against this from what I could see.

>2. Requiring IPv4 over IB to have a minimal MTU of 1500.
>(This effectively rules out all the IB MTUs less than 2K for
>running IPoIB.)

Clarified to be for IB UD transport service.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr  8 16:55:39 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06194
	for <ipoverib-archive@odin.ietf.org>; Mon, 8 Apr 2002 16:55:39 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA10686
	for ipoverib-archive@odin.ietf.org; Mon, 8 Apr 2002 16:55:42 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA09996;
	Mon, 8 Apr 2002 16:45:43 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA09960
	for <ipoverib@ns.ietf.org>; Mon, 8 Apr 2002 16:45:41 -0400 (EDT)
Received: from mail2.microsoft.com (mail2.microsoft.com [131.107.3.124])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA05810
	for <ipoverib@ietf.org>; Mon, 8 Apr 2002 16:45:36 -0400 (EDT)
Received: from INET-VRS-02.redmond.corp.microsoft.com ([157.54.8.110]) by mail2.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Mon, 8 Apr 2002 13:45:09 -0700
Received: from 157.54.6.197 by INET-VRS-02.redmond.corp.microsoft.com (InterScan E-Mail VirusWall NT); Mon, 08 Apr 2002 13:45:09 -0700
Received: from red-imc-01.redmond.corp.microsoft.com ([157.54.9.102]) by inet-hub-06.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Mon, 8 Apr 2002 13:45:08 -0700
Received: from win-imc-02.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.84]) by red-imc-01.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Mon, 8 Apr 2002 13:45:08 -0700
Received: from win-msg-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.52]) by win-imc-02.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(6.0.3590.0);
	 Mon, 8 Apr 2002 13:41:42 -0700
X-MimeOLE: Produced By Microsoft Exchange V6.0.6177.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Subject: RE: [Ipoverib] 4/9 deadline for a consensus call
Date: Mon, 8 Apr 2002 13:41:41 -0700
Message-ID: <2E33960095B58E40A4D3345AB9F65EC102450E0A@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Thread-Topic: [Ipoverib] 4/9 deadline for a consensus call
Thread-Index: AcHfKatbQlJ5aHPqR4mjTcNclHeRUQAEIPwg
From: "Aditya Dube" <adube@windows.microsoft.com>
To: <ipoverib@ietf.org>
X-OriginalArrivalTime: 08 Apr 2002 20:41:42.0099 (UTC) FILETIME=[CA2C6230:01C1DF3D]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id QAA09961
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 8bit


IMHO, Adding the EtherType will not cause the performance to degrade and
there is a strong advantage to having this field present. 

There are many corner cases and scenarios that use the EtherType field.
As market requirements change, the requirements for our products might
change to the point where one of these corner scenarios is needed.

Also, there has been a posting on the reflector where a company said
that the EtherType is a must for their products. I would not like the
Standard to preclude anyone from developing Infiniband products.

Aditya Dube

-----Original Message-----
From: Bill Strahm [mailto:bill@strahm.net] 
Sent: Monday, April 08, 2002 11:19 AM
To: Pete Zaitcev
Cc: Larsen, Roy K; 'Michael Krause'; ipoverib@ietf.org; bill@strahm.net
Subject: Re: [Ipoverib] 4/9 deadline for a consensus call

Just on a note from the WG chair...

By definition you ARE a member of the working group by subscribing
to the mailing list and offering the opinion below...

I am very interested in hearing what implementors have to say, I am
of the opinion that including Ethertype in the payload of the packet
is not the right thing to do, however, based on hearing some 
implementation reports from people who actually ARE doing the work, I
am very close to changing my mind again.  

Can you provide a quick summary on your position (I think I read that
you are against the Ethertype field, but I could be wrong) and
especially
any technical reasons for that opinion, such as I can/can't do this
if the field is/isn't there...

Bill
On Mon, Apr 08, 2002 at 02:00:34PM -0400, Pete Zaitcev wrote:
> I do not know if that would count, but I am against that,
> because to make a lower level driver to poke into something that
> belongs to upper levels seems a little retarded to me. My gut
> feeling (e.g. distilled expirience), suggests that such choice
> may also result in a performance hit down the road. Or may be not,
> we'll see.
> 
> Since I am NOT a member of the working group, but a mere implemntor
> of what was designed by smarter people (like you), I kept quiet.
> I am going to write some code and see if this works well.
> Unfortunately, it may be a little too late to change standards
> by that time...
> 
> -- Pete
> 
> > From: "Larsen, Roy K" <roy.k.larsen@intel.com>
> > Date: Mon, 8 Apr 2002 10:23:32 -0700 
> 
> > I recently went through the old mails on the EtherType issue and
found that
> > of those folks expressing an opinion, more wanted or were willing to
accept
> > the encapsulation field than were dead set against it.  There
certainly
> > wasn't a strong consensus against.  
> > 
> > Roy
> > 
> > -----Original Message-----
> > From: Michael Krause [mailto:krause@cup.hp.com]
> > Sent: Monday, April 08, 2002 7:22 AM
> > To: ipoverib@ietf.org
> > Cc: bill@strahm.net
> > Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
> > 
> > 
> > At 12:15 PM 4/6/2002 -0800, H.K. Jerry Chu wrote:
> > >At the Minneapolis/IETF meeting we had a "room concensus" on the
> > >following:
> > >
> > >1. Embedding EtherType in the IPoIB UD payload.
> > 
> > I've seen strong consensus on the reflector that indicates it should
not be 
> > used.  I don't want to resurface all of the arguments but there were
more 
> > against this from what I could see.
> > 
> > >2. Requiring IPv4 over IB to have a minimal MTU of 1500.
> > >(This effectively rules out all the IB MTUs less than 2K for
> > >running IPoIB.)
> > 
> > Clarified to be for IB UD transport service.
> > 
> > Mike

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr  8 21:52:21 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA14080
	for <ipoverib-archive@odin.ietf.org>; Mon, 8 Apr 2002 21:52:21 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id VAA00388
	for ipoverib-archive@odin.ietf.org; Mon, 8 Apr 2002 21:52:24 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id VAA00065;
	Mon, 8 Apr 2002 21:40:43 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id VAA00033
	for <ipoverib@optimus.ietf.org>; Mon, 8 Apr 2002 21:40:40 -0400 (EDT)
Received: from fgwmail6.fujitsu.co.jp (fgwmail6.fujitsu.co.jp [192.51.44.36])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA13982
	for <ipoverib@ietf.org>; Mon, 8 Apr 2002 21:40:36 -0400 (EDT)
Received: from m2.gw.fujitsu.co.jp by fgwmail6.fujitsu.co.jp (8.9.3/3.7W-MX0201-Fujitsu Gateway)
	id KAA29661; Tue, 9 Apr 2002 10:40:08 +0900 (JST)
	(envelope-from imai@giulia.ds.oaks.cs.fujitsu.co.jp)
Received: from dm2.oaks.cs.fujitsu.co.jp by m2.gw.fujitsu.co.jp (8.9.3/3.7W-0202-Fujitsu Domain Master)
	id KAA01547; Tue, 9 Apr 2002 10:40:07 +0900 (JST)
	(envelope-from imai@giulia.ds.oaks.cs.fujitsu.co.jp)
Received: from lovewars.ds.oaks.cs.fujitsu.co.jp by dm2.oaks.cs.fujitsu.co.jp (8.7.4+2.6Wbeta6/3.4W4-MXM960524)
	id KAA26649; Tue, 9 Apr 2002 10:40:06 +0900 (JST)
Received: from giulia.ds.oaks.cs.fujitsu.co.jp by lovewars.ds.oaks.cs.fujitsu.co.jp (5.0/6.4J.6)
	id AA04190; Tue, 9 Apr 2002 11:02:09 --900
Received: from giulia (imai@localhost)
	by giulia.ds.oaks.cs.fujitsu.co.jp (8.10.2+Sun/8.10.2) with ESMTP id g391e5f09797;
	Tue, 9 Apr 2002 10:40:05 +0900 (JST)
Message-Id: <200204090140.g391e5f09797@giulia.ds.oaks.cs.fujitsu.co.jp>
To: "Aditya Dube" <adube@windows.microsoft.com>
Cc: Pete Zaitcev <zaitcev@redhat.com>
Cc: ipoverib@ietf.org
From: akihiko_imai@cs.fujitsu.co.jp
Subject: Re: [Ipoverib] 4/9 deadline for a consensus call 
In-Reply-To: Your message of "Mon, 08 Apr 2002 13:41:41 JST."
             <2E33960095B58E40A4D3345AB9F65EC102450E0A@win-msg-01.wingroup.windeploy.ntdev.microsoft.com> 
Date: Tue, 09 Apr 2002 10:40:04 +0900
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

I have the same opinions as Aditya, about performance,etc.
And I also agree with Roy and Yaron, about consensus understanding.

Jerry and Bill wrote questions to Pete.
And I also have the same questions about Pete's comments as Jerry has.

Thanks,
Akihiko

---
>Date: Mon, 8 Apr 2002 13:41:41 -0700
>From: "Aditya Dube" <adube@windows.microsoft.com>
>
>IMHO, Adding the EtherType will not cause the performance to degrade and
>there is a strong advantage to having this field present. 
>
>There are many corner cases and scenarios that use the EtherType field.
>As market requirements change, the requirements for our products might
>change to the point where one of these corner scenarios is needed.
>
>Also, there has been a posting on the reflector where a company said
>that the EtherType is a must for their products. I would not like the
>Standard to preclude anyone from developing Infiniband products.
>
>Aditya Dube

---
>From: "Larsen, Roy K" <roy.k.larsen@intel.com>
>Date: Mon, 8 Apr 2002 10:23:32 -0700 
>
>I recently went through the old mails on the EtherType issue and found that
>of those folks expressing an opinion, more wanted or were willing to accept
>the encapsulation field than were dead set against it.  There certainly
>wasn't a strong consensus against.  
>
>Roy

---
>Date: Mon, 8 Apr 2002 11:29:27 -0700 (PDT)
>From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
>
>>I do not know if that would count, but I am against that,
>
>It's not clear to me what you are against? (Including EtherType?)
>
>>because to make a lower level driver to poke into something that
>>belongs to upper levels seems a little retarded to me. My gut
>
>But the above argument seems to favor EtherType.
>
>>feeling (e.g. distilled expirience), suggests that such choice
>>may also result in a performance hit down the road. Or may be not,
>>we'll see.
>>
>>Since I am NOT a member of the working group, but a mere implemntor
>
>IETF membership is determined by participation. So you have just
>become a member now :-).
>
>Jerry
>
>>of what was designed by smarter people (like you), I kept quiet.

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr  8 23:09:52 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA16185
	for <ipoverib-archive@odin.ietf.org>; Mon, 8 Apr 2002 23:09:52 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id XAA04415
	for ipoverib-archive@odin.ietf.org; Mon, 8 Apr 2002 23:09:52 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id WAA03579;
	Mon, 8 Apr 2002 22:58:46 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA05796
	for <ipoverib@ns.ietf.org>; Mon, 8 Apr 2002 09:50:41 -0400 (EDT)
Received: from CNRI.Reston.VA.US (localhost [127.0.0.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA23453;
	Mon, 8 Apr 2002 09:50:39 -0400 (EDT)
Message-Id: <200204081350.JAA23453@ietf.org>
Mime-Version: 1.0
Content-Type: Multipart/Mixed; Boundary="NextPart"
To: IETF-Announce: ;
Cc: ipoverib@ietf.org
From: Internet-Drafts@ietf.org
Reply-to: Internet-Drafts@ietf.org
Date: Mon, 08 Apr 2002 09:50:38 -0400
Subject: [Ipoverib] I-D ACTION:draft-ietf-ipoib-dhcp-over-infiniband-01.txt
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the IP over InfiniBand Working Group of the IETF.

	Title		: DHCP over InfiniBand
	Author(s)	: V. Kashyap
	Filename	: draft-ietf-ipoib-dhcp-over-infiniband-01.txt
	Pages		: 6
	Date		: 05-Apr-02
	
An InfiniBand network uses a link-layer addressing scheme that
is 20-bytes long. This is larger than the 16-bytes reserved for
the hardware address in DHCP/BOOTP message. The above
inequality imposes restrictions on the use of the DHCP message
fields when used over an IP over InfiniBand(IPoIB) network.
This document describes the use of DHCP message fields when
implementing DHCP over IPoIB.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-ipoib-dhcp-over-infiniband-01.txt

To remove yourself from the IETF Announcement list, send a message to 
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-ipoib-dhcp-over-infiniband-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt


Internet-Drafts can also be obtained by e-mail.

Send a message to:
	mailserv@ietf.org.
In the body type:
	"FILE /internet-drafts/draft-ietf-ipoib-dhcp-over-infiniband-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.
		
		
Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="mailserv@ietf.org"

Content-Type: text/plain
Content-ID:	<20020405101933.I-D@ietf.org>

ENCODING mime
FILE /internet-drafts/draft-ietf-ipoib-dhcp-over-infiniband-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-ipoib-dhcp-over-infiniband-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<20020405101933.I-D@ietf.org>

--OtherAccess--

--NextPart--




_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 00:09:12 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA16186
	for <ipoverib-archive@odin.ietf.org>; Mon, 8 Apr 2002 23:09:52 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id XAA04401
	for ipoverib-archive@odin.ietf.org; Mon, 8 Apr 2002 23:09:51 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id WAA03625;
	Mon, 8 Apr 2002 22:59:27 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id UAA24434
	for <ipoverib@optimus.ietf.org>; Mon, 8 Apr 2002 20:20:23 -0400 (EDT)
Received: from fgwmail7.fujitsu.co.jp (fgwmail7.fujitsu.co.jp [192.51.44.37])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA11190
	for <ipoverib@ietf.org>; Mon, 8 Apr 2002 20:20:18 -0400 (EDT)
Received: from m3.gw.fujitsu.co.jp by fgwmail7.fujitsu.co.jp (8.9.3/3.7W-MX0201-Fujitsu Gateway)
	id JAA16257; Tue, 9 Apr 2002 09:19:38 +0900 (JST)
	(envelope-from hiro.kishimoto@jp.fujitsu.com)
Received: from flabmail.flab.fujitsu.co.jp by m3.gw.fujitsu.co.jp (8.9.3/3.7W-0202-Fujitsu Domain Master)
	id JAA21791; Tue, 9 Apr 2002 09:19:37 +0900 (JST)
	(envelope-from hiro.kishimoto@jp.fujitsu.com)
Received: from localhost by flabmail.flab.fujitsu.co.jp (8.9.3/3.7W-020318-Fujitsu Labs. Domain Mail Master)
	id JAA22747; Tue, 9 Apr 2002 09:19:36 +0900 (JST)
To: adube@windows.microsoft.com
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] 4/9 deadline for a consensus call
In-Reply-To: <2E33960095B58E40A4D3345AB9F65EC102450E0A@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
References: <2E33960095B58E40A4D3345AB9F65EC102450E0A@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
X-Mailer: Mew version 1.94.2 on Emacs 20.7 / Mule 4.1 (AOI)
Mime-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20020409091047V.hiro.kishimoto@jp.fujitsu.com>
Date: Tue, 09 Apr 2002 09:10:47 +0900
From: Hiro Kishimoto <hiro.kishimoto@jp.fujitsu.com>
X-Dispatcher: imput version 20000228(IM140)
Lines: 120
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

I agree with Aditya,

The EtherType field has many benefits with negligible performance
hit. We have a couple of software products which require the EtherType
for straightforward design and implementation on InfiniBand.
---
Hiro Kishimoto


From: "Aditya Dube" <adube@windows.microsoft.com>
Subject: RE: [Ipoverib] 4/9 deadline for a consensus call
Date: Mon, 8 Apr 2002 13:41:41 -0700
Message-ID: <2E33960095B58E40A4D3345AB9F65EC102450E0A@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>

> 
> IMHO, Adding the EtherType will not cause the performance to degrade and
> there is a strong advantage to having this field present. 
> 
> There are many corner cases and scenarios that use the EtherType field.
> As market requirements change, the requirements for our products might
> change to the point where one of these corner scenarios is needed.
> 
> Also, there has been a posting on the reflector where a company said
> that the EtherType is a must for their products. I would not like the
> Standard to preclude anyone from developing Infiniband products.
> 
> Aditya Dube
> 
> -----Original Message-----
> From: Bill Strahm [mailto:bill@strahm.net] 
> Sent: Monday, April 08, 2002 11:19 AM
> To: Pete Zaitcev
> Cc: Larsen, Roy K; 'Michael Krause'; ipoverib@ietf.org; bill@strahm.net
> Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
> 
> Just on a note from the WG chair...
> 
> By definition you ARE a member of the working group by subscribing
> to the mailing list and offering the opinion below...
> 
> I am very interested in hearing what implementors have to say, I am
> of the opinion that including Ethertype in the payload of the packet
> is not the right thing to do, however, based on hearing some 
> implementation reports from people who actually ARE doing the work, I
> am very close to changing my mind again.  
> 
> Can you provide a quick summary on your position (I think I read that
> you are against the Ethertype field, but I could be wrong) and
> especially
> any technical reasons for that opinion, such as I can/can't do this
> if the field is/isn't there...
> 
> Bill
> On Mon, Apr 08, 2002 at 02:00:34PM -0400, Pete Zaitcev wrote:
> > I do not know if that would count, but I am against that,
> > because to make a lower level driver to poke into something that
> > belongs to upper levels seems a little retarded to me. My gut
> > feeling (e.g. distilled expirience), suggests that such choice
> > may also result in a performance hit down the road. Or may be not,
> > we'll see.
> > 
> > Since I am NOT a member of the working group, but a mere implemntor
> > of what was designed by smarter people (like you), I kept quiet.
> > I am going to write some code and see if this works well.
> > Unfortunately, it may be a little too late to change standards
> > by that time...
> > 
> > -- Pete
> > 
> > > From: "Larsen, Roy K" <roy.k.larsen@intel.com>
> > > Date: Mon, 8 Apr 2002 10:23:32 -0700 
> > 
> > > I recently went through the old mails on the EtherType issue and
> found that
> > > of those folks expressing an opinion, more wanted or were willing to
> accept
> > > the encapsulation field than were dead set against it.  There
> certainly
> > > wasn't a strong consensus against.  
> > > 
> > > Roy
> > > 
> > > -----Original Message-----
> > > From: Michael Krause [mailto:krause@cup.hp.com]
> > > Sent: Monday, April 08, 2002 7:22 AM
> > > To: ipoverib@ietf.org
> > > Cc: bill@strahm.net
> > > Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
> > > 
> > > 
> > > At 12:15 PM 4/6/2002 -0800, H.K. Jerry Chu wrote:
> > > >At the Minneapolis/IETF meeting we had a "room concensus" on the
> > > >following:
> > > >
> > > >1. Embedding EtherType in the IPoIB UD payload.
> > > 
> > > I've seen strong consensus on the reflector that indicates it should
> not be 
> > > used.  I don't want to resurface all of the arguments but there were
> more 
> > > against this from what I could see.
> > > 
> > > >2. Requiring IPv4 over IB to have a minimal MTU of 1500.
> > > >(This effectively rules out all the IB MTUs less than 2K for
> > > >running IPoIB.)
> > > 
> > > Clarified to be for IB UD transport service.
> > > 
> > > Mike
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 04:35:23 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA29476
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 04:35:23 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id EAA01009
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 04:35:25 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id EAA00867;
	Tue, 9 Apr 2002 04:31:43 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id EAA00805
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 04:31:40 -0400 (EDT)
Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA29435
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 04:31:36 -0400 (EDT)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e31.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id g398VcRW023298
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 04:31:38 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g398VYa28008;
	Tue, 9 Apr 2002 02:31:34 -0600
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g398VWM03964;
	Tue, 9 Apr 2002 01:31:32 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g398VT816762;
	Tue, 9 Apr 2002 01:31:29 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204090831.g398VT816762@eng4.beaverton.ibm.com>
To: ipoverib@ietf.org
Date: Tue, 9 Apr 2002 01:31:29 -0700 (PDT)
Cc: fenner@research.att.com
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Subject: [Ipoverib] IPoIB multicast forwarding
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit


Folks,

        We have defined a mapping from the IP addresses to the IB
        multicast groups. Use of different groups causes minimal
        replication at the switches and less load at the receivers.

        But there is a quirk in the way the groups are setup in
        InfiniBand - there is no way for any of the nodes to know if
        another has created an InfiniBand group. This is easy to fix
        in most cases - one checks with the SM. But it doesn't work
        out well for IPoIB packet forwarding.

        The router needs to forward the multicast packets beyond the
        local IP subnet if it knows there are recipients for it or in
        the case of some routing protocols it must do that unless it
        gets an update stating not to forward. Similarly, it needs to
        forward external packets to the local receivers if there are
        any receiver present. 

        Thus if there are IB multicast groups associated with an IP
        multicast group the router must know about them or the router
        must receive all multicast packets in the subnet via some
        other construct.

        Below are the three alternatives that I mentioned at the
        53rd IETF meet. 
	
        i) Define only one IB multicast group i.e. IB broadcast MGID.
        All IP multicast packets use this group.

        ii) The multicast IP packet is sent to both the associated IB
        multicast group and the IB mutlicast group corresponding to
        the all-IP routers IP group.

        iii) Sender always sends to corresonding IB group if it exists
        else it sends to the broadcast group.

             IP receivers create IB groups and send the IGMP/MLD
             groups on the broadcast MGID.

             Routers join relevant groups based on the IGMP report
             and routing protocol updates.


        I also put this problem to Bill Fenner (AD of routing area)
        and asked his advice on the alternatives. Of the three options
        he favored option (iii) above.

        Option (i) doesn't take advantage fo IB multicast at all and
        so should be looked at as a last resort solution only.

        Option (ii) requires the sender to send two copies of the same
        packet and 'deliberately' increases the chance of duplicates
        spreading across the network.


Suggested solution  in detail:


        1. Pure senders - at IB level check for the relevant IB group.
           If it exists then send to the IB group else send to the
           'all-routers' IB group. If there is no IB group
           corresponding to a particular IP group then it implies that
           there are no IP receivers on the subnet for that IP
           multicast group. 

           Note: a) Pure senders don't create IB groups. This is
           because there is no existing mechanism for the senders to
           let the routers know of the new groups.

        2. IP receivers: As per IGMP/MLD the first packet sent by a
           host on joining a multicast group for receiving
           (IP_ADD_MEMBERSHIP) is the multicast report.

	   The hosts, as a result of IP_ADD_MEMBERSHIP, do the following:

	   	check if the corresponding IB group exists. 

		if (IB group exists)
			IB_join the group and send the IGMP/MLD report

		else
			IB_Create the group
			IB_join the group
                        Send IGMP/MLD report to IB group corresponding
                        to all-routers (all-hosts or broadcast) IP
                        group (Note: only done when creating the IB
                        group) 

	3. IP routers IB_join MGIDs on:

		a) Finding a new IP group on the subnet in the IGMP/MLD report
                b) Determining a need to forward IP multicast packets
                   through a routing protocol



Comments ?


Vivek
			
-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 10:03:13 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA05781
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 10:03:07 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id KAA21574
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 10:03:08 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA20577;
	Tue, 9 Apr 2002 09:54:39 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA20543
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 09:54:33 -0400 (EDT)
Received: from ztxmail05.ztx.compaq.com (ztxmail05.ztx.compaq.com [161.114.1.209])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA05541
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 09:54:31 -0400 (EDT)
Received: from cceexg12.americas.cpqcorp.net (cceexg12.americas.cpqcorp.net [16.110.250.124])
	by ztxmail05.ztx.compaq.com (Postfix) with ESMTP
	id 95DD78D0; Tue,  9 Apr 2002 08:53:58 -0500 (CDT)
Received: from cceexc18.americas.cpqcorp.net ([16.110.250.64]) by cceexg12.americas.cpqcorp.net with Microsoft SMTPSVC(5.0.2195.2966);
	 Tue, 9 Apr 2002 08:53:58 -0500
X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3
Content-Class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Tue, 9 Apr 2002 08:53:57 -0500
Message-ID: <4D027986353D1341ADA4F2A4F8A170760E31A5@cceexc18.americas.cpqcorp.net>
Thread-Topic: [Ipoverib] IPoIB multicast forwarding
thread-index: AcHfoVi/0WDkR71oRguInLwHBEY3XwAKjKOw
From: "Culley, Paul" <Paul.Culley@COMPAQ.com>
To: "Vivek Kashyap" <kashyapv@us.ibm.com>, <ipoverib@ietf.org>
Cc: <fenner@research.att.com>
X-OriginalArrivalTime: 09 Apr 2002 13:53:58.0397 (UTC) FILETIME=[FF1572D0:01C1DFCD]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id JAA20544
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 8bit

I suspect that the problem can be made simpler; IP routers should always IB_join ALL IPoIB Multicast groups for a given interface.  The router will simply ask to join any groups in the same partition(s) it is assigned to.  Whether the IP router forwards the multicast packets will then depend on the IP routing protocol.

IB routers have the same problem, and may use the same solution...

Paul R. Culley
Compaq Sr. Fellow
281-514-5543


-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
Sent: Tuesday, April 09, 2002 3:31 AM
To: ipoverib@ietf.org
Cc: fenner@research.att.com
Subject: [Ipoverib] IPoIB multicast forwarding



Folks,

        We have defined a mapping from the IP addresses to the IB
        multicast groups. Use of different groups causes minimal
        replication at the switches and less load at the receivers.

        But there is a quirk in the way the groups are setup in
        InfiniBand - there is no way for any of the nodes to know if
        another has created an InfiniBand group. This is easy to fix
        in most cases - one checks with the SM. But it doesn't work
        out well for IPoIB packet forwarding.

        The router needs to forward the multicast packets beyond the
        local IP subnet if it knows there are recipients for it or in
        the case of some routing protocols it must do that unless it
        gets an update stating not to forward. Similarly, it needs to
        forward external packets to the local receivers if there are
        any receiver present. 

        Thus if there are IB multicast groups associated with an IP
        multicast group the router must know about them or the router
        must receive all multicast packets in the subnet via some
        other construct.

        Below are the three alternatives that I mentioned at the
        53rd IETF meet. 
	
        i) Define only one IB multicast group i.e. IB broadcast MGID.
        All IP multicast packets use this group.

        ii) The multicast IP packet is sent to both the associated IB
        multicast group and the IB mutlicast group corresponding to
        the all-IP routers IP group.

        iii) Sender always sends to corresonding IB group if it exists
        else it sends to the broadcast group.

             IP receivers create IB groups and send the IGMP/MLD
             groups on the broadcast MGID.

             Routers join relevant groups based on the IGMP report
             and routing protocol updates.


        I also put this problem to Bill Fenner (AD of routing area)
        and asked his advice on the alternatives. Of the three options
        he favored option (iii) above.

        Option (i) doesn't take advantage fo IB multicast at all and
        so should be looked at as a last resort solution only.

        Option (ii) requires the sender to send two copies of the same
        packet and 'deliberately' increases the chance of duplicates
        spreading across the network.


Suggested solution  in detail:


        1. Pure senders - at IB level check for the relevant IB group.
           If it exists then send to the IB group else send to the
           'all-routers' IB group. If there is no IB group
           corresponding to a particular IP group then it implies that
           there are no IP receivers on the subnet for that IP
           multicast group. 

           Note: a) Pure senders don't create IB groups. This is
           because there is no existing mechanism for the senders to
           let the routers know of the new groups.

        2. IP receivers: As per IGMP/MLD the first packet sent by a
           host on joining a multicast group for receiving
           (IP_ADD_MEMBERSHIP) is the multicast report.

	   The hosts, as a result of IP_ADD_MEMBERSHIP, do the following:

	   	check if the corresponding IB group exists. 

		if (IB group exists)
			IB_join the group and send the IGMP/MLD report

		else
			IB_Create the group
			IB_join the group
                        Send IGMP/MLD report to IB group corresponding
                        to all-routers (all-hosts or broadcast) IP
                        group (Note: only done when creating the IB
                        group) 

	3. IP routers IB_join MGIDs on:

		a) Finding a new IP group on the subnet in the IGMP/MLD report
                b) Determining a need to forward IP multicast packets
                   through a routing protocol



Comments ?


Vivek
			
-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 11:15:58 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA08546
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 11:15:58 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id LAA26436
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 11:16:00 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id LAA26244;
	Tue, 9 Apr 2002 11:11:46 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id LAA26215
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 11:11:44 -0400 (EDT)
Received: from palrel10.hp.com (palrel10.hp.com [156.153.255.245])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA08303
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 11:11:41 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel10.hp.com (Postfix) with ESMTP
	id B3907C00CCF; Tue,  9 Apr 2002 08:11:07 -0700 (PDT)
Received: from mkrause.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id IAA19406;
	Tue, 9 Apr 2002 08:11:04 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020409080212.02b07e00@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Tue, 09 Apr 2002 08:10:43 -0700
To: Hiro Kishimoto <hiro.kishimoto@jp.fujitsu.com>,
        adube@windows.microsoft.com
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] 4/9 deadline for a consensus call
Cc: ipoverib@ietf.org
In-Reply-To: <20020409091047V.hiro.kishimoto@jp.fujitsu.com>
References: <2E33960095B58E40A4D3345AB9F65EC102450E0A@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
 <2E33960095B58E40A4D3345AB9F65EC102450E0A@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 09:10 AM 4/9/2002 +0900, Hiro Kishimoto wrote:
>I agree with Aditya,
>
>The EtherType field has many benefits with negligible performance
>hit. We have a couple of software products which require the EtherType
>for straightforward design and implementation on InfiniBand.

Ethertype is to inform one of the next layer above the layer 2 
protocol.  In IP over IB one knows what this type is via the QP and hence 
the IP over IB specification always knows what the protocol is.  For those 
that want to support other layer 3 and above protocols, they must define an 
appropriate specification which is not the IP over IB specification whose 
charter is well understood.

While I can appreciate the desire of some to co-opt the IP over IB 
specification for other purposes that have nothing to do with IP, it simply 
is out of bounds from a charter perspective to include a requirement that 
does not align with the stated purpose of this workgroup.  There is no way 
to create interoperability if people start to encode Ethertype with 
anything but IPv4 or IPv6 values and I'm fairly certain that quite a few 
implementations will simply ignore this field if it is being used for any 
other purpose than IP - as an implementer I certainly have absolutely no 
intention of using it and any attempt to derive compliance / 
interoperability "branding" from its presence is strongly opposed by my 
company.

Mike

BTW, the whole reason for IB Raw and its use of Ethertype was to provide 
such a capability.  The only reason we are not using IB Raw was the lack of 
a sufficiently strong CRC for this type of traffic - be nice if people 
either stated a preference to have the IBTA fix this problem or get rid of 
the associated wire protocol. 


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 13:49:58 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA14033
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 13:49:58 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id NAA08113
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 13:49:59 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id NAA07741;
	Tue, 9 Apr 2002 13:45:57 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id NAA07700
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 13:45:53 -0400 (EDT)
Received: from strahm.net (adsl-67-115-29-109.dsl.sndg02.pacbell.net [67.115.29.109])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA13856
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 13:45:50 -0400 (EDT)
Received: (from bill@localhost)
	by strahm.net (8.11.6/8.11.6) id g39Hjt470897
	for ipoverib@ietf.org; Tue, 9 Apr 2002 10:45:55 -0700 (PDT)
	(envelope-from bill)
Date: Tue, 9 Apr 2002 10:45:54 -0700
From: Bill Strahm <bill@strahm.net>
To: ipoverib@ietf.org
Message-ID: <20020409104554.C70815@homegate.strahm.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
Subject: [Ipoverib] Post approvals
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Just a quick note.  

Currently our list only supports messages from subscribers.  All other messages 
are sent for approval and either Jerry, or I approve on topic messages.  This
protects you from all kinds of spam that none of us want to read (but I slog
through on a daily basis anyway)

The downside of this is occasionally people post from e-mail addresses that they
aren't subscribed from, and the mail is delayed until Jerry or I can get to it.
We do our best but it takes a while, I just wanted people to be aware that 
there isn't a conspiracy or anything on why some mail seems to be delayed.

If there are any questions about this, please feel free to e-mail me, or if it
is needed, the list, or our Area Directors.  

Bill

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 14:09:55 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA14696
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 14:09:55 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA09913
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 14:09:57 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA09599;
	Tue, 9 Apr 2002 14:05:59 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA09569
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 14:05:55 -0400 (EDT)
Received: from strahm.net (adsl-67-115-29-109.dsl.sndg02.pacbell.net [67.115.29.109])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA14559
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 14:05:52 -0400 (EDT)
Received: (from bill@localhost)
	by strahm.net (8.11.6/8.11.6) id g39I5n170955;
	Tue, 9 Apr 2002 11:05:49 -0700 (PDT)
	(envelope-from bill)
Date: Tue, 9 Apr 2002 11:05:47 -0700
From: Bill Strahm <bill@strahm.net>
To: "Culley, Paul" <Paul.Culley@COMPAQ.com>
Cc: Vivek Kashyap <kashyapv@us.ibm.com>, ipoverib@ietf.org,
        fenner@research.att.com
Subject: Re: [Ipoverib] IPoIB multicast forwarding
Message-ID: <20020409110547.D70815@homegate.strahm.net>
References: <4D027986353D1341ADA4F2A4F8A170760E31A5@cceexc18.americas.cpqcorp.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <4D027986353D1341ADA4F2A4F8A170760E31A5@cceexc18.americas.cpqcorp.net>; from Paul.Culley@COMPAQ.com on Tue, Apr 09, 2002 at 08:53:57AM -0500
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Ok, I am going to express a scary amount of ignorance here...

I know how multicast works on receive, I know how IGMP works to build the forwarding tree,
I know how flood/prune, RP Multicast Routing protocols work...

What I would like a pointer to is a simple this is how a first hop multicast router 
behaves.  An example on Ethernet would be acceptable, a generic example would be better.

The question I have is as a Multicast sender, I send a multicast packet to the first
hop router that is multicast aware.  Now, do I send that packet to its unicast MAC 
address, or do I send that to the mapped layer 2 broadcast address and let the IP 
layer rebroadcast to the broadcast address if it is needed on the local subnet.

Once I have that, everything else I understand

Bill
PS Bill Fenner, the IPoIB list is subscriber posting only.  If you haven't subscribed, please do 
so we can get the benefit of your wisdom...

On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley, Paul wrote:
> I suspect that the problem can be made simpler; IP routers should always IB_join ALL IPoIB Multicast groups for a given interface.  The router will simply ask to join any groups in the same partition(s) it is assigned to.  Whether the IP router forwards the multicast packets will then depend on the IP routing protocol.
> 
> IB routers have the same problem, and may use the same solution...
> 
> Paul R. Culley
> Compaq Sr. Fellow
> 281-514-5543
> 
> 
> -----Original Message-----
> From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> Sent: Tuesday, April 09, 2002 3:31 AM
> To: ipoverib@ietf.org
> Cc: fenner@research.att.com
> Subject: [Ipoverib] IPoIB multicast forwarding
> 
> 
> 
> Folks,
> 
>         We have defined a mapping from the IP addresses to the IB
>         multicast groups. Use of different groups causes minimal
>         replication at the switches and less load at the receivers.
> 
>         But there is a quirk in the way the groups are setup in
>         InfiniBand - there is no way for any of the nodes to know if
>         another has created an InfiniBand group. This is easy to fix
>         in most cases - one checks with the SM. But it doesn't work
>         out well for IPoIB packet forwarding.
> 
>         The router needs to forward the multicast packets beyond the
>         local IP subnet if it knows there are recipients for it or in
>         the case of some routing protocols it must do that unless it
>         gets an update stating not to forward. Similarly, it needs to
>         forward external packets to the local receivers if there are
>         any receiver present. 
> 
>         Thus if there are IB multicast groups associated with an IP
>         multicast group the router must know about them or the router
>         must receive all multicast packets in the subnet via some
>         other construct.
> 
>         Below are the three alternatives that I mentioned at the
>         53rd IETF meet. 
> 	
>         i) Define only one IB multicast group i.e. IB broadcast MGID.
>         All IP multicast packets use this group.
> 
>         ii) The multicast IP packet is sent to both the associated IB
>         multicast group and the IB mutlicast group corresponding to
>         the all-IP routers IP group.
> 
>         iii) Sender always sends to corresonding IB group if it exists
>         else it sends to the broadcast group.
> 
>              IP receivers create IB groups and send the IGMP/MLD
>              groups on the broadcast MGID.
> 
>              Routers join relevant groups based on the IGMP report
>              and routing protocol updates.
> 
> 
>         I also put this problem to Bill Fenner (AD of routing area)
>         and asked his advice on the alternatives. Of the three options
>         he favored option (iii) above.
> 
>         Option (i) doesn't take advantage fo IB multicast at all and
>         so should be looked at as a last resort solution only.
> 
>         Option (ii) requires the sender to send two copies of the same
>         packet and 'deliberately' increases the chance of duplicates
>         spreading across the network.
> 
> 
> Suggested solution  in detail:
> 
> 
>         1. Pure senders - at IB level check for the relevant IB group.
>            If it exists then send to the IB group else send to the
>            'all-routers' IB group. If there is no IB group
>            corresponding to a particular IP group then it implies that
>            there are no IP receivers on the subnet for that IP
>            multicast group. 
> 
>            Note: a) Pure senders don't create IB groups. This is
>            because there is no existing mechanism for the senders to
>            let the routers know of the new groups.
> 
>         2. IP receivers: As per IGMP/MLD the first packet sent by a
>            host on joining a multicast group for receiving
>            (IP_ADD_MEMBERSHIP) is the multicast report.
> 
> 	   The hosts, as a result of IP_ADD_MEMBERSHIP, do the following:
> 
> 	   	check if the corresponding IB group exists. 
> 
> 		if (IB group exists)
> 			IB_join the group and send the IGMP/MLD report
> 
> 		else
> 			IB_Create the group
> 			IB_join the group
>                         Send IGMP/MLD report to IB group corresponding
>                         to all-routers (all-hosts or broadcast) IP
>                         group (Note: only done when creating the IB
>                         group) 
> 
> 	3. IP routers IB_join MGIDs on:
> 
> 		a) Finding a new IP group on the subnet in the IGMP/MLD report
>                 b) Determining a need to forward IP multicast packets
>                    through a routing protocol
> 
> 
> 
> Comments ?
> 
> 
> Vivek
> 			
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 14:10:36 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA14756
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 14:10:36 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA09976
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 14:10:37 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA09658;
	Tue, 9 Apr 2002 14:06:19 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA09626
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 14:06:16 -0400 (EDT)
Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA14585
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 14:06:13 -0400 (EDT)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e31.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id g39I6DNr019310;
	Tue, 9 Apr 2002 14:06:13 -0400
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g39I5oD133194;
	Tue, 9 Apr 2002 12:05:50 -0600
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g39I5jX06680;
	Tue, 9 Apr 2002 11:05:45 -0700
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g39I5iZ23645;
	Tue, 9 Apr 2002 11:05:44 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204091805.g39I5iZ23645@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: Paul.Culley@COMPAQ.com (Culley, Paul)
Date: Tue, 9 Apr 2002 11:05:43 -0700 (PDT)
Cc: kashyapv@us.ibm.com (Vivek Kashyap), ipoverib@ietf.org,
        fenner@research.att.com
In-Reply-To: <4D027986353D1341ADA4F2A4F8A170760E31A5@cceexc18.americas.cpqcorp.net> from "Culley, Paul" at Apr 09, 2002 07:53:57 AM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> I suspect that the problem can be made simpler; IP routers should always IB_join ALL IPoIB Multicast groups for a given interface.  The router will simply ask to join any groups in the same partition(s) it is assigned to.  Whether the IP router forwards the multicast packets will then depend on the IP routing protocol.

That does not seem workable. How does the router know which groups to
join ? Note that the groups are not static - new ones can always be
created and the old ones deleted.

In my solution the IGMP/MLD report which tells the router about IP
joins is used to trigger the corresponding IB join. The same is the
case with routing protocols.

Vivek


> 
> IB routers have the same problem, and may use the same solution...
> 
> Paul R. Culley
> Compaq Sr. Fellow
> 281-514-5543
> 
> 
> -----Original Message-----
> From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> Sent: Tuesday, April 09, 2002 3:31 AM
> To: ipoverib@ietf.org
> Cc: fenner@research.att.com
> Subject: [Ipoverib] IPoIB multicast forwarding
> 
> 
> 
> Folks,
> 
>         We have defined a mapping from the IP addresses to the IB
>         multicast groups. Use of different groups causes minimal
>         replication at the switches and less load at the receivers.
> 
>         But there is a quirk in the way the groups are setup in
>         InfiniBand - there is no way for any of the nodes to know if
>         another has created an InfiniBand group. This is easy to fix
>         in most cases - one checks with the SM. But it doesn't work
>         out well for IPoIB packet forwarding.
> 
>         The router needs to forward the multicast packets beyond the
>         local IP subnet if it knows there are recipients for it or in
>         the case of some routing protocols it must do that unless it
>         gets an update stating not to forward. Similarly, it needs to
>         forward external packets to the local receivers if there are
>         any receiver present. 
> 
>         Thus if there are IB multicast groups associated with an IP
>         multicast group the router must know about them or the router
>         must receive all multicast packets in the subnet via some
>         other construct.
> 
>         Below are the three alternatives that I mentioned at the
>         53rd IETF meet. 
> 	
>         i) Define only one IB multicast group i.e. IB broadcast MGID.
>         All IP multicast packets use this group.
> 
>         ii) The multicast IP packet is sent to both the associated IB
>         multicast group and the IB mutlicast group corresponding to
>         the all-IP routers IP group.
> 
>         iii) Sender always sends to corresonding IB group if it exists
>         else it sends to the broadcast group.
> 
>              IP receivers create IB groups and send the IGMP/MLD
>              groups on the broadcast MGID.
> 
>              Routers join relevant groups based on the IGMP report
>              and routing protocol updates.
> 
> 
>         I also put this problem to Bill Fenner (AD of routing area)
>         and asked his advice on the alternatives. Of the three options
>         he favored option (iii) above.
> 
>         Option (i) doesn't take advantage fo IB multicast at all and
>         so should be looked at as a last resort solution only.
> 
>         Option (ii) requires the sender to send two copies of the same
>         packet and 'deliberately' increases the chance of duplicates
>         spreading across the network.
> 
> 
> Suggested solution  in detail:
> 
> 
>         1. Pure senders - at IB level check for the relevant IB group.
>            If it exists then send to the IB group else send to the
>            'all-routers' IB group. If there is no IB group
>            corresponding to a particular IP group then it implies that
>            there are no IP receivers on the subnet for that IP
>            multicast group. 
> 
>            Note: a) Pure senders don't create IB groups. This is
>            because there is no existing mechanism for the senders to
>            let the routers know of the new groups.
> 
>         2. IP receivers: As per IGMP/MLD the first packet sent by a
>            host on joining a multicast group for receiving
>            (IP_ADD_MEMBERSHIP) is the multicast report.
> 
> 	   The hosts, as a result of IP_ADD_MEMBERSHIP, do the following:
> 
> 	   	check if the corresponding IB group exists. 
> 
> 		if (IB group exists)
> 			IB_join the group and send the IGMP/MLD report
> 
> 		else
> 			IB_Create the group
> 			IB_join the group
>                         Send IGMP/MLD report to IB group corresponding
>                         to all-routers (all-hosts or broadcast) IP
>                         group (Note: only done when creating the IB
>                         group) 
> 
> 	3. IP routers IB_join MGIDs on:
> 
> 		a) Finding a new IP group on the subnet in the IGMP/MLD report
>                 b) Determining a need to forward IP multicast packets
>                    through a routing protocol
> 
> 
> 
> Comments ?
> 
> 
> Vivek
> 			
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 14:45:06 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA16491
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 14:45:06 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA12520
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 14:45:08 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA12207;
	Tue, 9 Apr 2002 14:40:51 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA12169
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 14:40:47 -0400 (EDT)
Received: from strahm.net (adsl-67-115-29-109.dsl.sndg02.pacbell.net [67.115.29.109])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA16303
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 14:40:43 -0400 (EDT)
Received: (from bill@localhost)
	by strahm.net (8.11.6/8.11.6) id g39IebB71073;
	Tue, 9 Apr 2002 11:40:37 -0700 (PDT)
	(envelope-from bill)
Date: Tue, 9 Apr 2002 11:40:37 -0700
From: Bill Strahm <bill@strahm.net>
To: Sukanta ganguly <sganguly@yahoo.com>
Cc: Bill Strahm <bill@strahm.net>, "Culley, Paul" <Paul.Culley@COMPAQ.com>,
        Vivek Kashyap <kashyapv@us.ibm.com>, ipoverib@ietf.org,
        fenner@research.att.com
Subject: Re: [Ipoverib] IPoIB multicast forwarding
Message-ID: <20020409114037.F70815@homegate.strahm.net>
References: <20020409110547.D70815@homegate.strahm.net> <20020409183622.77877.qmail@web12504.mail.yahoo.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <20020409183622.77877.qmail@web12504.mail.yahoo.com>; from sganguly@yahoo.com on Tue, Apr 09, 2002 at 11:36:22AM -0700
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Question then...

As an IP Multicast sender, I am not required to join 
the Multicast group... So now there has not been an
IGMP message telling the first hop router to join the
group either.  So the router has not told its layer
2 to join the broadcast address for the multicast group.

How does the first hop Multicast router know to join
this group and forward the traffic on to the multicast
tree (either RP, or Prune/Flood) if it can't receive
the initial packet...

This is the magic that I was wanting to know, I know how
I would make it work, but I am hoping Bill F. can tell me
the correct way to do it <G>

Bill Strahm

On Tue, Apr 09, 2002 at 11:36:22AM -0700, Sukanta ganguly wrote:
> Bill,
>   When you send in the Multicast packet (as a sender),
> at layer 3 you send it to the multicast IP address
> that the sender decided on. Your layer 2 module will
> decide on the next hop for that Multicast IP address.
> And a similar functioning will be build up as a chain.
> Note in this pacth while the senders original packet
> is moving to its destination which is the IP multicast
> address, the intermediate routers (which are multicast
> aware) will check to see if they have any IGMP joins
> they have received from any other subnets. If they
> have any such mapping they have built due to receive
> of IGMP joins then they will oblige that join by
> copying the packet to that joined destination. That's
> how the copy of the packet is enroute to its rightful
> receiver.
> 
> SG
> 
> P.S :: Dr Fenner is an authority on this and can
> provide some special comments 
> 
> 
> --- Bill Strahm <bill@strahm.net> wrote:
> > Ok, I am going to express a scary amount of
> > ignorance here...
> > 
> > I know how multicast works on receive, I know how
> > IGMP works to build the forwarding tree,
> > I know how flood/prune, RP Multicast Routing
> > protocols work...
> > 
> > What I would like a pointer to is a simple this is
> > how a first hop multicast router 
> > behaves.  An example on Ethernet would be
> > acceptable, a generic example would be better.
> > 
> > The question I have is as a Multicast sender, I send
> > a multicast packet to the first
> > hop router that is multicast aware.  Now, do I send
> > that packet to its unicast MAC 
> > address, or do I send that to the mapped layer 2
> > broadcast address and let the IP 
> > layer rebroadcast to the broadcast address if it is
> > needed on the local subnet.
> > 
> > Once I have that, everything else I understand
> > 
> > Bill
> > PS Bill Fenner, the IPoIB list is subscriber posting
> > only.  If you haven't subscribed, please do 
> > so we can get the benefit of your wisdom...
> > 
> > On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley,
> > Paul wrote:
> > > I suspect that the problem can be made simpler; IP
> > routers should always IB_join ALL IPoIB Multicast
> > groups for a given interface.  The router will
> > simply ask to join any groups in the same
> > partition(s) it is assigned to.  Whether the IP
> > router forwards the multicast packets will then
> > depend on the IP routing protocol.
> > > 
> > > IB routers have the same problem, and may use the
> > same solution...
> > > 
> > > Paul R. Culley
> > > Compaq Sr. Fellow
> > > 281-514-5543
> > > 
> > > 
> > > -----Original Message-----
> > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > > Sent: Tuesday, April 09, 2002 3:31 AM
> > > To: ipoverib@ietf.org
> > > Cc: fenner@research.att.com
> > > Subject: [Ipoverib] IPoIB multicast forwarding
> > > 
> > > 
> > > 
> > > Folks,
> > > 
> > >         We have defined a mapping from the IP
> > addresses to the IB
> > >         multicast groups. Use of different groups
> > causes minimal
> > >         replication at the switches and less load
> > at the receivers.
> > > 
> > >         But there is a quirk in the way the groups
> > are setup in
> > >         InfiniBand - there is no way for any of
> > the nodes to know if
> > >         another has created an InfiniBand group.
> > This is easy to fix
> > >         in most cases - one checks with the SM.
> > But it doesn't work
> > >         out well for IPoIB packet forwarding.
> > > 
> > >         The router needs to forward the multicast
> > packets beyond the
> > >         local IP subnet if it knows there are
> > recipients for it or in
> > >         the case of some routing protocols it must
> > do that unless it
> > >         gets an update stating not to forward.
> > Similarly, it needs to
> > >         forward external packets to the local
> > receivers if there are
> > >         any receiver present. 
> > > 
> > >         Thus if there are IB multicast groups
> > associated with an IP
> > >         multicast group the router must know about
> > them or the router
> > >         must receive all multicast packets in the
> > subnet via some
> > >         other construct.
> > > 
> > >         Below are the three alternatives that I
> > mentioned at the
> > >         53rd IETF meet. 
> > > 	
> > >         i) Define only one IB multicast group i.e.
> > IB broadcast MGID.
> > >         All IP multicast packets use this group.
> > > 
> > >         ii) The multicast IP packet is sent to
> > both the associated IB
> > >         multicast group and the IB mutlicast group
> > corresponding to
> > >         the all-IP routers IP group.
> > > 
> > >         iii) Sender always sends to corresonding
> > IB group if it exists
> > >         else it sends to the broadcast group.
> > > 
> > >              IP receivers create IB groups and
> > send the IGMP/MLD
> > >              groups on the broadcast MGID.
> > > 
> > >              Routers join relevant groups based on
> > the IGMP report
> > >              and routing protocol updates.
> > > 
> > > 
> > >         I also put this problem to Bill Fenner (AD
> > of routing area)
> > >         and asked his advice on the alternatives.
> > Of the three options
> > >         he favored option (iii) above.
> > > 
> > >         Option (i) doesn't take advantage fo IB
> > multicast at all and
> > >         so should be looked at as a last resort
> > solution only.
> > > 
> > >         Option (ii) requires the sender to send
> > two copies of the same
> > >         packet and 'deliberately' increases the
> > chance of duplicates
> > >         spreading across the network.
> > > 
> > > 
> > > Suggested solution  in detail:
> > > 
> > > 
> > >         1. Pure senders - at IB level check for
> > the relevant IB group.
> > >            If it exists then send to the IB group
> > else send to the
> > >            'all-routers' IB group. If there is no
> > IB group
> > >            corresponding to a particular IP group
> > then it implies that
> > >            there are no IP receivers on the subnet
> > for that IP
> > >            multicast group. 
> > > 
> > >            Note: a) Pure senders don't create IB
> > groups. This is
> > >            because there is no existing mechanism
> > for the senders to
> > >            let the routers know of the new groups.
> > > 
> > >         2. IP receivers: As per IGMP/MLD the first
> > packet sent by a
> > >            host on joining a multicast group for
> > receiving
> > >            (IP_ADD_MEMBERSHIP) is the multicast
> > report.
> > > 
> > > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> > do the following:
> > > 
> > > 	   	check if the corresponding IB group exists. 
> > > 
> > > 		if (IB group exists)
> > > 			IB_join the group and send the IGMP/MLD report
> > > 
> > > 		else
> > > 			IB_Create the group
> > > 			IB_join the group
> > >                         Send IGMP/MLD report to IB
> > group corresponding
> > >                         to all-routers (all-hosts
> > or broadcast) IP
> > >                         group (Note: only done
> > when creating the IB
> > >                         group) 
> > > 
> > > 	3. IP routers IB_join MGIDs on:
> > > 
> > > 		a) Finding a new IP group on the subnet in the
> > IGMP/MLD report
> > >                 b) Determining a need to forward
> > IP multicast packets
> > >                    through a routing protocol
> > > 
> > > 
> > > 
> > > Comments ?
> > > 
> > > 
> > > Vivek
> > 
> === message truncated ===
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Tax Center - online filing with TurboTax
> http://taxes.yahoo.com/

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 15:01:21 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA17162
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 15:01:21 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA13425
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 15:01:23 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA12864;
	Tue, 9 Apr 2002 14:55:05 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA12807
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 14:55:00 -0400 (EDT)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA16870
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 14:54:50 -0400 (EDT)
Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.37 2002/04/01 17:50:55 root Exp $) with ESMTP id g39IsIA15860
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 18:54:18 GMT
Received: from fmsmsxvs042.fm.intel.com (fmsmsxv042-1.fm.intel.com [132.233.48.110])
	by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.15 2002/04/01 17:51:48 root Exp $) with SMTP id g39IrRI20747
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 18:53:27 GMT
Received: from fmsmsx26.fm.intel.com ([132.233.42.26])
 by fmsmsxvs042.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002040911583630019
 ; Tue, 09 Apr 2002 11:58:36 -0700
Received: by fmsmsx26.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2HL0C56M>; Tue, 9 Apr 2002 11:54:43 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F6D@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Bill Strahm'" <bill@strahm.net>, Sukanta ganguly <sganguly@yahoo.com>
Cc: "Culley, Paul" <Paul.Culley@COMPAQ.com>,
        Vivek Kashyap<kashyapv@us.ibm.com>, ipoverib@ietf.org,
        fenner@research.att.com
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Tue, 9 Apr 2002 11:54:36 -0700 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

I was also wondering why the multicast sender needed to do anything more
that send to the multicast group (if it existed).  Unless a multicast router
is in promiscuous mode, wouldn't it have the same issues for an Ethernet
link?  Either it has set a MAC filter to receive the IP multicast addresses
or it will miss them.  Why is an IB link different?  If a multicast router
knew to set its MAC filter for a particular IP multicast address, then in an
IB link, wouldn't it have joined the appropriate IB multicast group?

Roy

-----Original Message-----
From: Bill Strahm [mailto:bill@strahm.net]
Sent: Tuesday, April 09, 2002 11:41 AM
To: Sukanta ganguly
Cc: Bill Strahm; Culley, Paul; Vivek Kashyap; ipoverib@ietf.org;
fenner@research.att.com
Subject: Re: [Ipoverib] IPoIB multicast forwarding


Question then...

As an IP Multicast sender, I am not required to join 
the Multicast group... So now there has not been an
IGMP message telling the first hop router to join the
group either.  So the router has not told its layer
2 to join the broadcast address for the multicast group.

How does the first hop Multicast router know to join
this group and forward the traffic on to the multicast
tree (either RP, or Prune/Flood) if it can't receive
the initial packet...

This is the magic that I was wanting to know, I know how
I would make it work, but I am hoping Bill F. can tell me
the correct way to do it <G>

Bill Strahm

On Tue, Apr 09, 2002 at 11:36:22AM -0700, Sukanta ganguly wrote:
> Bill,
>   When you send in the Multicast packet (as a sender),
> at layer 3 you send it to the multicast IP address
> that the sender decided on. Your layer 2 module will
> decide on the next hop for that Multicast IP address.
> And a similar functioning will be build up as a chain.
> Note in this pacth while the senders original packet
> is moving to its destination which is the IP multicast
> address, the intermediate routers (which are multicast
> aware) will check to see if they have any IGMP joins
> they have received from any other subnets. If they
> have any such mapping they have built due to receive
> of IGMP joins then they will oblige that join by
> copying the packet to that joined destination. That's
> how the copy of the packet is enroute to its rightful
> receiver.
> 
> SG
> 
> P.S :: Dr Fenner is an authority on this and can
> provide some special comments 
> 
> 
> --- Bill Strahm <bill@strahm.net> wrote:
> > Ok, I am going to express a scary amount of
> > ignorance here...
> > 
> > I know how multicast works on receive, I know how
> > IGMP works to build the forwarding tree,
> > I know how flood/prune, RP Multicast Routing
> > protocols work...
> > 
> > What I would like a pointer to is a simple this is
> > how a first hop multicast router 
> > behaves.  An example on Ethernet would be
> > acceptable, a generic example would be better.
> > 
> > The question I have is as a Multicast sender, I send
> > a multicast packet to the first
> > hop router that is multicast aware.  Now, do I send
> > that packet to its unicast MAC 
> > address, or do I send that to the mapped layer 2
> > broadcast address and let the IP 
> > layer rebroadcast to the broadcast address if it is
> > needed on the local subnet.
> > 
> > Once I have that, everything else I understand
> > 
> > Bill
> > PS Bill Fenner, the IPoIB list is subscriber posting
> > only.  If you haven't subscribed, please do 
> > so we can get the benefit of your wisdom...
> > 
> > On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley,
> > Paul wrote:
> > > I suspect that the problem can be made simpler; IP
> > routers should always IB_join ALL IPoIB Multicast
> > groups for a given interface.  The router will
> > simply ask to join any groups in the same
> > partition(s) it is assigned to.  Whether the IP
> > router forwards the multicast packets will then
> > depend on the IP routing protocol.
> > > 
> > > IB routers have the same problem, and may use the
> > same solution...
> > > 
> > > Paul R. Culley
> > > Compaq Sr. Fellow
> > > 281-514-5543
> > > 
> > > 
> > > -----Original Message-----
> > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > > Sent: Tuesday, April 09, 2002 3:31 AM
> > > To: ipoverib@ietf.org
> > > Cc: fenner@research.att.com
> > > Subject: [Ipoverib] IPoIB multicast forwarding
> > > 
> > > 
> > > 
> > > Folks,
> > > 
> > >         We have defined a mapping from the IP
> > addresses to the IB
> > >         multicast groups. Use of different groups
> > causes minimal
> > >         replication at the switches and less load
> > at the receivers.
> > > 
> > >         But there is a quirk in the way the groups
> > are setup in
> > >         InfiniBand - there is no way for any of
> > the nodes to know if
> > >         another has created an InfiniBand group.
> > This is easy to fix
> > >         in most cases - one checks with the SM.
> > But it doesn't work
> > >         out well for IPoIB packet forwarding.
> > > 
> > >         The router needs to forward the multicast
> > packets beyond the
> > >         local IP subnet if it knows there are
> > recipients for it or in
> > >         the case of some routing protocols it must
> > do that unless it
> > >         gets an update stating not to forward.
> > Similarly, it needs to
> > >         forward external packets to the local
> > receivers if there are
> > >         any receiver present. 
> > > 
> > >         Thus if there are IB multicast groups
> > associated with an IP
> > >         multicast group the router must know about
> > them or the router
> > >         must receive all multicast packets in the
> > subnet via some
> > >         other construct.
> > > 
> > >         Below are the three alternatives that I
> > mentioned at the
> > >         53rd IETF meet. 
> > > 	
> > >         i) Define only one IB multicast group i.e.
> > IB broadcast MGID.
> > >         All IP multicast packets use this group.
> > > 
> > >         ii) The multicast IP packet is sent to
> > both the associated IB
> > >         multicast group and the IB mutlicast group
> > corresponding to
> > >         the all-IP routers IP group.
> > > 
> > >         iii) Sender always sends to corresonding
> > IB group if it exists
> > >         else it sends to the broadcast group.
> > > 
> > >              IP receivers create IB groups and
> > send the IGMP/MLD
> > >              groups on the broadcast MGID.
> > > 
> > >              Routers join relevant groups based on
> > the IGMP report
> > >              and routing protocol updates.
> > > 
> > > 
> > >         I also put this problem to Bill Fenner (AD
> > of routing area)
> > >         and asked his advice on the alternatives.
> > Of the three options
> > >         he favored option (iii) above.
> > > 
> > >         Option (i) doesn't take advantage fo IB
> > multicast at all and
> > >         so should be looked at as a last resort
> > solution only.
> > > 
> > >         Option (ii) requires the sender to send
> > two copies of the same
> > >         packet and 'deliberately' increases the
> > chance of duplicates
> > >         spreading across the network.
> > > 
> > > 
> > > Suggested solution  in detail:
> > > 
> > > 
> > >         1. Pure senders - at IB level check for
> > the relevant IB group.
> > >            If it exists then send to the IB group
> > else send to the
> > >            'all-routers' IB group. If there is no
> > IB group
> > >            corresponding to a particular IP group
> > then it implies that
> > >            there are no IP receivers on the subnet
> > for that IP
> > >            multicast group. 
> > > 
> > >            Note: a) Pure senders don't create IB
> > groups. This is
> > >            because there is no existing mechanism
> > for the senders to
> > >            let the routers know of the new groups.
> > > 
> > >         2. IP receivers: As per IGMP/MLD the first
> > packet sent by a
> > >            host on joining a multicast group for
> > receiving
> > >            (IP_ADD_MEMBERSHIP) is the multicast
> > report.
> > > 
> > > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> > do the following:
> > > 
> > > 	   	check if the corresponding IB group exists. 
> > > 
> > > 		if (IB group exists)
> > > 			IB_join the group and send the IGMP/MLD report
> > > 
> > > 		else
> > > 			IB_Create the group
> > > 			IB_join the group
> > >                         Send IGMP/MLD report to IB
> > group corresponding
> > >                         to all-routers (all-hosts
> > or broadcast) IP
> > >                         group (Note: only done
> > when creating the IB
> > >                         group) 
> > > 
> > > 	3. IP routers IB_join MGIDs on:
> > > 
> > > 		a) Finding a new IP group on the subnet in the
> > IGMP/MLD report
> > >                 b) Determining a need to forward
> > IP multicast packets
> > >                    through a routing protocol
> > > 
> > > 
> > > 
> > > Comments ?
> > > 
> > > 
> > > Vivek
> > 
> === message truncated ===
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Tax Center - online filing with TurboTax
> http://taxes.yahoo.com/

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 15:06:49 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA17318
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 15:06:49 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA13854
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 15:06:51 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA13630;
	Tue, 9 Apr 2002 15:02:57 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA13591
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 15:02:53 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA17220
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 15:02:50 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g39J2ksX143438;
	Tue, 9 Apr 2002 15:02:46 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.00) with ESMTP id g39J2i138744;
	Tue, 9 Apr 2002 15:02:45 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g39J2gM23286;
	Tue, 9 Apr 2002 12:02:42 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g39J2eV24654;
	Tue, 9 Apr 2002 12:02:40 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204091902.g39J2eV24654@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: sganguly@yahoo.com (Sukanta ganguly)
Date: Tue, 9 Apr 2002 12:02:39 -0700 (PDT)
Cc: kashyapv@us.ibm.com (Vivek Kashyap), Paul.Culley@COMPAQ.com (Culley Paul),
        ipoverib@ietf.org, fenner@research.att.com
In-Reply-To: <20020409183002.74249.qmail@web12502.mail.yahoo.com> from "Sukanta ganguly" at Apr 09, 2002 10:30:02 AM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> The problem IP multicast joining/leaving is performed
> optimally becuase the routers implement the IGMP.
> For IPoIB have an optimal implementation of multicast,
> the IB node intending to join the multicast needs to
> express the IB multicast group ID to the appropriate
> IP multicast group ID.

        In this WG we have defined a mapping from the IP multicast
        addresses to IB multicast addresses. When I say
        'corresponding' or 'relevant' IB group I mean the IB mutlciast
        GID mapped to from the IP multicast address.

	Eg. 224.0.0.1 is mapped to FF12:<401B>:<P_Key>::1

>   On a similar account the originator of the IB
> multicast data should create a mapping of IB Multicast
> groupID to IP multicast groupID. Otherwise if a
> blanket link would flood the IP network with unwanted
> packets, since the forwarder would not know better and
> hence would take the most pessimitic approach an
> forward any and every multicast data.


        The reason the originator of the data is not being asked to
        create the group is because it cannot tell the IP router about
        the corresponding IB group the router needs to join. We cannot
        have the router parsing every mutlicast packet received to
        determine if it is a new group to IB_join.

        In any case if there are no IP receivers for the IP group then
        the packet will be either dropped by the router (after
        receiving it on the 'MGID corresponding to the all-routers IP
        address') or forwarded further. The IP router may receive
        pruning updates to remove the IP group and in that case it
        will drop any future packets on that IP address.

	Thus there is no flooding as you fear. 

        Things at the IP layer work exactly as in any other media. For
        IB the only aspect that is new is:

                IP receivers create corresponding IB groups. They also
                ensure a copy of the MLD/IGMP report is sent out on
                the *IB MGID corresponding to all-hosts IP address*

                IP routers join the IB group corresponding to the IP
                groups listed in the MLD/IGMP report. They may
                optionally choose to create and join IB groups based
                on IP groups they expect to forward out of the local
                subnet. 

> 
>   In order to not have any major infrastucture rework
> to get IPoIB and its multicast to work this needs to
> be done.
>   
>    Vivek, I read your three proposals and saw the
> commanility of sticking to onely one IB multicast
> group ID. Whay is that ? That means simultaneous IB
> multicast of data cannot be facilitated. Did I get
> that right?


         No I am not suggesting sticking to one IB group ID. That is
         option (i) only - and I don't support that option.

Vivek

> 
> SG
> 
> 
> --- Vivek Kashyap <kashyapv@us.ibm.com> wrote:
> > > 
> > > I suspect that the problem can be made simpler; IP
> > routers should always IB_join ALL IPoIB Multicast
> > groups for a given interface.  The router will
> > simply ask to join any groups in the same
> > partition(s) it is assigned to.  Whether the IP
> > router forwards the multicast packets will then
> > depend on the IP routing protocol.
> > 
> > That does not seem workable. How does the router
> > know which groups to
> > join ? Note that the groups are not static - new
> > ones can always be
> > created and the old ones deleted.
> > 
> > In my solution the IGMP/MLD report which tells the
> > router about IP
> > joins is used to trigger the corresponding IB join.
> > The same is the
> > case with routing protocols.
> > 
> > Vivek
> > 
> > 
> > > 
> > > IB routers have the same problem, and may use the
> > same solution...
> > > 
> > > Paul R. Culley
> > > Compaq Sr. Fellow
> > > 281-514-5543
> > > 
> > > 
> > > -----Original Message-----
> > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > > Sent: Tuesday, April 09, 2002 3:31 AM
> > > To: ipoverib@ietf.org
> > > Cc: fenner@research.att.com
> > > Subject: [Ipoverib] IPoIB multicast forwarding
> > > 
> > > 
> > > 
> > > Folks,
> > > 
> > >         We have defined a mapping from the IP
> > addresses to the IB
> > >         multicast groups. Use of different groups
> > causes minimal
> > >         replication at the switches and less load
> > at the receivers.
> > > 
> > >         But there is a quirk in the way the groups
> > are setup in
> > >         InfiniBand - there is no way for any of
> > the nodes to know if
> > >         another has created an InfiniBand group.
> > This is easy to fix
> > >         in most cases - one checks with the SM.
> > But it doesn't work
> > >         out well for IPoIB packet forwarding.
> > > 
> > >         The router needs to forward the multicast
> > packets beyond the
> > >         local IP subnet if it knows there are
> > recipients for it or in
> > >         the case of some routing protocols it must
> > do that unless it
> > >         gets an update stating not to forward.
> > Similarly, it needs to
> > >         forward external packets to the local
> > receivers if there are
> > >         any receiver present. 
> > > 
> > >         Thus if there are IB multicast groups
> > associated with an IP
> > >         multicast group the router must know about
> > them or the router
> > >         must receive all multicast packets in the
> > subnet via some
> > >         other construct.
> > > 
> > >         Below are the three alternatives that I
> > mentioned at the
> > >         53rd IETF meet. 
> > > 	
> > >         i) Define only one IB multicast group i.e.
> > IB broadcast MGID.
> > >         All IP multicast packets use this group.
> > > 
> > >         ii) The multicast IP packet is sent to
> > both the associated IB
> > >         multicast group and the IB mutlicast group
> > corresponding to
> > >         the all-IP routers IP group.
> > > 
> > >         iii) Sender always sends to corresonding
> > IB group if it exists
> > >         else it sends to the broadcast group.
> > > 
> > >              IP receivers create IB groups and
> > send the IGMP/MLD
> > >              groups on the broadcast MGID.
> > > 
> > >              Routers join relevant groups based on
> > the IGMP report
> > >              and routing protocol updates.
> > > 
> > > 
> > >         I also put this problem to Bill Fenner (AD
> > of routing area)
> > >         and asked his advice on the alternatives.
> > Of the three options
> > >         he favored option (iii) above.
> > > 
> > >         Option (i) doesn't take advantage fo IB
> > multicast at all and
> > >         so should be looked at as a last resort
> > solution only.
> > > 
> > >         Option (ii) requires the sender to send
> > two copies of the same
> > >         packet and 'deliberately' increases the
> > chance of duplicates
> > >         spreading across the network.
> > > 
> > > 
> > > Suggested solution  in detail:
> > > 
> > > 
> > >         1. Pure senders - at IB level check for
> > the relevant IB group.
> > >            If it exists then send to the IB group
> > else send to the
> > >            'all-routers' IB group. If there is no
> > IB group
> > >            corresponding to a particular IP group
> > then it implies that
> > >            there are no IP receivers on the subnet
> > for that IP
> > >            multicast group. 
> > > 
> > >            Note: a) Pure senders don't create IB
> > groups. This is
> > >            because there is no existing mechanism
> > for the senders to
> > >            let the routers know of the new groups.
> > > 
> > >         2. IP receivers: As per IGMP/MLD the first
> > packet sent by a
> > >            host on joining a multicast group for
> > receiving
> > >            (IP_ADD_MEMBERSHIP) is the multicast
> > report.
> > > 
> > > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> > do the following:
> > > 
> > > 	   	check if the corresponding IB group exists. 
> > > 
> > > 		if (IB group exists)
> > > 			IB_join the group and send the IGMP/MLD report
> > > 
> > > 		else
> > > 			IB_Create the group
> > > 			IB_join the group
> > >                         Send IGMP/MLD report to IB
> > group corresponding
> > >                         to all-routers (all-hosts
> > or broadcast) IP
> > >                         group (Note: only done
> > when creating the IB
> > >                         group) 
> > > 
> > > 	3. IP routers IB_join MGIDs on:
> > > 
> > > 		a) Finding a new IP group on the subnet in the
> > IGMP/MLD report
> > >                 b) Determining a need to forward
> > IP multicast packets
> > >                    through a routing protocol
> > > 
> > > 
> > > 
> > > Comments ?
> > > 
> > > 
> > > Vivek
> > > 			
> > > -- 
> > > Vivek Kashyap
> > > Linux Technology Center, IBM
> > > kashyapv@us.ibm.com 
> > > vivk@us.ibm.com 
> > > 503 578 3422 (o)
> > > 
> > > _______________________________________________
> > > IPoverIB mailing list
> > > IPoverIB@ietf.org
> > > https://www1.ietf.org/mailman/listinfo/ipoverib
> > > 
> > 
> > 
> === message truncated ===
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Tax Center - online filing with TurboTax
> http://taxes.yahoo.com/
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 15:34:37 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA18405
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 15:34:37 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA15500
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 15:34:39 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA14920;
	Tue, 9 Apr 2002 15:29:10 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA14852
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 15:29:05 -0400 (EDT)
Received: from kathmandu.sun.com (kathmandu.sun.com [192.18.98.36])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA18199
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 15:29:00 -0400 (EDT)
Received: from sunmail1.Sun.COM ([129.145.1.2])
	by kathmandu.sun.com (8.9.3+Sun/8.9.3) with ESMTP id NAA15765;
	Tue, 9 Apr 2002 13:28:45 -0600 (MDT)
Received: from jurassic.eng.sun.com (jurassic.Eng.Sun.COM [129.146.85.31])
	by sunmail1.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1-Sun.COM.mod.2) with ESMTP id MAA18578;
	Tue, 9 Apr 2002 12:29:26 -0700 (PDT)
Received: from taipei (taipei.Eng.Sun.COM [129.146.85.178])
	by jurassic.eng.sun.com (8.12.2+Sun/8.12.2) with SMTP id g39JSi35628548;
	Tue, 9 Apr 2002 12:28:44 -0700 (PDT)
Message-Id: <200204091928.g39JSi35628548@jurassic.eng.sun.com>
Date: Tue, 9 Apr 2002 12:28:40 -0700 (PDT)
From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Reply-To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Subject: RE: [Ipoverib] IPoIB multicast forwarding
To: bill@strahm.net, sganguly@yahoo.com, roy.k.larsen@intel.com
Cc: Paul.Culley@compaq.com, kashyapv@us.ibm.com, ipoverib@ietf.org,
        fenner@research.att.com
MIME-Version: 1.0
Content-Type: MULTIPART/mixed; BOUNDARY=Brace_of_Greyhounds_046_000
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc 
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

--Brace_of_Greyhounds_046_000
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: gI8DdjeG/xLsXAbIoYaIkg==

The whole problem is that IB doesn't support a promiscuous multicast mode.
This problem is described in the link-multicast draft and my proposed
solution to put the burden on the sender to send a copy to the all-router
MC group. Vivek is making a different proposal. Both have their pros and
cons. (Sorry I haven't got the time to follow all the details of the
discussion so far. Will do some catch up later.)

Of course if IBA provides a solution for a router to turn on promiscuous
mode (or equivalent ie recieving a copy of every MC pkt) then the problem
doesn't exist for us (IPoIB).

There is a related question on whether in IB a sender is required to
join a MC group to send. To me the 1.0 spec is clear on this and it
follows IP model. But some in IBTA disagree and it's been discussion
in IBTA. I've asked why makes the IB MC model different (i.e. requiring
join to send) from IP MC model. But it's IBTA's call to make the best
choice for IB.

I've enclosed the latest link-multicast (-01). Take a took at section
9 and 10. I'll republish the draft after the consensus call.

Jerry

>I was also wondering why the multicast sender needed to do anything more
>that send to the multicast group (if it existed).  Unless a multicast router
>is in promiscuous mode, wouldn't it have the same issues for an Ethernet
>link?  Either it has set a MAC filter to receive the IP multicast addresses
>or it will miss them.  Why is an IB link different?  If a multicast router
>knew to set its MAC filter for a particular IP multicast address, then in an
>IB link, wouldn't it have joined the appropriate IB multicast group?
>
>Roy
>
>-----Original Message-----
>From: Bill Strahm [mailto:bill@strahm.net]
>Sent: Tuesday, April 09, 2002 11:41 AM
>To: Sukanta ganguly
>Cc: Bill Strahm; Culley, Paul; Vivek Kashyap; ipoverib@ietf.org;
>fenner@research.att.com
>Subject: Re: [Ipoverib] IPoIB multicast forwarding
>
>
>Question then...
>
>As an IP Multicast sender, I am not required to join 
>the Multicast group... So now there has not been an
>IGMP message telling the first hop router to join the
>group either.  So the router has not told its layer
>2 to join the broadcast address for the multicast group.
>
>How does the first hop Multicast router know to join
>this group and forward the traffic on to the multicast
>tree (either RP, or Prune/Flood) if it can't receive
>the initial packet...
>
>This is the magic that I was wanting to know, I know how
>I would make it work, but I am hoping Bill F. can tell me
>the correct way to do it <G>
>
>Bill Strahm
>
>On Tue, Apr 09, 2002 at 11:36:22AM -0700, Sukanta ganguly wrote:
>> Bill,
>>   When you send in the Multicast packet (as a sender),
>> at layer 3 you send it to the multicast IP address
>> that the sender decided on. Your layer 2 module will
>> decide on the next hop for that Multicast IP address.
>> And a similar functioning will be build up as a chain.
>> Note in this pacth while the senders original packet
>> is moving to its destination which is the IP multicast
>> address, the intermediate routers (which are multicast
>> aware) will check to see if they have any IGMP joins
>> they have received from any other subnets. If they
>> have any such mapping they have built due to receive
>> of IGMP joins then they will oblige that join by
>> copying the packet to that joined destination. That's
>> how the copy of the packet is enroute to its rightful
>> receiver.
>> 
>> SG
>> 
>> P.S :: Dr Fenner is an authority on this and can
>> provide some special comments 
>> 
>> 
>> --- Bill Strahm <bill@strahm.net> wrote:
>> > Ok, I am going to express a scary amount of
>> > ignorance here...
>> > 
>> > I know how multicast works on receive, I know how
>> > IGMP works to build the forwarding tree,
>> > I know how flood/prune, RP Multicast Routing
>> > protocols work...
>> > 
>> > What I would like a pointer to is a simple this is
>> > how a first hop multicast router 
>> > behaves.  An example on Ethernet would be
>> > acceptable, a generic example would be better.
>> > 
>> > The question I have is as a Multicast sender, I send
>> > a multicast packet to the first
>> > hop router that is multicast aware.  Now, do I send
>> > that packet to its unicast MAC 
>> > address, or do I send that to the mapped layer 2
>> > broadcast address and let the IP 
>> > layer rebroadcast to the broadcast address if it is
>> > needed on the local subnet.
>> > 
>> > Once I have that, everything else I understand
>> > 
>> > Bill
>> > PS Bill Fenner, the IPoIB list is subscriber posting
>> > only.  If you haven't subscribed, please do 
>> > so we can get the benefit of your wisdom...
>> > 
>> > On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley,
>> > Paul wrote:
>> > > I suspect that the problem can be made simpler; IP
>> > routers should always IB_join ALL IPoIB Multicast
>> > groups for a given interface.  The router will
>> > simply ask to join any groups in the same
>> > partition(s) it is assigned to.  Whether the IP
>> > router forwards the multicast packets will then
>> > depend on the IP routing protocol.
>> > > 
>> > > IB routers have the same problem, and may use the
>> > same solution...
>> > > 
>> > > Paul R. Culley
>> > > Compaq Sr. Fellow
>> > > 281-514-5543
>> > > 
>> > > 
>> > > -----Original Message-----
>> > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
>> > > Sent: Tuesday, April 09, 2002 3:31 AM
>> > > To: ipoverib@ietf.org
>> > > Cc: fenner@research.att.com
>> > > Subject: [Ipoverib] IPoIB multicast forwarding
>> > > 
>> > > 
>> > > 
>> > > Folks,
>> > > 
>> > >         We have defined a mapping from the IP
>> > addresses to the IB
>> > >         multicast groups. Use of different groups
>> > causes minimal
>> > >         replication at the switches and less load
>> > at the receivers.
>> > > 
>> > >         But there is a quirk in the way the groups
>> > are setup in
>> > >         InfiniBand - there is no way for any of
>> > the nodes to know if
>> > >         another has created an InfiniBand group.
>> > This is easy to fix
>> > >         in most cases - one checks with the SM.
>> > But it doesn't work
>> > >         out well for IPoIB packet forwarding.
>> > > 
>> > >         The router needs to forward the multicast
>> > packets beyond the
>> > >         local IP subnet if it knows there are
>> > recipients for it or in
>> > >         the case of some routing protocols it must
>> > do that unless it
>> > >         gets an update stating not to forward.
>> > Similarly, it needs to
>> > >         forward external packets to the local
>> > receivers if there are
>> > >         any receiver present. 
>> > > 
>> > >         Thus if there are IB multicast groups
>> > associated with an IP
>> > >         multicast group the router must know about
>> > them or the router
>> > >         must receive all multicast packets in the
>> > subnet via some
>> > >         other construct.
>> > > 
>> > >         Below are the three alternatives that I
>> > mentioned at the
>> > >         53rd IETF meet. 
>> > > 	
>> > >         i) Define only one IB multicast group i.e.
>> > IB broadcast MGID.
>> > >         All IP multicast packets use this group.
>> > > 
>> > >         ii) The multicast IP packet is sent to
>> > both the associated IB
>> > >         multicast group and the IB mutlicast group
>> > corresponding to
>> > >         the all-IP routers IP group.
>> > > 
>> > >         iii) Sender always sends to corresonding
>> > IB group if it exists
>> > >         else it sends to the broadcast group.
>> > > 
>> > >              IP receivers create IB groups and
>> > send the IGMP/MLD
>> > >              groups on the broadcast MGID.
>> > > 
>> > >              Routers join relevant groups based on
>> > the IGMP report
>> > >              and routing protocol updates.
>> > > 
>> > > 
>> > >         I also put this problem to Bill Fenner (AD
>> > of routing area)
>> > >         and asked his advice on the alternatives.
>> > Of the three options
>> > >         he favored option (iii) above.
>> > > 
>> > >         Option (i) doesn't take advantage fo IB
>> > multicast at all and
>> > >         so should be looked at as a last resort
>> > solution only.
>> > > 
>> > >         Option (ii) requires the sender to send
>> > two copies of the same
>> > >         packet and 'deliberately' increases the
>> > chance of duplicates
>> > >         spreading across the network.
>> > > 
>> > > 
>> > > Suggested solution  in detail:
>> > > 
>> > > 
>> > >         1. Pure senders - at IB level check for
>> > the relevant IB group.
>> > >            If it exists then send to the IB group
>> > else send to the
>> > >            'all-routers' IB group. If there is no
>> > IB group
>> > >            corresponding to a particular IP group
>> > then it implies that
>> > >            there are no IP receivers on the subnet
>> > for that IP
>> > >            multicast group. 
>> > > 
>> > >            Note: a) Pure senders don't create IB
>> > groups. This is
>> > >            because there is no existing mechanism
>> > for the senders to
>> > >            let the routers know of the new groups.
>> > > 
>> > >         2. IP receivers: As per IGMP/MLD the first
>> > packet sent by a
>> > >            host on joining a multicast group for
>> > receiving
>> > >            (IP_ADD_MEMBERSHIP) is the multicast
>> > report.
>> > > 
>> > > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
>> > do the following:
>> > > 
>> > > 	   	check if the corresponding IB group exists. 
>> > > 
>> > > 		if (IB group exists)
>> > > 			IB_join the group and send the IGMP/MLD report
>> > > 
>> > > 		else
>> > > 			IB_Create the group
>> > > 			IB_join the group
>> > >                         Send IGMP/MLD report to IB
>> > group corresponding
>> > >                         to all-routers (all-hosts
>> > or broadcast) IP
>> > >                         group (Note: only done
>> > when creating the IB
>> > >                         group) 
>> > > 
>> > > 	3. IP routers IB_join MGIDs on:
>> > > 
>> > > 		a) Finding a new IP group on the subnet in the
>> > IGMP/MLD report
>> > >                 b) Determining a need to forward
>> > IP multicast packets
>> > >                    through a routing protocol
>> > > 
>> > > 
>> > > 
>> > > Comments ?
>> > > 
>> > > 
>> > > Vivek
>> > 
>> === message truncated ===
>> 
>> 
>> __________________________________________________
>> Do You Yahoo!?
>> Yahoo! Tax Center - online filing with TurboTax
>> http://taxes.yahoo.com/
>
>_______________________________________________
>IPoverIB mailing list
>IPoverIB@ietf.org
>https://www1.ietf.org/mailman/listinfo/ipoverib
>
>_______________________________________________
>IPoverIB mailing list
>IPoverIB@ietf.org
>https://www1.ietf.org/mailman/listinfo/ipoverib

--Brace_of_Greyhounds_046_000
Content-Type: TEXT/plain; name="draft-ietf-ipoib-link-multicast-01.txt"; charset=us-ascii; x-unix-mode=0664
Content-Description: draft-ietf-ipoib-link-multicast-01.txt
Content-MD5: c74p2Pkmky+PHeJfjsXIvg==







INTERNET-DRAFT                                            H.K. Jerry Chu
<draft-ietf-ipoib-link-multicast-01.txt>                Sun Microsystems
                                                           Vivek Kashyap
                                                                     IBM
Expires: September, 2002                                     March, 2002


             IP link and multicast over InfiniBand networks


Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that other
   groups may also distribute working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time. It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   Copyright (C) The Internet Society (date).  All Rights Reserved.


Abstract

   This document specifies a method for setting up IP subnets and
   multicast services over InfiniBand(TM) networks. Discussions in this
   document are applicable to both IPv4 and IPv6, unless explicitly
   specified. A separate document will cover unicast and encapsulation
   of IP datagrams over InfiniBand networks.


Table of Contents
   1.0     Introduction
   2.0     Terminology
   3.0     Basic IPoIB Transport - Unreliable Datagram
   4.0     IB Multicast Architecture
   5.0     IB Links vs IPoIB Links



Chu & Kashyap                                                   [Page 1]

draft-ietf-ipoib-link-multicast-01.txt                        March 2002


   6.0     Setting up an IPoIB Link
   6.1     Maximum Transmission Unit
   6.2     IPoIB Link Q_Key
   6.3     Other Link Attributes
   7.0     The IPoIB All-Node Multicast and Broadcast Group
   8.0     Mapping for other Multicast Groups
   9.0     Sending and Receiving IP Multicast Packets
   10.0    Support for IP Multicast Routing
   11.0    Security Considerations
   12.0    Acknowledgments
   13.0    References
   14.0    Author's Address
   15.0    Full Copyright Statement


1.0 Introduction

   InfiniBand Architecture (IBA) defines four layers of network services
   corresponding to layer one through layer four of the OSI reference
   model.  For the purpose of running IP over an InfiniBand (IB)
   network, the IB link, network, and transport layers collectively
   constitute the data link layer to the IP stack. One can find a
   general overview of IB architecture related to IP networks in
   [IPoIB_ARCH].

   This document will focus on the steps required to lay out an IP
   network on top of an IB network. It will describe all the elements an
   IP over InfiniBand (IPoIB) link consists of, how to configure its
   associated link attributes, and how to set up basic broadcast and
   multicast services on an IPoIB link. IPoIB link is the building block
   for an IP network to be decomposed into multiple subnets connected by
   IP routers. Subnetting allows the containment of broadcast traffic
   within a single subnet. It also provides certain degree of isolation
   between nodes on different subnets. The latter may be an important
   consideration for administration purpose.

2.0 Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

3.0 Basic IPoIB Transport - Unreliable Datagram

   InfiniBand defines four types of transport services [IBTA]. They are
   reliable connection, unreliable connection, reliable datagram,
   unreliable datagram. IBA also defines a special raw datagram service
   for encapsulation purpose. Both unreliable datagram and raw datagram



Chu & Kashyap                                                   [Page 2]

draft-ietf-ipoib-link-multicast-01.txt                        March 2002


   define support for multicast. They provide the basic transport
   mechanism that best matches the IP datagram paradigm.

   IB unreliable datagram provides many additional transport features
   such as the partition key (P_Key) protection, multiple queue pairs
   (QPs), and Q_Key protection. Moreover, it requires a 32-bit invariant
   CRC checksum, which provides a much stronger protection against data
   corruption, compared with the 16-bit CRC that a raw datagram carries.

   For these reasons, IB unreliable datagram is considered to be a much
   better choice as the basic IPoIB transport than the raw datagram, and
   is chosen as the default IPoIB transport mechanism ([IPoIB_ARCH],
   [IPoIB_ENCAP]).

4.0 IB Multicast Architecture

   The following discussion gives a short overview of the multicast
   architecture in InfiniBand. For a more complete description, the
   reader is referred to [IBTA] and [IPoIB_ARCH].

   IBA defines two layers of multicast services. Its link layer uses
   multicast LIDs (MLIDs), which are allocated by the Subnet Manager
   (SM) and fall in the range between 0xC0000 to 0xFFFE (approximately
   16k). MLIDs are used by IB switches to program their multicast
   forwarding tables. An IB switch implementation may support much fewer
   MLIDs in its forwarding table though.

   IB network layer uses multicast GIDs (MGIDs), which closely resemble
   IPv6 multicast addresses [AARCH] shown below.

   |   8    |  4 |  4 |                  112 bits                   |
   +------ -+----+----+---------------------------------------------+
   |11111111|flgs|scop|                  group ID                   |
   +--------+----+----+---------------------------------------------+

   [IPoIB_ARCH] describes each field in more details.

   Since every IB multicast packet is required to carry both LRH and
   GRH, a valid MGID and a valid MLID are both needed before a valid IB
   multicast packet can be constructed.

   An IB multicast group is uniquely identified by a valid MGID. Before
   a MGID can be used within an IB subnet, either as a destination
   address of a multicast packet, or representing a multicast group that
   an IB node can join, a "MCGroupRecord" corresponding to the MGID must
   be created through the Subnet Administrator (SA). Besides the MGID,
   the creator must supply values of the path MTU, Q_Key, TClass,
   FlowLabel, HopLimit that are appropriate for all the potential



Chu & Kashyap                                                   [Page 3]

draft-ietf-ipoib-link-multicast-01.txt                        March 2002


   clients of the multicast group to use. In return, SA will allocate a
   MLID to be used by switches in the local IB subnet.

   Unreliable multicast is defined by IBA as an optional functionality
   for channel adaptors (CAs) and switches. In today's IP technology,
   link multicast has become an indispensable function for better
   supporting a modern IP network. For this reason, it is required that
   an IPoIB fabric supports multicast. This includes all the CAs and
   switches that make up an IP network.

5.0 IB Links vs IPoIB Links

   A link segment on top of which an IP subnet can be configured is
   defined in [IPV6] as a communication facility or medium over which
   nodes can communicate at the "link" layer.  For most types of
   communication media, the boundary between different data link
   segments follows the physical topology of the network connectivity,
   and is pretty obvious. E.g. an Ethernet network connected by
   switches, hubs, or bridges usually forms a single link segment and
   broadcast/multicast domain.  Different Ethernet segments can be
   connected by IP routers at the network layer.

   InfiniBand defines its own link-layer and subnets consisting of nodes
   connected by IB switches. However, the IPoIB link boundary needs not
   follow the IB link boundary. Nodes residing on different IB subnets
   can still communicate directly with one another through IB routers at
   the InfiniBand network layer. This communication at the network layer
   applies to both unicast as well as multicast.

   The ultimate requirement for two nodes in the same IB fabric to
   communicate at the IB level, besides the physical connectivity, is a
   common P_Key.

   Partitioning in IB provides an isolation mechanism among nodes in an
   IB fabric, much like VLANs in an Ethernet network.  Each HCA (Host
   Channel Adaptor) port of an endnode contains a P_Key table of all the
   valid P_Keys the port is allowed to use. The P_Key table is set up by
   the SM of the local IB subnet.  Each QP is programmed with a P_Key
   from the local P_Key table.  This P_Key is carried in all the
   outgoing packets from the QP, and is used to compare against the
   P_Key of the incoming packets to the QP. Reception of an invalid
   P_Key causes the packet to be discarded. IB switches may optionally
   enforce partition checking too.

   Therefore P_Key and IB partition are the natural choice for defining
   IPoIB link boundary. It also affords much flexibility to the network
   administrators when multiple links are set up in a large network.




Chu & Kashyap                                                   [Page 4]

draft-ietf-ipoib-link-multicast-01.txt                        March 2002


6.0 Setting up an IPoIB Link

   A network administrator defines an IPoIB link by setting up an IB
   partition and assigning it a unique P_Key. An IB partition may or may
   not span multiple IB subnets, and whether it does or not is mostly
   transparent to IPoIB.

   Each node attached to the IB partition MUST have one of its HCAs
   assigned the P_Key to use. Note that the P_key table of a HCA port
   may contain many P_Keys. It is up to an implementation to define the
   method by which the P_Key relevant to a particular IPoIB subnet is
   determined and conveyed to the IPoIB stack. E.g. implementations can
   resort to a manual configuration to determine the P_key or a set of
   P_Keys for IPoIB use, and rely on DHCP [DHCP] on each IPoIB link to
   assign an IP subnet number.

   Once an IB partition is established for IPoIB use, the link MTU and
   Q_Key are two other important attributes that must be chosen before
   the IPoIB link can be configured.

6.1 Maximum Transmission Unit

   IB defines five permissible maximum payload sizes. They are 256, 512,
   1024, 2048 and 4096 bytes.  [IPV6] requires a link MTU of 1280 bytes
   or greater. This leaves only 2048 and 4096 bytes as two acceptable
   choices for IPv6. Channel adaptors supporting a maximum payload size
   less than the minimal MTU requirement can still expose an acceptable
   link MTU to IP through an adaptation layer that fragments larger
   messages into smaller IB packets, and reassembles them on the
   receiving end. But this must be done in a way that is totally
   transparent to the IP stack.

   It is up to the network administrator to select a link MTU to use
   when configuring an IPoIB link. The link MTU SHALL not be greater
   than the maximum payload size of any IB component that is part of the
   IPoIB link.  This includes IB switches, CAs, or routers.

   In general, a larger link MTU is preferred over smaller ones, and
   should be employed whenever possible to attain better throughput
   performance. One caveat is that once a link MTU is chosen for a given
   IPoIB link, nodes connected by CAs of a smaller maximum payload size
   won't be able to join the link unless the whole link and all the
   devices attached to it are reconfigured to use a smaller MTU.

   The flexibility of configuring a smaller than the full link MTU size
   also makes it easier for one to bridge an IPoIB link with an Ethernet
   link, by setting the MTU of all the connecting nodes to 1500 bytes.
   For IPv4, this may require a manual configuration of a MTU different



Chu & Kashyap                                                   [Page 5]

draft-ietf-ipoib-link-multicast-01.txt                        March 2002


   from the default, link MTU size on all the nodes belonging to an
   IPoIB link.  For IPv6, one can use the link MTU option of the router
   advertisement [DISC] to announce a smaller MTU to all the nodes.

   In case an IPoIB link spans more than one IB subnet, the IPoIB link
   MTU MUST not exceed the path MTU of any path connecting two nodes in
   the same IB partition. It is up to the network administrator to
   determine the appropriate path MTU value that will work for any node
   in the same IPoIB link.

6.2 IPoIB Link Q_Key

   A Q_Key is programmed by the source QP in every IB datagram, and is
   verified by the destination QP against the Q_Key it has been
   assigned. A Q_Key violation will cause the offending datagram to be
   dropped, and a Q_Key violation trap to be raised.

   A Q_Key must be selected to be used by all the QPs attached to an
   IPoIB link. It is recommended that a controlled Q_Key be used with
   the high order bit set. This is to prevent non-privileged software
   from fabricating and sending out bogus IP datagrams. All QPs
   configured to use on a given IPoIB link SHALL be assigned the same
   per-link Q_Key.

6.3 Other Link Attributes

   TClass, FlowLabel, and HopLimit are three other attributes that are
   required for an IPoIB link covering more than a single IB subnet.
   The selection of these values are implementation dependent. But it
   must take into account the topology of IB subnets comprising the
   IPoIB link to ensure successful communication between any two nodes
   in the same IPoIB link.

7.0 The IPoIB All-Node Multicast and Broadcast Group

   Once an IB partition is created with link attributes identified for
   an IPoIB link, the network administrator must create a special IB
   multicast group for every node on the IPoIB link to join. This is
   achieved through the creation of "MCGroupRecord" in each IB subnet
   that the IB partition encompasses, as described in section 4 above.

   The MGID will have the P_Key of the IB partition that defines the
   IPoIB link embedded in it. A special signature is also embedded to
   identify the MGID for IPoIB use only. For IPv4 over IB, the signature
   will be "0x401B". For IPv6 over IB, the signature will be "0x601B".

   For an IPv4 subnet, the MGID for this special IB multicast group
   SHALL have the following format:



Chu & Kashyap                                                   [Page 6]

draft-ietf-ipoib-link-multicast-01.txt                        March 2002


   |   8    |  4 |  4 |     16 bits     | 16 bits | 48 bits  | 32 bits |
   +--------+----+----+-----------------+---------+----------+---------+
   |11111111|0001|scop|<IPoIB signature>|< P_Key >|00.......0|<all 1's>|
   +--------+----+----+-----------------+---------+----------+---------+

   For an IPv6 subnet, the format of the MGID SHALL look like this:

   |   8    |  4 |  4 |     16 bits     | 16 bits |       80 bits      |
   +--------+----+----+-----------------+---------+--------------------+
   |11111111|0001|scop|<IPoIB signature>|< P_Key >|000.............0001|
   +--------+----+----+-----------------+---------+--------------------+

   As for the scop bits, if the IPoIB link is fully contained within a
   single IB subnet, the scop bits SHALL be set to 2 (link-local).
   Otherwise the scope will be set higher.

   A MCGroupRecord will be created with all the IPoIB link attributes
   described before. When a node is attached to an IPoIB link identified
   by a P_Key, it must look for a special, all-node multicast/broadcast
   group to join. This is done by constructing the MGID with the link
   P_Key and the IPoIB signature. The node SHOULD always look for a MGID
   of a link-local scope first before attempting one with a greater
   scope.

   Once the right MGID and MCGroupRecord are identified, the local node
   SHOULD use the link MTU recorded in the MCGroupRecord.  In case the
   link MTU is greater than the maximum payload size that the local HCA
   can support, the node can not join the IPoIB link and operate as an
   IP node. Otherwise the local node must join the special all-node
   multicast/broadcast group by calling the SA to create a
   MCMemberRecord corresponding to the MGID. The SA will return all the
   link attributes for the local node to use. The node MUST use these
   attributes in all future multicast operations to the local IPoIB
   link.  The broadcast group for IPv4 will serve to provide a broadcast
   service for protocol like ARP to use.

   In addition to the all-node multicast/broadcast group, an all-router
   multicast group SHOULD be created at link configuration time if an IP
   router will be attached to the link. This is to facilitate IP
   multicast operations described later. A MCGroupRecord for the all-
   router MGID must be created in every IB subnet that the IPoIB link
   encompasses. The format of the all-router MGID will be covered in
   next section.

8.0 Mapping for other Multicast Groups

   The support of general IP multicast [IPMULT] over IB is similar to
   the case of the special all-node multicast/broadcast group discussed



Chu & Kashyap                                                   [Page 7]

draft-ietf-ipoib-link-multicast-01.txt                        March 2002


   above. An algorithmic mapping is used so that given an IP multicast
   address, individual host can compute the corresponding IB multicast
   address (MGID) all by itself without having to consult an external
   entity. This also removes the need for an externally maintained IP to
   IB multicast mapping table.

   The IPoIB multicast mapping is defined as follows. The same mapping
   function is used for both IPv4 and IPv6 except the IPoIB signature
   field.

   |   8    |  4 |  4 |     16 bits     | 16 bits |      80 bits       |
   +------ -+----+----+-----------------+---------+--------------------+
   |11111111|0001|scop|<IPoIB signature>|< P_Key >|      group ID      |
   +--------+----+----+-----------------+---------+--------------------+

   Since a MGID allocated for transporting IP multicast datagrams is
   considered only a transient link-layer multicast address, all IB
   MGIDs allocated for IPoIB purpose SHOULD have T = 1. The scope bits
   SHALL be the same as that of the all-node MGID for the same IPoIB
   link.

   An IP multicast address is used together with a given IPoIB link
   P_Key to form the MGID of the IB multicast group. For IPv6 the lower
   80-bit of the group ID is used directly in the lower 80-bit of the
   MGID. For IPv4, the group ID is only 28-bit long and the rest of the
   80 bits are filled with 0.

   The rest of the bits are the same as those of the all-node MGID.
   E.g. on an IPoIB link that is fully contained within a single IB
   subnet with a P_Key of 8, the MGIDs for the all-router multicast
   group with group ID 2 [AARCH, IGMP2] are:

   FF12:401B:8:0:0:0:0:2

   or

   FF12:401B:8::2

   for IPv4 in a compressed format, and

   FF12:601B:8:0:0:0:0:2

   or

   FF12:601B:8::2

   for IPv6 in a compressed format.




Chu & Kashyap                                                   [Page 8]

draft-ietf-ipoib-link-multicast-01.txt                        March 2002


   A special case exists for the IPv4 limited broadcast address
   "255.255.255.255" [HOSTS]. The address SHALL be mapped to the
   broadcast MGID for IPv4 networks as described in section 7 above.
   Also the IPv6 all-node multicast address "FF0X::1" [AARCH] will be
   mapped to the the special all-node MGID for IPv6 networks.

9.0 Sending and Receiving IP Multicast Packets

   In order to send a packet destined for an IP multicast address, a
   node must first check for the existence of MCGroupRecord
   corresponding to the MGID of the outbound link. If one already
   exists, the MLID allocated by the SM for the MCGroupRecord is used as
   the DLID for the packet. Otherwise, it means no member exists on the
   local link.  The packet should be forwarded to the all-router
   multicast group to ensure the correct delivery of the packet to
   multicast listeners on remote networks. (See section 10 below.) If an
   all-router multicast group doesn't already exist, it implies no
   router presence on the local subnet. The packet can then be safely
   dropped.

   Note that the local node MUST be notified when an IB multicast group
   corresponding to the MGID ever comes into existence later. This
   signifies that an interested party just showed up on the local link
   and therefore must be copied.

   For a node to join an IP multicast group to receive IP multicast
   packets, it must first construct a MGID corresponding to the IP
   multicast group, using the rule described above. Note that it must
   remember the scope bits from the all-node MGID, and use the same
   scope in all the MGIDs it constructs.

   The local node then checks with SA to see if a MCGroupRecord
   corresponding to the MGID already exists. If not, one must be
   created.  The MCGroupRecord MUST be created with the IPoIB link MTU.
   For the rest of the attributes, it is recommended that it uses the
   same values from the all-node multicast/broadcast group corresponding
   to the link.

   Note that for an IPoIB link that spans more than one IB subnet
   connected by IB routers, adequate multicast forwarding support at the
   IB level is required for multicast packets to be forwarded properly
   to members in remote IB subnets. The specific mechanism for this will
   be covered in [IBTA], and is out of scope of this document.

   Once the IB multicast group is identified, the node must join the
   group, unless it is a member already, by calling the SA to create a
   MCMemberRecord corresponding to the MGID.  The join call enables SM
   to program local IB switches and routers with the new multicast



Chu & Kashyap                                                   [Page 9]

draft-ietf-ipoib-link-multicast-01.txt                        March 2002


   information.  Specifically it causes an IB switch to add the LID of
   the caller to its forwarding table entry corresponding to the MLID
   allocated for the group.  It also causes an IB router to attach
   itself to the IB multicast tree corresponding to the MGID.

   In case any of the above IB operations fails, a node MAY choose to
   simply join the all-router multicast group. This will ensure it
   receives a copy of every multicast packet on the local link. Nodes
   doing so MUST filter out those multicast packets that are of no
   interest to the local node.

   When a node leaves an IP multicast group, it SHOULD delete the
   MCMemberRecord from the SA. This allows the SA to free up related
   resources. SM should delete MCGroupRecords that are no longer in use,
   and free up the MLIDs allocated for them. The specific algorithm is
   implementation-dependent, and therefore is out of scope of this
   document.

10.0 Support for IP Multicast Routing

   IP multicast routing requires a router to receive a copy of every
   link multicast packet on a locally connected link [IPMULT, IP6MLD].
   For Ethernet this is usually done by turning on promiscuous multicast
   mode on a locally connected Ethernet interface.

   Unfortunately IBA does not support promiscuous multicast mode.
   Therefore the IPoIB driver should forward a copy of every outbound
   multicast datagram to the MGID corresponding to the all-router
   multicast group.  This is to ensure multicast packets be properly
   forwarded to remote IP networks, and applies to IP hosts as well as
   multicast routers.

11.0 Security Considerations

   All the operations for creating and configuring an IPoIB link
   described in this document, including assigning P_Keys to CAs,
   creating MCGroupRecords and MCMemberRecords in SA, creating and
   attaching QPs to IB multicast groups,... etc are privileged
   operations, and MUST be protected by the underlying operating system.
   This is to prevent malicious, non- privileged software from hijacking
   important resources and configurations.  E.g. A bogus all-node IPoIB
   multicast group may prevent a proper one from being created when the
   network administrator tries to set up a link.

   Controlled Q_Keys SHOULD be used in IB multicast groups in order to
   prevent non-privileged software from fabricating IP datagrams to
   send, as mentioned in section 6.2.




Chu & Kashyap                                                  [Page 10]

draft-ietf-ipoib-link-multicast-01.txt                        March 2002


12.0 Acknowledgments

   The authors would like to thank Bruce Beukema, David Brean, Dan
   Cassiday, Aditya Dube, Yaron Haviv, Thomas Narten, Erik Nordmark,
   Greg Pfister, Renato Recio, Satya Sharma, David L. Stevens, and Madhu
   Talluri for their suggestions and many clarifications on the IBA
   specification.

13.0 References

   [AARCH]   Hinden, R. and S. Deering "IP Version 6 Addressing
             Architecture", RFC 2373, July 1998.

   [DHCP]    R. Droms "Dynamic Host Configuration Protocol", RFC 2131,
             March 1997.

   [DISC]    Narten, T., Nordmark, E. and W. Simpson, "Neighbor
             Discovery for IP Version 6 (IPv6)", RFC 2461, December
             1998.

   [HOSTS]   Braden R., "Requirements for Internet Hosts --
             Communication Layers", RFC 1122, October 1989

   [IBTA]    InfiniBand Architecture Specification, Release 1.0.a by
             InfiniBand Trade Association at www.infinibandta.org

   [IGMP2]   Fenner W., "Internet Group Management Protocol, Version 2",
             RFC 2236, November 1997.

   [IPMULT]  Deering S., "Host Extensions for IP Multicasting", RFC
             1112, August 1989.

   [IPoIB_ARCH]  draft-ietf-ipoib-architecture-01.txt

   [IPoIB_ENCAP] draft-ietf-ipoib-ip-over-infiniband-00.txt

   [IPV6]    Deering, S. and R. Hinden, "Internet Protocol, Version 6
             (IPv6) Specification", RFC 2460, December 1998.

   [IP6MLD]  Deering S., Fenner W., Haberman B., "Multicast Listener
             Discovery (MLD) for IPv6", RFC 2710, October 1999.

   [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
             Requirement Levels", BCP 14, RFC 2119, March 1997.


14.0 Author's Address




Chu & Kashyap                                                  [Page 11]

draft-ietf-ipoib-link-multicast-01.txt                        March 2002


   H.K. Jerry Chu
   901 San Antonio Road, UMPK17-201
   Palo Alto, CA 94303-4900
   USA

   Phone: +1 650 786-5146
   EMail: jerry.chu@sun.com


   Vivek Kashyap
   IBM
   15450, SW Koll Parkway
   Beaverton, OR 97006

   Phone: 503 578 3422
   EMail: vivk@us.ibm.com



15.0 Full Copyright Statement

   Copyright (C) The Internet Society (2001>.  All Rights Reserved.

   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph are
   included on all such copies and derivative works.  However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.





Chu & Kashyap                                                  [Page 12]


--Brace_of_Greyhounds_046_000--

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 15:42:32 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA18842
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 15:42:32 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA15889
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 15:42:35 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA15756;
	Tue, 9 Apr 2002 15:37:40 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA15716
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 15:37:37 -0400 (EDT)
Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA18565
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 15:37:33 -0400 (EDT)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e31.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id g39JbYNr017916;
	Tue, 9 Apr 2002 15:37:34 -0400
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g39JbKD136776;
	Tue, 9 Apr 2002 13:37:20 -0600
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g39JbFX08392;
	Tue, 9 Apr 2002 12:37:15 -0700
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g39JbEO25031;
	Tue, 9 Apr 2002 12:37:14 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204091937.g39JbEO25031@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: bill@strahm.net (Bill Strahm)
Date: Tue, 9 Apr 2002 12:37:12 -0700 (PDT)
Cc: sganguly@yahoo.com (Sukanta ganguly), bill@strahm.net (Bill Strahm),
        Paul.Culley@compaq.com (Culley Paul),
        kashyapv@us.ibm.com (Vivek Kashyap), ipoverib@ietf.org,
        fenner@research.att.com
In-Reply-To: <20020409114037.F70815@homegate.strahm.net> from "Bill Strahm" at Apr 09, 2002 10:40:37 AM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

Bill,
        The multicast router does not 'join' any group at the IP
        level. The router sets the ethernet interface in all-multicast
        promiscuous mode. Thus the router gets all multicast packets.

        It filters or forwards the packets based on the multicast
        routing protocol updates.

Vivek


> 
> Question then...
> 
> As an IP Multicast sender, I am not required to join 
> the Multicast group... So now there has not been an
> IGMP message telling the first hop router to join the
> group either.  So the router has not told its layer
> 2 to join the broadcast address for the multicast group.
> 
> How does the first hop Multicast router know to join
> this group and forward the traffic on to the multicast
> tree (either RP, or Prune/Flood) if it can't receive
> the initial packet...
> 
> This is the magic that I was wanting to know, I know how
> I would make it work, but I am hoping Bill F. can tell me
> the correct way to do it <G>
> 
> Bill Strahm
> 
> On Tue, Apr 09, 2002 at 11:36:22AM -0700, Sukanta ganguly wrote:
> > Bill,
> >   When you send in the Multicast packet (as a sender),
> > at layer 3 you send it to the multicast IP address
> > that the sender decided on. Your layer 2 module will
> > decide on the next hop for that Multicast IP address.
> > And a similar functioning will be build up as a chain.
> > Note in this pacth while the senders original packet
> > is moving to its destination which is the IP multicast
> > address, the intermediate routers (which are multicast
> > aware) will check to see if they have any IGMP joins
> > they have received from any other subnets. If they
> > have any such mapping they have built due to receive
> > of IGMP joins then they will oblige that join by
> > copying the packet to that joined destination. That's
> > how the copy of the packet is enroute to its rightful
> > receiver.
> > 
> > SG
> > 
> > P.S :: Dr Fenner is an authority on this and can
> > provide some special comments 
> > 
> > 
> > --- Bill Strahm <bill@strahm.net> wrote:
> > > Ok, I am going to express a scary amount of
> > > ignorance here...
> > > 
> > > I know how multicast works on receive, I know how
> > > IGMP works to build the forwarding tree,
> > > I know how flood/prune, RP Multicast Routing
> > > protocols work...
> > > 
> > > What I would like a pointer to is a simple this is
> > > how a first hop multicast router 
> > > behaves.  An example on Ethernet would be
> > > acceptable, a generic example would be better.
> > > 
> > > The question I have is as a Multicast sender, I send
> > > a multicast packet to the first
> > > hop router that is multicast aware.  Now, do I send
> > > that packet to its unicast MAC 
> > > address, or do I send that to the mapped layer 2
> > > broadcast address and let the IP 
> > > layer rebroadcast to the broadcast address if it is
> > > needed on the local subnet.
> > > 
> > > Once I have that, everything else I understand
> > > 
> > > Bill
> > > PS Bill Fenner, the IPoIB list is subscriber posting
> > > only.  If you haven't subscribed, please do 
> > > so we can get the benefit of your wisdom...
> > > 
> > > On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley,
> > > Paul wrote:
> > > > I suspect that the problem can be made simpler; IP
> > > routers should always IB_join ALL IPoIB Multicast
> > > groups for a given interface.  The router will
> > > simply ask to join any groups in the same
> > > partition(s) it is assigned to.  Whether the IP
> > > router forwards the multicast packets will then
> > > depend on the IP routing protocol.
> > > > 
> > > > IB routers have the same problem, and may use the
> > > same solution...
> > > > 
> > > > Paul R. Culley
> > > > Compaq Sr. Fellow
> > > > 281-514-5543
> > > > 
> > > > 
> > > > -----Original Message-----
> > > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > > > Sent: Tuesday, April 09, 2002 3:31 AM
> > > > To: ipoverib@ietf.org
> > > > Cc: fenner@research.att.com
> > > > Subject: [Ipoverib] IPoIB multicast forwarding
> > > > 
> > > > 
> > > > 
> > > > Folks,
> > > > 
> > > >         We have defined a mapping from the IP
> > > addresses to the IB
> > > >         multicast groups. Use of different groups
> > > causes minimal
> > > >         replication at the switches and less load
> > > at the receivers.
> > > > 
> > > >         But there is a quirk in the way the groups
> > > are setup in
> > > >         InfiniBand - there is no way for any of
> > > the nodes to know if
> > > >         another has created an InfiniBand group.
> > > This is easy to fix
> > > >         in most cases - one checks with the SM.
> > > But it doesn't work
> > > >         out well for IPoIB packet forwarding.
> > > > 
> > > >         The router needs to forward the multicast
> > > packets beyond the
> > > >         local IP subnet if it knows there are
> > > recipients for it or in
> > > >         the case of some routing protocols it must
> > > do that unless it
> > > >         gets an update stating not to forward.
> > > Similarly, it needs to
> > > >         forward external packets to the local
> > > receivers if there are
> > > >         any receiver present. 
> > > > 
> > > >         Thus if there are IB multicast groups
> > > associated with an IP
> > > >         multicast group the router must know about
> > > them or the router
> > > >         must receive all multicast packets in the
> > > subnet via some
> > > >         other construct.
> > > > 
> > > >         Below are the three alternatives that I
> > > mentioned at the
> > > >         53rd IETF meet. 
> > > > 	
> > > >         i) Define only one IB multicast group i.e.
> > > IB broadcast MGID.
> > > >         All IP multicast packets use this group.
> > > > 
> > > >         ii) The multicast IP packet is sent to
> > > both the associated IB
> > > >         multicast group and the IB mutlicast group
> > > corresponding to
> > > >         the all-IP routers IP group.
> > > > 
> > > >         iii) Sender always sends to corresonding
> > > IB group if it exists
> > > >         else it sends to the broadcast group.
> > > > 
> > > >              IP receivers create IB groups and
> > > send the IGMP/MLD
> > > >              groups on the broadcast MGID.
> > > > 
> > > >              Routers join relevant groups based on
> > > the IGMP report
> > > >              and routing protocol updates.
> > > > 
> > > > 
> > > >         I also put this problem to Bill Fenner (AD
> > > of routing area)
> > > >         and asked his advice on the alternatives.
> > > Of the three options
> > > >         he favored option (iii) above.
> > > > 
> > > >         Option (i) doesn't take advantage fo IB
> > > multicast at all and
> > > >         so should be looked at as a last resort
> > > solution only.
> > > > 
> > > >         Option (ii) requires the sender to send
> > > two copies of the same
> > > >         packet and 'deliberately' increases the
> > > chance of duplicates
> > > >         spreading across the network.
> > > > 
> > > > 
> > > > Suggested solution  in detail:
> > > > 
> > > > 
> > > >         1. Pure senders - at IB level check for
> > > the relevant IB group.
> > > >            If it exists then send to the IB group
> > > else send to the
> > > >            'all-routers' IB group. If there is no
> > > IB group
> > > >            corresponding to a particular IP group
> > > then it implies that
> > > >            there are no IP receivers on the subnet
> > > for that IP
> > > >            multicast group. 
> > > > 
> > > >            Note: a) Pure senders don't create IB
> > > groups. This is
> > > >            because there is no existing mechanism
> > > for the senders to
> > > >            let the routers know of the new groups.
> > > > 
> > > >         2. IP receivers: As per IGMP/MLD the first
> > > packet sent by a
> > > >            host on joining a multicast group for
> > > receiving
> > > >            (IP_ADD_MEMBERSHIP) is the multicast
> > > report.
> > > > 
> > > > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> > > do the following:
> > > > 
> > > > 	   	check if the corresponding IB group exists. 
> > > > 
> > > > 		if (IB group exists)
> > > > 			IB_join the group and send the IGMP/MLD report
> > > > 
> > > > 		else
> > > > 			IB_Create the group
> > > > 			IB_join the group
> > > >                         Send IGMP/MLD report to IB
> > > group corresponding
> > > >                         to all-routers (all-hosts
> > > or broadcast) IP
> > > >                         group (Note: only done
> > > when creating the IB
> > > >                         group) 
> > > > 
> > > > 	3. IP routers IB_join MGIDs on:
> > > > 
> > > > 		a) Finding a new IP group on the subnet in the
> > > IGMP/MLD report
> > > >                 b) Determining a need to forward
> > > IP multicast packets
> > > >                    through a routing protocol
> > > > 
> > > > 
> > > > 
> > > > Comments ?
> > > > 
> > > > 
> > > > Vivek
> > > 
> > === message truncated ===
> > 
> > 
> > __________________________________________________
> > Do You Yahoo!?
> > Yahoo! Tax Center - online filing with TurboTax
> > http://taxes.yahoo.com/
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 15:59:13 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA19527
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 15:59:13 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA16621
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 15:59:16 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA16436;
	Tue, 9 Apr 2002 15:53:46 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA16402
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 15:53:43 -0400 (EDT)
Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA19347
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 15:53:39 -0400 (EDT)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e31.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id g39JrdNr042040;
	Tue, 9 Apr 2002 15:53:39 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g39JrED19280;
	Tue, 9 Apr 2002 13:53:14 -0600
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g39JrBM24618;
	Tue, 9 Apr 2002 12:53:11 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g39Jr9J25506;
	Tue, 9 Apr 2002 12:53:09 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204091953.g39Jr9J25506@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: Jerry.Chu@eng.sun.com
Date: Tue, 9 Apr 2002 12:53:07 -0700 (PDT)
Cc: bill@strahm.net, sganguly@yahoo.com, roy.k.larsen@intel.com,
        Paul.Culley@compaq.com, kashyapv@us.ibm.com, ipoverib@ietf.org,
        fenner@research.att.com
In-Reply-To: <200204091928.g39JSi35628548@jurassic.eng.sun.com> from "H.K. Jerry Chu" at Apr 09, 2002 11:28:40 AM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> The whole problem is that IB doesn't support a promiscuous multicast mode.
> This problem is described in the link-multicast draft and my proposed
> solution to put the burden on the sender to send a copy to the all-router
> MC group. Vivek is making a different proposal. Both have their pros and
> cons. (Sorry I haven't got the time to follow all the details of the
> discussion so far. Will do some catch up later.)
> 
> Of course if IBA provides a solution for a router to turn on promiscuous
> mode (or equivalent ie recieving a copy of every MC pkt) then the problem
> doesn't exist for us (IPoIB).
> 
> There is a related question on whether in IB a sender is required to
> join a MC group to send. To me the 1.0 spec is clear on this and it
> follows IP model. But some in IBTA disagree and it's been discussion
> in IBTA. I've asked why makes the IB MC model different (i.e. requiring
> join to send) from IP MC model. But it's IBTA's call to make the best
> choice for IB.

Why does this matter to IPoIB ? If the IB group is joined at the IB
layer or the IB group is not joined at the IB layer, there is no
effect on IPoIB. The driver will do the needful at the IB layer.

Anyway, this is an issue for IBTA to clarify. The IP forwarding
problem is a different one.

Vivek

> 
> I've enclosed the latest link-multicast (-01). Take a took at section
> 9 and 10. I'll republish the draft after the consensus call.
> 

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 16:06:45 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA20085
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 16:06:45 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA17445
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 16:06:47 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA17213;
	Tue, 9 Apr 2002 16:03:08 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA17170
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 16:03:05 -0400 (EDT)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA19750
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 16:02:59 -0400 (EDT)
Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.37 2002/04/01 17:50:55 root Exp $) with ESMTP id g39K2JU04652
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 20:02:19 GMT
Received: from fmsmsxvs042.fm.intel.com (fmsmsxv042-1.fm.intel.com [132.233.48.110])
	by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.15 2002/04/01 17:51:48 root Exp $) with SMTP id g39K1ST11692
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 20:01:28 GMT
Received: from fmsmsx26.fm.intel.com ([132.233.42.26])
 by fmsmsxvs042.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002040913064307199
 ; Tue, 09 Apr 2002 13:06:44 -0700
Received: by fmsmsx26.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2HL0C77Q>; Tue, 9 Apr 2002 13:02:50 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F6F@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Vivek Kashyap'" <kashyapv@us.ibm.com>, bill@strahm.net
Cc: sganguly@yahoo.com, bill@strahm.net, Paul.Culley@compaq.com,
        ipoverib@ietf.org, fenner@research.att.com
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Tue, 9 Apr 2002 13:02:44 -0700 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Ahh, that clarifies it for me.  However, from an implementation point of
view, the IPoIB link driver can not indicate that it supports promiscuous
mode, so one would presume that the OS network infrastructure will not use
that interface for multicast routing unless the OS network stack were
modified.  If you need to modify the network stack, why not simply alter it
so that it IB creates/joins all multicast groups it has received updates
for?  In other words, keep all the special processing and "fixes" local to
the router implementation and out of the common host IPoIB driver.

Roy

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
Sent: Tuesday, April 09, 2002 12:37 PM
To: bill@strahm.net
Cc: sganguly@yahoo.com; bill@strahm.net; Paul.Culley@compaq.com;
kashyapv@us.ibm.com; ipoverib@ietf.org; fenner@research.att.com
Subject: Re: [Ipoverib] IPoIB multicast forwarding


Bill,
        The multicast router does not 'join' any group at the IP
        level. The router sets the ethernet interface in all-multicast
        promiscuous mode. Thus the router gets all multicast packets.

        It filters or forwards the packets based on the multicast
        routing protocol updates.

Vivek


> 
> Question then...
> 
> As an IP Multicast sender, I am not required to join 
> the Multicast group... So now there has not been an
> IGMP message telling the first hop router to join the
> group either.  So the router has not told its layer
> 2 to join the broadcast address for the multicast group.
> 
> How does the first hop Multicast router know to join
> this group and forward the traffic on to the multicast
> tree (either RP, or Prune/Flood) if it can't receive
> the initial packet...
> 
> This is the magic that I was wanting to know, I know how
> I would make it work, but I am hoping Bill F. can tell me
> the correct way to do it <G>
> 
> Bill Strahm
> 
> On Tue, Apr 09, 2002 at 11:36:22AM -0700, Sukanta ganguly wrote:
> > Bill,
> >   When you send in the Multicast packet (as a sender),
> > at layer 3 you send it to the multicast IP address
> > that the sender decided on. Your layer 2 module will
> > decide on the next hop for that Multicast IP address.
> > And a similar functioning will be build up as a chain.
> > Note in this pacth while the senders original packet
> > is moving to its destination which is the IP multicast
> > address, the intermediate routers (which are multicast
> > aware) will check to see if they have any IGMP joins
> > they have received from any other subnets. If they
> > have any such mapping they have built due to receive
> > of IGMP joins then they will oblige that join by
> > copying the packet to that joined destination. That's
> > how the copy of the packet is enroute to its rightful
> > receiver.
> > 
> > SG
> > 
> > P.S :: Dr Fenner is an authority on this and can
> > provide some special comments 
> > 
> > 
> > --- Bill Strahm <bill@strahm.net> wrote:
> > > Ok, I am going to express a scary amount of
> > > ignorance here...
> > > 
> > > I know how multicast works on receive, I know how
> > > IGMP works to build the forwarding tree,
> > > I know how flood/prune, RP Multicast Routing
> > > protocols work...
> > > 
> > > What I would like a pointer to is a simple this is
> > > how a first hop multicast router 
> > > behaves.  An example on Ethernet would be
> > > acceptable, a generic example would be better.
> > > 
> > > The question I have is as a Multicast sender, I send
> > > a multicast packet to the first
> > > hop router that is multicast aware.  Now, do I send
> > > that packet to its unicast MAC 
> > > address, or do I send that to the mapped layer 2
> > > broadcast address and let the IP 
> > > layer rebroadcast to the broadcast address if it is
> > > needed on the local subnet.
> > > 
> > > Once I have that, everything else I understand
> > > 
> > > Bill
> > > PS Bill Fenner, the IPoIB list is subscriber posting
> > > only.  If you haven't subscribed, please do 
> > > so we can get the benefit of your wisdom...
> > > 
> > > On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley,
> > > Paul wrote:
> > > > I suspect that the problem can be made simpler; IP
> > > routers should always IB_join ALL IPoIB Multicast
> > > groups for a given interface.  The router will
> > > simply ask to join any groups in the same
> > > partition(s) it is assigned to.  Whether the IP
> > > router forwards the multicast packets will then
> > > depend on the IP routing protocol.
> > > > 
> > > > IB routers have the same problem, and may use the
> > > same solution...
> > > > 
> > > > Paul R. Culley
> > > > Compaq Sr. Fellow
> > > > 281-514-5543
> > > > 
> > > > 
> > > > -----Original Message-----
> > > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > > > Sent: Tuesday, April 09, 2002 3:31 AM
> > > > To: ipoverib@ietf.org
> > > > Cc: fenner@research.att.com
> > > > Subject: [Ipoverib] IPoIB multicast forwarding
> > > > 
> > > > 
> > > > 
> > > > Folks,
> > > > 
> > > >         We have defined a mapping from the IP
> > > addresses to the IB
> > > >         multicast groups. Use of different groups
> > > causes minimal
> > > >         replication at the switches and less load
> > > at the receivers.
> > > > 
> > > >         But there is a quirk in the way the groups
> > > are setup in
> > > >         InfiniBand - there is no way for any of
> > > the nodes to know if
> > > >         another has created an InfiniBand group.
> > > This is easy to fix
> > > >         in most cases - one checks with the SM.
> > > But it doesn't work
> > > >         out well for IPoIB packet forwarding.
> > > > 
> > > >         The router needs to forward the multicast
> > > packets beyond the
> > > >         local IP subnet if it knows there are
> > > recipients for it or in
> > > >         the case of some routing protocols it must
> > > do that unless it
> > > >         gets an update stating not to forward.
> > > Similarly, it needs to
> > > >         forward external packets to the local
> > > receivers if there are
> > > >         any receiver present. 
> > > > 
> > > >         Thus if there are IB multicast groups
> > > associated with an IP
> > > >         multicast group the router must know about
> > > them or the router
> > > >         must receive all multicast packets in the
> > > subnet via some
> > > >         other construct.
> > > > 
> > > >         Below are the three alternatives that I
> > > mentioned at the
> > > >         53rd IETF meet. 
> > > > 	
> > > >         i) Define only one IB multicast group i.e.
> > > IB broadcast MGID.
> > > >         All IP multicast packets use this group.
> > > > 
> > > >         ii) The multicast IP packet is sent to
> > > both the associated IB
> > > >         multicast group and the IB mutlicast group
> > > corresponding to
> > > >         the all-IP routers IP group.
> > > > 
> > > >         iii) Sender always sends to corresonding
> > > IB group if it exists
> > > >         else it sends to the broadcast group.
> > > > 
> > > >              IP receivers create IB groups and
> > > send the IGMP/MLD
> > > >              groups on the broadcast MGID.
> > > > 
> > > >              Routers join relevant groups based on
> > > the IGMP report
> > > >              and routing protocol updates.
> > > > 
> > > > 
> > > >         I also put this problem to Bill Fenner (AD
> > > of routing area)
> > > >         and asked his advice on the alternatives.
> > > Of the three options
> > > >         he favored option (iii) above.
> > > > 
> > > >         Option (i) doesn't take advantage fo IB
> > > multicast at all and
> > > >         so should be looked at as a last resort
> > > solution only.
> > > > 
> > > >         Option (ii) requires the sender to send
> > > two copies of the same
> > > >         packet and 'deliberately' increases the
> > > chance of duplicates
> > > >         spreading across the network.
> > > > 
> > > > 
> > > > Suggested solution  in detail:
> > > > 
> > > > 
> > > >         1. Pure senders - at IB level check for
> > > the relevant IB group.
> > > >            If it exists then send to the IB group
> > > else send to the
> > > >            'all-routers' IB group. If there is no
> > > IB group
> > > >            corresponding to a particular IP group
> > > then it implies that
> > > >            there are no IP receivers on the subnet
> > > for that IP
> > > >            multicast group. 
> > > > 
> > > >            Note: a) Pure senders don't create IB
> > > groups. This is
> > > >            because there is no existing mechanism
> > > for the senders to
> > > >            let the routers know of the new groups.
> > > > 
> > > >         2. IP receivers: As per IGMP/MLD the first
> > > packet sent by a
> > > >            host on joining a multicast group for
> > > receiving
> > > >            (IP_ADD_MEMBERSHIP) is the multicast
> > > report.
> > > > 
> > > > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> > > do the following:
> > > > 
> > > > 	   	check if the corresponding IB group exists. 
> > > > 
> > > > 		if (IB group exists)
> > > > 			IB_join the group and send the IGMP/MLD
report
> > > > 
> > > > 		else
> > > > 			IB_Create the group
> > > > 			IB_join the group
> > > >                         Send IGMP/MLD report to IB
> > > group corresponding
> > > >                         to all-routers (all-hosts
> > > or broadcast) IP
> > > >                         group (Note: only done
> > > when creating the IB
> > > >                         group) 
> > > > 
> > > > 	3. IP routers IB_join MGIDs on:
> > > > 
> > > > 		a) Finding a new IP group on the subnet in the
> > > IGMP/MLD report
> > > >                 b) Determining a need to forward
> > > IP multicast packets
> > > >                    through a routing protocol
> > > > 
> > > > 
> > > > 
> > > > Comments ?
> > > > 
> > > > 
> > > > Vivek
> > > 
> > === message truncated ===
> > 
> > 
> > __________________________________________________
> > Do You Yahoo!?
> > Yahoo! Tax Center - online filing with TurboTax
> > http://taxes.yahoo.com/
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 16:25:54 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA21198
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 16:25:53 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA18236
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 16:25:56 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA18118;
	Tue, 9 Apr 2002 16:21:47 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA18050
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 16:21:43 -0400 (EDT)
Received: from e33.esmtp.ibm.com (e33.co.us.ibm.com [32.97.110.131])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA20923
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 16:21:39 -0400 (EDT)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e33.esmtp.ibm.com (8.12.2/8.12.2) with ESMTP id g39KLeh0028428;
	Tue, 9 Apr 2002 16:21:40 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g39KKMD143100;
	Tue, 9 Apr 2002 14:20:22 -0600
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g39KKHM25410;
	Tue, 9 Apr 2002 13:20:17 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g39KKFW25942;
	Tue, 9 Apr 2002 13:20:15 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204092020.g39KKFW25942@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: roy.k.larsen@intel.com (Larsen, Roy K)
Date: Tue, 9 Apr 2002 13:20:14 -0700 (PDT)
Cc: ipoverib@ietf.org
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4F6F@ORSMSX108> from "Larsen, Roy K" at Apr 09, 2002 12:02:44 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> Ahh, that clarifies it for me.  However, from an implementation point of
> view, the IPoIB link driver can not indicate that it supports promiscuous
> mode, so one would presume that the OS network infrastructure will not use
> that interface for multicast routing unless the OS network stack were
> modified.  If you need to modify the network stack, why not simply alter it
> so that it IB creates/joins all multicast groups it has received updates
> for?  In other words, keep all the special processing and "fixes" local to
> the router implementation and out of the common host IPoIB driver.

Aren't you saying the same thing as the solution I have presented?

        The router knows the groups to forward to based on IGMP/MLD
        reports and multicast routing protocol updates. It therefore
        IB_joins (or creates in some cases) the corresponding IB
        MGIDs. 

For the above to occur the MLD/IGMP report must reach the router. So,
the host must send it to the router.

        The host network stack is not effected. The solution is
        limited to the host's interface driver. As I had mentioned in
        the presentation at the IETF meet: the IP_ADD_MEMBERSHIP call
        causes the host IP stack to tell the interface
        (ethernet/IPoIB/FDDI/Token ring etc.) of the IP multicast
        group. In the case of ethernet the ethernet driver updates the
        device/internal tables to include the corresponding ethernet
        mutlicast group. In the case of IPoIB the driver lets the
        SM/SA know (create if group doesn't exist; join if it
        exists). 

So, the solution is limited to a) routers' IPoIB module b) in the
IPoIB interface driver of the host.


Vivek
> 
> Roy
> 
> -----Original Message-----
> From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> Sent: Tuesday, April 09, 2002 12:37 PM
> To: bill@strahm.net
> Cc: sganguly@yahoo.com; bill@strahm.net; Paul.Culley@compaq.com;
> kashyapv@us.ibm.com; ipoverib@ietf.org; fenner@research.att.com
> Subject: Re: [Ipoverib] IPoIB multicast forwarding
> 
> 
> Bill,
>         The multicast router does not 'join' any group at the IP
>         level. The router sets the ethernet interface in all-multicast
>         promiscuous mode. Thus the router gets all multicast packets.
> 
>         It filters or forwards the packets based on the multicast
>         routing protocol updates.
> 
> Vivek
> 
> 
> > 
> > Question then...
> > 
> > As an IP Multicast sender, I am not required to join 
> > the Multicast group... So now there has not been an
> > IGMP message telling the first hop router to join the
> > group either.  So the router has not told its layer
> > 2 to join the broadcast address for the multicast group.
> > 
> > How does the first hop Multicast router know to join
> > this group and forward the traffic on to the multicast
> > tree (either RP, or Prune/Flood) if it can't receive
> > the initial packet...
> > 
> > This is the magic that I was wanting to know, I know how
> > I would make it work, but I am hoping Bill F. can tell me
> > the correct way to do it <G>
> > 
> > Bill Strahm
> > 
> > On Tue, Apr 09, 2002 at 11:36:22AM -0700, Sukanta ganguly wrote:
> > > Bill,
> > >   When you send in the Multicast packet (as a sender),
> > > at layer 3 you send it to the multicast IP address
> > > that the sender decided on. Your layer 2 module will
> > > decide on the next hop for that Multicast IP address.
> > > And a similar functioning will be build up as a chain.
> > > Note in this pacth while the senders original packet
> > > is moving to its destination which is the IP multicast
> > > address, the intermediate routers (which are multicast
> > > aware) will check to see if they have any IGMP joins
> > > they have received from any other subnets. If they
> > > have any such mapping they have built due to receive
> > > of IGMP joins then they will oblige that join by
> > > copying the packet to that joined destination. That's
> > > how the copy of the packet is enroute to its rightful
> > > receiver.
> > > 
> > > SG
> > > 
> > > P.S :: Dr Fenner is an authority on this and can
> > > provide some special comments 
> > > 
> > > 
> > > --- Bill Strahm <bill@strahm.net> wrote:
> > > > Ok, I am going to express a scary amount of
> > > > ignorance here...
> > > > 
> > > > I know how multicast works on receive, I know how
> > > > IGMP works to build the forwarding tree,
> > > > I know how flood/prune, RP Multicast Routing
> > > > protocols work...
> > > > 
> > > > What I would like a pointer to is a simple this is
> > > > how a first hop multicast router 
> > > > behaves.  An example on Ethernet would be
> > > > acceptable, a generic example would be better.
> > > > 
> > > > The question I have is as a Multicast sender, I send
> > > > a multicast packet to the first
> > > > hop router that is multicast aware.  Now, do I send
> > > > that packet to its unicast MAC 
> > > > address, or do I send that to the mapped layer 2
> > > > broadcast address and let the IP 
> > > > layer rebroadcast to the broadcast address if it is
> > > > needed on the local subnet.
> > > > 
> > > > Once I have that, everything else I understand
> > > > 
> > > > Bill
> > > > PS Bill Fenner, the IPoIB list is subscriber posting
> > > > only.  If you haven't subscribed, please do 
> > > > so we can get the benefit of your wisdom...
> > > > 
> > > > On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley,
> > > > Paul wrote:
> > > > > I suspect that the problem can be made simpler; IP
> > > > routers should always IB_join ALL IPoIB Multicast
> > > > groups for a given interface.  The router will
> > > > simply ask to join any groups in the same
> > > > partition(s) it is assigned to.  Whether the IP
> > > > router forwards the multicast packets will then
> > > > depend on the IP routing protocol.
> > > > > 
> > > > > IB routers have the same problem, and may use the
> > > > same solution...
> > > > > 
> > > > > Paul R. Culley
> > > > > Compaq Sr. Fellow
> > > > > 281-514-5543
> > > > > 
> > > > > 
> > > > > -----Original Message-----
> > > > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > > > > Sent: Tuesday, April 09, 2002 3:31 AM
> > > > > To: ipoverib@ietf.org
> > > > > Cc: fenner@research.att.com
> > > > > Subject: [Ipoverib] IPoIB multicast forwarding
> > > > > 
> > > > > 
> > > > > 
> > > > > Folks,
> > > > > 
> > > > >         We have defined a mapping from the IP
> > > > addresses to the IB
> > > > >         multicast groups. Use of different groups
> > > > causes minimal
> > > > >         replication at the switches and less load
> > > > at the receivers.
> > > > > 
> > > > >         But there is a quirk in the way the groups
> > > > are setup in
> > > > >         InfiniBand - there is no way for any of
> > > > the nodes to know if
> > > > >         another has created an InfiniBand group.
> > > > This is easy to fix
> > > > >         in most cases - one checks with the SM.
> > > > But it doesn't work
> > > > >         out well for IPoIB packet forwarding.
> > > > > 
> > > > >         The router needs to forward the multicast
> > > > packets beyond the
> > > > >         local IP subnet if it knows there are
> > > > recipients for it or in
> > > > >         the case of some routing protocols it must
> > > > do that unless it
> > > > >         gets an update stating not to forward.
> > > > Similarly, it needs to
> > > > >         forward external packets to the local
> > > > receivers if there are
> > > > >         any receiver present. 
> > > > > 
> > > > >         Thus if there are IB multicast groups
> > > > associated with an IP
> > > > >         multicast group the router must know about
> > > > them or the router
> > > > >         must receive all multicast packets in the
> > > > subnet via some
> > > > >         other construct.
> > > > > 
> > > > >         Below are the three alternatives that I
> > > > mentioned at the
> > > > >         53rd IETF meet. 
> > > > > 	
> > > > >         i) Define only one IB multicast group i.e.
> > > > IB broadcast MGID.
> > > > >         All IP multicast packets use this group.
> > > > > 
> > > > >         ii) The multicast IP packet is sent to
> > > > both the associated IB
> > > > >         multicast group and the IB mutlicast group
> > > > corresponding to
> > > > >         the all-IP routers IP group.
> > > > > 
> > > > >         iii) Sender always sends to corresonding
> > > > IB group if it exists
> > > > >         else it sends to the broadcast group.
> > > > > 
> > > > >              IP receivers create IB groups and
> > > > send the IGMP/MLD
> > > > >              groups on the broadcast MGID.
> > > > > 
> > > > >              Routers join relevant groups based on
> > > > the IGMP report
> > > > >              and routing protocol updates.
> > > > > 
> > > > > 
> > > > >         I also put this problem to Bill Fenner (AD
> > > > of routing area)
> > > > >         and asked his advice on the alternatives.
> > > > Of the three options
> > > > >         he favored option (iii) above.
> > > > > 
> > > > >         Option (i) doesn't take advantage fo IB
> > > > multicast at all and
> > > > >         so should be looked at as a last resort
> > > > solution only.
> > > > > 
> > > > >         Option (ii) requires the sender to send
> > > > two copies of the same
> > > > >         packet and 'deliberately' increases the
> > > > chance of duplicates
> > > > >         spreading across the network.
> > > > > 
> > > > > 
> > > > > Suggested solution  in detail:
> > > > > 
> > > > > 
> > > > >         1. Pure senders - at IB level check for
> > > > the relevant IB group.
> > > > >            If it exists then send to the IB group
> > > > else send to the
> > > > >            'all-routers' IB group. If there is no
> > > > IB group
> > > > >            corresponding to a particular IP group
> > > > then it implies that
> > > > >            there are no IP receivers on the subnet
> > > > for that IP
> > > > >            multicast group. 
> > > > > 
> > > > >            Note: a) Pure senders don't create IB
> > > > groups. This is
> > > > >            because there is no existing mechanism
> > > > for the senders to
> > > > >            let the routers know of the new groups.
> > > > > 
> > > > >         2. IP receivers: As per IGMP/MLD the first
> > > > packet sent by a
> > > > >            host on joining a multicast group for
> > > > receiving
> > > > >            (IP_ADD_MEMBERSHIP) is the multicast
> > > > report.
> > > > > 
> > > > > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> > > > do the following:
> > > > > 
> > > > > 	   	check if the corresponding IB group exists. 
> > > > > 
> > > > > 		if (IB group exists)
> > > > > 			IB_join the group and send the IGMP/MLD
> report
> > > > > 
> > > > > 		else
> > > > > 			IB_Create the group
> > > > > 			IB_join the group
> > > > >                         Send IGMP/MLD report to IB
> > > > group corresponding
> > > > >                         to all-routers (all-hosts
> > > > or broadcast) IP
> > > > >                         group (Note: only done
> > > > when creating the IB
> > > > >                         group) 
> > > > > 
> > > > > 	3. IP routers IB_join MGIDs on:
> > > > > 
> > > > > 		a) Finding a new IP group on the subnet in the
> > > > IGMP/MLD report
> > > > >                 b) Determining a need to forward
> > > > IP multicast packets
> > > > >                    through a routing protocol
> > > > > 
> > > > > 
> > > > > 
> > > > > Comments ?
> > > > > 
> > > > > 
> > > > > Vivek
> > > > 
> > > === message truncated ===
> > > 
> > > 
> > > __________________________________________________
> > > Do You Yahoo!?
> > > Yahoo! Tax Center - online filing with TurboTax
> > > http://taxes.yahoo.com/
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 19:37:22 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA25744
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 19:37:22 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA29032
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 19:37:22 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA28389;
	Tue, 9 Apr 2002 19:29:57 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA28355
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 19:29:50 -0400 (EDT)
Received: from mxout1.netvision.net.il (mxout1.netvision.net.il [194.90.9.20])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA25643
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 19:29:48 -0400 (EDT)
Received: from vaio505he ([12.65.132.145]) by mxout1.netvision.net.il
 (iPlanet Messaging Server 5.1 (built Sep  5 2001))
 with ESMTPA id <0GUB00F46QKJT0@mxout1.netvision.net.il> for ipoverib@ietf.org;
 Wed, 10 Apr 2002 02:29:14 +0300 (IDT)
Date: Tue, 09 Apr 2002 18:11:55 +0200
From: Yaron Haviv <yaronh@voltaire.com>
Subject: RE: [Ipoverib] HW address - VRRP Protocol
In-reply-to: <D9223EB959A5D511A98F00508B68C20C06BC4F69@ORSMSX108>
To: ipoverib@ietf.org, "'Larsen, Roy K'" <roy.k.larsen@intel.com>,
        yaronh@voltaire.com, "'H.K. Jerry Chu'" <Jerry.Chu@eng.sun.com>,
        kashyapv@us.ibm.com
Cc: "'Yousif, Mazin S'" <mazin.s.yousif@intel.com>
Reply-to: yaronh@voltaire.com
Message-id: <004801c1e01e$7b7f3720$cad05c8b@vaio505he>
MIME-version: 1.0
X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4807.1700
X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2910.0)
Content-type: text/plain; charset=iso-8859-1
Content-transfer-encoding: 7BIT
Importance: Normal
X-Priority: 3 (Normal)
X-MSMail-priority: Normal
Content-Transfer-Encoding: 7BIT
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7BIT

Recently I raised the issue of VRRP protocol and HW addresses

The problem in VRRP over IB is that the HW addresses are well known
addresses
Currently such well-known HW addresses are not supported in IB, also VRRP HW
addresses are 48-bit

We can easily add an IB well known VRRP address, which will obey the EUI
rules (see my original email)
The main problem is that in the case of a fail-over, the backup node needs
to take over the original HW address
(It could easily do so in Ethernet by sending out any packet on the wire
that will generate an update in the spanning tree, nothing should be done at
the host in a fail-over situation, it doesn't involve updating the ARP cache
since the HW address and IP are the same)

This could either be done by just taking over the GID, keep a different LID,
or by taking over both the GID and LID which will avoid the problem of
invalid cached path information
The GID/LID take over capability can serve IB not just for IPoIB but also to
any other fail-over mechanism (nodes, routers, etc')
it probably involves having the SM update the subnet routing tree, and
forwarding tables in switches, it might also consists on the already defined
multi-pathing IB mechanisms (LMC, APM, ..)
any suggestions for solutions are welcomed !!!

VRRP is a very important protocol, which is the prevailing standard for
routers fail-over, supported by all major vendors
IB, which is known for its high-availability, cannot allow it self not to
support VRRP

Yaron

>-----Original Message-----
>From:  ipoverib-admin@ietf.org [mailto:ipoverib-admin@ietf.org]  On Behalf
>Of Yaron Haviv
>Sent:  Sunday, March 24, 2002 2:09 PM
>To:    'ipoverib@ietf.org'
>Subject:       [Ipoverib] HW address - VRRP Protocol
>
>Since Jerry asked for more protocols that relate to the HW address, here is
>another one
>VRRP - Virtual Router Redundancy Protocol (RFC 2338)
>Is a very important protocol for implementing default gateway fail-over and
>load distribution
>An updated draft at -
>http://www.ietf.org/internet-drafts/draft-ietf-vrrp-spec-v2-06.txt
><http://www.ietf.org/internet-drafts/draft-ietf-vrrp-spec-v2-06.txt>
>
>It defines a well known 48 bit (Ethernet) HW address
(00-00-5E-00-01-{VRID})
>Since I believe IB community wouldn't like to harm VRRP functionality, we
>need to probably provide well known addresses to VRRP over IB (IPoIB),
there
>might be more issues around the GID to LID mapping, and crossing IB subnet
>Example well known IPoIB VRRP address:
>Use 48 to 64 bit EUI encapsulation method for the new GID address
>00-00-5E-FF-FF-00-01-{VRID} maybe add a well known QP to complete the 20
>byte address e.g. QP 2 (since 0 = SMI, 1 = GSI)
>We might need to deal with it at the SM (Path Record) somehow
>Yaron
>
>_______________________________________________
>IPoverIB mailing list
>IPoverIB@ietf.org
>https://www1.ietf.org/mailman/listinfo/ipoverib


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 19:37:23 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA25758
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 19:37:23 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA29046
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 19:37:24 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA28509;
	Tue, 9 Apr 2002 19:30:16 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA28477
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 19:30:13 -0400 (EDT)
Received: from mxout2.netvision.net.il (mxout2.netvision.net.il [194.90.9.21])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA25645
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 19:30:11 -0400 (EDT)
Received: from vaio505he ([12.65.132.145]) by mxout2.netvision.net.il
 (iPlanet Messaging Server 5.1 (built Sep  5 2001))
 with ESMTPA id <0GUB005CHQLD7C@mxout2.netvision.net.il> for ipoverib@ietf.org;
 Wed, 10 Apr 2002 02:29:42 +0300 (IDT)
Date: Tue, 09 Apr 2002 19:12:24 +0200
From: Yaron Haviv <yaronh@voltaire.com>
Subject: [ipoverib] IP Multicast and IGMP's
To: ipoverib@ietf.org
Cc: "'Vivek Kashyap'" <vivk@us.ibm.com>,
        "'H.K. Jerry Chu'" <Jerry.Chu@eng.sun.com>
Reply-to: yaronh@voltaire.com
Message-id: <004b01c1e01e$8aaa1e90$cad05c8b@vaio505he>
MIME-version: 1.0
X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4807.1700
X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2910.0)
Content-type: text/plain; charset=windows-1255
Content-transfer-encoding: 7BIT
Importance: Normal
X-Priority: 3 (Normal)
X-MSMail-priority: Normal
Content-Transfer-Encoding: 7BIT
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7BIT

This email refers to previous emails by Vivek raising the problems presented
by IP Multicast and proposes 2 possible solutions

In the Ethernet model, there is no explicit Join in Layer 2
Most Ethernet switches treat Multicast as Broadcast, the few that don't do
"IGMP Snooping" and peek into the layer 3 data
Learning the multicast tree and filtering traffic (however this method is
note widely used)
This way joining or leaving an MC Group doesn't involve any changes to the
fabric
The receiving nodes does the filtering to make sure only the appropriate MC
data is received

IB model is different since you must explicitly join the group, not to
mention all the info you need such as MTU, MLID, etc'
So you cannot just send an IP packet like in Ethernet and it works, you must
first join the club, and also get the MCGroup info

We have 2 options, one is to treat the multicast GID's like broadcasts (use
a broadcast LID, or the same IPoIB partition MLID)
Any time we see a Multicast IP address or HW (GID) address, we attach the
IPoIB broadcast MLID & MTU to it and send it away
This will also make sure that all IPoIB nodes will receive the packet, we
just need to make sure that the HW doesn't through away the packets by
either have a multicast promiscuous mode (not filter MGID's, just MLID's),
or by adding a new MGID to the HW on each join to a new IP Multicast group

The other method is to implement an "IGMP Snooping" like mechanism at the
IPoIB driver, when an IGMP message is sent, first contact the SM and join
the group, as well as get the required path info (MTU, MLID, ..), and
program the HCA
When the multicast message will be sent the path would already be set, if
for some reason the sender is not yet a part of the MC Group, it could first
contact the SM and join
It should also be noted that the snooping should be done on arriving IGMP
messages, since the IP stack might choose not to send an IGMP message since
it sees that the subnet already joined the group and the router is already
aware of it.
(IGMP's are mainly used for the routers o know if there is ANY node in this
subnet belonging to the IP Multicast group)

I personally prefer the first approach, since it is much simpler to
implement, and it also follows the common Ethernet model

Yaron



_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 23:20:23 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA00647
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 23:20:23 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id XAA10080
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 23:20:26 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id XAA09912;
	Tue, 9 Apr 2002 23:16:32 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA11012
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 14:30:06 -0400 (EDT)
Received: from web12502.mail.yahoo.com (web12502.mail.yahoo.com [216.136.173.194])
	by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA15695
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 14:30:03 -0400 (EDT)
Message-ID: <20020409183002.74249.qmail@web12502.mail.yahoo.com>
Received: from [66.122.195.194] by web12502.mail.yahoo.com via HTTP; Tue, 09 Apr 2002 11:30:02 PDT
Date: Tue, 9 Apr 2002 11:30:02 -0700 (PDT)
From: Sukanta ganguly <sganguly@yahoo.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: Vivek Kashyap <kashyapv@us.ibm.com>,
        "Culley, Paul" <Paul.Culley@COMPAQ.com>
Cc: Vivek Kashyap <kashyapv@us.ibm.com>, ipoverib@ietf.org,
        fenner@research.att.com
In-Reply-To: <200204091805.g39I5iZ23645@eng4.beaverton.ibm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

The problem IP multicast joining/leaving is performed
optimally becuase the routers implement the IGMP.
For IPoIB have an optimal implementation of multicast,
the IB node intending to join the multicast needs to
express the IB multicast group ID to the appropriate
IP multicast group ID.
  On a similar account the originator of the IB
multicast data should create a mapping of IB Multicast
groupID to IP multicast groupID. Otherwise if a
blanket link would flood the IP network with unwanted
packets, since the forwarder would not know better and
hence would take the most pessimitic approach an
forward any and every multicast data.

  In order to not have any major infrastucture rework
to get IPoIB and its multicast to work this needs to
be done.
  
   Vivek, I read your three proposals and saw the
commanility of sticking to onely one IB multicast
group ID. Whay is that ? That means simultaneous IB
multicast of data cannot be facilitated. Did I get
that right?

SG


--- Vivek Kashyap <kashyapv@us.ibm.com> wrote:
> > 
> > I suspect that the problem can be made simpler; IP
> routers should always IB_join ALL IPoIB Multicast
> groups for a given interface.  The router will
> simply ask to join any groups in the same
> partition(s) it is assigned to.  Whether the IP
> router forwards the multicast packets will then
> depend on the IP routing protocol.
> 
> That does not seem workable. How does the router
> know which groups to
> join ? Note that the groups are not static - new
> ones can always be
> created and the old ones deleted.
> 
> In my solution the IGMP/MLD report which tells the
> router about IP
> joins is used to trigger the corresponding IB join.
> The same is the
> case with routing protocols.
> 
> Vivek
> 
> 
> > 
> > IB routers have the same problem, and may use the
> same solution...
> > 
> > Paul R. Culley
> > Compaq Sr. Fellow
> > 281-514-5543
> > 
> > 
> > -----Original Message-----
> > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > Sent: Tuesday, April 09, 2002 3:31 AM
> > To: ipoverib@ietf.org
> > Cc: fenner@research.att.com
> > Subject: [Ipoverib] IPoIB multicast forwarding
> > 
> > 
> > 
> > Folks,
> > 
> >         We have defined a mapping from the IP
> addresses to the IB
> >         multicast groups. Use of different groups
> causes minimal
> >         replication at the switches and less load
> at the receivers.
> > 
> >         But there is a quirk in the way the groups
> are setup in
> >         InfiniBand - there is no way for any of
> the nodes to know if
> >         another has created an InfiniBand group.
> This is easy to fix
> >         in most cases - one checks with the SM.
> But it doesn't work
> >         out well for IPoIB packet forwarding.
> > 
> >         The router needs to forward the multicast
> packets beyond the
> >         local IP subnet if it knows there are
> recipients for it or in
> >         the case of some routing protocols it must
> do that unless it
> >         gets an update stating not to forward.
> Similarly, it needs to
> >         forward external packets to the local
> receivers if there are
> >         any receiver present. 
> > 
> >         Thus if there are IB multicast groups
> associated with an IP
> >         multicast group the router must know about
> them or the router
> >         must receive all multicast packets in the
> subnet via some
> >         other construct.
> > 
> >         Below are the three alternatives that I
> mentioned at the
> >         53rd IETF meet. 
> > 	
> >         i) Define only one IB multicast group i.e.
> IB broadcast MGID.
> >         All IP multicast packets use this group.
> > 
> >         ii) The multicast IP packet is sent to
> both the associated IB
> >         multicast group and the IB mutlicast group
> corresponding to
> >         the all-IP routers IP group.
> > 
> >         iii) Sender always sends to corresonding
> IB group if it exists
> >         else it sends to the broadcast group.
> > 
> >              IP receivers create IB groups and
> send the IGMP/MLD
> >              groups on the broadcast MGID.
> > 
> >              Routers join relevant groups based on
> the IGMP report
> >              and routing protocol updates.
> > 
> > 
> >         I also put this problem to Bill Fenner (AD
> of routing area)
> >         and asked his advice on the alternatives.
> Of the three options
> >         he favored option (iii) above.
> > 
> >         Option (i) doesn't take advantage fo IB
> multicast at all and
> >         so should be looked at as a last resort
> solution only.
> > 
> >         Option (ii) requires the sender to send
> two copies of the same
> >         packet and 'deliberately' increases the
> chance of duplicates
> >         spreading across the network.
> > 
> > 
> > Suggested solution  in detail:
> > 
> > 
> >         1. Pure senders - at IB level check for
> the relevant IB group.
> >            If it exists then send to the IB group
> else send to the
> >            'all-routers' IB group. If there is no
> IB group
> >            corresponding to a particular IP group
> then it implies that
> >            there are no IP receivers on the subnet
> for that IP
> >            multicast group. 
> > 
> >            Note: a) Pure senders don't create IB
> groups. This is
> >            because there is no existing mechanism
> for the senders to
> >            let the routers know of the new groups.
> > 
> >         2. IP receivers: As per IGMP/MLD the first
> packet sent by a
> >            host on joining a multicast group for
> receiving
> >            (IP_ADD_MEMBERSHIP) is the multicast
> report.
> > 
> > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> do the following:
> > 
> > 	   	check if the corresponding IB group exists. 
> > 
> > 		if (IB group exists)
> > 			IB_join the group and send the IGMP/MLD report
> > 
> > 		else
> > 			IB_Create the group
> > 			IB_join the group
> >                         Send IGMP/MLD report to IB
> group corresponding
> >                         to all-routers (all-hosts
> or broadcast) IP
> >                         group (Note: only done
> when creating the IB
> >                         group) 
> > 
> > 	3. IP routers IB_join MGIDs on:
> > 
> > 		a) Finding a new IP group on the subnet in the
> IGMP/MLD report
> >                 b) Determining a need to forward
> IP multicast packets
> >                    through a routing protocol
> > 
> > 
> > 
> > Comments ?
> > 
> > 
> > Vivek
> > 			
> > -- 
> > Vivek Kashyap
> > Linux Technology Center, IBM
> > kashyapv@us.ibm.com 
> > vivk@us.ibm.com 
> > 503 578 3422 (o)
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> 
> 
=== message truncated ===


__________________________________________________
Do You Yahoo!?
Yahoo! Tax Center - online filing with TurboTax
http://taxes.yahoo.com/


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr  9 23:21:14 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA00691
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 23:21:14 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id XAA10115
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 23:21:17 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id XAA09868;
	Tue, 9 Apr 2002 23:16:29 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA11133
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 14:30:26 -0400 (EDT)
Received: from web12503.mail.yahoo.com (web12503.mail.yahoo.com [216.136.173.195])
	by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA15706
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 14:30:18 -0400 (EDT)
Message-ID: <20020409183019.16441.qmail@web12503.mail.yahoo.com>
Received: from [66.122.195.194] by web12503.mail.yahoo.com via HTTP; Tue, 09 Apr 2002 11:30:19 PDT
Date: Tue, 9 Apr 2002 11:30:19 -0700 (PDT)
From: Sukanta ganguly <sganguly@yahoo.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: Vivek Kashyap <kashyapv@us.ibm.com>,
        "Culley, Paul" <Paul.Culley@COMPAQ.com>
Cc: Vivek Kashyap <kashyapv@us.ibm.com>, ipoverib@ietf.org,
        fenner@research.att.com
In-Reply-To: <200204091805.g39I5iZ23645@eng4.beaverton.ibm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

The problem IP multicast joining/leaving is performed
optimally becuase the routers implement the IGMP.
For IPoIB have an optimal implementation of multicast,
the IB node intending to join the multicast needs to
express the IB multicast group ID to the appropriate
IP multicast group ID.
  On a similar account the originator of the IB
multicast data should create a mapping of IB Multicast
groupID to IP multicast groupID. Otherwise if a
blanket link would flood the IP network with unwanted
packets, since the forwarder would not know better and
hence would take the most pessimitic approach an
forward any and every multicast data.

  In order to not have any major infrastucture rework
to get IPoIB and its multicast to work this needs to
be done.
  
   Vivek, I read your three proposals and saw the
commanility of sticking to onely one IB multicast
group ID. Whay is that ? That means simultaneous IB
multicast of data cannot be facilitated. Did I get
that right?

SG


--- Vivek Kashyap <kashyapv@us.ibm.com> wrote:
> > 
> > I suspect that the problem can be made simpler; IP
> routers should always IB_join ALL IPoIB Multicast
> groups for a given interface.  The router will
> simply ask to join any groups in the same
> partition(s) it is assigned to.  Whether the IP
> router forwards the multicast packets will then
> depend on the IP routing protocol.
> 
> That does not seem workable. How does the router
> know which groups to
> join ? Note that the groups are not static - new
> ones can always be
> created and the old ones deleted.
> 
> In my solution the IGMP/MLD report which tells the
> router about IP
> joins is used to trigger the corresponding IB join.
> The same is the
> case with routing protocols.
> 
> Vivek
> 
> 
> > 
> > IB routers have the same problem, and may use the
> same solution...
> > 
> > Paul R. Culley
> > Compaq Sr. Fellow
> > 281-514-5543
> > 
> > 
> > -----Original Message-----
> > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > Sent: Tuesday, April 09, 2002 3:31 AM
> > To: ipoverib@ietf.org
> > Cc: fenner@research.att.com
> > Subject: [Ipoverib] IPoIB multicast forwarding
> > 
> > 
> > 
> > Folks,
> > 
> >         We have defined a mapping from the IP
> addresses to the IB
> >         multicast groups. Use of different groups
> causes minimal
> >         replication at the switches and less load
> at the receivers.
> > 
> >         But there is a quirk in the way the groups
> are setup in
> >         InfiniBand - there is no way for any of
> the nodes to know if
> >         another has created an InfiniBand group.
> This is easy to fix
> >         in most cases - one checks with the SM.
> But it doesn't work
> >         out well for IPoIB packet forwarding.
> > 
> >         The router needs to forward the multicast
> packets beyond the
> >         local IP subnet if it knows there are
> recipients for it or in
> >         the case of some routing protocols it must
> do that unless it
> >         gets an update stating not to forward.
> Similarly, it needs to
> >         forward external packets to the local
> receivers if there are
> >         any receiver present. 
> > 
> >         Thus if there are IB multicast groups
> associated with an IP
> >         multicast group the router must know about
> them or the router
> >         must receive all multicast packets in the
> subnet via some
> >         other construct.
> > 
> >         Below are the three alternatives that I
> mentioned at the
> >         53rd IETF meet. 
> > 	
> >         i) Define only one IB multicast group i.e.
> IB broadcast MGID.
> >         All IP multicast packets use this group.
> > 
> >         ii) The multicast IP packet is sent to
> both the associated IB
> >         multicast group and the IB mutlicast group
> corresponding to
> >         the all-IP routers IP group.
> > 
> >         iii) Sender always sends to corresonding
> IB group if it exists
> >         else it sends to the broadcast group.
> > 
> >              IP receivers create IB groups and
> send the IGMP/MLD
> >              groups on the broadcast MGID.
> > 
> >              Routers join relevant groups based on
> the IGMP report
> >              and routing protocol updates.
> > 
> > 
> >         I also put this problem to Bill Fenner (AD
> of routing area)
> >         and asked his advice on the alternatives.
> Of the three options
> >         he favored option (iii) above.
> > 
> >         Option (i) doesn't take advantage fo IB
> multicast at all and
> >         so should be looked at as a last resort
> solution only.
> > 
> >         Option (ii) requires the sender to send
> two copies of the same
> >         packet and 'deliberately' increases the
> chance of duplicates
> >         spreading across the network.
> > 
> > 
> > Suggested solution  in detail:
> > 
> > 
> >         1. Pure senders - at IB level check for
> the relevant IB group.
> >            If it exists then send to the IB group
> else send to the
> >            'all-routers' IB group. If there is no
> IB group
> >            corresponding to a particular IP group
> then it implies that
> >            there are no IP receivers on the subnet
> for that IP
> >            multicast group. 
> > 
> >            Note: a) Pure senders don't create IB
> groups. This is
> >            because there is no existing mechanism
> for the senders to
> >            let the routers know of the new groups.
> > 
> >         2. IP receivers: As per IGMP/MLD the first
> packet sent by a
> >            host on joining a multicast group for
> receiving
> >            (IP_ADD_MEMBERSHIP) is the multicast
> report.
> > 
> > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> do the following:
> > 
> > 	   	check if the corresponding IB group exists. 
> > 
> > 		if (IB group exists)
> > 			IB_join the group and send the IGMP/MLD report
> > 
> > 		else
> > 			IB_Create the group
> > 			IB_join the group
> >                         Send IGMP/MLD report to IB
> group corresponding
> >                         to all-routers (all-hosts
> or broadcast) IP
> >                         group (Note: only done
> when creating the IB
> >                         group) 
> > 
> > 	3. IP routers IB_join MGIDs on:
> > 
> > 		a) Finding a new IP group on the subnet in the
> IGMP/MLD report
> >                 b) Determining a need to forward
> IP multicast packets
> >                    through a routing protocol
> > 
> > 
> > 
> > Comments ?
> > 
> > 
> > Vivek
> > 			
> > -- 
> > Vivek Kashyap
> > Linux Technology Center, IBM
> > kashyapv@us.ibm.com 
> > vivk@us.ibm.com 
> > 503 578 3422 (o)
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> 
> 
=== message truncated ===


__________________________________________________
Do You Yahoo!?
Yahoo! Tax Center - online filing with TurboTax
http://taxes.yahoo.com/


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 10 00:09:20 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA00652
	for <ipoverib-archive@odin.ietf.org>; Tue, 9 Apr 2002 23:20:23 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id XAA10081
	for ipoverib-archive@odin.ietf.org; Tue, 9 Apr 2002 23:20:26 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id XAA09862;
	Tue, 9 Apr 2002 23:16:29 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA11991
	for <ipoverib@optimus.ietf.org>; Tue, 9 Apr 2002 14:36:28 -0400 (EDT)
Received: from web12504.mail.yahoo.com (web12504.mail.yahoo.com [216.136.173.196])
	by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA16028
	for <ipoverib@ietf.org>; Tue, 9 Apr 2002 14:36:21 -0400 (EDT)
Message-ID: <20020409183622.77877.qmail@web12504.mail.yahoo.com>
Received: from [66.122.195.194] by web12504.mail.yahoo.com via HTTP; Tue, 09 Apr 2002 11:36:22 PDT
Date: Tue, 9 Apr 2002 11:36:22 -0700 (PDT)
From: Sukanta ganguly <sganguly@yahoo.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: Bill Strahm <bill@strahm.net>, "Culley, Paul" <Paul.Culley@COMPAQ.com>
Cc: Vivek Kashyap <kashyapv@us.ibm.com>, ipoverib@ietf.org,
        fenner@research.att.com
In-Reply-To: <20020409110547.D70815@homegate.strahm.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Bill,
  When you send in the Multicast packet (as a sender),
at layer 3 you send it to the multicast IP address
that the sender decided on. Your layer 2 module will
decide on the next hop for that Multicast IP address.
And a similar functioning will be build up as a chain.
Note in this pacth while the senders original packet
is moving to its destination which is the IP multicast
address, the intermediate routers (which are multicast
aware) will check to see if they have any IGMP joins
they have received from any other subnets. If they
have any such mapping they have built due to receive
of IGMP joins then they will oblige that join by
copying the packet to that joined destination. That's
how the copy of the packet is enroute to its rightful
receiver.

SG

P.S :: Dr Fenner is an authority on this and can
provide some special comments 


--- Bill Strahm <bill@strahm.net> wrote:
> Ok, I am going to express a scary amount of
> ignorance here...
> 
> I know how multicast works on receive, I know how
> IGMP works to build the forwarding tree,
> I know how flood/prune, RP Multicast Routing
> protocols work...
> 
> What I would like a pointer to is a simple this is
> how a first hop multicast router 
> behaves.  An example on Ethernet would be
> acceptable, a generic example would be better.
> 
> The question I have is as a Multicast sender, I send
> a multicast packet to the first
> hop router that is multicast aware.  Now, do I send
> that packet to its unicast MAC 
> address, or do I send that to the mapped layer 2
> broadcast address and let the IP 
> layer rebroadcast to the broadcast address if it is
> needed on the local subnet.
> 
> Once I have that, everything else I understand
> 
> Bill
> PS Bill Fenner, the IPoIB list is subscriber posting
> only.  If you haven't subscribed, please do 
> so we can get the benefit of your wisdom...
> 
> On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley,
> Paul wrote:
> > I suspect that the problem can be made simpler; IP
> routers should always IB_join ALL IPoIB Multicast
> groups for a given interface.  The router will
> simply ask to join any groups in the same
> partition(s) it is assigned to.  Whether the IP
> router forwards the multicast packets will then
> depend on the IP routing protocol.
> > 
> > IB routers have the same problem, and may use the
> same solution...
> > 
> > Paul R. Culley
> > Compaq Sr. Fellow
> > 281-514-5543
> > 
> > 
> > -----Original Message-----
> > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > Sent: Tuesday, April 09, 2002 3:31 AM
> > To: ipoverib@ietf.org
> > Cc: fenner@research.att.com
> > Subject: [Ipoverib] IPoIB multicast forwarding
> > 
> > 
> > 
> > Folks,
> > 
> >         We have defined a mapping from the IP
> addresses to the IB
> >         multicast groups. Use of different groups
> causes minimal
> >         replication at the switches and less load
> at the receivers.
> > 
> >         But there is a quirk in the way the groups
> are setup in
> >         InfiniBand - there is no way for any of
> the nodes to know if
> >         another has created an InfiniBand group.
> This is easy to fix
> >         in most cases - one checks with the SM.
> But it doesn't work
> >         out well for IPoIB packet forwarding.
> > 
> >         The router needs to forward the multicast
> packets beyond the
> >         local IP subnet if it knows there are
> recipients for it or in
> >         the case of some routing protocols it must
> do that unless it
> >         gets an update stating not to forward.
> Similarly, it needs to
> >         forward external packets to the local
> receivers if there are
> >         any receiver present. 
> > 
> >         Thus if there are IB multicast groups
> associated with an IP
> >         multicast group the router must know about
> them or the router
> >         must receive all multicast packets in the
> subnet via some
> >         other construct.
> > 
> >         Below are the three alternatives that I
> mentioned at the
> >         53rd IETF meet. 
> > 	
> >         i) Define only one IB multicast group i.e.
> IB broadcast MGID.
> >         All IP multicast packets use this group.
> > 
> >         ii) The multicast IP packet is sent to
> both the associated IB
> >         multicast group and the IB mutlicast group
> corresponding to
> >         the all-IP routers IP group.
> > 
> >         iii) Sender always sends to corresonding
> IB group if it exists
> >         else it sends to the broadcast group.
> > 
> >              IP receivers create IB groups and
> send the IGMP/MLD
> >              groups on the broadcast MGID.
> > 
> >              Routers join relevant groups based on
> the IGMP report
> >              and routing protocol updates.
> > 
> > 
> >         I also put this problem to Bill Fenner (AD
> of routing area)
> >         and asked his advice on the alternatives.
> Of the three options
> >         he favored option (iii) above.
> > 
> >         Option (i) doesn't take advantage fo IB
> multicast at all and
> >         so should be looked at as a last resort
> solution only.
> > 
> >         Option (ii) requires the sender to send
> two copies of the same
> >         packet and 'deliberately' increases the
> chance of duplicates
> >         spreading across the network.
> > 
> > 
> > Suggested solution  in detail:
> > 
> > 
> >         1. Pure senders - at IB level check for
> the relevant IB group.
> >            If it exists then send to the IB group
> else send to the
> >            'all-routers' IB group. If there is no
> IB group
> >            corresponding to a particular IP group
> then it implies that
> >            there are no IP receivers on the subnet
> for that IP
> >            multicast group. 
> > 
> >            Note: a) Pure senders don't create IB
> groups. This is
> >            because there is no existing mechanism
> for the senders to
> >            let the routers know of the new groups.
> > 
> >         2. IP receivers: As per IGMP/MLD the first
> packet sent by a
> >            host on joining a multicast group for
> receiving
> >            (IP_ADD_MEMBERSHIP) is the multicast
> report.
> > 
> > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> do the following:
> > 
> > 	   	check if the corresponding IB group exists. 
> > 
> > 		if (IB group exists)
> > 			IB_join the group and send the IGMP/MLD report
> > 
> > 		else
> > 			IB_Create the group
> > 			IB_join the group
> >                         Send IGMP/MLD report to IB
> group corresponding
> >                         to all-routers (all-hosts
> or broadcast) IP
> >                         group (Note: only done
> when creating the IB
> >                         group) 
> > 
> > 	3. IP routers IB_join MGIDs on:
> > 
> > 		a) Finding a new IP group on the subnet in the
> IGMP/MLD report
> >                 b) Determining a need to forward
> IP multicast packets
> >                    through a routing protocol
> > 
> > 
> > 
> > Comments ?
> > 
> > 
> > Vivek
> 
=== message truncated ===


__________________________________________________
Do You Yahoo!?
Yahoo! Tax Center - online filing with TurboTax
http://taxes.yahoo.com/


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 10 13:05:59 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA25328
	for <ipoverib-archive@odin.ietf.org>; Wed, 10 Apr 2002 13:05:54 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id NAA05880
	for ipoverib-archive@odin.ietf.org; Wed, 10 Apr 2002 13:05:44 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA05179;
	Wed, 10 Apr 2002 12:58:38 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA05147
	for <ipoverib@optimus.ietf.org>; Wed, 10 Apr 2002 12:58:33 -0400 (EDT)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA25184
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 12:58:27 -0400 (EDT)
Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.38 2002/04/09 21:20:23 root Exp $) with ESMTP id g3AGvtu13104
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 16:57:55 GMT
Received: from fmsmsxvs042.fm.intel.com (fmsmsxv042-1.fm.intel.com [132.233.48.110])
	by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.15 2002/04/01 17:51:48 root Exp $) with SMTP id g3AGv4X10873
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 16:57:04 GMT
Received: from fmsmsx29.FM.INTEL.COM ([132.233.42.29])
 by fmsmsxvs042.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002041010022124304
 ; Wed, 10 Apr 2002 10:02:21 -0700
Received: by fmsmsx29.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2LKBWNYX>; Wed, 10 Apr 2002 09:58:27 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F70@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Vivek Kashyap'" <kashyapv@us.ibm.com>
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Wed, 10 Apr 2002 09:58:22 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

/rkl
embedded
rkl/

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
Sent: Tuesday, April 09, 2002 1:20 PM
To: roy.k.larsen@intel.com
Cc: ipoverib@ietf.org
Subject: Re: [Ipoverib] IPoIB multicast forwarding


> 
> Ahh, that clarifies it for me.  However, from an implementation point of
> view, the IPoIB link driver can not indicate that it supports promiscuous
> mode, so one would presume that the OS network infrastructure will not use
> that interface for multicast routing unless the OS network stack were
> modified.  If you need to modify the network stack, why not simply alter
it
> so that it IB creates/joins all multicast groups it has received updates
> for?  In other words, keep all the special processing and "fixes" local to
> the router implementation and out of the common host IPoIB driver.

Aren't you saying the same thing as the solution I have presented?

/rkl
I am utterly confused so I don't know.  I was trying to remove the need of a
multicast sender to broadcast a frame if the multicast group did not exist.
Your response to Bill said that multicast router does not join any group at
the IP level.  However, below you state that it will IB_join based on
IGMP/MLD reports.  Isn't that at the IP level?  If not, what is going to
prompt the IPoIB driver to do the IB_join?

If I understood Yaron's proposals correctly, in one he suggests simply
broadcast all multicast frames on IB fabrics.  When considering the overhead
and relative complexity of IB multicast management, this is very compelling
to me.  But I obviously need to get much smarter on IGMP before contributing
anything meaningful to this thread.

The other point I was trying to make was one of implementation.  An IPoIB
driver should not indicate that it supports promiscuous/all-multicast
filtering unless it truly can.  So, if routers using IGMP protocols require
this capability of a network interface, there are going to be some software
integration issues.

Roy
rkl/

        The router knows the groups to forward to based on IGMP/MLD
        reports and multicast routing protocol updates. It therefore
        IB_joins (or creates in some cases) the corresponding IB
        MGIDs. 

For the above to occur the MLD/IGMP report must reach the router. So,
the host must send it to the router.

        The host network stack is not effected. The solution is
        limited to the host's interface driver. As I had mentioned in
        the presentation at the IETF meet: the IP_ADD_MEMBERSHIP call
        causes the host IP stack to tell the interface
        (ethernet/IPoIB/FDDI/Token ring etc.) of the IP multicast
        group. In the case of ethernet the ethernet driver updates the
        device/internal tables to include the corresponding ethernet
        mutlicast group. In the case of IPoIB the driver lets the
        SM/SA know (create if group doesn't exist; join if it
        exists). 

So, the solution is limited to a) routers' IPoIB module b) in the
IPoIB interface driver of the host.


Vivek
> 
> Roy
> 
> -----Original Message-----
> From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> Sent: Tuesday, April 09, 2002 12:37 PM
> To: bill@strahm.net
> Cc: sganguly@yahoo.com; bill@strahm.net; Paul.Culley@compaq.com;
> kashyapv@us.ibm.com; ipoverib@ietf.org; fenner@research.att.com
> Subject: Re: [Ipoverib] IPoIB multicast forwarding
> 
> 
> Bill,
>         The multicast router does not 'join' any group at the IP
>         level. The router sets the ethernet interface in all-multicast
>         promiscuous mode. Thus the router gets all multicast packets.
> 
>         It filters or forwards the packets based on the multicast
>         routing protocol updates.
> 
> Vivek
> 
> 
> > 
> > Question then...
> > 
> > As an IP Multicast sender, I am not required to join 
> > the Multicast group... So now there has not been an
> > IGMP message telling the first hop router to join the
> > group either.  So the router has not told its layer
> > 2 to join the broadcast address for the multicast group.
> > 
> > How does the first hop Multicast router know to join
> > this group and forward the traffic on to the multicast
> > tree (either RP, or Prune/Flood) if it can't receive
> > the initial packet...
> > 
> > This is the magic that I was wanting to know, I know how
> > I would make it work, but I am hoping Bill F. can tell me
> > the correct way to do it <G>
> > 
> > Bill Strahm
> > 
> > On Tue, Apr 09, 2002 at 11:36:22AM -0700, Sukanta ganguly wrote:
> > > Bill,
> > >   When you send in the Multicast packet (as a sender),
> > > at layer 3 you send it to the multicast IP address
> > > that the sender decided on. Your layer 2 module will
> > > decide on the next hop for that Multicast IP address.
> > > And a similar functioning will be build up as a chain.
> > > Note in this pacth while the senders original packet
> > > is moving to its destination which is the IP multicast
> > > address, the intermediate routers (which are multicast
> > > aware) will check to see if they have any IGMP joins
> > > they have received from any other subnets. If they
> > > have any such mapping they have built due to receive
> > > of IGMP joins then they will oblige that join by
> > > copying the packet to that joined destination. That's
> > > how the copy of the packet is enroute to its rightful
> > > receiver.
> > > 
> > > SG
> > > 
> > > P.S :: Dr Fenner is an authority on this and can
> > > provide some special comments 
> > > 
> > > 
> > > --- Bill Strahm <bill@strahm.net> wrote:
> > > > Ok, I am going to express a scary amount of
> > > > ignorance here...
> > > > 
> > > > I know how multicast works on receive, I know how
> > > > IGMP works to build the forwarding tree,
> > > > I know how flood/prune, RP Multicast Routing
> > > > protocols work...
> > > > 
> > > > What I would like a pointer to is a simple this is
> > > > how a first hop multicast router 
> > > > behaves.  An example on Ethernet would be
> > > > acceptable, a generic example would be better.
> > > > 
> > > > The question I have is as a Multicast sender, I send
> > > > a multicast packet to the first
> > > > hop router that is multicast aware.  Now, do I send
> > > > that packet to its unicast MAC 
> > > > address, or do I send that to the mapped layer 2
> > > > broadcast address and let the IP 
> > > > layer rebroadcast to the broadcast address if it is
> > > > needed on the local subnet.
> > > > 
> > > > Once I have that, everything else I understand
> > > > 
> > > > Bill
> > > > PS Bill Fenner, the IPoIB list is subscriber posting
> > > > only.  If you haven't subscribed, please do 
> > > > so we can get the benefit of your wisdom...
> > > > 
> > > > On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley,
> > > > Paul wrote:
> > > > > I suspect that the problem can be made simpler; IP
> > > > routers should always IB_join ALL IPoIB Multicast
> > > > groups for a given interface.  The router will
> > > > simply ask to join any groups in the same
> > > > partition(s) it is assigned to.  Whether the IP
> > > > router forwards the multicast packets will then
> > > > depend on the IP routing protocol.
> > > > > 
> > > > > IB routers have the same problem, and may use the
> > > > same solution...
> > > > > 
> > > > > Paul R. Culley
> > > > > Compaq Sr. Fellow
> > > > > 281-514-5543
> > > > > 
> > > > > 
> > > > > -----Original Message-----
> > > > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > > > > Sent: Tuesday, April 09, 2002 3:31 AM
> > > > > To: ipoverib@ietf.org
> > > > > Cc: fenner@research.att.com
> > > > > Subject: [Ipoverib] IPoIB multicast forwarding
> > > > > 
> > > > > 
> > > > > 
> > > > > Folks,
> > > > > 
> > > > >         We have defined a mapping from the IP
> > > > addresses to the IB
> > > > >         multicast groups. Use of different groups
> > > > causes minimal
> > > > >         replication at the switches and less load
> > > > at the receivers.
> > > > > 
> > > > >         But there is a quirk in the way the groups
> > > > are setup in
> > > > >         InfiniBand - there is no way for any of
> > > > the nodes to know if
> > > > >         another has created an InfiniBand group.
> > > > This is easy to fix
> > > > >         in most cases - one checks with the SM.
> > > > But it doesn't work
> > > > >         out well for IPoIB packet forwarding.
> > > > > 
> > > > >         The router needs to forward the multicast
> > > > packets beyond the
> > > > >         local IP subnet if it knows there are
> > > > recipients for it or in
> > > > >         the case of some routing protocols it must
> > > > do that unless it
> > > > >         gets an update stating not to forward.
> > > > Similarly, it needs to
> > > > >         forward external packets to the local
> > > > receivers if there are
> > > > >         any receiver present. 
> > > > > 
> > > > >         Thus if there are IB multicast groups
> > > > associated with an IP
> > > > >         multicast group the router must know about
> > > > them or the router
> > > > >         must receive all multicast packets in the
> > > > subnet via some
> > > > >         other construct.
> > > > > 
> > > > >         Below are the three alternatives that I
> > > > mentioned at the
> > > > >         53rd IETF meet. 
> > > > > 	
> > > > >         i) Define only one IB multicast group i.e.
> > > > IB broadcast MGID.
> > > > >         All IP multicast packets use this group.
> > > > > 
> > > > >         ii) The multicast IP packet is sent to
> > > > both the associated IB
> > > > >         multicast group and the IB mutlicast group
> > > > corresponding to
> > > > >         the all-IP routers IP group.
> > > > > 
> > > > >         iii) Sender always sends to corresonding
> > > > IB group if it exists
> > > > >         else it sends to the broadcast group.
> > > > > 
> > > > >              IP receivers create IB groups and
> > > > send the IGMP/MLD
> > > > >              groups on the broadcast MGID.
> > > > > 
> > > > >              Routers join relevant groups based on
> > > > the IGMP report
> > > > >              and routing protocol updates.
> > > > > 
> > > > > 
> > > > >         I also put this problem to Bill Fenner (AD
> > > > of routing area)
> > > > >         and asked his advice on the alternatives.
> > > > Of the three options
> > > > >         he favored option (iii) above.
> > > > > 
> > > > >         Option (i) doesn't take advantage fo IB
> > > > multicast at all and
> > > > >         so should be looked at as a last resort
> > > > solution only.
> > > > > 
> > > > >         Option (ii) requires the sender to send
> > > > two copies of the same
> > > > >         packet and 'deliberately' increases the
> > > > chance of duplicates
> > > > >         spreading across the network.
> > > > > 
> > > > > 
> > > > > Suggested solution  in detail:
> > > > > 
> > > > > 
> > > > >         1. Pure senders - at IB level check for
> > > > the relevant IB group.
> > > > >            If it exists then send to the IB group
> > > > else send to the
> > > > >            'all-routers' IB group. If there is no
> > > > IB group
> > > > >            corresponding to a particular IP group
> > > > then it implies that
> > > > >            there are no IP receivers on the subnet
> > > > for that IP
> > > > >            multicast group. 
> > > > > 
> > > > >            Note: a) Pure senders don't create IB
> > > > groups. This is
> > > > >            because there is no existing mechanism
> > > > for the senders to
> > > > >            let the routers know of the new groups.
> > > > > 
> > > > >         2. IP receivers: As per IGMP/MLD the first
> > > > packet sent by a
> > > > >            host on joining a multicast group for
> > > > receiving
> > > > >            (IP_ADD_MEMBERSHIP) is the multicast
> > > > report.
> > > > > 
> > > > > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> > > > do the following:
> > > > > 
> > > > > 	   	check if the corresponding IB group exists. 
> > > > > 
> > > > > 		if (IB group exists)
> > > > > 			IB_join the group and send the IGMP/MLD
> report
> > > > > 
> > > > > 		else
> > > > > 			IB_Create the group
> > > > > 			IB_join the group
> > > > >                         Send IGMP/MLD report to IB
> > > > group corresponding
> > > > >                         to all-routers (all-hosts
> > > > or broadcast) IP
> > > > >                         group (Note: only done
> > > > when creating the IB
> > > > >                         group) 
> > > > > 
> > > > > 	3. IP routers IB_join MGIDs on:
> > > > > 
> > > > > 		a) Finding a new IP group on the subnet in the
> > > > IGMP/MLD report
> > > > >                 b) Determining a need to forward
> > > > IP multicast packets
> > > > >                    through a routing protocol
> > > > > 
> > > > > 
> > > > > 
> > > > > Comments ?
> > > > > 
> > > > > 
> > > > > Vivek
> > > > 
> > > === message truncated ===
> > > 
> > > 
> > > __________________________________________________
> > > Do You Yahoo!?
> > > Yahoo! Tax Center - online filing with TurboTax
> > > http://taxes.yahoo.com/
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 10 14:36:14 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA28847
	for <ipoverib-archive@odin.ietf.org>; Wed, 10 Apr 2002 14:36:09 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA12139
	for ipoverib-archive@odin.ietf.org; Wed, 10 Apr 2002 14:36:12 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA11911;
	Wed, 10 Apr 2002 14:31:56 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA11883
	for <ipoverib@optimus.ietf.org>; Wed, 10 Apr 2002 14:31:53 -0400 (EDT)
Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA28643
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 14:31:46 -0400 (EDT)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e31.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3AIVkNr012890;
	Wed, 10 Apr 2002 14:31:47 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3AIVYv100098;
	Wed, 10 Apr 2002 12:31:35 -0600
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g3AIVVM10568;
	Wed, 10 Apr 2002 11:31:31 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3AIVT419521;
	Wed, 10 Apr 2002 11:31:29 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204101831.g3AIVT419521@eng4.beaverton.ibm.com>
Subject: Re: [ipoverib] IP Multicast and IGMP's
To: yaronh@voltaire.com
Date: Wed, 10 Apr 2002 11:31:29 -0700 (PDT)
Cc: ipoverib@ietf.org, vivk@us.ibm.com ('Vivek Kashyap'),
        Jerry.Chu@eng.sun.com ('H.K. Jerry Chu')
In-Reply-To: <004b01c1e01e$8aaa1e90$cad05c8b@vaio505he> from "Yaron Haviv" at Apr 09, 2002 06:12:24 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> IB model is different since you must explicitly join the group, not to
> mention all the info you need such as MTU, MLID, etc'
> So you cannot just send an IP packet like in Ethernet and it works, you must
> first join the club, and also get the MCGroup info
> 
> We have 2 options, one is to treat the multicast GID's like broadcasts (use
> a broadcast LID, or the same IPoIB partition MLID)
> Any time we see a Multicast IP address or HW (GID) address, we attach the
> IPoIB broadcast MLID & MTU to it and send it away
> This will also make sure that all IPoIB nodes will receive the packet, we
> just need to make sure that the HW doesn't through away the packets by
> either have a multicast promiscuous mode (not filter MGID's, just MLID's),
> or by adding a new MGID to the HW on each join to a new IP Multicast group

        This is equivalent to proposal (i) that I presented. Use a
        single hardware broadcast group. 

        Such a solution just ignores all the benefits of having
        multiple multicast groups. All the packets are broadcast and
        all are received by everybody. They then filter them in
        software. 

        I would suggest that we go for a solution that allows us to
        benefit from the multicast support available from the
        hardware. 

> The other method is to implement an "IGMP Snooping" like mechanism at the

        The reason I didn't pursue 'IGMP snooping' is because it
        requires all IPoIB drivers - whether in routers or end hosts -
        to parse every multicast packet or at least every multicast
        packet received on all-routers MGID - for IGMP/MLD packets and
        then take action.

        Therefore I came up with the solution that I presented which
        requires much more limited changes and does not require
        looking at every IP multicast packet to see if it is a valid
        IGMP/MLD report. 
> 
> IPoIB driver, when an IGMP message is sent, first contact the SM and join
> the group, as well as get the required path info (MTU, MLID, ..), and
> program the HCA


        This is the same as saying that when IP_ADD_MEMBERSHIP is done
        i.e. an IP node decides to join an IP group then IB_Join the
        corresponding IB group. [ Note that MTU is not going to be
        different - we have mandated that all MTUs in all the groups
        have to be the same as that for the broadcast_GID.]

	Then send an IGMP report.  


        -- we are in sync till the above. This is where I have
        suggested the additional function given below:

        What I have suggested is that the IP router will not receive
        the report unless it is sent out on the broadcast_GID or on
        the all-routers-GID. It is better to send it out on the
        broadcast_GID so that all other hosts receive it too.


        A little enhancement:

        In my proposal the node sends the IGMP report on the MGID
        corresponding to the broadcast/all-routers multicast IP
        address only when the node IB_creates the IB group. Since, at
        that time the router does not know of the new group.

        If it is just IP_joininig the group it only needs to IB_Join
        the group at the IB layer but the IGMP report is then sent to
        the corresponding IB MGID and not the broadcast MGID.


> When the multicast message will be sent the path would already be set, if
> for some reason the sender is not yet a part of the MC Group, it could first
> contact the SM and join
> It should also be noted that the snooping should be done on arriving IGMP
> messages, since the IP stack might choose not to send an IGMP message since
> it sees that the subnet already joined the group and the router is already
> aware of it.
> (IGMP's are mainly used for the routers o know if there is ANY node in this
> subnet belonging to the IP Multicast group)

        In my proposal the rest of the solution lies with the router.
        The router gets the IGMP packet and parses it in the normal
        way. It even sets the IP groups to forward to in the normal
        way. 


        ***The router is being asked to merely IB_Join the IB group when
        it hears of a new IP multicast group.**


        Note that I have bounced this off of Bill Fenner(routing area
        AD) and he thinks it is ok to ask the router to do the above.

	We could combine the soluion in another way:

        Keep all the requirements on the sender and the host that I
        have said already.

Vivek

> 
> I personally prefer the first approach, since it is much simpler to
> implement, and it also follows the common Ethernet model
> 
> Yaron
> 
> 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 10 14:49:11 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA29413
	for <ipoverib-archive@odin.ietf.org>; Wed, 10 Apr 2002 14:49:11 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA12652
	for ipoverib-archive@odin.ietf.org; Wed, 10 Apr 2002 14:49:14 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA12503;
	Wed, 10 Apr 2002 14:45:17 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA12436
	for <ipoverib@optimus.ietf.org>; Wed, 10 Apr 2002 14:45:09 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA29310
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 14:45:05 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3AIj7sX014416;
	Wed, 10 Apr 2002 14:45:07 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.00) with ESMTP id g3AIj5M239778;
	Wed, 10 Apr 2002 14:45:06 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g3AIj3M10952;
	Wed, 10 Apr 2002 11:45:03 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3AIj1U19679;
	Wed, 10 Apr 2002 11:45:01 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204101845.g3AIj1U19679@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: roy.k.larsen@intel.com (Larsen, Roy K)
Date: Wed, 10 Apr 2002 11:45:00 -0700 (PDT)
Cc: kashyapv@us.ibm.com ('Vivek Kashyap'), ipoverib@ietf.org
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4F70@ORSMSX108> from "Larsen, Roy K" at Apr 10, 2002 08:58:22 AM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> > 
> > Ahh, that clarifies it for me.  However, from an implementation point of
> > view, the IPoIB link driver can not indicate that it supports promiscuous
> > mode, so one would presume that the OS network infrastructure will not use
> > that interface for multicast routing unless the OS network stack were
> > modified.  If you need to modify the network stack, why not simply alter
> it
> > so that it IB creates/joins all multicast groups it has received updates
> > for?  In other words, keep all the special processing and "fixes" local to
> > the router implementation and out of the common host IPoIB driver.
> 
> Aren't you saying the same thing as the solution I have presented?
> 
> /rkl
> I am utterly confused so I don't know.  I was trying to remove the need of a
> multicast sender to broadcast a frame if the multicast group did not exist.

        Actually, it is not broadcast it is sent to all-routers MGID.
        So it is received by routers only. Since there are no
        receivers there is no need to send it on the broadcast MGID.

> Your response to Bill said that multicast router does not join any group at
> the IP level.  However, below you state that it will IB_join based on
> IGMP/MLD reports.  Isn't that at the IP level?  If not, what is going to
> prompt the IPoIB driver to do the IB_join?

        The IGMP/MLD report lets the router know the IP multicast
        groups that exist on an interface. I am suggesting that the
        router then give a command to the interface to add the
        corresponding IB group to its list. This command, at the IPoIB
        driver translates into an IB_join for the corresponding IB
        group. 

> 
> If I understood Yaron's proposals correctly, in one he suggests simply
> broadcast all multicast frames on IB fabrics.  When considering the overhead
> and relative complexity of IB multicast management, this is very compelling
> to me.  

        It is compelling to you to broadcast everything and yet you
        are bothered that a sender will broadcast(though it won't) its
        packets if it can't IB_join the relevant group ??? 

But I obviously need to get much smarter on IGMP before contributing
> anything meaningful to this thread.
> 
> The other point I was trying to make was one of implementation.  An IPoIB
> driver should not indicate that it supports promiscuous/all-multicast
> filtering unless it truly can.  So, if routers using IGMP protocols require
> this capability of a network interface, there are going to be some software
> integration issues.

        This does not arise with respect to the solution I have
        indicated. Neither the router nor the end hosts are being
        asked to support such a mode. 
	
        The solution hinges on the IP multicast senders sending to
        groups that exist - IB group corresponding to the IP group or
        the all-routers MGID. 
	
        The IP multicast receivers create IB groups if they don't
        exist. When they create a group they must ensure that the
        IGMP/MLD report is received by the routers. And so instead of
        sending it to the group they just created they send it to the
        MGID corresponding to the all-routers IP address.

        The IP routers receive packets only on MGIDs they are members
        of - this includes the broadcast/all-routers MGIDs and the
        MGIDs they have joined. When they parse an IGMP/MLD report and
        find a new IP group they IB_join the corrresponding IB group.


Vivek

> 
> Roy
> rkl/
> 
>         The router knows the groups to forward to based on IGMP/MLD
>         reports and multicast routing protocol updates. It therefore
>         IB_joins (or creates in some cases) the corresponding IB
>         MGIDs. 
> 
> For the above to occur the MLD/IGMP report must reach the router. So,
> the host must send it to the router.
> 
>         The host network stack is not effected. The solution is
>         limited to the host's interface driver. As I had mentioned in
>         the presentation at the IETF meet: the IP_ADD_MEMBERSHIP call
>         causes the host IP stack to tell the interface
>         (ethernet/IPoIB/FDDI/Token ring etc.) of the IP multicast
>         group. In the case of ethernet the ethernet driver updates the
>         device/internal tables to include the corresponding ethernet
>         mutlicast group. In the case of IPoIB the driver lets the
>         SM/SA know (create if group doesn't exist; join if it
>         exists). 
> 
> So, the solution is limited to a) routers' IPoIB module b) in the
> IPoIB interface driver of the host.
> 
> 
> Vivek
> > 
> > Roy
> > 
> > -----Original Message-----
> > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > Sent: Tuesday, April 09, 2002 12:37 PM
> > To: bill@strahm.net
> > Cc: sganguly@yahoo.com; bill@strahm.net; Paul.Culley@compaq.com;
> > kashyapv@us.ibm.com; ipoverib@ietf.org; fenner@research.att.com
> > Subject: Re: [Ipoverib] IPoIB multicast forwarding
> > 
> > 
> > Bill,
> >         The multicast router does not 'join' any group at the IP
> >         level. The router sets the ethernet interface in all-multicast
> >         promiscuous mode. Thus the router gets all multicast packets.
> > 
> >         It filters or forwards the packets based on the multicast
> >         routing protocol updates.
> > 
> > Vivek
> > 
> > 
> > > 
> > > Question then...
> > > 
> > > As an IP Multicast sender, I am not required to join 
> > > the Multicast group... So now there has not been an
> > > IGMP message telling the first hop router to join the
> > > group either.  So the router has not told its layer
> > > 2 to join the broadcast address for the multicast group.
> > > 
> > > How does the first hop Multicast router know to join
> > > this group and forward the traffic on to the multicast
> > > tree (either RP, or Prune/Flood) if it can't receive
> > > the initial packet...
> > > 
> > > This is the magic that I was wanting to know, I know how
> > > I would make it work, but I am hoping Bill F. can tell me
> > > the correct way to do it <G>
> > > 
> > > Bill Strahm
> > > 
> > > On Tue, Apr 09, 2002 at 11:36:22AM -0700, Sukanta ganguly wrote:
> > > > Bill,
> > > >   When you send in the Multicast packet (as a sender),
> > > > at layer 3 you send it to the multicast IP address
> > > > that the sender decided on. Your layer 2 module will
> > > > decide on the next hop for that Multicast IP address.
> > > > And a similar functioning will be build up as a chain.
> > > > Note in this pacth while the senders original packet
> > > > is moving to its destination which is the IP multicast
> > > > address, the intermediate routers (which are multicast
> > > > aware) will check to see if they have any IGMP joins
> > > > they have received from any other subnets. If they
> > > > have any such mapping they have built due to receive
> > > > of IGMP joins then they will oblige that join by
> > > > copying the packet to that joined destination. That's
> > > > how the copy of the packet is enroute to its rightful
> > > > receiver.
> > > > 
> > > > SG
> > > > 
> > > > P.S :: Dr Fenner is an authority on this and can
> > > > provide some special comments 
> > > > 
> > > > 
> > > > --- Bill Strahm <bill@strahm.net> wrote:
> > > > > Ok, I am going to express a scary amount of
> > > > > ignorance here...
> > > > > 
> > > > > I know how multicast works on receive, I know how
> > > > > IGMP works to build the forwarding tree,
> > > > > I know how flood/prune, RP Multicast Routing
> > > > > protocols work...
> > > > > 
> > > > > What I would like a pointer to is a simple this is
> > > > > how a first hop multicast router 
> > > > > behaves.  An example on Ethernet would be
> > > > > acceptable, a generic example would be better.
> > > > > 
> > > > > The question I have is as a Multicast sender, I send
> > > > > a multicast packet to the first
> > > > > hop router that is multicast aware.  Now, do I send
> > > > > that packet to its unicast MAC 
> > > > > address, or do I send that to the mapped layer 2
> > > > > broadcast address and let the IP 
> > > > > layer rebroadcast to the broadcast address if it is
> > > > > needed on the local subnet.
> > > > > 
> > > > > Once I have that, everything else I understand
> > > > > 
> > > > > Bill
> > > > > PS Bill Fenner, the IPoIB list is subscriber posting
> > > > > only.  If you haven't subscribed, please do 
> > > > > so we can get the benefit of your wisdom...
> > > > > 
> > > > > On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley,
> > > > > Paul wrote:
> > > > > > I suspect that the problem can be made simpler; IP
> > > > > routers should always IB_join ALL IPoIB Multicast
> > > > > groups for a given interface.  The router will
> > > > > simply ask to join any groups in the same
> > > > > partition(s) it is assigned to.  Whether the IP
> > > > > router forwards the multicast packets will then
> > > > > depend on the IP routing protocol.
> > > > > > 
> > > > > > IB routers have the same problem, and may use the
> > > > > same solution...
> > > > > > 
> > > > > > Paul R. Culley
> > > > > > Compaq Sr. Fellow
> > > > > > 281-514-5543
> > > > > > 
> > > > > > 
> > > > > > -----Original Message-----
> > > > > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > > > > > Sent: Tuesday, April 09, 2002 3:31 AM
> > > > > > To: ipoverib@ietf.org
> > > > > > Cc: fenner@research.att.com
> > > > > > Subject: [Ipoverib] IPoIB multicast forwarding
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > Folks,
> > > > > > 
> > > > > >         We have defined a mapping from the IP
> > > > > addresses to the IB
> > > > > >         multicast groups. Use of different groups
> > > > > causes minimal
> > > > > >         replication at the switches and less load
> > > > > at the receivers.
> > > > > > 
> > > > > >         But there is a quirk in the way the groups
> > > > > are setup in
> > > > > >         InfiniBand - there is no way for any of
> > > > > the nodes to know if
> > > > > >         another has created an InfiniBand group.
> > > > > This is easy to fix
> > > > > >         in most cases - one checks with the SM.
> > > > > But it doesn't work
> > > > > >         out well for IPoIB packet forwarding.
> > > > > > 
> > > > > >         The router needs to forward the multicast
> > > > > packets beyond the
> > > > > >         local IP subnet if it knows there are
> > > > > recipients for it or in
> > > > > >         the case of some routing protocols it must
> > > > > do that unless it
> > > > > >         gets an update stating not to forward.
> > > > > Similarly, it needs to
> > > > > >         forward external packets to the local
> > > > > receivers if there are
> > > > > >         any receiver present. 
> > > > > > 
> > > > > >         Thus if there are IB multicast groups
> > > > > associated with an IP
> > > > > >         multicast group the router must know about
> > > > > them or the router
> > > > > >         must receive all multicast packets in the
> > > > > subnet via some
> > > > > >         other construct.
> > > > > > 
> > > > > >         Below are the three alternatives that I
> > > > > mentioned at the
> > > > > >         53rd IETF meet. 
> > > > > > 	
> > > > > >         i) Define only one IB multicast group i.e.
> > > > > IB broadcast MGID.
> > > > > >         All IP multicast packets use this group.
> > > > > > 
> > > > > >         ii) The multicast IP packet is sent to
> > > > > both the associated IB
> > > > > >         multicast group and the IB mutlicast group
> > > > > corresponding to
> > > > > >         the all-IP routers IP group.
> > > > > > 
> > > > > >         iii) Sender always sends to corresonding
> > > > > IB group if it exists
> > > > > >         else it sends to the broadcast group.
> > > > > > 
> > > > > >              IP receivers create IB groups and
> > > > > send the IGMP/MLD
> > > > > >              groups on the broadcast MGID.
> > > > > > 
> > > > > >              Routers join relevant groups based on
> > > > > the IGMP report
> > > > > >              and routing protocol updates.
> > > > > > 
> > > > > > 
> > > > > >         I also put this problem to Bill Fenner (AD
> > > > > of routing area)
> > > > > >         and asked his advice on the alternatives.
> > > > > Of the three options
> > > > > >         he favored option (iii) above.
> > > > > > 
> > > > > >         Option (i) doesn't take advantage fo IB
> > > > > multicast at all and
> > > > > >         so should be looked at as a last resort
> > > > > solution only.
> > > > > > 
> > > > > >         Option (ii) requires the sender to send
> > > > > two copies of the same
> > > > > >         packet and 'deliberately' increases the
> > > > > chance of duplicates
> > > > > >         spreading across the network.
> > > > > > 
> > > > > > 
> > > > > > Suggested solution  in detail:
> > > > > > 
> > > > > > 
> > > > > >         1. Pure senders - at IB level check for
> > > > > the relevant IB group.
> > > > > >            If it exists then send to the IB group
> > > > > else send to the
> > > > > >            'all-routers' IB group. If there is no
> > > > > IB group
> > > > > >            corresponding to a particular IP group
> > > > > then it implies that
> > > > > >            there are no IP receivers on the subnet
> > > > > for that IP
> > > > > >            multicast group. 
> > > > > > 
> > > > > >            Note: a) Pure senders don't create IB
> > > > > groups. This is
> > > > > >            because there is no existing mechanism
> > > > > for the senders to
> > > > > >            let the routers know of the new groups.
> > > > > > 
> > > > > >         2. IP receivers: As per IGMP/MLD the first
> > > > > packet sent by a
> > > > > >            host on joining a multicast group for
> > > > > receiving
> > > > > >            (IP_ADD_MEMBERSHIP) is the multicast
> > > > > report.
> > > > > > 
> > > > > > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> > > > > do the following:
> > > > > > 
> > > > > > 	   	check if the corresponding IB group exists. 
> > > > > > 
> > > > > > 		if (IB group exists)
> > > > > > 			IB_join the group and send the IGMP/MLD
> > report
> > > > > > 
> > > > > > 		else
> > > > > > 			IB_Create the group
> > > > > > 			IB_join the group
> > > > > >                         Send IGMP/MLD report to IB
> > > > > group corresponding
> > > > > >                         to all-routers (all-hosts
> > > > > or broadcast) IP
> > > > > >                         group (Note: only done
> > > > > when creating the IB
> > > > > >                         group) 
> > > > > > 
> > > > > > 	3. IP routers IB_join MGIDs on:
> > > > > > 
> > > > > > 		a) Finding a new IP group on the subnet in the
> > > > > IGMP/MLD report
> > > > > >                 b) Determining a need to forward
> > > > > IP multicast packets
> > > > > >                    through a routing protocol
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > Comments ?
> > > > > > 
> > > > > > 
> > > > > > Vivek
> > > > > 
> > > > === message truncated ===
> > > > 
> > > > 
> > > > __________________________________________________
> > > > Do You Yahoo!?
> > > > Yahoo! Tax Center - online filing with TurboTax
> > > > http://taxes.yahoo.com/
> > > 
> > > _______________________________________________
> > > IPoverIB mailing list
> > > IPoverIB@ietf.org
> > > https://www1.ietf.org/mailman/listinfo/ipoverib
> > > 
> > 
> > 
> > -- 
> > Vivek Kashyap
> > Linux Technology Center, IBM
> > kashyapv@us.ibm.com 
> > vivk@us.ibm.com 
> > 503 578 3422 (o)
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 10 15:50:03 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA00797
	for <ipoverib-archive@odin.ietf.org>; Wed, 10 Apr 2002 15:50:02 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA16102
	for ipoverib-archive@odin.ietf.org; Wed, 10 Apr 2002 15:50:06 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA15793;
	Wed, 10 Apr 2002 15:44:52 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA15753
	for <ipoverib@optimus.ietf.org>; Wed, 10 Apr 2002 15:44:47 -0400 (EDT)
Received: from hermes.fm.intel.com (fmr01.intel.com [192.55.52.18])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA00583
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 15:41:50 -0400 (EDT)
Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37])
	by hermes.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.38 2002/04/09 21:20:23 root Exp $) with ESMTP id g3AJh5o20568
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 19:43:05 GMT
Received: from fmsmsxvs040.fm.intel.com (fmsmsxv040-1.fm.intel.com [132.233.48.108])
	by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.15 2002/04/01 17:51:48 root Exp $) with SMTP id g3AJeSm02153
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 19:40:28 GMT
Received: from fmsmsx28.fm.intel.com ([132.233.42.28])
 by fmsmsxvs040.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002041012411700565
 ; Wed, 10 Apr 2002 12:41:17 -0700
Received: by fmsmsx28.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2KDVW6FH>; Wed, 10 Apr 2002 12:41:53 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F73@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Vivek Kashyap'" <kashyapv@us.ibm.com>
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Wed, 10 Apr 2002 12:41:50 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

/rkl
embedded
rkl/

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
Sent: Wednesday, April 10, 2002 11:45 AM
To: roy.k.larsen@intel.com
Cc: kashyapv@us.ibm.com; ipoverib@ietf.org
Subject: Re: [Ipoverib] IPoIB multicast forwarding


> 
> > 
> > Ahh, that clarifies it for me.  However, from an implementation point of
> > view, the IPoIB link driver can not indicate that it supports
promiscuous
> > mode, so one would presume that the OS network infrastructure will not
use
> > that interface for multicast routing unless the OS network stack were
> > modified.  If you need to modify the network stack, why not simply alter
> it
> > so that it IB creates/joins all multicast groups it has received updates
> > for?  In other words, keep all the special processing and "fixes" local
to
> > the router implementation and out of the common host IPoIB driver.
> 
> Aren't you saying the same thing as the solution I have presented?
> 
> /rkl
> I am utterly confused so I don't know.  I was trying to remove the need of
a
> multicast sender to broadcast a frame if the multicast group did not
exist.

        Actually, it is not broadcast it is sent to all-routers MGID.
        So it is received by routers only. Since there are no
        receivers there is no need to send it on the broadcast MGID.

> Your response to Bill said that multicast router does not join any group
at
> the IP level.  However, below you state that it will IB_join based on
> IGMP/MLD reports.  Isn't that at the IP level?  If not, what is going to
> prompt the IPoIB driver to do the IB_join?

        The IGMP/MLD report lets the router know the IP multicast
        groups that exist on an interface. I am suggesting that the
        router then give a command to the interface to add the
        corresponding IB group to its list. This command, at the IPoIB
        driver translates into an IB_join for the corresponding IB
        group. 

> 
> If I understood Yaron's proposals correctly, in one he suggests simply
> broadcast all multicast frames on IB fabrics.  When considering the
overhead
> and relative complexity of IB multicast management, this is very
compelling
> to me.  

        It is compelling to you to broadcast everything and yet you
        are bothered that a sender will broadcast(though it won't) its
        packets if it can't IB_join the relevant group ???
/rkl
I am bothered by complexity for arguably marginal benefit.  I am bothered
that the IPoIB encapsulation protocol is requiring existing OS and user mode
protocol implementations to be modified to fit our needs (DHCP, now IGMP,
and Yaron has brought up VRRP).  We should be striving to keep our impact to
link encapsulation.  When given the choice between a compromise on the usage
of some link capabilities and requiring changes in existing OS stacks and
network applications, I will always come down on the side of compromise.
For companies that don't supply a complete vertical solution, supplying
IPoIB related products that require modifications to the core OS and
application environment leave them at a distinct disadvantage.  This type of
environment does not lend itself to the wide adoption of IBA for general
networking.

Roy
rkl/

But I obviously need to get much smarter on IGMP before contributing
> anything meaningful to this thread.
> 
> The other point I was trying to make was one of implementation.  An IPoIB
> driver should not indicate that it supports promiscuous/all-multicast
> filtering unless it truly can.  So, if routers using IGMP protocols
require
> this capability of a network interface, there are going to be some
software
> integration issues.

        This does not arise with respect to the solution I have
        indicated. Neither the router nor the end hosts are being
        asked to support such a mode. 
	
        The solution hinges on the IP multicast senders sending to
        groups that exist - IB group corresponding to the IP group or
        the all-routers MGID. 
	
        The IP multicast receivers create IB groups if they don't
        exist. When they create a group they must ensure that the
        IGMP/MLD report is received by the routers. And so instead of
        sending it to the group they just created they send it to the
        MGID corresponding to the all-routers IP address.

        The IP routers receive packets only on MGIDs they are members
        of - this includes the broadcast/all-routers MGIDs and the
        MGIDs they have joined. When they parse an IGMP/MLD report and
        find a new IP group they IB_join the corrresponding IB group.


Vivek

> 
> Roy
> rkl/
> 
>         The router knows the groups to forward to based on IGMP/MLD
>         reports and multicast routing protocol updates. It therefore
>         IB_joins (or creates in some cases) the corresponding IB
>         MGIDs. 
> 
> For the above to occur the MLD/IGMP report must reach the router. So,
> the host must send it to the router.
> 
>         The host network stack is not effected. The solution is
>         limited to the host's interface driver. As I had mentioned in
>         the presentation at the IETF meet: the IP_ADD_MEMBERSHIP call
>         causes the host IP stack to tell the interface
>         (ethernet/IPoIB/FDDI/Token ring etc.) of the IP multicast
>         group. In the case of ethernet the ethernet driver updates the
>         device/internal tables to include the corresponding ethernet
>         mutlicast group. In the case of IPoIB the driver lets the
>         SM/SA know (create if group doesn't exist; join if it
>         exists). 
> 
> So, the solution is limited to a) routers' IPoIB module b) in the
> IPoIB interface driver of the host.
> 
> 
> Vivek
> > 
> > Roy
> > 
> > -----Original Message-----
> > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > Sent: Tuesday, April 09, 2002 12:37 PM
> > To: bill@strahm.net
> > Cc: sganguly@yahoo.com; bill@strahm.net; Paul.Culley@compaq.com;
> > kashyapv@us.ibm.com; ipoverib@ietf.org; fenner@research.att.com
> > Subject: Re: [Ipoverib] IPoIB multicast forwarding
> > 
> > 
> > Bill,
> >         The multicast router does not 'join' any group at the IP
> >         level. The router sets the ethernet interface in all-multicast
> >         promiscuous mode. Thus the router gets all multicast packets.
> > 
> >         It filters or forwards the packets based on the multicast
> >         routing protocol updates.
> > 
> > Vivek
> > 
> > 
> > > 
> > > Question then...
> > > 
> > > As an IP Multicast sender, I am not required to join 
> > > the Multicast group... So now there has not been an
> > > IGMP message telling the first hop router to join the
> > > group either.  So the router has not told its layer
> > > 2 to join the broadcast address for the multicast group.
> > > 
> > > How does the first hop Multicast router know to join
> > > this group and forward the traffic on to the multicast
> > > tree (either RP, or Prune/Flood) if it can't receive
> > > the initial packet...
> > > 
> > > This is the magic that I was wanting to know, I know how
> > > I would make it work, but I am hoping Bill F. can tell me
> > > the correct way to do it <G>
> > > 
> > > Bill Strahm
> > > 
> > > On Tue, Apr 09, 2002 at 11:36:22AM -0700, Sukanta ganguly wrote:
> > > > Bill,
> > > >   When you send in the Multicast packet (as a sender),
> > > > at layer 3 you send it to the multicast IP address
> > > > that the sender decided on. Your layer 2 module will
> > > > decide on the next hop for that Multicast IP address.
> > > > And a similar functioning will be build up as a chain.
> > > > Note in this pacth while the senders original packet
> > > > is moving to its destination which is the IP multicast
> > > > address, the intermediate routers (which are multicast
> > > > aware) will check to see if they have any IGMP joins
> > > > they have received from any other subnets. If they
> > > > have any such mapping they have built due to receive
> > > > of IGMP joins then they will oblige that join by
> > > > copying the packet to that joined destination. That's
> > > > how the copy of the packet is enroute to its rightful
> > > > receiver.
> > > > 
> > > > SG
> > > > 
> > > > P.S :: Dr Fenner is an authority on this and can
> > > > provide some special comments 
> > > > 
> > > > 
> > > > --- Bill Strahm <bill@strahm.net> wrote:
> > > > > Ok, I am going to express a scary amount of
> > > > > ignorance here...
> > > > > 
> > > > > I know how multicast works on receive, I know how
> > > > > IGMP works to build the forwarding tree,
> > > > > I know how flood/prune, RP Multicast Routing
> > > > > protocols work...
> > > > > 
> > > > > What I would like a pointer to is a simple this is
> > > > > how a first hop multicast router 
> > > > > behaves.  An example on Ethernet would be
> > > > > acceptable, a generic example would be better.
> > > > > 
> > > > > The question I have is as a Multicast sender, I send
> > > > > a multicast packet to the first
> > > > > hop router that is multicast aware.  Now, do I send
> > > > > that packet to its unicast MAC 
> > > > > address, or do I send that to the mapped layer 2
> > > > > broadcast address and let the IP 
> > > > > layer rebroadcast to the broadcast address if it is
> > > > > needed on the local subnet.
> > > > > 
> > > > > Once I have that, everything else I understand
> > > > > 
> > > > > Bill
> > > > > PS Bill Fenner, the IPoIB list is subscriber posting
> > > > > only.  If you haven't subscribed, please do 
> > > > > so we can get the benefit of your wisdom...
> > > > > 
> > > > > On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley,
> > > > > Paul wrote:
> > > > > > I suspect that the problem can be made simpler; IP
> > > > > routers should always IB_join ALL IPoIB Multicast
> > > > > groups for a given interface.  The router will
> > > > > simply ask to join any groups in the same
> > > > > partition(s) it is assigned to.  Whether the IP
> > > > > router forwards the multicast packets will then
> > > > > depend on the IP routing protocol.
> > > > > > 
> > > > > > IB routers have the same problem, and may use the
> > > > > same solution...
> > > > > > 
> > > > > > Paul R. Culley
> > > > > > Compaq Sr. Fellow
> > > > > > 281-514-5543
> > > > > > 
> > > > > > 
> > > > > > -----Original Message-----
> > > > > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > > > > > Sent: Tuesday, April 09, 2002 3:31 AM
> > > > > > To: ipoverib@ietf.org
> > > > > > Cc: fenner@research.att.com
> > > > > > Subject: [Ipoverib] IPoIB multicast forwarding
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > Folks,
> > > > > > 
> > > > > >         We have defined a mapping from the IP
> > > > > addresses to the IB
> > > > > >         multicast groups. Use of different groups
> > > > > causes minimal
> > > > > >         replication at the switches and less load
> > > > > at the receivers.
> > > > > > 
> > > > > >         But there is a quirk in the way the groups
> > > > > are setup in
> > > > > >         InfiniBand - there is no way for any of
> > > > > the nodes to know if
> > > > > >         another has created an InfiniBand group.
> > > > > This is easy to fix
> > > > > >         in most cases - one checks with the SM.
> > > > > But it doesn't work
> > > > > >         out well for IPoIB packet forwarding.
> > > > > > 
> > > > > >         The router needs to forward the multicast
> > > > > packets beyond the
> > > > > >         local IP subnet if it knows there are
> > > > > recipients for it or in
> > > > > >         the case of some routing protocols it must
> > > > > do that unless it
> > > > > >         gets an update stating not to forward.
> > > > > Similarly, it needs to
> > > > > >         forward external packets to the local
> > > > > receivers if there are
> > > > > >         any receiver present. 
> > > > > > 
> > > > > >         Thus if there are IB multicast groups
> > > > > associated with an IP
> > > > > >         multicast group the router must know about
> > > > > them or the router
> > > > > >         must receive all multicast packets in the
> > > > > subnet via some
> > > > > >         other construct.
> > > > > > 
> > > > > >         Below are the three alternatives that I
> > > > > mentioned at the
> > > > > >         53rd IETF meet. 
> > > > > > 	
> > > > > >         i) Define only one IB multicast group i.e.
> > > > > IB broadcast MGID.
> > > > > >         All IP multicast packets use this group.
> > > > > > 
> > > > > >         ii) The multicast IP packet is sent to
> > > > > both the associated IB
> > > > > >         multicast group and the IB mutlicast group
> > > > > corresponding to
> > > > > >         the all-IP routers IP group.
> > > > > > 
> > > > > >         iii) Sender always sends to corresonding
> > > > > IB group if it exists
> > > > > >         else it sends to the broadcast group.
> > > > > > 
> > > > > >              IP receivers create IB groups and
> > > > > send the IGMP/MLD
> > > > > >              groups on the broadcast MGID.
> > > > > > 
> > > > > >              Routers join relevant groups based on
> > > > > the IGMP report
> > > > > >              and routing protocol updates.
> > > > > > 
> > > > > > 
> > > > > >         I also put this problem to Bill Fenner (AD
> > > > > of routing area)
> > > > > >         and asked his advice on the alternatives.
> > > > > Of the three options
> > > > > >         he favored option (iii) above.
> > > > > > 
> > > > > >         Option (i) doesn't take advantage fo IB
> > > > > multicast at all and
> > > > > >         so should be looked at as a last resort
> > > > > solution only.
> > > > > > 
> > > > > >         Option (ii) requires the sender to send
> > > > > two copies of the same
> > > > > >         packet and 'deliberately' increases the
> > > > > chance of duplicates
> > > > > >         spreading across the network.
> > > > > > 
> > > > > > 
> > > > > > Suggested solution  in detail:
> > > > > > 
> > > > > > 
> > > > > >         1. Pure senders - at IB level check for
> > > > > the relevant IB group.
> > > > > >            If it exists then send to the IB group
> > > > > else send to the
> > > > > >            'all-routers' IB group. If there is no
> > > > > IB group
> > > > > >            corresponding to a particular IP group
> > > > > then it implies that
> > > > > >            there are no IP receivers on the subnet
> > > > > for that IP
> > > > > >            multicast group. 
> > > > > > 
> > > > > >            Note: a) Pure senders don't create IB
> > > > > groups. This is
> > > > > >            because there is no existing mechanism
> > > > > for the senders to
> > > > > >            let the routers know of the new groups.
> > > > > > 
> > > > > >         2. IP receivers: As per IGMP/MLD the first
> > > > > packet sent by a
> > > > > >            host on joining a multicast group for
> > > > > receiving
> > > > > >            (IP_ADD_MEMBERSHIP) is the multicast
> > > > > report.
> > > > > > 
> > > > > > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> > > > > do the following:
> > > > > > 
> > > > > > 	   	check if the corresponding IB group exists. 
> > > > > > 
> > > > > > 		if (IB group exists)
> > > > > > 			IB_join the group and send the IGMP/MLD
> > report
> > > > > > 
> > > > > > 		else
> > > > > > 			IB_Create the group
> > > > > > 			IB_join the group
> > > > > >                         Send IGMP/MLD report to IB
> > > > > group corresponding
> > > > > >                         to all-routers (all-hosts
> > > > > or broadcast) IP
> > > > > >                         group (Note: only done
> > > > > when creating the IB
> > > > > >                         group) 
> > > > > > 
> > > > > > 	3. IP routers IB_join MGIDs on:
> > > > > > 
> > > > > > 		a) Finding a new IP group on the subnet in the
> > > > > IGMP/MLD report
> > > > > >                 b) Determining a need to forward
> > > > > IP multicast packets
> > > > > >                    through a routing protocol
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > Comments ?
> > > > > > 
> > > > > > 
> > > > > > Vivek
> > > > > 
> > > > === message truncated ===
> > > > 
> > > > 
> > > > __________________________________________________
> > > > Do You Yahoo!?
> > > > Yahoo! Tax Center - online filing with TurboTax
> > > > http://taxes.yahoo.com/
> > > 
> > > _______________________________________________
> > > IPoverIB mailing list
> > > IPoverIB@ietf.org
> > > https://www1.ietf.org/mailman/listinfo/ipoverib
> > > 
> > 
> > 
> > -- 
> > Vivek Kashyap
> > Linux Technology Center, IBM
> > kashyapv@us.ibm.com 
> > vivk@us.ibm.com 
> > 503 578 3422 (o)
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 10 16:31:11 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA02021
	for <ipoverib-archive@odin.ietf.org>; Wed, 10 Apr 2002 16:31:11 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA18541
	for ipoverib-archive@odin.ietf.org; Wed, 10 Apr 2002 16:31:15 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA18099;
	Wed, 10 Apr 2002 16:26:57 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA18032
	for <ipoverib@optimus.ietf.org>; Wed, 10 Apr 2002 16:26:53 -0400 (EDT)
Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA01936
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 16:26:45 -0400 (EDT)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e31.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3AKQkNr023952;
	Wed, 10 Apr 2002 16:26:47 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3AKQMv110678;
	Wed, 10 Apr 2002 14:26:22 -0600
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g3AKQJM13759;
	Wed, 10 Apr 2002 13:26:19 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3AKQH521604;
	Wed, 10 Apr 2002 13:26:17 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204102026.g3AKQH521604@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: roy.k.larsen@intel.com (Larsen, Roy K)
Date: Wed, 10 Apr 2002 13:26:15 -0700 (PDT)
Cc: kashyapv@us.ibm.com ('Vivek Kashyap'), ipoverib@ietf.org
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4F73@ORSMSX108> from "Larsen, Roy K" at Apr 10, 2002 11:41:50 AM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> > If I understood Yaron's proposals correctly, in one he suggests simply
> > broadcast all multicast frames on IB fabrics.  When considering the
> overhead
> > and relative complexity of IB multicast management, this is very
> compelling
> > to me.  
> 
>         It is compelling to you to broadcast everything and yet you
>         are bothered that a sender will broadcast(though it won't) its
>         packets if it can't IB_join the relevant group ???
> /rkl
> I am bothered by complexity for arguably marginal benefit.  I am bothered
> that the IPoIB encapsulation protocol is requiring existing OS and user mode
> protocol implementations to be modified to fit our needs (DHCP, now IGMP,
> and Yaron has brought up VRRP).  We should be striving to keep our impact to
> link encapsulation.  When given the choice between a compromise on the usage
> of some link capabilities and requiring changes in existing OS stacks and
> network applications, I will always come down on the side of compromise.
> For companies that don't supply a complete vertical solution, supplying
> IPoIB related products that require modifications to the core OS and
> application environment leave them at a distinct disadvantage.  This type of
> environment does not lend itself to the wide adoption of IBA for general
> networking.
> 
> Roy
> rkl/

        Can you point to the exact 'complexity' that bothers you in
        the proposed solution? What 'core' OS or application setup in
        your implementation is adversely effected ?

        The proposal is taking care to keep the impact minimal while
        utilising the benefit offered. It effects only those that want
        to implement IPoIB and not others. It does not effect the
        protocols themselves (DHCP/IGMP are left untouched). 
	
        By dumbing down the solution to a broadcast link we will loose
        all the benefit that could have accrued to us. I would rather
        not have unneccesary broadcasts take away the bandwidth from
        my storage data. I also would rather that the unnecessary
        broadcasts do not fill my receive buffers and take up CPU time
        in filtering the irrelevant packets. I would rather that the
        IB switches don't get overwhelmed replicating packets out
        every port for broadcast but rather do it only for the links
        that really need such replication - implying use separate
        multicast groups as far as possible. Not all considerations
        need to be software specific; we also need to look at other
        issues. 

        As I said, I did ask about the impact on the routers and was
        told it was an acceptable requirement. 

Vivek

> 
> But I obviously need to get much smarter on IGMP before contributing
> > anything meaningful to this thread.
> > 
> > The other point I was trying to make was one of implementation.  An IPoIB
> > driver should not indicate that it supports promiscuous/all-multicast
> > filtering unless it truly can.  So, if routers using IGMP protocols
> require
> > this capability of a network interface, there are going to be some
> software
> > integration issues.
> 
>         This does not arise with respect to the solution I have
>         indicated. Neither the router nor the end hosts are being
>         asked to support such a mode. 
> 	
>         The solution hinges on the IP multicast senders sending to
>         groups that exist - IB group corresponding to the IP group or
>         the all-routers MGID. 
> 	
>         The IP multicast receivers create IB groups if they don't
>         exist. When they create a group they must ensure that the
>         IGMP/MLD report is received by the routers. And so instead of
>         sending it to the group they just created they send it to the
>         MGID corresponding to the all-routers IP address.
> 
>         The IP routers receive packets only on MGIDs they are members
>         of - this includes the broadcast/all-routers MGIDs and the
>         MGIDs they have joined. When they parse an IGMP/MLD report and
>         find a new IP group they IB_join the corrresponding IB group.
> 
> 
> Vivek
> 
> > 
> > Roy
> > rkl/
> > 
> >         The router knows the groups to forward to based on IGMP/MLD
> >         reports and multicast routing protocol updates. It therefore
> >         IB_joins (or creates in some cases) the corresponding IB
> >         MGIDs. 
> > 
> > For the above to occur the MLD/IGMP report must reach the router. So,
> > the host must send it to the router.
> > 
> >         The host network stack is not effected. The solution is
> >         limited to the host's interface driver. As I had mentioned in
> >         the presentation at the IETF meet: the IP_ADD_MEMBERSHIP call
> >         causes the host IP stack to tell the interface
> >         (ethernet/IPoIB/FDDI/Token ring etc.) of the IP multicast
> >         group. In the case of ethernet the ethernet driver updates the
> >         device/internal tables to include the corresponding ethernet
> >         mutlicast group. In the case of IPoIB the driver lets the
> >         SM/SA know (create if group doesn't exist; join if it
> >         exists). 
> > 
> > So, the solution is limited to a) routers' IPoIB module b) in the
> > IPoIB interface driver of the host.
> > 
> > 
> > Vivek
> > > 
> > > Roy
> > > 
> > > -----Original Message-----
> > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > > Sent: Tuesday, April 09, 2002 12:37 PM
> > > To: bill@strahm.net
> > > Cc: sganguly@yahoo.com; bill@strahm.net; Paul.Culley@compaq.com;
> > > kashyapv@us.ibm.com; ipoverib@ietf.org; fenner@research.att.com
> > > Subject: Re: [Ipoverib] IPoIB multicast forwarding
> > > 
> > > 
> > > Bill,
> > >         The multicast router does not 'join' any group at the IP
> > >         level. The router sets the ethernet interface in all-multicast
> > >         promiscuous mode. Thus the router gets all multicast packets.
> > > 
> > >         It filters or forwards the packets based on the multicast
> > >         routing protocol updates.
> > > 
> > > Vivek
> > > 
> > > 
> > > > 
> > > > Question then...
> > > > 
> > > > As an IP Multicast sender, I am not required to join 
> > > > the Multicast group... So now there has not been an
> > > > IGMP message telling the first hop router to join the
> > > > group either.  So the router has not told its layer
> > > > 2 to join the broadcast address for the multicast group.
> > > > 
> > > > How does the first hop Multicast router know to join
> > > > this group and forward the traffic on to the multicast
> > > > tree (either RP, or Prune/Flood) if it can't receive
> > > > the initial packet...
> > > > 
> > > > This is the magic that I was wanting to know, I know how
> > > > I would make it work, but I am hoping Bill F. can tell me
> > > > the correct way to do it <G>
> > > > 
> > > > Bill Strahm
> > > > 
> > > > On Tue, Apr 09, 2002 at 11:36:22AM -0700, Sukanta ganguly wrote:
> > > > > Bill,
> > > > >   When you send in the Multicast packet (as a sender),
> > > > > at layer 3 you send it to the multicast IP address
> > > > > that the sender decided on. Your layer 2 module will
> > > > > decide on the next hop for that Multicast IP address.
> > > > > And a similar functioning will be build up as a chain.
> > > > > Note in this pacth while the senders original packet
> > > > > is moving to its destination which is the IP multicast
> > > > > address, the intermediate routers (which are multicast
> > > > > aware) will check to see if they have any IGMP joins
> > > > > they have received from any other subnets. If they
> > > > > have any such mapping they have built due to receive
> > > > > of IGMP joins then they will oblige that join by
> > > > > copying the packet to that joined destination. That's
> > > > > how the copy of the packet is enroute to its rightful
> > > > > receiver.
> > > > > 
> > > > > SG
> > > > > 
> > > > > P.S :: Dr Fenner is an authority on this and can
> > > > > provide some special comments 
> > > > > 
> > > > > 
> > > > > --- Bill Strahm <bill@strahm.net> wrote:
> > > > > > Ok, I am going to express a scary amount of
> > > > > > ignorance here...
> > > > > > 
> > > > > > I know how multicast works on receive, I know how
> > > > > > IGMP works to build the forwarding tree,
> > > > > > I know how flood/prune, RP Multicast Routing
> > > > > > protocols work...
> > > > > > 
> > > > > > What I would like a pointer to is a simple this is
> > > > > > how a first hop multicast router 
> > > > > > behaves.  An example on Ethernet would be
> > > > > > acceptable, a generic example would be better.
> > > > > > 
> > > > > > The question I have is as a Multicast sender, I send
> > > > > > a multicast packet to the first
> > > > > > hop router that is multicast aware.  Now, do I send
> > > > > > that packet to its unicast MAC 
> > > > > > address, or do I send that to the mapped layer 2
> > > > > > broadcast address and let the IP 
> > > > > > layer rebroadcast to the broadcast address if it is
> > > > > > needed on the local subnet.
> > > > > > 
> > > > > > Once I have that, everything else I understand
> > > > > > 
> > > > > > Bill
> > > > > > PS Bill Fenner, the IPoIB list is subscriber posting
> > > > > > only.  If you haven't subscribed, please do 
> > > > > > so we can get the benefit of your wisdom...
> > > > > > 
> > > > > > On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley,
> > > > > > Paul wrote:
> > > > > > > I suspect that the problem can be made simpler; IP
> > > > > > routers should always IB_join ALL IPoIB Multicast
> > > > > > groups for a given interface.  The router will
> > > > > > simply ask to join any groups in the same
> > > > > > partition(s) it is assigned to.  Whether the IP
> > > > > > router forwards the multicast packets will then
> > > > > > depend on the IP routing protocol.
> > > > > > > 
> > > > > > > IB routers have the same problem, and may use the
> > > > > > same solution...
> > > > > > > 
> > > > > > > Paul R. Culley
> > > > > > > Compaq Sr. Fellow
> > > > > > > 281-514-5543
> > > > > > > 
> > > > > > > 
> > > > > > > -----Original Message-----
> > > > > > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > > > > > > Sent: Tuesday, April 09, 2002 3:31 AM
> > > > > > > To: ipoverib@ietf.org
> > > > > > > Cc: fenner@research.att.com
> > > > > > > Subject: [Ipoverib] IPoIB multicast forwarding
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > Folks,
> > > > > > > 
> > > > > > >         We have defined a mapping from the IP
> > > > > > addresses to the IB
> > > > > > >         multicast groups. Use of different groups
> > > > > > causes minimal
> > > > > > >         replication at the switches and less load
> > > > > > at the receivers.
> > > > > > > 
> > > > > > >         But there is a quirk in the way the groups
> > > > > > are setup in
> > > > > > >         InfiniBand - there is no way for any of
> > > > > > the nodes to know if
> > > > > > >         another has created an InfiniBand group.
> > > > > > This is easy to fix
> > > > > > >         in most cases - one checks with the SM.
> > > > > > But it doesn't work
> > > > > > >         out well for IPoIB packet forwarding.
> > > > > > > 
> > > > > > >         The router needs to forward the multicast
> > > > > > packets beyond the
> > > > > > >         local IP subnet if it knows there are
> > > > > > recipients for it or in
> > > > > > >         the case of some routing protocols it must
> > > > > > do that unless it
> > > > > > >         gets an update stating not to forward.
> > > > > > Similarly, it needs to
> > > > > > >         forward external packets to the local
> > > > > > receivers if there are
> > > > > > >         any receiver present. 
> > > > > > > 
> > > > > > >         Thus if there are IB multicast groups
> > > > > > associated with an IP
> > > > > > >         multicast group the router must know about
> > > > > > them or the router
> > > > > > >         must receive all multicast packets in the
> > > > > > subnet via some
> > > > > > >         other construct.
> > > > > > > 
> > > > > > >         Below are the three alternatives that I
> > > > > > mentioned at the
> > > > > > >         53rd IETF meet. 
> > > > > > > 	
> > > > > > >         i) Define only one IB multicast group i.e.
> > > > > > IB broadcast MGID.
> > > > > > >         All IP multicast packets use this group.
> > > > > > > 
> > > > > > >         ii) The multicast IP packet is sent to
> > > > > > both the associated IB
> > > > > > >         multicast group and the IB mutlicast group
> > > > > > corresponding to
> > > > > > >         the all-IP routers IP group.
> > > > > > > 
> > > > > > >         iii) Sender always sends to corresonding
> > > > > > IB group if it exists
> > > > > > >         else it sends to the broadcast group.
> > > > > > > 
> > > > > > >              IP receivers create IB groups and
> > > > > > send the IGMP/MLD
> > > > > > >              groups on the broadcast MGID.
> > > > > > > 
> > > > > > >              Routers join relevant groups based on
> > > > > > the IGMP report
> > > > > > >              and routing protocol updates.
> > > > > > > 
> > > > > > > 
> > > > > > >         I also put this problem to Bill Fenner (AD
> > > > > > of routing area)
> > > > > > >         and asked his advice on the alternatives.
> > > > > > Of the three options
> > > > > > >         he favored option (iii) above.
> > > > > > > 
> > > > > > >         Option (i) doesn't take advantage fo IB
> > > > > > multicast at all and
> > > > > > >         so should be looked at as a last resort
> > > > > > solution only.
> > > > > > > 
> > > > > > >         Option (ii) requires the sender to send
> > > > > > two copies of the same
> > > > > > >         packet and 'deliberately' increases the
> > > > > > chance of duplicates
> > > > > > >         spreading across the network.
> > > > > > > 
> > > > > > > 
> > > > > > > Suggested solution  in detail:
> > > > > > > 
> > > > > > > 
> > > > > > >         1. Pure senders - at IB level check for
> > > > > > the relevant IB group.
> > > > > > >            If it exists then send to the IB group
> > > > > > else send to the
> > > > > > >            'all-routers' IB group. If there is no
> > > > > > IB group
> > > > > > >            corresponding to a particular IP group
> > > > > > then it implies that
> > > > > > >            there are no IP receivers on the subnet
> > > > > > for that IP
> > > > > > >            multicast group. 
> > > > > > > 
> > > > > > >            Note: a) Pure senders don't create IB
> > > > > > groups. This is
> > > > > > >            because there is no existing mechanism
> > > > > > for the senders to
> > > > > > >            let the routers know of the new groups.
> > > > > > > 
> > > > > > >         2. IP receivers: As per IGMP/MLD the first
> > > > > > packet sent by a
> > > > > > >            host on joining a multicast group for
> > > > > > receiving
> > > > > > >            (IP_ADD_MEMBERSHIP) is the multicast
> > > > > > report.
> > > > > > > 
> > > > > > > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> > > > > > do the following:
> > > > > > > 
> > > > > > > 	   	check if the corresponding IB group exists. 
> > > > > > > 
> > > > > > > 		if (IB group exists)
> > > > > > > 			IB_join the group and send the IGMP/MLD
> > > report
> > > > > > > 
> > > > > > > 		else
> > > > > > > 			IB_Create the group
> > > > > > > 			IB_join the group
> > > > > > >                         Send IGMP/MLD report to IB
> > > > > > group corresponding
> > > > > > >                         to all-routers (all-hosts
> > > > > > or broadcast) IP
> > > > > > >                         group (Note: only done
> > > > > > when creating the IB
> > > > > > >                         group) 
> > > > > > > 
> > > > > > > 	3. IP routers IB_join MGIDs on:
> > > > > > > 
> > > > > > > 		a) Finding a new IP group on the subnet in the
> > > > > > IGMP/MLD report
> > > > > > >                 b) Determining a need to forward
> > > > > > IP multicast packets
> > > > > > >                    through a routing protocol
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > Comments ?
> > > > > > > 
> > > > > > > 
> > > > > > > Vivek
> > > > > > 
> > > > > === message truncated ===
> > > > > 
> > > > > 
> > > > > __________________________________________________
> > > > > Do You Yahoo!?
> > > > > Yahoo! Tax Center - online filing with TurboTax
> > > > > http://taxes.yahoo.com/
> > > > 
> > > > _______________________________________________
> > > > IPoverIB mailing list
> > > > IPoverIB@ietf.org
> > > > https://www1.ietf.org/mailman/listinfo/ipoverib
> > > > 
> > > 
> > > 
> > > -- 
> > > Vivek Kashyap
> > > Linux Technology Center, IBM
> > > kashyapv@us.ibm.com 
> > > vivk@us.ibm.com 
> > > 503 578 3422 (o)
> > > 
> > > _______________________________________________
> > > IPoverIB mailing list
> > > IPoverIB@ietf.org
> > > https://www1.ietf.org/mailman/listinfo/ipoverib
> > > 
> > 
> > 
> > -- 
> > Vivek Kashyap
> > Linux Technology Center, IBM
> > kashyapv@us.ibm.com 
> > vivk@us.ibm.com 
> > 503 578 3422 (o)
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 10 16:38:34 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA02173
	for <ipoverib-archive@odin.ietf.org>; Wed, 10 Apr 2002 16:38:34 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA18872
	for ipoverib-archive@odin.ietf.org; Wed, 10 Apr 2002 16:38:38 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA18766;
	Wed, 10 Apr 2002 16:34:48 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA18733
	for <ipoverib@optimus.ietf.org>; Wed, 10 Apr 2002 16:34:35 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA02093
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 16:34:30 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3AKYJsX049214;
	Wed, 10 Apr 2002 16:34:19 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.00) with ESMTP id g3AKYFM101188;
	Wed, 10 Apr 2002 16:34:15 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g3AKYCM14096;
	Wed, 10 Apr 2002 13:34:12 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3AKY9K21774;
	Wed, 10 Apr 2002 13:34:09 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204102034.g3AKY9K21774@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
To: zaitcev@redhat.com (Pete Zaitcev)
Date: Wed, 10 Apr 2002 13:34:09 -0700 (PDT)
Cc: roy.k.larsen@intel.com (Larsen Roy K),
        krause@cup.hp.com ('Michael Krause'), ipoverib@ietf.org,
        bill@strahm.net
In-Reply-To: <20020408140034.A24355@devserv.devel.redhat.com> from "Pete Zaitcev" at Apr 08, 2002 01:00:34 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

I agree with Pete. I too don't see any benefit in adding type field
in the payload.

Vivek

> 
> I do not know if that would count, but I am against that,
> because to make a lower level driver to poke into something that
> belongs to upper levels seems a little retarded to me. My gut
> feeling (e.g. distilled expirience), suggests that such choice
> may also result in a performance hit down the road. Or may be not,
> we'll see.
> 
> Since I am NOT a member of the working group, but a mere implemntor
> of what was designed by smarter people (like you), I kept quiet.
> I am going to write some code and see if this works well.
> Unfortunately, it may be a little too late to change standards
> by that time...
> 
> -- Pete
> 
> > From: "Larsen, Roy K" <roy.k.larsen@intel.com>
> > Date: Mon, 8 Apr 2002 10:23:32 -0700 
> 
> > I recently went through the old mails on the EtherType issue and found that
> > of those folks expressing an opinion, more wanted or were willing to accept
> > the encapsulation field than were dead set against it.  There certainly
> > wasn't a strong consensus against.  
> > 
> > Roy
> > 
> > -----Original Message-----
> > From: Michael Krause [mailto:krause@cup.hp.com]
> > Sent: Monday, April 08, 2002 7:22 AM
> > To: ipoverib@ietf.org
> > Cc: bill@strahm.net
> > Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
> > 
> > 
> > At 12:15 PM 4/6/2002 -0800, H.K. Jerry Chu wrote:
> > >At the Minneapolis/IETF meeting we had a "room concensus" on the
> > >following:
> > >
> > >1. Embedding EtherType in the IPoIB UD payload.
> > 
> > I've seen strong consensus on the reflector that indicates it should not be 
> > used.  I don't want to resurface all of the arguments but there were more 
> > against this from what I could see.
> > 
> > >2. Requiring IPv4 over IB to have a minimal MTU of 1500.
> > >(This effectively rules out all the IB MTUs less than 2K for
> > >running IPoIB.)
> > 
> > Clarified to be for IB UD transport service.
> > 
> > Mike
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 10 17:00:27 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA02506
	for <ipoverib-archive@odin.ietf.org>; Wed, 10 Apr 2002 17:00:26 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id RAA20052
	for ipoverib-archive@odin.ietf.org; Wed, 10 Apr 2002 17:00:30 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA19211;
	Wed, 10 Apr 2002 16:49:38 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA19173
	for <ipoverib@optimus.ietf.org>; Wed, 10 Apr 2002 16:49:35 -0400 (EDT)
Received: from hermes.fm.intel.com (fmr01.intel.com [192.55.52.18])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA02361
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 16:49:22 -0400 (EDT)
Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39])
	by hermes.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.38 2002/04/09 21:20:23 root Exp $) with ESMTP id g3AKocX09735
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 20:50:38 GMT
Received: from fmsmsxvs040.fm.intel.com (fmsmsxv040-1.fm.intel.com [132.233.48.108])
	by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.15 2002/04/01 17:51:48 root Exp $) with SMTP id g3AKowo12677
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 20:50:58 GMT
Received: from fmsmsx28.fm.intel.com ([132.233.42.28])
 by fmsmsxvs040.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002041013484920903
 ; Wed, 10 Apr 2002 13:48:49 -0700
Received: by fmsmsx28.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2KDVW9A3>; Wed, 10 Apr 2002 13:49:25 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F74@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: zaitcev@redhat.com
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] 4/9 deadline for a consensus call
Date: Wed, 10 Apr 2002 13:49:23 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Humm, when Pete says he doesn't want to poke into something that belongs to
the upper levels, do you think he was talking about an IPoIB encapsulation
field?  I'm guessing he's referring to peeking into the IP header.

Pete, could you clarify your position?  The options before us are adding an
explicit LSAP (Ethertype) to the IPoIB encapsulation header to demultiplex
ARP and IPv4 frames or looking into the upper nibble of the first bytes of
the payload to infer whether it is an ARP or IPv4 frame.

Roy


-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
Sent: Wednesday, April 10, 2002 1:34 PM
To: zaitcev@redhat.com
Cc: roy.k.larsen@intel.com; krause@cup.hp.com; ipoverib@ietf.org;
bill@strahm.net
Subject: Re: [Ipoverib] 4/9 deadline for a consensus call


I agree with Pete. I too don't see any benefit in adding type field
in the payload.

Vivek

> 
> I do not know if that would count, but I am against that,
> because to make a lower level driver to poke into something that
> belongs to upper levels seems a little retarded to me. My gut
> feeling (e.g. distilled expirience), suggests that such choice
> may also result in a performance hit down the road. Or may be not,
> we'll see.
> 
> Since I am NOT a member of the working group, but a mere implemntor
> of what was designed by smarter people (like you), I kept quiet.
> I am going to write some code and see if this works well.
> Unfortunately, it may be a little too late to change standards
> by that time...
> 
> -- Pete
> 
> > From: "Larsen, Roy K" <roy.k.larsen@intel.com>
> > Date: Mon, 8 Apr 2002 10:23:32 -0700 
> 
> > I recently went through the old mails on the EtherType issue and found
that
> > of those folks expressing an opinion, more wanted or were willing to
accept
> > the encapsulation field than were dead set against it.  There certainly
> > wasn't a strong consensus against.  
> > 
> > Roy
> > 
> > -----Original Message-----
> > From: Michael Krause [mailto:krause@cup.hp.com]
> > Sent: Monday, April 08, 2002 7:22 AM
> > To: ipoverib@ietf.org
> > Cc: bill@strahm.net
> > Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
> > 
> > 
> > At 12:15 PM 4/6/2002 -0800, H.K. Jerry Chu wrote:
> > >At the Minneapolis/IETF meeting we had a "room concensus" on the
> > >following:
> > >
> > >1. Embedding EtherType in the IPoIB UD payload.
> > 
> > I've seen strong consensus on the reflector that indicates it should not
be 
> > used.  I don't want to resurface all of the arguments but there were
more 
> > against this from what I could see.
> > 
> > >2. Requiring IPv4 over IB to have a minimal MTU of 1500.
> > >(This effectively rules out all the IB MTUs less than 2K for
> > >running IPoIB.)
> > 
> > Clarified to be for IB UD transport service.
> > 
> > Mike
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 10 18:41:53 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA04560
	for <ipoverib-archive@odin.ietf.org>; Wed, 10 Apr 2002 18:41:52 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id SAA25349
	for ipoverib-archive@odin.ietf.org; Wed, 10 Apr 2002 18:41:56 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA25198;
	Wed, 10 Apr 2002 18:36:57 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA25163
	for <ipoverib@optimus.ietf.org>; Wed, 10 Apr 2002 18:36:55 -0400 (EDT)
Received: from devserv.devel.redhat.com (nat-pool-rdu.redhat.com [66.187.233.200])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA04304
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 18:36:49 -0400 (EDT)
Received: (from zaitcev@localhost)
	by devserv.devel.redhat.com (8.11.6/8.11.0) id g3AMa5T07639;
	Wed, 10 Apr 2002 18:36:05 -0400
Date: Wed, 10 Apr 2002 18:36:05 -0400
From: Pete Zaitcev <zaitcev@redhat.com>
To: Bill Strahm <bill@strahm.net>
Cc: Pete Zaitcev <zaitcev@redhat.com>,
        "Larsen, Roy K" <roy.k.larsen@intel.com>, ipoverib@ietf.org
Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
Message-ID: <20020410183605.A4540@devserv.devel.redhat.com>
References: <D9223EB959A5D511A98F00508B68C20C06BC4F66@ORSMSX108> <20020408140034.A24355@devserv.devel.redhat.com> <20020408111900.A68455@homegate.strahm.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <20020408111900.A68455@homegate.strahm.net>; from bill@strahm.net on Mon, Apr 08, 2002 at 11:19:00AM -0700
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

I think you saw a typical "first mail" blunder from me. Naturally
I expected that different QPNs would be used for IPv4 and ARP,
but since then I observed that it is not so. The clause 6.1.1
says "All IP communication to the relevant IPoIB interface
MUST be directed to this QPN. In the case of IPv4 subnets
[...] ARP reply packets are also directed to the same QPN".

Please discount my mail. I hope I did not cause you to blow
the 4/9 deadline?

-- Pete

> Date: Mon, 8 Apr 2002 11:19:00 -0700
> From: Bill Strahm <bill@strahm.net>

> By definition you ARE a member of the working group by subscribing
> to the mailing list and offering the opinion below...

> I am very interested in hearing what implementors have to say, I am
> of the opinion that including Ethertype in the payload of the packet
> is not the right thing to do, however, based on hearing some 
> implementation reports from people who actually ARE doing the work, I
> am very close to changing my mind again.  
> 
> Can you provide a quick summary on your position (I think I read that
> you are against the Ethertype field, but I could be wrong) and especially
> any technical reasons for that opinion, such as I can/can't do this
> if the field is/isn't there...
> 
> Bill

> On Mon, Apr 08, 2002 at 02:00:34PM -0400, Pete Zaitcev wrote:
> > I do not know if that would count, but I am against that,
> > because to make a lower level driver to poke into something that
> > belongs to upper levels seems a little retarded to me. My gut
> > feeling (e.g. distilled expirience), suggests that such choice
> > may also result in a performance hit down the road. Or may be not,
> > we'll see.
> > 
> > Since I am NOT a member of the working group, but a mere implemntor
> > of what was designed by smarter people (like you), I kept quiet.
> > I am going to write some code and see if this works well.
> > Unfortunately, it may be a little too late to change standards
> > by that time...
> > 
> > -- Pete
> > 
> > > From: "Larsen, Roy K" <roy.k.larsen@intel.com>
> > > Date: Mon, 8 Apr 2002 10:23:32 -0700 
> > 
> > > I recently went through the old mails on the EtherType issue and found that
> > > of those folks expressing an opinion, more wanted or were willing to accept
> > > the encapsulation field than were dead set against it.  There certainly
> > > wasn't a strong consensus against.  
> > > 
> > > Roy

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 10 18:51:46 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA04662
	for <ipoverib-archive@odin.ietf.org>; Wed, 10 Apr 2002 18:51:46 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id SAA25578
	for ipoverib-archive@odin.ietf.org; Wed, 10 Apr 2002 18:51:49 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA25329;
	Wed, 10 Apr 2002 18:41:12 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA25291
	for <ipoverib@optimus.ietf.org>; Wed, 10 Apr 2002 18:41:09 -0400 (EDT)
Received: from strahm.net (adsl-67-115-29-109.dsl.sndg02.pacbell.net [67.115.29.109])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA04552
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 18:41:05 -0400 (EDT)
Received: (from bill@localhost)
	by strahm.net (8.11.6/8.11.6) id g3AMgFT82346;
	Wed, 10 Apr 2002 15:42:15 -0700 (PDT)
	(envelope-from bill)
Date: Wed, 10 Apr 2002 15:42:14 -0700
From: Bill Strahm <bill@strahm.net>
To: Pete Zaitcev <zaitcev@redhat.com>
Cc: Bill Strahm <bill@strahm.net>, "Larsen, Roy K" <roy.k.larsen@intel.com>,
        ipoverib@ietf.org
Subject: Re: [Ipoverib] 4/9 deadline for a consensus call
Message-ID: <20020410154214.H81646@homegate.strahm.net>
References: <D9223EB959A5D511A98F00508B68C20C06BC4F66@ORSMSX108> <20020408140034.A24355@devserv.devel.redhat.com> <20020408111900.A68455@homegate.strahm.net> <20020410183605.A4540@devserv.devel.redhat.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <20020410183605.A4540@devserv.devel.redhat.com>; from zaitcev@redhat.com on Wed, Apr 10, 2002 at 06:36:05PM -0400
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

I'd just like to be very clear here, since we are going for concensus
here...
Can I get you (And anyone else who wants to chime in as well) to complete
the following sentence

I am [for|against] including the Ethertype header in the IPoIB encapsulation
draft.

That way Jerry and I do not have to try to interpret very differing
language and guess at what people are really trying to say...

Bill

On Wed, Apr 10, 2002 at 06:36:05PM -0400, Pete Zaitcev wrote:
> I think you saw a typical "first mail" blunder from me. Naturally
> I expected that different QPNs would be used for IPv4 and ARP,
> but since then I observed that it is not so. The clause 6.1.1
> says "All IP communication to the relevant IPoIB interface
> MUST be directed to this QPN. In the case of IPv4 subnets
> [...] ARP reply packets are also directed to the same QPN".
> 
> Please discount my mail. I hope I did not cause you to blow
> the 4/9 deadline?
> 
> -- Pete
> 
> > Date: Mon, 8 Apr 2002 11:19:00 -0700
> > From: Bill Strahm <bill@strahm.net>
> 
> > By definition you ARE a member of the working group by subscribing
> > to the mailing list and offering the opinion below...
> 
> > I am very interested in hearing what implementors have to say, I am
> > of the opinion that including Ethertype in the payload of the packet
> > is not the right thing to do, however, based on hearing some 
> > implementation reports from people who actually ARE doing the work, I
> > am very close to changing my mind again.  
> > 
> > Can you provide a quick summary on your position (I think I read that
> > you are against the Ethertype field, but I could be wrong) and especially
> > any technical reasons for that opinion, such as I can/can't do this
> > if the field is/isn't there...
> > 
> > Bill
> 
> > On Mon, Apr 08, 2002 at 02:00:34PM -0400, Pete Zaitcev wrote:
> > > I do not know if that would count, but I am against that,
> > > because to make a lower level driver to poke into something that
> > > belongs to upper levels seems a little retarded to me. My gut
> > > feeling (e.g. distilled expirience), suggests that such choice
> > > may also result in a performance hit down the road. Or may be not,
> > > we'll see.
> > > 
> > > Since I am NOT a member of the working group, but a mere implemntor
> > > of what was designed by smarter people (like you), I kept quiet.
> > > I am going to write some code and see if this works well.
> > > Unfortunately, it may be a little too late to change standards
> > > by that time...
> > > 
> > > -- Pete
> > > 
> > > > From: "Larsen, Roy K" <roy.k.larsen@intel.com>
> > > > Date: Mon, 8 Apr 2002 10:23:32 -0700 
> > > 
> > > > I recently went through the old mails on the EtherType issue and found that
> > > > of those folks expressing an opinion, more wanted or were willing to accept
> > > > the encapsulation field than were dead set against it.  There certainly
> > > > wasn't a strong consensus against.  
> > > > 
> > > > Roy

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 10 19:33:36 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA05161
	for <ipoverib-archive@odin.ietf.org>; Wed, 10 Apr 2002 19:33:35 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA27628
	for ipoverib-archive@odin.ietf.org; Wed, 10 Apr 2002 19:33:39 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA27066;
	Wed, 10 Apr 2002 19:28:48 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA27034
	for <ipoverib@optimus.ietf.org>; Wed, 10 Apr 2002 19:28:44 -0400 (EDT)
Received: from hebe.or.intel.com (jffdns02.or.intel.com [134.134.248.4])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA05121
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 19:28:39 -0400 (EDT)
Received: from orsmsxvs040.jf.intel.com (orsmsxvs040.jf.intel.com [192.168.65.206])
	by hebe.or.intel.com (8.11.6/8.11.6/d: solo.mc,v 1.33 2002/04/09 21:19:32 root Exp $) with SMTP id g3ANSBG16718
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 23:28:11 GMT
Received: from orsmsx26.jf.intel.com ([192.168.65.26])
 by orsmsxvs040.jf.intel.com (NAVGW 2.5.1.16) with SMTP id M2002041016350123630
 ; Wed, 10 Apr 2002 16:35:01 -0700
Received: by orsmsx26.jf.intel.com with Internet Mail Service (5.5.2653.19)
	id <2J3VKGBW>; Wed, 10 Apr 2002 16:28:11 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F75@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Vivek Kashyap'" <kashyapv@us.ibm.com>
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Wed, 10 Apr 2002 16:28:09 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

<snip>

>         It is compelling to you to broadcast everything and yet you
>         are bothered that a sender will broadcast(though it won't) its
>         packets if it can't IB_join the relevant group ???
> /rkl
> I am bothered by complexity for arguably marginal benefit.  I am bothered
> that the IPoIB encapsulation protocol is requiring existing OS and user
mode
> protocol implementations to be modified to fit our needs (DHCP, now IGMP,
> and Yaron has brought up VRRP).  We should be striving to keep our impact
to
> link encapsulation.  When given the choice between a compromise on the
usage
> of some link capabilities and requiring changes in existing OS stacks and
> network applications, I will always come down on the side of compromise.
> For companies that don't supply a complete vertical solution, supplying
> IPoIB related products that require modifications to the core OS and
> application environment leave them at a distinct disadvantage.  This type
of
> environment does not lend itself to the wide adoption of IBA for general
> networking.
> 
> Roy
> rkl/

        Can you point to the exact 'complexity' that bothers you in
        the proposed solution? What 'core' OS or application setup in
        your implementation is adversely effected ?

/rkl
Certainly there is complexity in managing IBA multicast groups.  It requires
statefull transmits.  This will be amplified if the LWG makes it necessary
to do "join to send" operations.  But, if we could faithfully support
multicast, I guess that's the price we pay.  But then we add the requirement
that says transform a multicast request from layer 3 into an all-routers
multicast if the group doesn't exist and, on a device type basis, have the
interface join/leave multicast groups on multicast routers when the IGMP
implementation receives IGMP/MLD reports.  If it could all go away with
broadcast-always, I'd tend to lean that way.
rkl/

        The proposal is taking care to keep the impact minimal while
        utilising the benefit offered. It effects only those that want
        to implement IPoIB and not others. It does not effect the
        protocols themselves (DHCP/IGMP are left untouched).

/rkl
When layer 3 transmit requests are not performed consistently, we are
technically changing the wire protocol and something I think we should not
disregarded lightly.  But for the sake of this discussion, my concern is not
about changing the wire protocols, it is requiring changes to their
implementation.  As you stated in a previous reply:

        "The IGMP/MLD report lets the router know the IP multicast
         groups that exist on an interface. I am suggesting that the
         router then give a command to the interface to add the
         corresponding IB group to its list. This command, at the IPoIB
         driver translates into an IB_join for the corresponding IB
         group."

So, we need IPoIB aware DHCP clients and now IPoIB aware IGMP
implementations.  We'll see what happens with VRRP.  It's not that any one
thing is a big deal, its just that the slope is getting pretty slippery.  My
vision is one that does not require a vertically aware IPoIB operating
environment.  Just an OS-standard network link driver that encapsulated IP
frames.  I am a realist and know that it is not quite that simple.  But I'll
tend to be a voice that pushes the group in that direction.
rkl/

        By dumbing down the solution to a broadcast link we will loose
        all the benefit that could have accrued to us. I would rather
        not have unneccesary broadcasts take away the bandwidth from
        my storage data. I also would rather that the unnecessary
        broadcasts do not fill my receive buffers and take up CPU time
        in filtering the irrelevant packets. I would rather that the
        IB switches don't get overwhelmed replicating packets out
        every port for broadcast but rather do it only for the links
        that really need such replication - implying use separate
        multicast groups as far as possible. Not all considerations
        need to be software specific; we also need to look at other
        issues.

/rkl
Technically, those are all valid concerns.  I don't have a good feel for how
heavily multicast is used in the data center, but of course it will all be
moot if the data center admin finds it far simpler, from an OS integration
and compatibility standpoint, to deploy emerging Ethernet technologies
instead of IBA.  This is my paramount concern.

Roy
rkl/
        As I said, I did ask about the impact on the routers and was
        told it was an acceptable requirement. 

Vivek
<snip>

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 10 21:53:41 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA07902
	for <ipoverib-archive@odin.ietf.org>; Wed, 10 Apr 2002 21:53:40 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id VAA03670
	for ipoverib-archive@odin.ietf.org; Wed, 10 Apr 2002 21:53:43 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id VAA03276;
	Wed, 10 Apr 2002 21:45:52 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id VAA03246
	for <ipoverib@optimus.ietf.org>; Wed, 10 Apr 2002 21:45:50 -0400 (EDT)
Received: from kathmandu.sun.com (kathmandu.sun.com [192.18.98.36])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA07740
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 21:45:46 -0400 (EDT)
Received: from sunmail1.Sun.COM ([129.145.1.2])
	by kathmandu.sun.com (8.9.3+Sun/8.9.3) with ESMTP id TAA15296
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 19:45:49 -0600 (MDT)
Received: from jurassic.eng.sun.com (jurassic.Eng.Sun.COM [129.146.89.50])
	by sunmail1.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1-Sun.COM.mod.2) with ESMTP id SAA13223
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 18:46:33 -0700 (PDT)
Received: from taipei (taipei.Eng.Sun.COM [129.146.85.178])
	by jurassic.eng.sun.com (8.12.2+Sun/8.12.2) with SMTP id g3B1jm35953423
	for <ipoverib@ietf.org>; Wed, 10 Apr 2002 18:45:48 -0700 (PDT)
Message-Id: <200204110145.g3B1jm35953423@jurassic.eng.sun.com>
Date: Wed, 10 Apr 2002 18:45:41 -0700 (PDT)
From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Reply-To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
To: ipoverib@ietf.org
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: SkgovZjLYJMZ+rum2+J3ew==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc 
Subject: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Folks,

We have passed the deadline for people to express their opinions.
For 2 and 3 (see the mail I sent on Sat. below) there seems to be
a unanimous support for them. For EtherType (1) the opinions remain
split.

Last week I asked people to bring "new, convincing evidence" when
arguing for or against EtherType. But unfortunately in the past few
days all I heard was either old arguments, or people reiterating their
positions. Since no one has brought forward any new, convincing
evidence against EtherType, and in both the mailing list and the
f2f meeting Bill and I feel there have been more people for EtherType
than against it, plus a number of people requesting EtherType are
actually working on real implementation, I hereby declare EtherType
to be the rough consensus of the WG to be included in the UD payload
of every IPoIB packet.

Over the past couple of days a couple of senior IBTA members have
reiterated their positions against EtherType. But in my opinion the
decision of including EtherType in the UD payload is at least partly
as the result of the IBTA failing to address people's need to have a
field in IB identifying the type of the data being encapsulated in
the UD payload. Some argued that the QP is supposed to serve that
purpose, but we are not even doing that for IPv4, ARP/RARP and IPv6.
I think the argument is rather weak.

I hope the debate on EtherType can stop now and we all move on to
work on other important issues. Vivek and I will update the relevant
drafts to reflect the decisions.

Your friendly co-chairs Bill & Jerry

----------------------------------------------------------------
Date: Sat, 6 Apr 2002 12:15:18 -0800 (PST)
From: "H.K. Jerry Chu" <hkchu@jurassic>
Subject: 4/9 deadline for a consensus call
To: ipoverib@ietf.org
Cc: bill@strahm.net
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.2 SunOS 5.8 sun4u sparc
Mime-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: u8in9Sq4ww3pAVDBKAA13A==
Content-Length: 940
Status:  O
X-Status: $$$$
X-UID: 0000000306

At the Minneapolis/IETF meeting we had a "room concensus" on the
following:

1. Embedding EtherType in the IPoIB UD payload.
2. Requiring IPv4 over IB to have a minimal MTU of 1500.
(This effectively rules out all the IB MTUs less than 2K for
running IPoIB.)
3. Using DHCP broadcast flag to solve the chaddr length problem
so the IPoIB hardware address stays as QPN+GID.

During the past two weeks since the meeting minutes was published
I have not heard of any objection or different opinion. The DHCPv4
over IB work seems to be going well too. So I'm inclined to declare
the above as WG concensus. But I'm willing to wait til next Tuesday
(4/9) COW for anyone to voice a different opinion. If you do decide
to object, I'd urge you to bring forward a strong case, e.g. new,
convincing evidence that shows the above is really problematic. We
have spent enough time bashing these topics it's time for us to
move on.                       
			       
Your friendly co-chair


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr 11 03:57:03 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA21687
	for <ipoverib-archive@odin.ietf.org>; Thu, 11 Apr 2002 03:57:02 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id DAA00900
	for ipoverib-archive@odin.ietf.org; Thu, 11 Apr 2002 03:57:04 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id DAA29894;
	Thu, 11 Apr 2002 03:31:56 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id DAA29840
	for <ipoverib@optimus.ietf.org>; Thu, 11 Apr 2002 03:31:52 -0400 (EDT)
Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA21421
	for <ipoverib@ietf.org>; Thu, 11 Apr 2002 03:31:47 -0400 (EDT)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e31.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3B7Vhsp014888;
	Thu, 11 Apr 2002 03:31:43 -0400
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3B7Vgo35912;
	Thu, 11 Apr 2002 01:31:42 -0600
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g3B7VfX30864;
	Thu, 11 Apr 2002 00:31:42 -0700
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3B7VfZ00279;
	Thu, 11 Apr 2002 00:31:41 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204110731.g3B7VfZ00279@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: roy.k.larsen@intel.com (Larsen, Roy K)
Date: Thu, 11 Apr 2002 00:31:40 -0700 (PDT)
Cc: ipoverib@ietf.org
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4F75@ORSMSX108> from "Larsen, Roy K" at Apr 10, 2002 03:28:09 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit


> /rkl
> Certainly there is complexity in managing IBA multicast groups.  It requires
> statefull transmits.  This will be amplified if the LWG makes it necessary
> to do "join to send" operations.  

        Every media driver(including Ethernet) keeps a list of
        multicast groups if the device can't handle it all by itself.
        There is nothing special about IPoIB driver keeping track of
        the groups.

        No complexity is added by 'join to send' or no 'join to send'
        since the driver still needs to know the MLID to use for a
        particular MGID. It needs to keep track of the MLIDs.
        We write IPoIB to the specification that is available. If it
        says join to send is required then we do that, if it says join
        is not required then we don't. It in no way effects the
        discussion.   

But, if we could faithfully support
> multicast, I guess that's the price we pay.  
 
        We are supporting multicast faithfully if my proposal is
        implemented. That is the idea. If we use a comon broadcast
        group - we no longer support multicast.


But then we add the requirement
> that says transform a multicast request from layer 3 into an all-routers
> multicast if the group doesn't exist and, on a device type basis, have the

        That is an incorrect statement. 

        There are two layers IP and IB. Please note - the IP packets
        are not modified. When the IP group is joined the IPoIB driver
        needs to create the corresponding IB group or join the
        corresponding IB group. This mapping is specified in
        draft-ietf-ipoib-link-and-multicast-00.txt. 

        The same thing happens in Ethernet. The ethernet driver is
        asked to join the corresponding ethernet group. It programmes
        the interface accordingly. In the case of InfiniBand the SM/SA
        is contacted.

        If the IB group is created then the IGMP packet is also sent
        to *IB group* corresponding to the IP all-routers group. 

> 
>         The proposal is taking care to keep the impact minimal while
>         utilising the benefit offered. It effects only those that want
>         to implement IPoIB and not others. It does not effect the
>         protocols themselves (DHCP/IGMP are left untouched).
> 
> /rkl
> When layer 3 transmit requests are not performed consistently, we are
> technically changing the wire protocol and something I think we should not
> disregarded lightly.  

	See above.
	

> 
> So, we need IPoIB aware DHCP clients and now IPoIB aware IGMP
> implementations.  We'll see what happens with VRRP.  It's not that any one

        We have gone ahead and chosen a radically different technology
        to support IP on. So it is unreasonable to expect media aware
        components to work unchanged.

> 
>         By dumbing down the solution to a broadcast link we will loose
>         all the benefit that could have accrued to us. I would rather
>         not have unneccesary broadcasts take away the bandwidth from
>         my storage data. I also would rather that the unnecessary
>         broadcasts do not fill my receive buffers and take up CPU time
>         in filtering the irrelevant packets. I would rather that the
>         IB switches don't get overwhelmed replicating packets out
>         every port for broadcast but rather do it only for the links
>         that really need such replication - implying use separate
>         multicast groups as far as possible. Not all considerations
>         need to be software specific; we also need to look at other
>         issues.
> 
> /rkl
> Technically, those are all valid concerns.  I don't have a good feel for how
> heavily multicast is used in the data center, but of course it will all be
> moot if the data center admin finds it far simpler, from an OS integration
> and compatibility standpoint, to deploy emerging Ethernet technologies
> instead of IBA.  This is my paramount concern.


        It is better to support these technical reasons for they are
        more likely to ward off any assault by other technologies than
        a dumbed down solution.

        I don't think the complexity of InfiniBand is in any way
        affected by reducing the multicast funtion. InfiniBand is
        complex. Reducing some function is likely to have more adverse
        an effect on market/administrator sentiment than a delta
        change in 'complexity'.


Vivek 

> 
> Roy
> rkl/
>         As I said, I did ask about the impact on the routers and was
>         told it was an acceptable requirement. 
> 
> Vivek
> <snip>
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr 11 10:46:42 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA28579
	for <ipoverib-archive@odin.ietf.org>; Thu, 11 Apr 2002 10:46:37 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id KAA20296
	for ipoverib-archive@odin.ietf.org; Thu, 11 Apr 2002 10:46:41 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA19155;
	Thu, 11 Apr 2002 10:30:26 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA19092
	for <ipoverib@optimus.ietf.org>; Thu, 11 Apr 2002 10:30:21 -0400 (EDT)
Received: from palrel13.hp.com (palrel13.hp.com [156.153.255.238])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA28190
	for <ipoverib@ietf.org>; Thu, 11 Apr 2002 10:30:17 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel13.hp.com (Postfix) with ESMTP
	id A8CC2400659; Thu, 11 Apr 2002 07:29:49 -0700 (PDT)
Received: from mkrause.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id HAA10553;
	Thu, 11 Apr 2002 07:29:48 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020411062218.02a0eca8@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Thu, 11 Apr 2002 06:33:06 -0700
To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>, ipoverib@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
In-Reply-To: <200204110145.g3B1jm35953423@jurassic.eng.sun.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 06:45 PM 4/10/2002 -0700, H.K. Jerry Chu wrote:
>Folks,
>
>We have passed the deadline for people to express their opinions.
>For 2 and 3 (see the mail I sent on Sat. below) there seems to be
>a unanimous support for them. For EtherType (1) the opinions remain
>split.
>
>Last week I asked people to bring "new, convincing evidence" when
>arguing for or against EtherType. But unfortunately in the past few
>days all I heard was either old arguments, or people reiterating their
>positions. Since no one has brought forward any new, convincing
>evidence against EtherType, and in both the mailing list and the
>f2f meeting Bill and I feel there have been more people for EtherType
>than against it, plus a number of people requesting EtherType are
>actually working on real implementation, I hereby declare EtherType
>to be the rough consensus of the WG to be included in the UD payload
>of every IPoIB packet.

What is the specification of Ethertype other than it is 16-bits of space 
followed by the next set of headers?  Is an implementation required to 
process this field if it does not want to do so?  Is Ethertype required for 
all IB transport services that may support IP traffic - it seems that it 
must be to remain consistent from an implementation point of view.

>Over the past couple of days a couple of senior IBTA members have
>reiterated their positions against EtherType. But in my opinion the
>decision of including EtherType in the UD payload is at least partly
>as the result of the IBTA failing to address people's need to have a
>field in IB identifying the type of the data being encapsulated in
>the UD payload.

In essence, the consensus is that the IP over IB workgroup is defining the 
official Raw protocol on behalf of the IBTA since in many people's opinion 
the IBTA screwed up.  As such, it would be beneficial to simply write a 
draft with this as the stated goal so that anyone can use it independent of 
IP over IB's charter for such purposes and then have the IP over IB 
specification reference said draft.  This will be much cleaner and at least 
make everyone who may want to implement such a service not have to go 
searching within an IP over IB specification to find this non-IP focused 
functionality.

>Some argued that the QP is supposed to serve that
>purpose, but we are not even doing that for IPv4, ARP/RARP and IPv6.
>I think the argument is rather weak.

Just as weak as hijacking one spec to include functionality that has 
nothing to do with the stated charter of this workgroup.  Consensus has 
been reached so I'll let it die.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr 11 10:49:32 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA28645
	for <ipoverib-archive@odin.ietf.org>; Thu, 11 Apr 2002 10:49:32 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id KAA20469
	for ipoverib-archive@odin.ietf.org; Thu, 11 Apr 2002 10:49:35 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA19197;
	Thu, 11 Apr 2002 10:30:28 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA19100
	for <ipoverib@optimus.ietf.org>; Thu, 11 Apr 2002 10:30:22 -0400 (EDT)
Received: from palrel11.hp.com (palrel11.hp.com [156.153.255.246])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA28192
	for <ipoverib@ietf.org>; Thu, 11 Apr 2002 10:30:18 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel11.hp.com (Postfix) with ESMTP id 9C49B60074B
	for <ipoverib@ietf.org>; Thu, 11 Apr 2002 07:29:50 -0700 (PDT)
Received: from mkrause.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id HAA10566
	for <ipoverib@ietf.org>; Thu, 11 Apr 2002 07:29:49 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020411063924.02a86a88@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Thu, 11 Apr 2002 07:19:38 -0700
To: ipoverib@ietf.org
From: Michael Krause <krause@cup.hp.com>
In-Reply-To: <200204110731.g3B7VfZ00279@eng4.beaverton.ibm.com>
References: <D9223EB959A5D511A98F00508B68C20C06BC4F75@ORSMSX108>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Subject: [Ipoverib] Renamed: Layer 3 multicast forwarding
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org


Let's make this simple:

- IP is one type of non-IB based layer 3 functionality that may operate 
over an IB fabric.  Given the need to support Ethertype, the solutions 
proposed within this workgroup should be applicable to all layer 3 
protocols to avoid having to rework everything anew for each layer 3 
protocol that is desired to be supported.

- If a non-IB layer 3 router is attached to an IB subnet, then it must 
communicate to the subnet's SA that it is present.  Note: a layer 3 router 
is not an IB router; it is an IB TCA that provides layer 3-focused 
connectivity to the non-IB based fabric attached to the other side (this is 
no different than an Ethernet or Fibre Channel attached TCA other than it 
is IB multicast capable as a base requirement).

- The TCA uses the IB multicast management services to determine all of the 
IB multicast groups that are operational within the fabric.  While it is 
outside the current IB specification, the IB management software 
implementation will have to include a field or fields to identify what the 
layer 3 or application service is associated with a given IB multicast 
group (e.g. for a group supporting IP protocols, it will need to also 
supply the list of IP multicast groups that are bound to this IB multicast 
group and some method must be defined to create such bindings - this could 
be done within this workgroup, within the IBTA, or left as a value add for 
IB management suppliers).

- Upon determining what multicast groups support its layer 3 services, the 
TCA will contact the SA to have the following occur:

(1) For each partition supporting the stated layer 3 traffic, a multicast 
tree is expanded to include this router.  This includes programming the 
switch P_Keys, the TCA P_Key table, and all relevant TCA attributes.

(2) The layer 3 TCA always performs a full join of the layer 3 focused IB 
multicast groups (i.e. to send and receive packets).  If there is no one 
within the IB fabric that wishes to receive packets, then the IB switch 
acts as a filter mechanism since the layer 3 TCA just forwards all unicast 
/ multicast traffic from the external fabric to the appropriate IB set of 
endnodes based on its decoding and mapping of the layer 3 headers.  Note: 
The layer 3 TCA operates in promiscuous mode for all of its communication.

(3) For Unicast traffic, the TCA participate in the appropriate layer 3 
protocols to advertise its services as a forwarding agent for layer 3 (this 
requires no additional specification from this workgroup to enable).

- All of the discussion of the layer 3 protocols and how they discover 
things from the outside world is rather irrelevant as this is the TCAs job 
to deal with and not the IBTA or this workgroup's responsibility.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr 11 10:58:12 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA29033
	for <ipoverib-archive@odin.ietf.org>; Thu, 11 Apr 2002 10:58:12 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id KAA21267
	for ipoverib-archive@odin.ietf.org; Thu, 11 Apr 2002 10:58:12 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA20363;
	Thu, 11 Apr 2002 10:47:44 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA20330
	for <ipoverib@optimus.ietf.org>; Thu, 11 Apr 2002 10:47:41 -0400 (EDT)
Received: from vieo.com (vieo.com [216.30.79.131])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA28614
	for <ipoverib@ietf.org>; Thu, 11 Apr 2002 10:47:37 -0400 (EDT)
Received: (from root@localhost)
	by vieo.com (8.11.2/8.11.2) id g3BEl7J96689
	for ipoverib@ietf.org; Thu, 11 Apr 2002 09:47:07 -0500 (CDT)
	(envelope-from rronald@VIEO.com)
Received: from Fishhead (dhcp-10-2-15-164.vieo.com [10.2.15.164])
	by vieo.com (8.11.2/8.11.2) with SMTP id g3BEl5O96607;
	Thu, 11 Apr 2002 09:47:05 -0500 (CDT)
	(envelope-from rronald@VIEO.com)
Message-ID: <00ee01c1e167$b0c1da60$a40f020a@Fishhead>
From: "Roger Ronald" <rronald@VIEO.com>
To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>, <ipoverib@ietf.org>,
        "Michael Krause" <krause@cup.hp.com>
References: <5.1.0.14.2.20020411062218.02a0eca8@esalpha2.cup.hp.com>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
Date: Thu, 11 Apr 2002 09:46:39 -0500
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
X-scanner: scanned by Inflex 0.1.5c+ on vieo.com
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

Sounds roughly like a proposal for a "snap" header.

Mike's idea would not change anything in the byte stream (all the same bits
would be sent and dealt with in the same way) but would 
make Ethertype a link encapsulation header consisting of a single
field.

It's a nice idea since it provides all the bits that every one wants,
but it puts them into the proper layering.

RR
----- Original Message ----- 
From: "Michael Krause" <krause@cup.hp.com>
To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>; <ipoverib@ietf.org>
Sent: Thursday, April 11, 2002 8:33 AM
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4


> At 06:45 PM 4/10/2002 -0700, H.K. Jerry Chu wrote:
> >Folks,
> >
<snip>

> In essence, the consensus is that the IP over IB workgroup is defining the 
> official Raw protocol on behalf of the IBTA since in many people's opinion 
> the IBTA screwed up.  As such, it would be beneficial to simply write a 
> draft with this as the stated goal so that anyone can use it independent of 
> IP over IB's charter for such purposes and then have the IP over IB 
> specification reference said draft.  This will be much cleaner and at least 
> make everyone who may want to implement such a service not have to go 
> searching within an IP over IB specification to find this non-IP focused 
> functionality.
> 
<snip> 
> Mike
> 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr 11 16:46:29 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA14610
	for <ipoverib-archive@odin.ietf.org>; Thu, 11 Apr 2002 16:46:28 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA11873
	for ipoverib-archive@odin.ietf.org; Thu, 11 Apr 2002 16:46:31 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA11628;
	Thu, 11 Apr 2002 16:35:41 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA11595
	for <ipoverib@optimus.ietf.org>; Thu, 11 Apr 2002 16:35:39 -0400 (EDT)
Received: from nwkea-mail-2.sun.com (nwkea-mail-2.sun.com [192.18.42.14])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA14333
	for <ipoverib@ietf.org>; Thu, 11 Apr 2002 16:35:35 -0400 (EDT)
Received: from sunmail1.Sun.COM ([129.145.1.2])
	by nwkea-mail-2.sun.com (8.9.3+Sun/8.9.3) with ESMTP id NAA18758;
	Thu, 11 Apr 2002 13:35:03 -0700 (PDT)
Received: from jurassic.eng.sun.com (jurassic.Eng.Sun.COM [129.146.82.166])
	by sunmail1.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1-Sun.COM.mod.2) with ESMTP id NAA28315;
	Thu, 11 Apr 2002 13:35:48 -0700 (PDT)
Received: from taipei (taipei.Eng.Sun.COM [129.146.85.178])
	by jurassic.eng.sun.com (8.12.2+Sun/8.12.2) with SMTP id g3BKZ235207024;
	Thu, 11 Apr 2002 13:35:02 -0700 (PDT)
Message-Id: <200204112035.g3BKZ235207024@jurassic.eng.sun.com>
Date: Thu, 11 Apr 2002 13:34:54 -0700 (PDT)
From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Reply-To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
To: ipoverib@ietf.org, krause@cup.hp.com
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: GenAJUOSmMHlgivkPxdJZA==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc 
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

>X-Sender: krause@esalpha2.cup.hp.com
>Date: Thu, 11 Apr 2002 06:33:06 -0700

<snip>

>>Last week I asked people to bring "new, convincing evidence" when
>>arguing for or against EtherType. But unfortunately in the past few
>>days all I heard was either old arguments, or people reiterating their
>>positions. Since no one has brought forward any new, convincing
>>evidence against EtherType, and in both the mailing list and the
>>f2f meeting Bill and I feel there have been more people for EtherType
>>than against it, plus a number of people requesting EtherType are
>>actually working on real implementation, I hereby declare EtherType
>>to be the rough consensus of the WG to be included in the UD payload
>>of every IPoIB packet.
>
>What is the specification of Ethertype other than it is 16-bits of space 
>followed by the next set of headers?

To keep 4-byte alignment I suggest to have Ethertype residing in the
lower (network byte-ordering) 2 bytes with the upper 16 bits marked
as reserved. It'd use the values for ARP, IPv4 and IPv6 from the
"EtherType" defined by IANA. Beyond these three protocols I'm not
sure we (IPoIB WG) want or even can say much.

>Is an implementation required to 
>process this field if it does not want to do so?

IMO the sender MUST incorporate EtherType correctly in all the
outgoing ARP/IPv4/IPv6 pkts (including both unicast and multicast).
The receiver SHOULD but is not required to demux based on EtherType.
This is partly implementation detail. If a receiver can correctly
demux without examining EtherType, why do we care?

>Is Ethertype required for 
>all IB transport services that may support IP traffic - it seems that it 
>must be to remain consistent from an implementation point of view.

Agreed. But since the consensus call was made specifically for UD,
for non-UD people can speak up if they have a different opinion.

>
>>Over the past couple of days a couple of senior IBTA members have
>>reiterated their positions against EtherType. But in my opinion the
>>decision of including EtherType in the UD payload is at least partly
>>as the result of the IBTA failing to address people's need to have a
>>field in IB identifying the type of the data being encapsulated in
>>the UD payload.
>
>In essence, the consensus is that the IP over IB workgroup is defining the 
>official Raw protocol on behalf of the IBTA since in many people's opinion 
>the IBTA screwed up.  As such, it would be beneficial to simply write a 
>draft with this as the stated goal so that anyone can use it independent of 
>IP over IB's charter for such purposes and then have the IP over IB 
>specification reference said draft.

Agreed. But since the draft will address a generic IB issue rather than
IPoIB issue, IBTA seems to be the right place for such a document, not
IETF.

>  This will be much cleaner and at least 
>make everyone who may want to implement such a service not have to go 
>searching within an IP over IB specification to find this non-IP focused 
>functionality.

Agreed.

>
>>Some argued that the QP is supposed to serve that
>>purpose, but we are not even doing that for IPv4, ARP/RARP and IPv6.
>>I think the argument is rather weak.
>
>Just as weak as hijacking one spec to include functionality that has 
>nothing to do with the stated charter of this workgroup.  Consensus has 
>been reached so I'll let it die.

Had IBTA fixed raw datagram to be used by IPoIB as the default IB
transport, or had we adopted SIDR to map ARP/IPv4/IPv6 to different
QPs, we wouldn't have gone down this path. Failing both, Ethertype
seemed inevitable.

Jerry

>
>Mike
>


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr 11 17:04:21 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA14930
	for <ipoverib-archive@odin.ietf.org>; Thu, 11 Apr 2002 17:04:21 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id RAA13411
	for ipoverib-archive@odin.ietf.org; Thu, 11 Apr 2002 17:04:24 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA12309;
	Thu, 11 Apr 2002 16:57:10 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA12267
	for <ipoverib@optimus.ietf.org>; Thu, 11 Apr 2002 16:57:07 -0400 (EDT)
Received: from nwkea-mail-1.sun.com (nwkea-mail-1.sun.com [192.18.42.13])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA14782
	for <ipoverib@ietf.org>; Thu, 11 Apr 2002 16:57:02 -0400 (EDT)
Received: from phys-ha10nwka.ebay.sun.com ([129.150.189.210])
	by nwkea-mail-1.sun.com (8.9.3+Sun/8.9.3) with ESMTP id NAA18943;
	Thu, 11 Apr 2002 13:56:24 -0700 (PDT)
Received: from endeavour (endeavour [129.150.143.50])
	by phys-ha10nwka.ebay.sun.com (8.8.8+Sun/8.8.8/ENSMAIL,v2.2) with SMTP id NAA28454;
	Thu, 11 Apr 2002 13:56:24 -0700 (PDT)
Message-Id: <200204112056.NAA28454@phys-ha10nwka.ebay.sun.com>
Date: Thu, 11 Apr 2002 13:56:24 -0700 (PDT)
From: Ajoy C Siddabathuni <Ajoy.Siddabathuni@Sun.COM>
Reply-To: Ajoy C Siddabathuni <Ajoy.Siddabathuni@Sun.COM>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
To: ipoverib@ietf.org, krause@cup.hp.com, Jerry.Chu@eng.sun.com
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: OictEsCPSWUJjuYIuyw8lg==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.3.4p_5 SunOS 5.7 sun4u sparc 
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Jerry,

>To keep 4-byte alignment I suggest to have Ethertype residing in the
>lower (network byte-ordering) 2 bytes with the upper 16 bits marked
>as reserved. It'd use the values for ARP, IPv4 and IPv6 from the
>"EtherType" defined by IANA. Beyond these three protocols I'm not
>sure we (IPoIB WG) want or even can say much.

Would this by definition allow implementations to use LLC/SNAP headers if
required?  In the cIPoverATM (see rfc2225) world, the LLC/SNAP
is the default packet format for IP datagrams.

I dont necessarily have an opinion one way or the other, but having the
alignment definetly helps.

~Ajoy


>Date: Thu, 11 Apr 2002 13:34:54 -0700 (PDT)
>From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
>Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
>To: ipoverib@ietf.org, krause@cup.hp.com
>MIME-Version: 1.0
>Content-MD5: GenAJUOSmMHlgivkPxdJZA==
>X-Mailman-Version: 1.0
>List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
>X-BeenThere: ipoverib@ietf.org
>
>>X-Sender: krause@esalpha2.cup.hp.com
>>Date: Thu, 11 Apr 2002 06:33:06 -0700
>
><snip>
>
>>>Last week I asked people to bring "new, convincing evidence" when
>>>arguing for or against EtherType. But unfortunately in the past few
>>>days all I heard was either old arguments, or people reiterating their
>>>positions. Since no one has brought forward any new, convincing
>>>evidence against EtherType, and in both the mailing list and the
>>>f2f meeting Bill and I feel there have been more people for EtherType
>>>than against it, plus a number of people requesting EtherType are
>>>actually working on real implementation, I hereby declare EtherType
>>>to be the rough consensus of the WG to be included in the UD payload
>>>of every IPoIB packet.
>>
>>What is the specification of Ethertype other than it is 16-bits of space 
>>followed by the next set of headers?
>
>To keep 4-byte alignment I suggest to have Ethertype residing in the
>lower (network byte-ordering) 2 bytes with the upper 16 bits marked
>as reserved. It'd use the values for ARP, IPv4 and IPv6 from the
>"EtherType" defined by IANA. Beyond these three protocols I'm not
>sure we (IPoIB WG) want or even can say much.
>
>>Is an implementation required to 
>>process this field if it does not want to do so?
>
>IMO the sender MUST incorporate EtherType correctly in all the
>outgoing ARP/IPv4/IPv6 pkts (including both unicast and multicast).
>The receiver SHOULD but is not required to demux based on EtherType.
>This is partly implementation detail. If a receiver can correctly
>demux without examining EtherType, why do we care?
>
>>Is Ethertype required for 
>>all IB transport services that may support IP traffic - it seems that it 
>>must be to remain consistent from an implementation point of view.
>
>Agreed. But since the consensus call was made specifically for UD,
>for non-UD people can speak up if they have a different opinion.
>
>>
>>>Over the past couple of days a couple of senior IBTA members have
>>>reiterated their positions against EtherType. But in my opinion the
>>>decision of including EtherType in the UD payload is at least partly
>>>as the result of the IBTA failing to address people's need to have a
>>>field in IB identifying the type of the data being encapsulated in
>>>the UD payload.
>>
>>In essence, the consensus is that the IP over IB workgroup is defining the 
>>official Raw protocol on behalf of the IBTA since in many people's opinion 
>>the IBTA screwed up.  As such, it would be beneficial to simply write a 
>>draft with this as the stated goal so that anyone can use it independent of 
>>IP over IB's charter for such purposes and then have the IP over IB 
>>specification reference said draft.
>
>Agreed. But since the draft will address a generic IB issue rather than
>IPoIB issue, IBTA seems to be the right place for such a document, not
>IETF.
>
>>  This will be much cleaner and at least 
>>make everyone who may want to implement such a service not have to go 
>>searching within an IP over IB specification to find this non-IP focused 
>>functionality.
>
>Agreed.
>
>>
>>>Some argued that the QP is supposed to serve that
>>>purpose, but we are not even doing that for IPv4, ARP/RARP and IPv6.
>>>I think the argument is rather weak.
>>
>>Just as weak as hijacking one spec to include functionality that has 
>>nothing to do with the stated charter of this workgroup.  Consensus has 
>>been reached so I'll let it die.
>
>Had IBTA fixed raw datagram to be used by IPoIB as the default IB
>transport, or had we adopted SIDR to map ARP/IPv4/IPv6 to different
>QPs, we wouldn't have gone down this path. Failing both, Ethertype
>seemed inevitable.
>
>Jerry
>
>>
>>Mike
>>
>
>
>_______________________________________________
>IPoverIB mailing list
>IPoverIB@ietf.org
>https://www1.ietf.org/mailman/listinfo/ipoverib


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr 11 18:44:07 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA18027
	for <ipoverib-archive@odin.ietf.org>; Thu, 11 Apr 2002 18:44:07 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id SAA19602
	for ipoverib-archive@odin.ietf.org; Thu, 11 Apr 2002 18:44:09 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA17670;
	Thu, 11 Apr 2002 18:36:24 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA17642
	for <ipoverib@optimus.ietf.org>; Thu, 11 Apr 2002 18:36:23 -0400 (EDT)
Received: from palrel13.hp.com (palrel13.hp.com [156.153.255.238])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA17301
	for <ipoverib@ietf.org>; Thu, 11 Apr 2002 18:36:08 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel13.hp.com (Postfix) with ESMTP id C1F16400245
	for <ipoverib@ietf.org>; Thu, 11 Apr 2002 15:35:37 -0700 (PDT)
Received: from mkrause.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id PAA10734
	for <ipoverib@ietf.org>; Thu, 11 Apr 2002 15:35:37 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020411152824.02ab6b20@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Thu, 11 Apr 2002 15:35:02 -0700
To: ipoverib@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
In-Reply-To: <200204112035.g3BKZ235207024@jurassic.eng.sun.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 01:34 PM 4/11/2002 -0700, H.K. Jerry Chu wrote:

> >What is the specification of Ethertype other than it is 16-bits of space
> >followed by the next set of headers?
>
>To keep 4-byte alignment I suggest to have Ethertype residing in the
>lower (network byte-ordering) 2 bytes with the upper 16 bits marked
>as reserved. It'd use the values for ARP, IPv4 and IPv6 from the
>"EtherType" defined by IANA. Beyond these three protocols I'm not
>sure we (IPoIB WG) want or even can say much.

I see no reason to further waste 2 more bytes on this thus increasing small 
packet overheads even further.  The Ethertype is going to be delivered 
along with some portion of the LRH / GRH / DETH / etc. which depending upon 
the local implementation may result in a varying amount of data.  This 
field really has nothing to do with the wire protocol's desire to be 4-byte 
aligned.

> >Is an implementation required to
> >process this field if it does not want to do so?
>
>IMO the sender MUST incorporate EtherType correctly in all the
>outgoing ARP/IPv4/IPv6 pkts (including both unicast and multicast).
>The receiver SHOULD but is not required to demux based on EtherType.
>This is partly implementation detail. If a receiver can correctly
>demux without examining EtherType, why do we care?

Just needs to be stated what people must incorporate in terms of the wire 
protocol and what they can ignore if desired.


> >Is Ethertype required for
> >all IB transport services that may support IP traffic - it seems that it
> >must be to remain consistent from an implementation point of view.
>
>Agreed. But since the consensus call was made specifically for UD,
>for non-UD people can speak up if they have a different opinion.

Make it one way or another for all transport services so at least people 
don't have to do it multiple ways.


> >
> >>Over the past couple of days a couple of senior IBTA members have
> >>reiterated their positions against EtherType. But in my opinion the
> >>decision of including EtherType in the UD payload is at least partly
> >>as the result of the IBTA failing to address people's need to have a
> >>field in IB identifying the type of the data being encapsulated in
> >>the UD payload.
> >
> >In essence, the consensus is that the IP over IB workgroup is defining the
> >official Raw protocol on behalf of the IBTA since in many people's opinion
> >the IBTA screwed up.  As such, it would be beneficial to simply write a
> >draft with this as the stated goal so that anyone can use it independent of
> >IP over IB's charter for such purposes and then have the IP over IB
> >specification reference said draft.
>
>Agreed. But since the draft will address a generic IB issue rather than
>IPoIB issue, IBTA seems to be the right place for such a document, not
>IETF.

This isn't an IBTA issue at all - this is about the IETF defining how a 
payload will be interpreted with some portion being control and some 
portion being data.  There is no effort within the IBTA to codify this 
packet layout as part of the IB specification therefore, it you want to use 
this for non-IP protocols in addition to IP, then you should codify this in 
an IETF standard.

>Had IBTA fixed raw datagram to be used by IPoIB as the default IB
>transport, or had we adopted SIDR to map ARP/IPv4/IPv6 to different
>QPs, we wouldn't have gone down this path. Failing both, Ethertype
>seemed inevitable.

There are some who did not want to fix it so it will not be fixed and I'm 
of the opinion it should be removed for the IB specification in its 
entirety.  But I'm just the co-chair and only have 1 vote just like any 
other company within the IBTA in this regard.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Fri Apr 12 00:51:38 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA20318
	for <ipoverib-archive@odin.ietf.org>; Fri, 12 Apr 2002 00:51:37 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id AAA10086
	for ipoverib-archive@odin.ietf.org; Fri, 12 Apr 2002 00:51:38 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id AAA09692;
	Fri, 12 Apr 2002 00:37:07 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id AAA09660
	for <ipoverib@optimus.ietf.org>; Fri, 12 Apr 2002 00:37:05 -0400 (EDT)
Received: from mail6.microsoft.com (mail6.microsoft.com [131.107.3.126])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA18941
	for <ipoverib@ietf.org>; Fri, 12 Apr 2002 00:37:03 -0400 (EDT)
Received: from inet-vrs-06.redmond.corp.microsoft.com ([157.54.6.201]) by mail6.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Thu, 11 Apr 2002 21:36:33 -0700
Received: from 157.54.8.155 by inet-vrs-06.redmond.corp.microsoft.com (InterScan E-Mail VirusWall NT); Thu, 11 Apr 2002 21:36:33 -0700
Received: from red-imc-04.redmond.corp.microsoft.com ([157.54.2.168]) by inet-hub-04.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Thu, 11 Apr 2002 21:36:33 -0700
Received: from win-imc-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.39]) by red-imc-04.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.2966);
	 Thu, 11 Apr 2002 21:36:31 -0700
Received: from win-msg-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.52]) by win-imc-01.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(6.0.3604.0);
	 Thu, 11 Apr 2002 21:32:47 -0700
X-MimeOLE: Produced By Microsoft Exchange V6.0.6177.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Thu, 11 Apr 2002 21:32:46 -0700
Message-ID: <2E33960095B58E40A4D3345AB9F65EC102450E20@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Thread-Topic: [Ipoverib] IPoIB multicast forwarding
Thread-Index: AcHfoIDv5mARjGDHRde4/uv8kJ4eXQCOZimQ
From: "Aditya Dube" <adube@windows.microsoft.com>
To: <ipoverib@ietf.org>, <kashyapv@us.ibm.com>
Cc: <fenner@research.att.com>
X-OriginalArrivalTime: 12 Apr 2002 04:32:47.0301 (UTC) FILETIME=[18C92F50:01C1E1DB]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id AAA09661
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 8bit

Hello Vivek

I have a question. Could you clarify what will happen in the following
scenario in the third scheme that was proposed below:

Pure senders - at IB level check for the relevant IB group.
           If it exists then send to the IB group else send to the
           'all-routers' IB group.

What happens if a Receiver creates the 'relevant IB group' after the
sender has made the check? 

Aditya Dube
-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
Sent: Tuesday, April 09, 2002 1:31 AM
To: ipoverib@ietf.org
Cc: fenner@research.att.com
Subject: [Ipoverib] IPoIB multicast forwarding


Folks,

        We have defined a mapping from the IP addresses to the IB
        multicast groups. Use of different groups causes minimal
        replication at the switches and less load at the receivers.

        But there is a quirk in the way the groups are setup in
        InfiniBand - there is no way for any of the nodes to know if
        another has created an InfiniBand group. This is easy to fix
        in most cases - one checks with the SM. But it doesn't work
        out well for IPoIB packet forwarding.

        The router needs to forward the multicast packets beyond the
        local IP subnet if it knows there are recipients for it or in
        the case of some routing protocols it must do that unless it
        gets an update stating not to forward. Similarly, it needs to
        forward external packets to the local receivers if there are
        any receiver present. 

        Thus if there are IB multicast groups associated with an IP
        multicast group the router must know about them or the router
        must receive all multicast packets in the subnet via some
        other construct.

        Below are the three alternatives that I mentioned at the
        53rd IETF meet. 
	
        i) Define only one IB multicast group i.e. IB broadcast MGID.
        All IP multicast packets use this group.

        ii) The multicast IP packet is sent to both the associated IB
        multicast group and the IB mutlicast group corresponding to
        the all-IP routers IP group.

        iii) Sender always sends to corresonding IB group if it exists
        else it sends to the broadcast group.

             IP receivers create IB groups and send the IGMP/MLD
             groups on the broadcast MGID.

             Routers join relevant groups based on the IGMP report
             and routing protocol updates.


        I also put this problem to Bill Fenner (AD of routing area)
        and asked his advice on the alternatives. Of the three options
        he favored option (iii) above.

        Option (i) doesn't take advantage fo IB multicast at all and
        so should be looked at as a last resort solution only.

        Option (ii) requires the sender to send two copies of the same
        packet and 'deliberately' increases the chance of duplicates
        spreading across the network.


Suggested solution  in detail:


        1. Pure senders - at IB level check for the relevant IB group.
           If it exists then send to the IB group else send to the
           'all-routers' IB group. If there is no IB group
           corresponding to a particular IP group then it implies that
           there are no IP receivers on the subnet for that IP
           multicast group. 

           Note: a) Pure senders don't create IB groups. This is
           because there is no existing mechanism for the senders to
           let the routers know of the new groups.

        2. IP receivers: As per IGMP/MLD the first packet sent by a
           host on joining a multicast group for receiving
           (IP_ADD_MEMBERSHIP) is the multicast report.

	   The hosts, as a result of IP_ADD_MEMBERSHIP, do the
following:

	   	check if the corresponding IB group exists. 

		if (IB group exists)
			IB_join the group and send the IGMP/MLD report

		else
			IB_Create the group
			IB_join the group
                        Send IGMP/MLD report to IB group corresponding
                        to all-routers (all-hosts or broadcast) IP
                        group (Note: only done when creating the IB
                        group) 

	3. IP routers IB_join MGIDs on:

		a) Finding a new IP group on the subnet in the IGMP/MLD
report
                b) Determining a need to forward IP multicast packets
                   through a routing protocol



Comments ?


Vivek
			
-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Fri Apr 12 01:00:19 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA21360
	for <ipoverib-archive@odin.ietf.org>; Fri, 12 Apr 2002 01:00:19 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id BAA10473
	for ipoverib-archive@odin.ietf.org; Fri, 12 Apr 2002 01:00:20 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id AAA10242;
	Fri, 12 Apr 2002 00:53:37 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id AAA10209
	for <ipoverib@optimus.ietf.org>; Fri, 12 Apr 2002 00:53:35 -0400 (EDT)
Received: from palrel11.hp.com (palrel11.hp.com [156.153.255.246])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA20563
	for <ipoverib@ietf.org>; Fri, 12 Apr 2002 00:53:33 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel11.hp.com (Postfix) with ESMTP
	id 6A9B7600749; Thu, 11 Apr 2002 21:52:50 -0700 (PDT)
Received: from mkrause.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id VAA12502;
	Thu, 11 Apr 2002 21:52:49 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020411215148.02ad9d38@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Thu, 11 Apr 2002 21:52:37 -0700
To: "Aditya Dube" <adube@windows.microsoft.com>, <ipoverib@ietf.org>,
        <kashyapv@us.ibm.com>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Cc: <fenner@research.att.com>
In-Reply-To: <2E33960095B58E40A4D3345AB9F65EC102450E20@win-msg-01.wingro
 up.windeploy.ntdev.microsoft.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 09:32 PM 4/11/2002 -0700, Aditya Dube wrote:
>Hello Vivek
>
>I have a question. Could you clarify what will happen in the following
>scenario in the third scheme that was proposed below:
>
>Pure senders - at IB level check for the relevant IB group.
>            If it exists then send to the IB group else send to the
>            'all-routers' IB group.
>
>What happens if a Receiver creates the 'relevant IB group' after the
>sender has made the check?

There should not be such an option in the first place: the relevant group 
should always exist or else nothing is sent.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Fri Apr 12 03:33:08 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA15673
	for <ipoverib-archive@odin.ietf.org>; Fri, 12 Apr 2002 03:33:08 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id DAA28136
	for ipoverib-archive@odin.ietf.org; Fri, 12 Apr 2002 03:33:11 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id DAA27152;
	Fri, 12 Apr 2002 03:18:10 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id DAA27120
	for <ipoverib@optimus.ietf.org>; Fri, 12 Apr 2002 03:18:07 -0400 (EDT)
Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA13831
	for <ipoverib@ietf.org>; Fri, 12 Apr 2002 03:18:02 -0400 (EDT)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e31.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3C7FAsp009506;
	Fri, 12 Apr 2002 03:15:11 -0400
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3C7F9W156696;
	Fri, 12 Apr 2002 01:15:09 -0600
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g3C7F8X31586;
	Fri, 12 Apr 2002 00:15:08 -0700
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3C7F8j26893;
	Fri, 12 Apr 2002 00:15:08 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204120715.g3C7F8j26893@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: krause@cup.hp.com (Michael Krause)
Date: Fri, 12 Apr 2002 00:15:07 -0700 (PDT)
Cc: adube@windows.microsoft.com (Aditya Dube), ipoverib@ietf.org,
        kashyapv@us.ibm.com, fenner@research.att.com
In-Reply-To: <5.1.0.14.2.20020411215148.02ad9d38@esalpha2.cup.hp.com> from "Michael Krause" at Apr 11, 2002 08:52:37 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> At 09:32 PM 4/11/2002 -0700, Aditya Dube wrote:
> >Hello Vivek
> >
> >I have a question. Could you clarify what will happen in the following
> >scenario in the third scheme that was proposed below:
> >
> >Pure senders - at IB level check for the relevant IB group.
> >            If it exists then send to the IB group else send to the
> >            'all-routers' IB group.
> >
> >What happens if a Receiver creates the 'relevant IB group' after the
> >sender has made the check?
> 
> There should not be such an option in the first place: the relevant group 
> should always exist or else nothing is sent.

        Unfortunately we do have to have such an option. There maybe
        no IP group receiver and hence IB group on the subnet. But the
        router may still need to receive the packet to forward out.

        The router could create the IB group if it knew it has to
        forward for the relevant IP group. However, the router does
        not know this for the first multicast datagram it sends for
        this group(DVMRP). So, we need to ensure that it receives the
        packet and therefore the need to have the option to the
        all-routers IB group.

        A receiver creating the group after the sender checks is no
        different than the receiver creating the group after a
        hypothetical atomic check and send of a multicast packet by a
        sender. I thought about suggesting that sender sends to the
        broadcast-GID but then decided against it - why punish
        everyone for a once in a while race condition that can go
        either way even if the sender could do atomic 'check (i.e.
        join) and send'.


        For the next packet the sender will find the group and from
        then onwards directly send on the group.

Vivek



> 
> Mike
> 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 12 14:15:25 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA28817
	for <ipoverib-archive@odin.ietf.org>; Fri, 12 Apr 2002 14:15:24 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA02597
	for ipoverib-archive@odin.ietf.org; Fri, 12 Apr 2002 14:15:25 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA02297;
	Fri, 12 Apr 2002 14:08:10 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA02263
	for <ipoverib@ns.ietf.org>; Fri, 12 Apr 2002 14:08:07 -0400 (EDT)
Received: from mail2.microsoft.com (mail2.microsoft.com [131.107.3.124])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA27915
	for <ipoverib@ietf.org>; Fri, 12 Apr 2002 14:08:04 -0400 (EDT)
Received: from INET-VRS-02.redmond.corp.microsoft.com ([157.54.8.110]) by mail2.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Fri, 12 Apr 2002 11:07:31 -0700
Received: from 157.54.8.109 by INET-VRS-02.redmond.corp.microsoft.com (InterScan E-Mail VirusWall NT); Fri, 12 Apr 2002 11:07:30 -0700
Received: from red-imc-02.redmond.corp.microsoft.com ([157.54.9.107]) by inet-hub-02.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Fri, 12 Apr 2002 11:07:30 -0700
Received: from win-imc-02.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.84]) by red-imc-02.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Fri, 12 Apr 2002 11:07:30 -0700
Received: from win-msg-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.52]) by win-imc-02.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(6.0.3590.0);
	 Fri, 12 Apr 2002 11:03:58 -0700
X-MimeOLE: Produced By Microsoft Exchange V6.0.6177.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Fri, 12 Apr 2002 11:03:59 -0700
Message-ID: <2E33960095B58E40A4D3345AB9F65EC105D2C94F@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Thread-Topic: [Ipoverib] IPoIB multicast forwarding
Thread-Index: AcHh8bfp/tIZFosGQB2Kejc4Lo+1LAAP9k4Q
From: "Aditya Dube" <adube@windows.microsoft.com>
To: <ipoverib@ietf.org>
X-OriginalArrivalTime: 12 Apr 2002 18:03:58.0701 (UTC) FILETIME=[6B3371D0:01C1E24C]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id OAA02264
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 8bit


Vivek

As the sender will only send packets to the all-router MGID after doing
the check, how will a receiver who creates the MGroup on the same subnet
as the sender ever receive the multicast packets that it should get?


On Ethernet the sender does no check. The sender will simply send
packets to corresponding Ethernet multicast address. Whenever, the
receiver wants it can start listening for the multicast address it is
interested in.

As you state, on Ethernet the router will be in promiscuous multicast
receive mode so the router will keep get all the multicast packets that
it needs to route. 


-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
Sent: Friday, April 12, 2002 12:15 AM
To: Michael Krause
Cc: Aditya Dube; ipoverib@ietf.org; kashyapv@us.ibm.com;
fenner@research.att.com
Subject: Re: [Ipoverib] IPoIB multicast forwarding

> 
> At 09:32 PM 4/11/2002 -0700, Aditya Dube wrote:
> >Hello Vivek
> >
> >I have a question. Could you clarify what will happen in the
following
> >scenario in the third scheme that was proposed below:
> >
> >Pure senders - at IB level check for the relevant IB group.
> >            If it exists then send to the IB group else send to the
> >            'all-routers' IB group.
> >
> >What happens if a Receiver creates the 'relevant IB group' after the
> >sender has made the check?
> 
> There should not be such an option in the first place: the relevant
group 
> should always exist or else nothing is sent.

        Unfortunately we do have to have such an option. There maybe
        no IP group receiver and hence IB group on the subnet. But the
        router may still need to receive the packet to forward out.

        The router could create the IB group if it knew it has to
        forward for the relevant IP group. However, the router does
        not know this for the first multicast datagram it sends for
        this group(DVMRP). So, we need to ensure that it receives the
        packet and therefore the need to have the option to the
        all-routers IB group.

        A receiver creating the group after the sender checks is no
        different than the receiver creating the group after a
        hypothetical atomic check and send of a multicast packet by a
        sender. I thought about suggesting that sender sends to the
        broadcast-GID but then decided against it - why punish
        everyone for a once in a while race condition that can go
        either way even if the sender could do atomic 'check (i.e.
        join) and send'.


        For the next packet the sender will find the group and from
        then onwards directly send on the group.

Vivek



> 
> Mike
> 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 12 16:23:37 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA02984
	for <ipoverib-archive@odin.ietf.org>; Fri, 12 Apr 2002 16:23:37 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA10780
	for ipoverib-archive@odin.ietf.org; Fri, 12 Apr 2002 16:23:39 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA09897;
	Fri, 12 Apr 2002 16:07:28 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA09855
	for <ipoverib@ns.ietf.org>; Fri, 12 Apr 2002 16:07:25 -0400 (EDT)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA02588
	for <ipoverib@ietf.org>; Fri, 12 Apr 2002 16:07:21 -0400 (EDT)
Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.38 2002/04/09 21:20:23 root Exp $) with ESMTP id g3CK6lO27585
	for <ipoverib@ietf.org>; Fri, 12 Apr 2002 20:06:47 GMT
Received: from fmsmsxvs042.fm.intel.com (fmsmsxv042-1.fm.intel.com [132.233.48.110])
	by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.15 2002/04/01 17:51:48 root Exp $) with SMTP id g3CK90309151
	for <ipoverib@ietf.org>; Fri, 12 Apr 2002 20:09:00 GMT
Received: from fmsmsx29.FM.INTEL.COM ([132.233.42.29])
 by fmsmsxvs042.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002041213111906992
 for <ipoverib@ietf.org>; Fri, 12 Apr 2002 13:11:19 -0700
Received: by fmsmsx29.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2LKBZAT3>; Fri, 12 Apr 2002 13:07:22 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F83@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Aditya Dube'" <adube@windows.microsoft.com>, ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Fri, 12 Apr 2002 13:07:21 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Aditya,

The way I understand it, the sender will need to check each time a multicast
send is attempted.  When a receiver is available on the subnet, the target
multicast group will exist.  When the multicast group has been deleted (i.e.
the receiving application has exited), the check will fail and the sender
will revert back to sending to the all-routers MGID.  Unfortunately, the
sender can't assume that once a receiver was available it will always be
available, nor can it assume that the state of multicast routers will stay
consistent.  Thus the need for persistent checks with the SM before each
multicast send.

The question I have is what happens in the case where there are two
multicast receives in the enterprise.  For example in a fault tolerant or
load balancing application.  Suppose there are three subnets (A, B, and C)
bridged by two multicast router( AB, BC).  There are two multicast receivers
(one on A and one on C) listing for the same multicast packets.  Finally,
assume there are no senders on subnet B.  If either router goes down during
the operation of the multicast application, how will it know to join the
appropriate multicast group when it comes back up?  It will not receive any
redirected all-router multicasts since senders will find a multicast receive
in each subnet?  If it can know in this situation, why the need to have a
sender redirect to the all-routers MGID at any time?

Roy

-----Original Message-----
From: Aditya Dube [mailto:adube@windows.microsoft.com]
Sent: Friday, April 12, 2002 11:04 AM
To: ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast forwarding



Vivek

As the sender will only send packets to the all-router MGID after doing
the check, how will a receiver who creates the MGroup on the same subnet
as the sender ever receive the multicast packets that it should get?


On Ethernet the sender does no check. The sender will simply send
packets to corresponding Ethernet multicast address. Whenever, the
receiver wants it can start listening for the multicast address it is
interested in.

As you state, on Ethernet the router will be in promiscuous multicast
receive mode so the router will keep get all the multicast packets that
it needs to route. 


-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
Sent: Friday, April 12, 2002 12:15 AM
To: Michael Krause
Cc: Aditya Dube; ipoverib@ietf.org; kashyapv@us.ibm.com;
fenner@research.att.com
Subject: Re: [Ipoverib] IPoIB multicast forwarding

> 
> At 09:32 PM 4/11/2002 -0700, Aditya Dube wrote:
> >Hello Vivek
> >
> >I have a question. Could you clarify what will happen in the
following
> >scenario in the third scheme that was proposed below:
> >
> >Pure senders - at IB level check for the relevant IB group.
> >            If it exists then send to the IB group else send to the
> >            'all-routers' IB group.
> >
> >What happens if a Receiver creates the 'relevant IB group' after the
> >sender has made the check?
> 
> There should not be such an option in the first place: the relevant
group 
> should always exist or else nothing is sent.

        Unfortunately we do have to have such an option. There maybe
        no IP group receiver and hence IB group on the subnet. But the
        router may still need to receive the packet to forward out.

        The router could create the IB group if it knew it has to
        forward for the relevant IP group. However, the router does
        not know this for the first multicast datagram it sends for
        this group(DVMRP). So, we need to ensure that it receives the
        packet and therefore the need to have the option to the
        all-routers IB group.

        A receiver creating the group after the sender checks is no
        different than the receiver creating the group after a
        hypothetical atomic check and send of a multicast packet by a
        sender. I thought about suggesting that sender sends to the
        broadcast-GID but then decided against it - why punish
        everyone for a once in a while race condition that can go
        either way even if the sender could do atomic 'check (i.e.
        join) and send'.


        For the next packet the sender will find the group and from
        then onwards directly send on the group.

Vivek



> 
> Mike
> 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 12 16:50:02 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA03956
	for <ipoverib-archive@odin.ietf.org>; Fri, 12 Apr 2002 16:50:02 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA12481
	for ipoverib-archive@odin.ietf.org; Fri, 12 Apr 2002 16:49:28 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA12324;
	Fri, 12 Apr 2002 16:44:20 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA12250
	for <ipoverib@ns.ietf.org>; Fri, 12 Apr 2002 16:44:15 -0400 (EDT)
Received: from palrel12.hp.com (palrel12.hp.com [156.153.255.237])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA03801
	for <ipoverib@ietf.org>; Fri, 12 Apr 2002 16:44:10 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel12.hp.com (Postfix) with ESMTP
	id 1F45EE00DEA; Fri, 12 Apr 2002 13:43:43 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id NAA18013;
	Fri, 12 Apr 2002 13:43:42 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020412134113.02a5ae28@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Fri, 12 Apr 2002 13:43:34 -0700
To: "Larsen, Roy K" <roy.k.larsen@intel.com>,
        "'Aditya Dube'" <adube@windows.microsoft.com>, ipoverib@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast forwarding
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4F83@ORSMSX108>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 01:07 PM 4/12/2002 -0700, Larsen, Roy K wrote:
>Aditya,
>
>The way I understand it, the sender will need to check each time a multicast
>send is attempted.

This would be an unacceptable performance overhead.

>When a receiver is available on the subnet, the target
>multicast group will exist.  When the multicast group has been deleted (i.e.
>the receiving application has exited), the check will fail and the sender
>will revert back to sending to the all-routers MGID.  Unfortunately, the
>sender can't assume that once a receiver was available it will always be
>available, nor can it assume that the state of multicast routers will stay
>consistent.  Thus the need for persistent checks with the SM before each
>multicast send.

Very impractical to implement.

>The question I have is what happens in the case where there are two
>multicast receives in the enterprise.  For example in a fault tolerant or
>load balancing application.  Suppose there are three subnets (A, B, and C)
>bridged by two multicast router( AB, BC).  There are two multicast receivers
>(one on A and one on C) listing for the same multicast packets.  Finally,
>assume there are no senders on subnet B.  If either router goes down during
>the operation of the multicast application, how will it know to join the
>appropriate multicast group when it comes back up?  It will not receive any
>redirected all-router multicasts since senders will find a multicast receive
>in each subnet?  If it can know in this situation, why the need to have a
>sender redirect to the all-routers MGID at any time?

The IP TCAs would be responsible for sorting this out.  From an IB 
perspective, these TCAs would all participate in the IB multicast 
operations thus they would each receive a packet.  The TCA would be then 
determine whether they inject the packet or not based on the appropriate IP 
requirements.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 12 16:50:04 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA03970
	for <ipoverib-archive@odin.ietf.org>; Fri, 12 Apr 2002 16:50:04 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA12467
	for ipoverib-archive@odin.ietf.org; Fri, 12 Apr 2002 16:49:19 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA12289;
	Fri, 12 Apr 2002 16:44:17 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA12234
	for <ipoverib@ns.ietf.org>; Fri, 12 Apr 2002 16:44:14 -0400 (EDT)
Received: from palrel12.hp.com (palrel12.hp.com [156.153.255.237])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA03800
	for <ipoverib@ietf.org>; Fri, 12 Apr 2002 16:44:10 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel12.hp.com (Postfix) with ESMTP
	id AF3FAE00D9C; Fri, 12 Apr 2002 13:43:42 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id NAA18007;
	Fri, 12 Apr 2002 13:43:42 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020412133049.02991258@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Fri, 12 Apr 2002 13:40:54 -0700
To: Vivek Kashyap <kashyapv@us.ibm.com>
From: Michael Krause <krause@cup.hp.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
Cc: adube@windows.microsoft.com (Aditya Dube), ipoverib@ietf.org,
        fenner@research.att.com
In-Reply-To: <200204120715.g3C7F8j26893@eng4.beaverton.ibm.com>
References: <5.1.0.14.2.20020411215148.02ad9d38@esalpha2.cup.hp.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 12:15 AM 4/12/2002 -0700, Vivek Kashyap wrote:
> >
> > There should not be such an option in the first place: the relevant group
> > should always exist or else nothing is sent.
>
>         Unfortunately we do have to have such an option. There maybe
>         no IP group receiver and hence IB group on the subnet. But the
>         router may still need to receive the packet to forward out.

If an IP multicast group is to be used on an IB fabric, then there must 
exist an IB multicast group.

>         The router could create the IB group if it knew it has to
>         forward for the relevant IP group. However, the router does
>         not know this for the first multicast datagram it sends for
>         this group(DVMRP). So, we need to ensure that it receives the
>         packet and therefore the need to have the option to the
>         all-routers IB group.

A management agent above the SA can track whether there are routers that 
also support IP to non-IB fabrics, i.e. there is an IB TCA that acts as an 
IP router (it isn't a router in the IB use of the term).  This management 
agent can determine whether to include this TCA in any IB multicast group 
or not.  As such, the management agent not the TCA creates the IB multicast 
group.  When an IB endnode wishes to use multicast, it would communicate to 
the SA and find that there is a corresponding multicast group for the 
service it is trying to deliver and if so, a join would be performed and 
the packets would be automatically forwarded to all members which may now 
include these IB TCAs.  It is strictly policy as to whether the management 
agent adds in these TCAs into all multicast groups or not.  The application 
though continues to use the same multicast group identifier whether such 
TCAs are present or not.

>         A receiver creating the group after the sender checks is no
>         different than the receiver creating the group after a
>         hypothetical atomic check and send of a multicast packet by a
>         sender. I thought about suggesting that sender sends to the
>         broadcast-GID but then decided against it - why punish
>         everyone for a once in a while race condition that can go
>         either way even if the sender could do atomic 'check (i.e.
>         join) and send'.

Broadcast is not required for the sending endnode.  All router multicast is 
not required for any of this to work; the sender injects packets to the 
selected IP multicast group which maps to a corresponding set of 
participating CAs, thus no race condition exists, no additional 
notification is required, etc.

>         For the next packet the sender will find the group and from
>         then onwards directly send on the group.

How exactly would it find out anything in this regard?  There is no 
mechanism to perform a dynamic update of all participating endnodes and 
there should not be except to indicate the multicast group no longer exists 
or a prune is performed for this endnode.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 12 20:34:34 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA07361
	for <ipoverib-archive@odin.ietf.org>; Fri, 12 Apr 2002 20:34:33 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id UAA23285
	for ipoverib-archive@odin.ietf.org; Fri, 12 Apr 2002 20:34:37 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id UAA22471;
	Fri, 12 Apr 2002 20:18:59 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id UAA22439
	for <ipoverib@ns.ietf.org>; Fri, 12 Apr 2002 20:18:57 -0400 (EDT)
Received: from nwkea-mail-1.sun.com (nwkea-mail-1.sun.com [192.18.42.13])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA07236
	for <ipoverib@ietf.org>; Fri, 12 Apr 2002 20:18:53 -0400 (EDT)
Received: from sunmail1.Sun.COM ([129.145.1.2])
	by nwkea-mail-1.sun.com (8.9.3+Sun/8.9.3) with ESMTP id RAA03662;
	Fri, 12 Apr 2002 17:18:25 -0700 (PDT)
Received: from jurassic.eng.sun.com (jurassic.Eng.Sun.COM [129.146.86.38])
	by sunmail1.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1-Sun.COM.mod.2) with ESMTP id RAA12138;
	Fri, 12 Apr 2002 17:18:24 -0700 (PDT)
Received: from taipei (taipei.Eng.Sun.COM [129.146.85.178])
	by jurassic.eng.sun.com (8.12.2+Sun/8.12.2) with SMTP id g3D0IN35465293;
	Fri, 12 Apr 2002 17:18:23 -0700 (PDT)
Message-Id: <200204130018.g3D0IN35465293@jurassic.eng.sun.com>
Date: Fri, 12 Apr 2002 17:18:11 -0700 (PDT)
From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Reply-To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
To: ipoverib@ietf.org, krause@cup.hp.com, Ajoy.Siddabathuni@Sun.COM
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: 0uMjWimDVowHnlG2pdkSNg==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc 
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Ajoy,

This sounds reasonable to me. But someone needs to make a concrete
proposal here, e.g., writing a draft. (Mike?)

Jerry

>Jerry,
>
>>To keep 4-byte alignment I suggest to have Ethertype residing in the
>>lower (network byte-ordering) 2 bytes with the upper 16 bits marked
>>as reserved. It'd use the values for ARP, IPv4 and IPv6 from the
>>"EtherType" defined by IANA. Beyond these three protocols I'm not
>>sure we (IPoIB WG) want or even can say much.
>
>Would this by definition allow implementations to use LLC/SNAP headers if
>required?  In the cIPoverATM (see rfc2225) world, the LLC/SNAP
>is the default packet format for IP datagrams.
>
>I dont necessarily have an opinion one way or the other, but having the
>alignment definetly helps.
>
>~Ajoy
>
>
>>Date: Thu, 11 Apr 2002 13:34:54 -0700 (PDT)
>>From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
>>Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
>>To: ipoverib@ietf.org, krause@cup.hp.com
>>MIME-Version: 1.0
>>Content-MD5: GenAJUOSmMHlgivkPxdJZA==
>>X-Mailman-Version: 1.0
>>List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
>>X-BeenThere: ipoverib@ietf.org
>>
>>>X-Sender: krause@esalpha2.cup.hp.com
>>>Date: Thu, 11 Apr 2002 06:33:06 -0700
>>
>><snip>
>>
>>>>Last week I asked people to bring "new, convincing evidence" when
>>>>arguing for or against EtherType. But unfortunately in the past few
>>>>days all I heard was either old arguments, or people reiterating their
>>>>positions. Since no one has brought forward any new, convincing
>>>>evidence against EtherType, and in both the mailing list and the
>>>>f2f meeting Bill and I feel there have been more people for EtherType
>>>>than against it, plus a number of people requesting EtherType are
>>>>actually working on real implementation, I hereby declare EtherType
>>>>to be the rough consensus of the WG to be included in the UD payload
>>>>of every IPoIB packet.
>>>
>>>What is the specification of Ethertype other than it is 16-bits of space 
>>>followed by the next set of headers?
>>
>>To keep 4-byte alignment I suggest to have Ethertype residing in the
>>lower (network byte-ordering) 2 bytes with the upper 16 bits marked
>>as reserved. It'd use the values for ARP, IPv4 and IPv6 from the
>>"EtherType" defined by IANA. Beyond these three protocols I'm not
>>sure we (IPoIB WG) want or even can say much.
>>
>>>Is an implementation required to 
>>>process this field if it does not want to do so?
>>
>>IMO the sender MUST incorporate EtherType correctly in all the
>>outgoing ARP/IPv4/IPv6 pkts (including both unicast and multicast).
>>The receiver SHOULD but is not required to demux based on EtherType.
>>This is partly implementation detail. If a receiver can correctly
>>demux without examining EtherType, why do we care?
>>
>>>Is Ethertype required for 
>>>all IB transport services that may support IP traffic - it seems that it 
>>>must be to remain consistent from an implementation point of view.
>>
>>Agreed. But since the consensus call was made specifically for UD,
>>for non-UD people can speak up if they have a different opinion.
>>
>>>
>>>>Over the past couple of days a couple of senior IBTA members have
>>>>reiterated their positions against EtherType. But in my opinion the
>>>>decision of including EtherType in the UD payload is at least partly
>>>>as the result of the IBTA failing to address people's need to have a
>>>>field in IB identifying the type of the data being encapsulated in
>>>>the UD payload.
>>>
>>>In essence, the consensus is that the IP over IB workgroup is defining the 
>>>official Raw protocol on behalf of the IBTA since in many people's opinion 
>>>the IBTA screwed up.  As such, it would be beneficial to simply write a 
>>>draft with this as the stated goal so that anyone can use it independent of 
>>>IP over IB's charter for such purposes and then have the IP over IB 
>>>specification reference said draft.
>>
>>Agreed. But since the draft will address a generic IB issue rather than
>>IPoIB issue, IBTA seems to be the right place for such a document, not
>>IETF.
>>
>>>  This will be much cleaner and at least 
>>>make everyone who may want to implement such a service not have to go 
>>>searching within an IP over IB specification to find this non-IP focused 
>>>functionality.
>>
>>Agreed.
>>
>>>
>>>>Some argued that the QP is supposed to serve that
>>>>purpose, but we are not even doing that for IPv4, ARP/RARP and IPv6.
>>>>I think the argument is rather weak.
>>>
>>>Just as weak as hijacking one spec to include functionality that has 
>>>nothing to do with the stated charter of this workgroup.  Consensus has 
>>>been reached so I'll let it die.
>>
>>Had IBTA fixed raw datagram to be used by IPoIB as the default IB
>>transport, or had we adopted SIDR to map ARP/IPv4/IPv6 to different
>>QPs, we wouldn't have gone down this path. Failing both, Ethertype
>>seemed inevitable.
>>
>>Jerry
>>
>>>
>>>Mike
>>>
>>
>>
>>_______________________________________________
>>IPoverIB mailing list
>>IPoverIB@ietf.org
>>https://www1.ietf.org/mailman/listinfo/ipoverib
>


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Sun Apr 14 04:43:06 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA15644
	for <ipoverib-archive@odin.ietf.org>; Sun, 14 Apr 2002 04:43:05 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id EAA20322
	for ipoverib-archive@odin.ietf.org; Sun, 14 Apr 2002 04:43:07 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id EAA20233;
	Sun, 14 Apr 2002 04:38:37 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id EAA20203
	for <ipoverib@optimus.ietf.org>; Sun, 14 Apr 2002 04:38:35 -0400 (EDT)
Received: from ntserver.voltaire.com (Volter-FW.ser.netvision.net.il [212.143.107.30])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA15610
	for <ipoverib@ietf.org>; Sun, 14 Apr 2002 04:38:31 -0400 (EDT)
Received: by NTSERVER with Internet Mail Service (5.5.2653.19)
	id <2TWT2LSN>; Sun, 14 Apr 2002 10:31:40 +0200
Message-ID: <20083A3BAEF9D211BDB600805F8B14F301CBF614@NTSERVER>
From: Yaron Haviv <yaronh@voltaire.com>
To: "'H.K. Jerry Chu'" <Jerry.Chu@eng.sun.com>,
        "'ipoverib@ietf.org'"
	 <ipoverib@ietf.org>,
        "'krause@cup.hp.com'" <krause@cup.hp.com>,
        "'Ajoy.Siddabathuni@Sun.COM'" <Ajoy.Siddabathuni@Sun.COM>
Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
Date: Sun, 14 Apr 2002 10:31:39 +0200
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Jerry,

We must stop expending the definition somewhere so we get the spec out 
This is IPoIB we are talking about, there is no need for LLC/SNAP, and we
just need the Ethertype 
This is also the consensus, as I understand

Adding LLC/Snap adds features that are not needed for IPoIB, and just add
redundant complexity 
Lets stick to adding the Ethertype and 16 bits reserved, unless there is
something mandatory which violates the clean operation of IPoIB 

Yaron


-----Original Message-----
From: ipoverib-admin@ietf.org [mailto:ipoverib-admin@ietf.org]On Behalf Of
H.K. Jerry Chu
Sent: Saturday, April 13, 2002 2:18 AM
To: ipoverib@ietf.org; krause@cup.hp.com; Ajoy.Siddabathuni@Sun.COM
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4

Ajoy,

This sounds reasonable to me. But someone needs to make a concrete
proposal here, e.g., writing a draft. (Mike?)

Jerry

>Jerry,
>
>>To keep 4-byte alignment I suggest to have Ethertype residing in the
>>lower (network byte-ordering) 2 bytes with the upper 16 bits marked
>>as reserved. It'd use the values for ARP, IPv4 and IPv6 from the
>>"EtherType" defined by IANA. Beyond these three protocols I'm not
>>sure we (IPoIB WG) want or even can say much.
>
>Would this by definition allow implementations to use LLC/SNAP headers if
>required?  In the cIPoverATM (see rfc2225) world, the LLC/SNAP
>is the default packet format for IP datagrams.
>
>I dont necessarily have an opinion one way or the other, but having the
>alignment definetly helps.
>
>~Ajoy
>
>
>>Date: Thu, 11 Apr 2002 13:34:54 -0700 (PDT)
>>From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
>>Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
>>To: ipoverib@ietf.org, krause@cup.hp.com
>>MIME-Version: 1.0
>>Content-MD5: GenAJUOSmMHlgivkPxdJZA==
>>X-Mailman-Version: 1.0
>>List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
>>X-BeenThere: ipoverib@ietf.org
>>
>>>X-Sender: krause@esalpha2.cup.hp.com
>>>Date: Thu, 11 Apr 2002 06:33:06 -0700
>>
>><snip>
>>
>>>>Last week I asked people to bring "new, convincing evidence" when
>>>>arguing for or against EtherType. But unfortunately in the past few
>>>>days all I heard was either old arguments, or people reiterating their
>>>>positions. Since no one has brought forward any new, convincing
>>>>evidence against EtherType, and in both the mailing list and the
>>>>f2f meeting Bill and I feel there have been more people for EtherType
>>>>than against it, plus a number of people requesting EtherType are
>>>>actually working on real implementation, I hereby declare EtherType
>>>>to be the rough consensus of the WG to be included in the UD payload
>>>>of every IPoIB packet.
>>>
>>>What is the specification of Ethertype other than it is 16-bits of space
>>>followed by the next set of headers?
>>
>>To keep 4-byte alignment I suggest to have Ethertype residing in the
>>lower (network byte-ordering) 2 bytes with the upper 16 bits marked
>>as reserved. It'd use the values for ARP, IPv4 and IPv6 from the
>>"EtherType" defined by IANA. Beyond these three protocols I'm not
>>sure we (IPoIB WG) want or even can say much.
>>
>>>Is an implementation required to
>>>process this field if it does not want to do so?
>>
>>IMO the sender MUST incorporate EtherType correctly in all the
>>outgoing ARP/IPv4/IPv6 pkts (including both unicast and multicast).
>>The receiver SHOULD but is not required to demux based on EtherType.
>>This is partly implementation detail. If a receiver can correctly
>>demux without examining EtherType, why do we care?
>>
>>>Is Ethertype required for
>>>all IB transport services that may support IP traffic - it seems that it
>>>must be to remain consistent from an implementation point of view.
>>
>>Agreed. But since the consensus call was made specifically for UD,
>>for non-UD people can speak up if they have a different opinion.
>>
>>>
>>>>Over the past couple of days a couple of senior IBTA members have
>>>>reiterated their positions against EtherType. But in my opinion the
>>>>decision of including EtherType in the UD payload is at least partly
>>>>as the result of the IBTA failing to address people's need to have a
>>>>field in IB identifying the type of the data being encapsulated in
>>>>the UD payload.
>>>
>>>In essence, the consensus is that the IP over IB workgroup is defining
the
>>>official Raw protocol on behalf of the IBTA since in many people's
opinion
>>>the IBTA screwed up.  As such, it would be beneficial to simply write a
>>>draft with this as the stated goal so that anyone can use it independent
of
>>>IP over IB's charter for such purposes and then have the IP over IB
>>>specification reference said draft.
>>
>>Agreed. But since the draft will address a generic IB issue rather than
>>IPoIB issue, IBTA seems to be the right place for such a document, not
>>IETF.
>>
>>>  This will be much cleaner and at least
>>>make everyone who may want to implement such a service not have to go
>>>searching within an IP over IB specification to find this non-IP focused
>>>functionality.
>>
>>Agreed.
>>
>>>
>>>>Some argued that the QP is supposed to serve that
>>>>purpose, but we are not even doing that for IPv4, ARP/RARP and IPv6.
>>>>I think the argument is rather weak.
>>>
>>>Just as weak as hijacking one spec to include functionality that has
>>>nothing to do with the stated charter of this workgroup.  Consensus has
>>>been reached so I'll let it die.
>>
>>Had IBTA fixed raw datagram to be used by IPoIB as the default IB
>>transport, or had we adopted SIDR to map ARP/IPv4/IPv6 to different
>>QPs, we wouldn't have gone down this path. Failing both, Ethertype
>>seemed inevitable.
>>
>>Jerry
>>
>>>
>>>Mike
>>>
>>
>>
>>_______________________________________________
>>IPoverIB mailing list
>>IPoverIB@ietf.org
>>https://www1.ietf.org/mailman/listinfo/ipoverib
>


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr 15 13:48:14 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA29738
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 13:48:14 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id NAA01750
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 13:48:15 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id NAA00435;
	Mon, 15 Apr 2002 13:26:22 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id NAA00399
	for <ipoverib@optimus.ietf.org>; Mon, 15 Apr 2002 13:26:19 -0400 (EDT)
Received: from palrel10.hp.com (palrel10.hp.com [156.153.255.245])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA28846
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 13:26:17 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel10.hp.com (Postfix) with ESMTP
	id 7E0D8C004DA; Mon, 15 Apr 2002 10:25:48 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id KAA05903;
	Mon, 15 Apr 2002 10:25:48 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020415101832.02a650e0@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Mon, 15 Apr 2002 10:20:49 -0700
To: Yaron Haviv <yaronh@voltaire.com>,
        "'H.K. Jerry Chu'" <Jerry.Chu@eng.sun.com>,
        "'ipoverib@ietf.org'" <ipoverib@ietf.org>,
        "'Ajoy.Siddabathuni@Sun.COM'" <Ajoy.Siddabathuni@Sun.COM>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
In-Reply-To: <20083A3BAEF9D211BDB600805F8B14F301CBF614@NTSERVER>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
>Jerry,
>
>We must stop expending the definition somewhere so we get the spec out
>This is IPoIB we are talking about, there is no need for LLC/SNAP, and we
>just need the Ethertype

The consensus is that people want to have the full range of Ethertype 
values supported.  As such, this needs to be specified independent of the 
IP over IB specification so that everyone may use these values without 
requiring them to implement IP over IB.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr 15 14:52:04 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA02720
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 14:52:02 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA05633
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 14:52:04 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA05111;
	Mon, 15 Apr 2002 14:39:24 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA05078
	for <ipoverib@optimus.ietf.org>; Mon, 15 Apr 2002 14:39:21 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA02202
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 14:39:18 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3FIdJLQ112150;
	Mon, 15 Apr 2002 14:39:19 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3FId02107290;
	Mon, 15 Apr 2002 14:39:03 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g3FIcjM22001;
	Mon, 15 Apr 2002 11:38:46 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3FIchL26216;
	Mon, 15 Apr 2002 11:38:43 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204151838.g3FIchL26216@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: adube@windows.microsoft.com (Aditya Dube)
Date: Mon, 15 Apr 2002 11:38:42 -0700 (PDT)
Cc: ipoverib@ietf.org
In-Reply-To: <2E33960095B58E40A4D3345AB9F65EC105D2C94F@win-msg-01.wingroup.windeploy.ntdev.microsoft.com> from "Aditya Dube" at Apr 12, 2002 10:03:59 AM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> 
> Vivek
> 
> As the sender will only send packets to the all-router MGID after doing
> the check, how will a receiver who creates the MGroup on the same subnet
> as the sender ever receive the multicast packets that it should get?


        The suggested solution is as follows. The basic idea is
        certainly workable. Hopefully, after some discussion we can
        refine it to cover all the issues that crop up.


	Sender:
		got IP multicast packet to send
			determine the corresponding IB group
			if already 'IB_joined' or 'recorded'
				IB_send to it

			else if IB group exists
				IB_join/record MGID locally
				IB_send to it
			else
				
				IB_send to all-routers MGID

	
	Receiver:
		want to receive IP multicast beloging to an IP group 
			check for corresponding IB group
			if IB group exists 
				IB_join it
				IB_send report to group
			else
				IB_create IB group
				IB_send report to all-routers MGID
				(IB_send report to group)

	
	Router:
		always receive on broadcast/all-hosts and all-routers
		if IGMP/MLD indicates new group
			IB_join corresponding IB group
		if routing protocol indicates new group
			IB_create corresponding IB group 


        Now, a sender must check for the IB group everytime it is sending
        to an IB group it doesn't know about (no prior IB_join/record
        with it). But once the group exists and the sender has a
        packet to send, it will detect the presence and join the
        group. From then on it will always send to that MGID. The
        receiver will get the packets since it (or another receiver)
        created the IB group.

        In other situation where the router determines that there are
        external receivers it creates the group too. The sender will
        therefore find this and join the group.

        The sender needs to continuously check for the IB group only
        in case is where the router doesn't need to forward the packet
        and there are no receivers in the local subnet. I don't think
        this will be too often. Is this case of concern ?



Vivek

<snip>

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr 15 15:24:34 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA03754
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 15:24:33 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA07609
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 15:24:36 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA07160;
	Mon, 15 Apr 2002 15:12:09 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA07122
	for <ipoverib@optimus.ietf.org>; Mon, 15 Apr 2002 15:12:06 -0400 (EDT)
Received: from mxout2.netvision.net.il (mxout2.netvision.net.il [194.90.9.21])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA03462
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 15:12:03 -0400 (EDT)
Received: from vaio505he ([139.92.208.180]) by mxout2.netvision.net.il
 (iPlanet Messaging Server 5.2 (built Feb 21 2002))
 with ESMTPA id <0GUM00C65IN6FB@mxout2.netvision.net.il> for ipoverib@ietf.org;
 Mon, 15 Apr 2002 22:11:33 +0300 (IDT)
Date: Mon, 15 Apr 2002 21:12:24 +0200
From: Yaron Haviv <yaronh@voltaire.com>
Subject: RE: [Ipoverib] IPoIB multicast forwarding
In-reply-to: <200204151838.g3FIchL26216@eng4.beaverton.ibm.com>
To: "'Vivek Kashyap'" <kashyapv@us.ibm.com>,
        "'Aditya Dube'" <adube@windows.microsoft.com>
Cc: ipoverib@ietf.org
Reply-to: yaronh@voltaire.com
Message-id: <001e01c1e4b1$79f1ab30$b4d05c8b@vaio505he>
MIME-version: 1.0
X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4807.1700
X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2910.0)
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7BIT
Importance: Normal
X-Priority: 3 (Normal)
X-MSMail-priority: Normal
Content-Transfer-Encoding: 7BIT
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7BIT

Vivek,

Since checking the Group any time you send a multicast packet is very costly
We can adopt some kind of a broadcast notification message sent by the
router, this way the node will update its caches

The process goes:
	Sender wants to send multicast
	No Group so sends to all Router Group
	(Can optionally create the group (like Mike suggested) still sending the
packet to all-routers without delaying the packet, since we need to first
have the router joining this group prior to our send)

Now we need someone to tell us that the router is registered to this group
already
This could be done by either the router or by the SM

now there are two possible scenarios
1 - 	Router receives the packet and sees there is a need for a new group
	it creates the group, and then sends a broadcast message saying this group
was formed
	(Can optionally send a unicast back to the sender)
	Sender updates its cache

2- 	SM sees a create/join from the router and than send a notification to
all other nodes
	this enables updating the cache

Updating PathRecord cache is probably an essential IBA requirement too when
dealing with the unicast traffic
It could use a similar notification mechanism

Yaron

-----Original Message-----
From: ipoverib-admin@ietf.org [mailto:ipoverib-admin@ietf.org]On Behalf Of
Vivek Kashyap
Sent: Monday, April 15, 2002 8:39 PM
To: Aditya Dube
Cc: ipoverib@ietf.org
Subject: Re: [Ipoverib] IPoIB multicast forwarding

>
>
> Vivek
>
> As the sender will only send packets to the all-router MGID after doing
> the check, how will a receiver who creates the MGroup on the same subnet
> as the sender ever receive the multicast packets that it should get?


        The suggested solution is as follows. The basic idea is
        certainly workable. Hopefully, after some discussion we can
        refine it to cover all the issues that crop up.


        Sender:
                got IP multicast packet to send
                        determine the corresponding IB group
                        if already 'IB_joined' or 'recorded'
                                IB_send to it

                        else if IB group exists
                                IB_join/record MGID locally
                                IB_send to it
                        else

                                IB_send to all-routers MGID


        Receiver:
                want to receive IP multicast beloging to an IP group
                        check for corresponding IB group
                        if IB group exists
                                IB_join it
                                IB_send report to group
                        else
                                IB_create IB group
                                IB_send report to all-routers MGID
                                (IB_send report to group)


        Router:
                always receive on broadcast/all-hosts and all-routers
                if IGMP/MLD indicates new group
                        IB_join corresponding IB group
                if routing protocol indicates new group
                        IB_create corresponding IB group


        Now, a sender must check for the IB group everytime it is sending
        to an IB group it doesn't know about (no prior IB_join/record
        with it). But once the group exists and the sender has a
        packet to send, it will detect the presence and join the
        group. From then on it will always send to that MGID. The
        receiver will get the packets since it (or another receiver)
        created the IB group.

        In other situation where the router determines that there are
        external receivers it creates the group too. The sender will
        therefore find this and join the group.

        The sender needs to continuously check for the IB group only
        in case is where the router doesn't need to forward the packet
        and there are no receivers in the local subnet. I don't think
        this will be too often. Is this case of concern ?



Vivek

<snip>

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr 15 15:52:43 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04708
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 15:52:43 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA09380
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 15:52:46 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA09077;
	Mon, 15 Apr 2002 15:47:32 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA09043
	for <ipoverib@optimus.ietf.org>; Mon, 15 Apr 2002 15:47:28 -0400 (EDT)
Received: from palrel10.hp.com (palrel10.hp.com [156.153.255.245])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04536
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 15:47:25 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel10.hp.com (Postfix) with ESMTP
	id 45C00C0004C; Mon, 15 Apr 2002 12:46:53 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id MAA00706;
	Mon, 15 Apr 2002 12:46:52 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020415124415.02a94c60@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Mon, 15 Apr 2002 12:46:47 -0700
To: yaronh@voltaire.com, "'Vivek Kashyap'" <kashyapv@us.ibm.com>,
        "'Aditya Dube'" <adube@windows.microsoft.com>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Cc: ipoverib@ietf.org
In-Reply-To: <001e01c1e4b1$79f1ab30$b4d05c8b@vaio505he>
References: <200204151838.g3FIchL26216@eng4.beaverton.ibm.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 09:12 PM 4/15/2002 +0200, Yaron Haviv wrote:
>Vivek,
>
>Since checking the Group any time you send a multicast packet is very costly
>We can adopt some kind of a broadcast notification message sent by the
>router, this way the node will update its caches

How would an application distinguish between what is now a control message 
independent of the multicast group and application messages which were part 
of the multicast group's work?  I prefer to leave all of this to the IB 
management realm to solve (as described in my other e-mail) and just have 
IP focus on IP operations.  The mappings should be obtained once and only 
upon an update from a management entity within the IB framework would this 
mapping be changed.  In general, I find much of the discussion to date as 
making things more complex / difficult for endnodes to implement when it 
should not be so.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr 15 15:59:50 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA05032
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 15:59:50 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA09677
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 15:59:53 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA09515;
	Mon, 15 Apr 2002 15:55:51 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA09481
	for <ipoverib@optimus.ietf.org>; Mon, 15 Apr 2002 15:55:49 -0400 (EDT)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04868
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 15:55:43 -0400 (EDT)
Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.39 2002/04/15 17:47:23 root Exp $) with ESMTP id g3FJsXt13786
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 19:54:33 GMT
Received: from fmsmsxvs040.fm.intel.com (fmsmsxv040-1.fm.intel.com [132.233.48.108])
	by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.16 2002/04/15 17:47:00 root Exp $) with SMTP id g3FJuvX23650
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 19:56:57 GMT
Received: from fmsmsx28.fm.intel.com ([132.233.42.28])
 by fmsmsxvs040.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002041512543122081
 for <ipoverib@ietf.org>; Mon, 15 Apr 2002 12:54:31 -0700
Received: by fmsmsx28.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2KDV654N>; Mon, 15 Apr 2002 12:55:07 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F8A@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Mon, 15 Apr 2002 12:54:02 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Can someone clarify for me the semantics of a join and delete operation with
respect to the following:

If a group has been created by node X and pure multicast sender Y has
detected that fact and recorded the MGID/MLID without joining, what is the
state of the MGID/MLID after X deletes the group?  If the group is then
created again, by X or a different node, must the SM assign the same MLID?
If not, node Y is holding a stale MLID which would argue that it must always
consult the SM before sending a multicast frame.

Along those same lines, if node Y had joined the multicast group X created
and then X deletes it, does the SM delete the multicast group and update IB
switches at that time or is the group reference counted such that as long as
a node has joined, switches are not reprogrammed until the last reference
does a leave?  If the former, we would have the same stale MLID problem.  If
the latter, doesn't a node crash in this situation preserve an "open"
multicast group indefinitely? 

Roy

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
Sent: Monday, April 15, 2002 11:39 AM
To: adube@windows.microsoft.com
Cc: ipoverib@ietf.org
Subject: Re: [Ipoverib] IPoIB multicast forwarding


> 
> 
> Vivek
> 
> As the sender will only send packets to the all-router MGID after doing
> the check, how will a receiver who creates the MGroup on the same subnet
> as the sender ever receive the multicast packets that it should get?


        The suggested solution is as follows. The basic idea is
        certainly workable. Hopefully, after some discussion we can
        refine it to cover all the issues that crop up.


	Sender:
		got IP multicast packet to send
			determine the corresponding IB group
			if already 'IB_joined' or 'recorded'
				IB_send to it

			else if IB group exists
				IB_join/record MGID locally
				IB_send to it
			else
				
				IB_send to all-routers MGID

	
	Receiver:
		want to receive IP multicast beloging to an IP group 
			check for corresponding IB group
			if IB group exists 
				IB_join it
				IB_send report to group
			else
				IB_create IB group
				IB_send report to all-routers MGID
				(IB_send report to group)

	
	Router:
		always receive on broadcast/all-hosts and all-routers
		if IGMP/MLD indicates new group
			IB_join corresponding IB group
		if routing protocol indicates new group
			IB_create corresponding IB group 


        Now, a sender must check for the IB group everytime it is sending
        to an IB group it doesn't know about (no prior IB_join/record
        with it). But once the group exists and the sender has a
        packet to send, it will detect the presence and join the
        group. From then on it will always send to that MGID. The
        receiver will get the packets since it (or another receiver)
        created the IB group.

        In other situation where the router determines that there are
        external receivers it creates the group too. The sender will
        therefore find this and join the group.

        The sender needs to continuously check for the IB group only
        in case is where the router doesn't need to forward the packet
        and there are no receivers in the local subnet. I don't think
        this will be too often. Is this case of concern ?



Vivek

<snip>

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr 15 16:02:21 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA05106
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 16:02:21 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA10221
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 16:02:24 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA09626;
	Mon, 15 Apr 2002 15:58:30 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA09573
	for <ipoverib@optimus.ietf.org>; Mon, 15 Apr 2002 15:58:27 -0400 (EDT)
Received: from mail4.microsoft.com (mail4.microsoft.com [131.107.3.122])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04969
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 15:58:23 -0400 (EDT)
Received: from inet-vrs-04.redmond.corp.microsoft.com ([157.54.8.154]) by mail4.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Mon, 15 Apr 2002 12:57:55 -0700
Received: from 157.54.5.25 by inet-vrs-04.redmond.corp.microsoft.com (InterScan E-Mail VirusWall NT); Mon, 15 Apr 2002 12:57:55 -0700
Received: from red-imc-01.redmond.corp.microsoft.com ([157.54.9.102]) by inet-hub-03.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Mon, 15 Apr 2002 12:57:54 -0700
Received: from win-imc-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.39]) by red-imc-01.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Mon, 15 Apr 2002 12:57:55 -0700
Received: from win-msg-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.52]) by win-imc-01.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(6.0.3604.0);
	 Mon, 15 Apr 2002 12:54:18 -0700
X-MimeOLE: Produced By Microsoft Exchange V6.0.6177.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Mon, 15 Apr 2002 12:54:18 -0700
Message-ID: <2E33960095B58E40A4D3345AB9F65EC102450E2C@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Thread-Topic: [Ipoverib] IPoIB multicast forwarding
Thread-Index: AcHkrGVtDUcTqionRBaGDC/meWwgVQACpYvQ
From: "Aditya Dube" <adube@windows.microsoft.com>
To: "Vivek Kashyap" <kashyapv@us.ibm.com>
Cc: <ipoverib@ietf.org>
X-OriginalArrivalTime: 15 Apr 2002 19:54:18.0964 (UTC) FILETIME=[546CA940:01C1E4B7]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id PAA09574
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 8bit

Hi Vivek,

After reading through your response, I realized that there is one more
case to consider: 

Sender does a check for IB Mcast Group...check succeeds because a
receiver has created the group.

Now, the receiver decides to leave the group and the IB MGroup gets
deleted.

The Sender needs detect this and revert back to sending the multicast
packets to the All-router Address.

I think in the scheme we are discussing, the sender will always have to
check for the existence of the IB Group just because of the edge cases.

Aditya Dube

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
Sent: Monday, April 15, 2002 11:39 AM
To: Aditya Dube
Cc: ipoverib@ietf.org
Subject: Re: [Ipoverib] IPoIB multicast forwarding

> 
> 
> Vivek
> 
> As the sender will only send packets to the all-router MGID after
doing
> the check, how will a receiver who creates the MGroup on the same
subnet
> as the sender ever receive the multicast packets that it should get?


        The suggested solution is as follows. The basic idea is
        certainly workable. Hopefully, after some discussion we can
        refine it to cover all the issues that crop up.


	Sender:
		got IP multicast packet to send
			determine the corresponding IB group
			if already 'IB_joined' or 'recorded'
				IB_send to it

			else if IB group exists
				IB_join/record MGID locally
				IB_send to it
			else
				
				IB_send to all-routers MGID

	
	Receiver:
		want to receive IP multicast beloging to an IP group 
			check for corresponding IB group
			if IB group exists 
				IB_join it
				IB_send report to group
			else
				IB_create IB group
				IB_send report to all-routers MGID
				(IB_send report to group)

	
	Router:
		always receive on broadcast/all-hosts and all-routers
		if IGMP/MLD indicates new group
			IB_join corresponding IB group
		if routing protocol indicates new group
			IB_create corresponding IB group 


        Now, a sender must check for the IB group everytime it is
sending
        to an IB group it doesn't know about (no prior IB_join/record
        with it). But once the group exists and the sender has a
        packet to send, it will detect the presence and join the
        group. From then on it will always send to that MGID. The
        receiver will get the packets since it (or another receiver)
        created the IB group.

        In other situation where the router determines that there are
        external receivers it creates the group too. The sender will
        therefore find this and join the group.

        The sender needs to continuously check for the IB group only
        in case is where the router doesn't need to forward the packet
        and there are no receivers in the local subnet. I don't think
        this will be too often. Is this case of concern ?



Vivek

<snip>

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr 15 16:16:47 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA05493
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 16:16:47 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA11138
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 16:16:50 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA10772;
	Mon, 15 Apr 2002 16:11:22 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA10733
	for <ipoverib@optimus.ietf.org>; Mon, 15 Apr 2002 16:11:18 -0400 (EDT)
Received: from palrel11.hp.com (palrel11.hp.com [156.153.255.246])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA05341
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 16:11:08 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel11.hp.com (Postfix) with ESMTP
	id 81B90600D47; Mon, 15 Apr 2002 13:10:35 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id NAA04984;
	Mon, 15 Apr 2002 13:10:35 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020415130922.02b07be0@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Mon, 15 Apr 2002 13:10:31 -0700
To: "Aditya Dube" <adube@windows.microsoft.com>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Cc: <ipoverib@ietf.org>
In-Reply-To: <2E33960095B58E40A4D3345AB9F65EC102450E2C@win-msg-01.wingro
 up.windeploy.ntdev.microsoft.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 12:54 PM 4/15/2002 -0700, Aditya Dube wrote:
>Hi Vivek,
>
>After reading through your response, I realized that there is one more
>case to consider:
>
>Sender does a check for IB Mcast Group...check succeeds because a
>receiver has created the group.
>
>Now, the receiver decides to leave the group and the IB MGroup gets
>deleted.
>
>The Sender needs detect this and revert back to sending the multicast
>packets to the All-router Address.
>
>I think in the scheme we are discussing, the sender will always have to
>check for the existence of the IB Group just because of the edge cases.

Yet another example of why use of an all router address is a bad idea.  The 
IB multicast group can be created and any node can be added / deleted 
(including a TCA providing IP routing services) without requiring any 
notification of address change, etc.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr 15 16:29:12 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA05947
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 16:29:12 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA11725
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 16:29:12 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA11461;
	Mon, 15 Apr 2002 16:24:59 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA11431
	for <ipoverib@optimus.ietf.org>; Mon, 15 Apr 2002 16:24:57 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA05824
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 16:24:52 -0400 (EDT)
Received: from southrelay02.raleigh.ibm.com (southrelay02.raleigh.us.ibm.com [9.37.3.209])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3FKOrLQ107578;
	Mon, 15 Apr 2002 16:24:54 -0400
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by southrelay02.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3FKOpv121006;
	Mon, 15 Apr 2002 16:24:51 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g3FKOpX10751;
	Mon, 15 Apr 2002 13:24:51 -0700
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3FKOo021476;
	Mon, 15 Apr 2002 13:24:50 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204152024.g3FKOo021476@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: adube@windows.microsoft.com (Aditya Dube)
Date: Mon, 15 Apr 2002 13:24:49 -0700 (PDT)
Cc: kashyapv@us.ibm.com (Vivek Kashyap), ipoverib@ietf.org
In-Reply-To: <2E33960095B58E40A4D3345AB9F65EC102450E2C@win-msg-01.wingroup.windeploy.ntdev.microsoft.com> from "Aditya Dube" at Apr 15, 2002 11:54:18 AM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit


> 
> Hi Vivek,
> 
> After reading through your response, I realized that there is one more
> case to consider: 
> 
> Sender does a check for IB Mcast Group...check succeeds because a
> receiver has created the group.
> 
> Now, the receiver decides to leave the group and the IB MGroup gets
> deleted.
> 
> The Sender needs detect this and revert back to sending the multicast
> packets to the All-router Address.
> 
> I think in the scheme we are discussing, the sender will always have to
> check for the existence of the IB Group just because of the edge cases.

        I have made some assumptions on IB deletion. Let us assume
        that the sender has joined the group (I am in the group that
        believes for IB to work right all have to join - no-join-send
        does not work universally).

        The receiver will not delete the group just leave. The
        sender's 'check' really is 'join'. The sender is a member of
        the group - the group cannot really be deleted until all
        members leave. The sender can age its entries of the groups it
        has joined and after certain idle time 'leave' the group.

        Also, if the receiver goes through the process, it will first
        have sent a report to the router. THe router is also a member
        of the group.

        We can also have it that the last guy leaving the group delete
        it. IF someone joins it just as we are deleting it then the
        deletion should fail if the membership count is greater than
        one. 

        So, if the sender has joined the group, I don't think that it
        will have to check the group everytime it sends a packet.


Vivek

> 
> Aditya Dube
> 
> -----Original Message-----
> From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
> Sent: Monday, April 15, 2002 11:39 AM
> To: Aditya Dube
> Cc: ipoverib@ietf.org
> Subject: Re: [Ipoverib] IPoIB multicast forwarding
> 
> > 
> > 
> > Vivek
> > 
> > As the sender will only send packets to the all-router MGID after
> doing
> > the check, how will a receiver who creates the MGroup on the same
> subnet
> > as the sender ever receive the multicast packets that it should get?
> 
> 
>         The suggested solution is as follows. The basic idea is
>         certainly workable. Hopefully, after some discussion we can
>         refine it to cover all the issues that crop up.
> 
> 
> 	Sender:
> 		got IP multicast packet to send
> 			determine the corresponding IB group
> 			if already 'IB_joined' or 'recorded'
> 				IB_send to it
> 
> 			else if IB group exists
> 				IB_join/record MGID locally
> 				IB_send to it
> 			else
> 				
> 				IB_send to all-routers MGID
> 
> 	
> 	Receiver:
> 		want to receive IP multicast beloging to an IP group 
> 			check for corresponding IB group
> 			if IB group exists 
> 				IB_join it
> 				IB_send report to group
> 			else
> 				IB_create IB group
> 				IB_send report to all-routers MGID
> 				(IB_send report to group)
> 
> 	
> 	Router:
> 		always receive on broadcast/all-hosts and all-routers
> 		if IGMP/MLD indicates new group
> 			IB_join corresponding IB group
> 		if routing protocol indicates new group
> 			IB_create corresponding IB group 
> 
> 
>         Now, a sender must check for the IB group everytime it is
> sending
>         to an IB group it doesn't know about (no prior IB_join/record
>         with it). But once the group exists and the sender has a
>         packet to send, it will detect the presence and join the
>         group. From then on it will always send to that MGID. The
>         receiver will get the packets since it (or another receiver)
>         created the IB group.
> 
>         In other situation where the router determines that there are
>         external receivers it creates the group too. The sender will
>         therefore find this and join the group.
> 
>         The sender needs to continuously check for the IB group only
>         in case is where the router doesn't need to forward the packet
>         and there are no receivers in the local subnet. I don't think
>         this will be too often. Is this case of concern ?
> 
> 
> 
> Vivek
> 
> <snip>
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr 15 16:48:01 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06342
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 16:48:00 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA13212
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 16:48:04 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA12815;
	Mon, 15 Apr 2002 16:41:47 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA12753
	for <ipoverib@optimus.ietf.org>; Mon, 15 Apr 2002 16:41:44 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06195
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 16:41:36 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3FKfcLQ088908;
	Mon, 15 Apr 2002 16:41:38 -0400
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3FKfa265928;
	Mon, 15 Apr 2002 16:41:37 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g3FKfaX11064;
	Mon, 15 Apr 2002 13:41:36 -0700
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3FKfZ721910;
	Mon, 15 Apr 2002 13:41:35 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204152041.g3FKfZ721910@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: yaronh@voltaire.com
Date: Mon, 15 Apr 2002 13:41:34 -0700 (PDT)
Cc: kashyapv@us.ibm.com ('Vivek Kashyap'),
        adube@windows.microsoft.com ('Aditya Dube'), ipoverib@ietf.org
In-Reply-To: <001e01c1e4b1$79f1ab30$b4d05c8b@vaio505he> from "Yaron Haviv" at Apr 15, 2002 08:12:24 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> Vivek,
> 
> Since checking the Group any time you send a multicast packet is very costly
> We can adopt some kind of a broadcast notification message sent by the
> router, this way the node will update its caches
> 
> The process goes:
> 	Sender wants to send multicast
> 	No Group so sends to all Router Group

	OK. This is exactly what I have written.

        Note: once the group is created (see the soution in the
        previous mail) we no longer have to do anything of this type.
        The checking is only for the time the sender has not joined
        the group.

         Sender:
                 got IP multicast packet to send
                         determine the corresponding IB group
                         if already 'IB_joined' or 'recorded' ----(a)
                                 IB_send to it
 
                         else if IB group exists             -----(b)
                                 IB_join/record MGID locally
                                 IB_send to it
                         else
 
                                IB_send to all-routers MGID   -----(c)
 
        We need to evaluate if case (c) will be a common case and if
        it is will it last for long. Only then can we say it is
        costly. 

	We can modify the solution a bit though:

                Let the sender send to the all-hosts MGID if it
                doesn't have the MGID recorded or the group doesn't
                exist. Since it is sending ot all-hosts MGID the
                packet will be received by all multicast hosts
                (routers/receivers). Thus the sender can check for new
                groups periodically and not with every new packet to
                the given group.

> 	(Can optionally create the group (like Mike suggested) still sending the
> packet to all-routers without delaying the packet, since we need to first
> have the router joining this group prior to our send)

        How will the router know to join ? The reason I suggested
        using the IGMP report is since it is a direct indication of
        the group coming into being. For a sender there is no IP level
        or IB level hook to tell anyone that a group is created.

> 
> Now we need someone to tell us that the router is registered to this group
> already
> This could be done by either the router or by the SM
> 
> now there are two possible scenarios
> 1 - 	Router receives the packet and sees there is a need for a new group
> 	it creates the group, and then sends a broadcast message saying this group
> was formed
> 	(Can optionally send a unicast back to the sender)
> 	Sender updates its cache

	I take you mean:

        Router receives the IP_packet-in-all-router-MGID. The router's
        IPoIB driver determines the new MGID based on the IP_group.

        Now, the IPoIB driver of the router sends a <broadcast packet>
        - has to be at IB level - and defined by a standard. This is
        received by all IPoIB drivers and they create the group and
        add it.


	I would modify this to:

                The router's IPoIB driver creates the group. Next time
                the sender will check and find the group exists and
                join it.

        This also is workable. It asks the router's IPoIB driver to be
        a little more aggressive. The rest of the solution can stay as
        I have suggested earlier. The difference is that we are
        ensuring that the router always create a group. The negative
        is that too many groups may be created, even if one IP packet
        was sent to the group, whether one needs them or not.

Vivek


> 
> 2- 	SM sees a create/join from the router and than send a notification to
> all other nodes
> 	this enables updating the cache
> 
        If IB solution provided such updates we wouldn't need any
        IPoIB level solutions. It doesn't. There is no point in
        discussing updates to IBA in IPoIB WG; they need to be looked
        into in the IBA WGs.


> Updating PathRecord cache is probably an essential IBA requirement too when
> dealing with the unicast traffic
> It could use a similar notification mechanism
> 
> Yaron
> 
> -----Original Message-----
> From: ipoverib-admin@ietf.org [mailto:ipoverib-admin@ietf.org]On Behalf Of
> Vivek Kashyap
> Sent: Monday, April 15, 2002 8:39 PM
> To: Aditya Dube
> Cc: ipoverib@ietf.org
> Subject: Re: [Ipoverib] IPoIB multicast forwarding
> 
> >
> >
> > Vivek
> >
> > As the sender will only send packets to the all-router MGID after doing
> > the check, how will a receiver who creates the MGroup on the same subnet
> > as the sender ever receive the multicast packets that it should get?
> 
> 
>         The suggested solution is as follows. The basic idea is
>         certainly workable. Hopefully, after some discussion we can
>         refine it to cover all the issues that crop up.
> 
> 
>         Sender:
>                 got IP multicast packet to send
>                         determine the corresponding IB group
>                         if already 'IB_joined' or 'recorded'
>                                 IB_send to it
> 
>                         else if IB group exists
>                                 IB_join/record MGID locally
>                                 IB_send to it
>                         else
> 
>                                 IB_send to all-routers MGID
> 
> 
>         Receiver:
>                 want to receive IP multicast beloging to an IP group
>                         check for corresponding IB group
>                         if IB group exists
>                                 IB_join it
>                                 IB_send report to group
>                         else
>                                 IB_create IB group
>                                 IB_send report to all-routers MGID
>                                 (IB_send report to group)
> 
> 
>         Router:
>                 always receive on broadcast/all-hosts and all-routers
>                 if IGMP/MLD indicates new group
>                         IB_join corresponding IB group
>                 if routing protocol indicates new group
>                         IB_create corresponding IB group
> 
> 
>         Now, a sender must check for the IB group everytime it is sending
>         to an IB group it doesn't know about (no prior IB_join/record
>         with it). But once the group exists and the sender has a
>         packet to send, it will detect the presence and join the
>         group. From then on it will always send to that MGID. The
>         receiver will get the packets since it (or another receiver)
>         created the IB group.
> 
>         In other situation where the router determines that there are
>         external receivers it creates the group too. The sender will
>         therefore find this and join the group.
> 
>         The sender needs to continuously check for the IB group only
>         in case is where the router doesn't need to forward the packet
>         and there are no receivers in the local subnet. I don't think
>         this will be too often. Is this case of concern ?
> 
> 
> 
> Vivek
> 
> <snip>
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 15 17:23:10 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA06971
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 17:23:10 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id RAA15550
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 17:23:14 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA15334;
	Mon, 15 Apr 2002 17:17:32 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA15288
	for <ipoverib@ns.ietf.org>; Mon, 15 Apr 2002 17:17:14 -0400 (EDT)
Received: from vieo.com (vieo.com [216.30.79.131])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA06878
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 17:17:10 -0400 (EDT)
Received: (from root@localhost)
	by vieo.com (8.11.2/8.11.2) id g3FLGgH37379
	for ipoverib@ietf.org; Mon, 15 Apr 2002 16:16:42 -0500 (CDT)
	(envelope-from rronald@VIEO.com)
Received: from Fishhead (dhcp-10-1-15-187.vieo.com [10.1.15.187])
	by vieo.com (8.11.2/8.11.2) with SMTP id g3FLGeO37203;
	Mon, 15 Apr 2002 16:16:40 -0500 (CDT)
	(envelope-from rronald@VIEO.com)
Message-ID: <012501c1e4c2$c7337070$bb0f010a@Fishhead>
From: "Roger Ronald" <rronald@VIEO.com>
To: "Yaron Haviv" <yaronh@voltaire.com>,
        "'H.K. Jerry Chu'" <Jerry.Chu@eng.sun.com>, <ipoverib@ietf.org>,
        <Ajoy.Siddabathuni@Sun.COM>, "Michael Krause" <krause@cup.hp.com>
References: <5.1.0.14.2.20020415101832.02a650e0@esalpha2.cup.hp.com>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
Date: Mon, 15 Apr 2002 16:16:14 -0500
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
X-scanner: scanned by Inflex 0.1.5c+ on vieo.com
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

Let me be the first to suggest that we call Ethertype, by itself, the
ETOL (Ethertype Only Link) Header.

I never meant to suggest that we add the existing LLC/SNAP or IEEE 802.3 LE
header...just that Ethertype (by itself) in this case serves essentially the same purpose.


RR
----- Original Message -----
From: "Michael Krause" <krause@cup.hp.com>
To: "Yaron Haviv" <yaronh@voltaire.com>; "'H.K. Jerry Chu'" <Jerry.Chu@eng.sun.com>;
<ipoverib@ietf.org>; <Ajoy.Siddabathuni@Sun.COM>
Sent: Monday, April 15, 2002 12:20 PM
Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4


> At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
> >Jerry,
> >
> >We must stop expending the definition somewhere so we get the spec out
> >This is IPoIB we are talking about, there is no need for LLC/SNAP, and we
> >just need the Ethertype
>
> The consensus is that people want to have the full range of Ethertype
> values supported.  As such, this needs to be specified independent of the
> IP over IB specification so that everyone may use these values without
> requiring them to implement IP over IB.
>
> Mike
>
>
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
>
>


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 15 17:31:29 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA07203
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 17:31:29 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id RAA16247
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 17:31:33 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA15639;
	Mon, 15 Apr 2002 17:25:24 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA15606
	for <ipoverib@ns.ietf.org>; Mon, 15 Apr 2002 17:25:22 -0400 (EDT)
Received: from kathmandu.sun.com (kathmandu.sun.com [192.18.98.36])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA06993
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 17:25:17 -0400 (EDT)
Received: from phys-ha10nwka.ebay.sun.com ([129.150.140.210])
	by kathmandu.sun.com (8.9.3+Sun/8.9.3) with ESMTP id PAA17126;
	Mon, 15 Apr 2002 15:25:05 -0600 (MDT)
Received: from endeavour (endeavour [129.150.143.50])
	by phys-ha10nwka.ebay.sun.com (8.8.8+Sun/8.8.8/ENSMAIL,v2.2) with SMTP id OAA20747;
	Mon, 15 Apr 2002 14:25:04 -0700 (PDT)
Message-Id: <200204152125.OAA20747@phys-ha10nwka.ebay.sun.com>
Date: Mon, 15 Apr 2002 14:25:04 -0700 (PDT)
From: Ajoy C Siddabathuni <Ajoy.Siddabathuni@Sun.COM>
Reply-To: Ajoy C Siddabathuni <Ajoy.Siddabathuni@Sun.COM>
Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
To: yaronh@voltaire.com, Jerry.Chu@eng.sun.com, ipoverib@ietf.org,
        Ajoy.Siddabathuni@Sun.COM, krause@cup.hp.com
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: w4skXZ4TzBsB72sV/Nw/VQ==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.3.4p_5 SunOS 5.7 sun4u sparc 
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org


Most IPoverXXX protocols defined so far defined their wire formats
such that they are extensible to other protocols if need be or such
that they co-existed with other protocols sharing the same resources.
I guess the whole argument behind having an ethertype field was to use
it to multiplex disparate protocols over the same QP or for legacy purposes;
if this is really the case, then I dont see why LLC/SNAP need not be 
supported. 

Frame formats -

a) If LLC/SNAP was used then you automatically get a 32bit-word alignment
   for the IP-Datagram.
   
   The frame would look something like --
   
   LRH (8bytes)
   GRH (40 bytes)*
   BTH (12 bytes)
   DETH (8 bytes)
               LLC       OUI     EthType
   LLC/SNAP <AA-AA-03><00-00-00><xx-xx>   (8bytes)
   IPv4/IPv6 Datagram
   ICRC
   VCRC
   
b) Embedding just the  ethtype - the frame would look like -- also gives
   you the alignment - but is a non-std way of doing things (with reserved
   fields etc in the format).

   LRH (8bytes)
   GRH (40 bytes)*
   BTH (12 bytes)
   DETH (8 bytes)
   
    Rsvd   EthType
   <??-??><xx-xx>  (4bytes)
   IPv4/IPv6 Datagram
   ICRC
   VCRC
----

My opinion on this is that if EthType is ever supported, it should be
supported using the std mechanism (a) to be consistent with how other
IPoverXXX protocols have been defined in the past. (See RFCs -ATM: 2225,
1577, 1483 SMDS:1209 FR:1294)

If the WG feels that in the foreseeable future, only IP will ever be
supported, then yes there is no need for the Ethtype - ie. leave it alone,
you can still get the IP datagram 32bit aligned right after DETH. On the
other hand, if the WG feels that we do it the way our ancestors have done it
there is enough argument already in place for supporting llc/snap hdrs.

~Ajoy


>X-Sender: krause@esalpha2.cup.hp.com
>Date: Mon, 15 Apr 2002 10:20:49 -0700
>To: Yaron Haviv <yaronh@voltaire.com>, "'H.K. Jerry Chu'" 
<Jerry.Chu@eng.sun.com>, "'ipoverib@ietf.org'" <ipoverib@ietf.org>, 
"'Ajoy.Siddabathuni@Sun.COM'" <Ajoy.Siddabathuni@Sun.COM>
>From: Michael Krause <krause@cup.hp.com>
>Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
>Mime-Version: 1.0
>X-Mailman-Version: 1.0
>List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
>X-BeenThere: ipoverib@ietf.org
>
>At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
>>Jerry,
>>
>>We must stop expending the definition somewhere so we get the spec out
>>This is IPoIB we are talking about, there is no need for LLC/SNAP, and we
>>just need the Ethertype
>
>The consensus is that people want to have the full range of Ethertype 
>values supported.  As such, this needs to be specified independent of the 
>IP over IB specification so that everyone may use these values without 
>requiring them to implement IP over IB.
>
>Mike
>
>
>_______________________________________________
>IPoverIB mailing list
>IPoverIB@ietf.org
>https://www1.ietf.org/mailman/listinfo/ipoverib


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 15 17:50:22 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA07486
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 17:50:22 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id RAA16744
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 17:50:26 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA16662;
	Mon, 15 Apr 2002 17:46:15 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA16634
	for <ipoverib@ns.ietf.org>; Mon, 15 Apr 2002 17:46:09 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA07440
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 17:46:04 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3FLk6LQ044758;
	Mon, 15 Apr 2002 17:46:06 -0400
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3FLk5271642;
	Mon, 15 Apr 2002 17:46:05 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g3FLk4X12416;
	Mon, 15 Apr 2002 14:46:04 -0700
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3FLjtP25570;
	Mon, 15 Apr 2002 14:45:55 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204152145.g3FLjtP25570@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
To: rronald@VIEO.com (Roger Ronald)
Date: Mon, 15 Apr 2002 14:45:54 -0700 (PDT)
Cc: yaronh@voltaire.com (Yaron Haviv),
        Jerry.Chu@eng.sun.com ('H.K. Jerry Chu'), ipoverib@ietf.org,
        Ajoy.Siddabathuni@sun.com, krause@cup.hp.com (Michael Krause)
In-Reply-To: <012501c1e4c2$c7337070$bb0f010a@Fishhead> from "Roger Ronald" at Apr 15, 2002 03:16:14 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

I agree. LLC/SNAP doesn't provide any useful information that is not
available by a type field. And that is all we need.


Vivek


> 
> Let me be the first to suggest that we call Ethertype, by itself, the
> ETOL (Ethertype Only Link) Header.
> 
> I never meant to suggest that we add the existing LLC/SNAP or IEEE 802.3 LE
> header...just that Ethertype (by itself) in this case serves essentially the same purpose.
> 
> 
> RR
> ----- Original Message -----
> From: "Michael Krause" <krause@cup.hp.com>
> To: "Yaron Haviv" <yaronh@voltaire.com>; "'H.K. Jerry Chu'" <Jerry.Chu@eng.sun.com>;
> <ipoverib@ietf.org>; <Ajoy.Siddabathuni@Sun.COM>
> Sent: Monday, April 15, 2002 12:20 PM
> Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
> 
> 
> > At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
> > >Jerry,
> > >
> > >We must stop expending the definition somewhere so we get the spec out
> > >This is IPoIB we are talking about, there is no need for LLC/SNAP, and we
> > >just need the Ethertype
> >
> > The consensus is that people want to have the full range of Ethertype
> > values supported.  As such, this needs to be specified independent of the
> > IP over IB specification so that everyone may use these values without
> > requiring them to implement IP over IB.
> >
> > Mike
> >
> >
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> >
> >
> 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 15 18:20:40 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA08057
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 18:20:40 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id SAA18019
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 18:20:45 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA17909;
	Mon, 15 Apr 2002 18:14:43 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA17873
	for <ipoverib@ns.ietf.org>; Mon, 15 Apr 2002 18:14:40 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA08013
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 18:14:35 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3FMC4LQ086602;
	Mon, 15 Apr 2002 18:12:04 -0400
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3FMBx271204;
	Mon, 15 Apr 2002 18:12:00 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g3FMBuX12894;
	Mon, 15 Apr 2002 15:11:56 -0700
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3FMBu926014;
	Mon, 15 Apr 2002 15:11:56 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204152211.g3FMBu926014@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
To: Ajoy.Siddabathuni@sun.com
Date: Mon, 15 Apr 2002 15:11:54 -0700 (PDT)
Cc: yaronh@voltaire.com, Jerry.Chu@eng.sun.com, ipoverib@ietf.org,
        krause@cup.hp.com
In-Reply-To: <200204152125.OAA20747@phys-ha10nwka.ebay.sun.com> from "Ajoy C Siddabathuni" at Apr 15, 2002 01:25:04 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> 
> Most IPoverXXX protocols defined so far defined their wire formats
> such that they are extensible to other protocols if need be or such
> that they co-existed with other protocols sharing the same resources.
> I guess the whole argument behind having an ethertype field was to use
> it to multiplex disparate protocols over the same QP or for legacy purposes;
> if this is really the case, then I dont see why LLC/SNAP need not be 
> supported. 

It takes another 4 bytes and provides no new information. The issues
you indicate above are equally addressed by a 'type' field.

We can get alignment at a 32-bit word by using a) no header b) 4 byte
header c) 8 byte header d) a 4*n, n=3.... header.

'No header case' is disallowed as per WG consensus. Otherwise, the
4-byte ethertype field is equally viable as the 8 byte LLC/SNAP field.
It has the advantage of being 4 bytes smaller.

> 
> Frame formats -
> 
> a) If LLC/SNAP was used then you automatically get a 32bit-word alignment
>    for the IP-Datagram.
>    
>    The frame would look something like --
>    
>    LRH (8bytes)
>    GRH (40 bytes)*
>    BTH (12 bytes)
>    DETH (8 bytes)
>                LLC       OUI     EthType
>    LLC/SNAP <AA-AA-03><00-00-00><xx-xx>   (8bytes)
>    IPv4/IPv6 Datagram
>    ICRC
>    VCRC
>    
> b) Embedding just the  ethtype - the frame would look like -- also gives
>    you the alignment - but is a non-std way of doing things (with reserved
>    fields etc in the format).
> 
>    LRH (8bytes)
>    GRH (40 bytes)*
>    BTH (12 bytes)
>    DETH (8 bytes)
>    
>     Rsvd   EthType
>    <??-??><xx-xx>  (4bytes)
>    IPv4/IPv6 Datagram
>    ICRC
>    VCRC
> ----
> 
> My opinion on this is that if EthType is ever supported, it should be
> supported using the std mechanism (a) to be consistent with how other
> IPoverXXX protocols have been defined in the past. (See RFCs -ATM: 2225,
> 1577, 1483 SMDS:1209 FR:1294)

How about RFC2734 ? There is nothing standard about this.

[ As another example one can even use a 2 byte header if it works out
for the hardware -i.e. if one can DMA to a 2 byte boundary, the IP
header will land at a 4 byte boundary. ]

Vivek

> 
> If the WG feels that in the foreseeable future, only IP will ever be
> supported, then yes there is no need for the Ethtype - ie. leave it alone,
> you can still get the IP datagram 32bit aligned right after DETH. On the
> other hand, if the WG feels that we do it the way our ancestors have done it
> there is enough argument already in place for supporting llc/snap hdrs.
> 
> ~Ajoy
> 
> 
> >X-Sender: krause@esalpha2.cup.hp.com
> >Date: Mon, 15 Apr 2002 10:20:49 -0700
> >To: Yaron Haviv <yaronh@voltaire.com>, "'H.K. Jerry Chu'" 
> <Jerry.Chu@eng.sun.com>, "'ipoverib@ietf.org'" <ipoverib@ietf.org>, 
> "'Ajoy.Siddabathuni@Sun.COM'" <Ajoy.Siddabathuni@Sun.COM>
> >From: Michael Krause <krause@cup.hp.com>
> >Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
> >Mime-Version: 1.0
> >X-Mailman-Version: 1.0
> >List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
> >X-BeenThere: ipoverib@ietf.org
> >
> >At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
> >>Jerry,
> >>
> >>We must stop expending the definition somewhere so we get the spec out
> >>This is IPoIB we are talking about, there is no need for LLC/SNAP, and we
> >>just need the Ethertype
> >
> >The consensus is that people want to have the full range of Ethertype 
> >values supported.  As such, this needs to be specified independent of the 
> >IP over IB specification so that everyone may use these values without 
> >requiring them to implement IP over IB.
> >
> >Mike
> >
> >
> >_______________________________________________
> >IPoverIB mailing list
> >IPoverIB@ietf.org
> >https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 15 18:36:15 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA08560
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 18:36:11 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id SAA19682
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 18:36:16 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA18842;
	Mon, 15 Apr 2002 18:27:39 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA18807
	for <ipoverib@ns.ietf.org>; Mon, 15 Apr 2002 18:27:36 -0400 (EDT)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA08225
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 18:27:08 -0400 (EDT)
Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.39 2002/04/15 17:47:23 root Exp $) with ESMTP id g3FMQUk06936
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 22:26:30 GMT
Received: from fmsmsxvs040.fm.intel.com (fmsmsxv040-1.fm.intel.com [132.233.48.108])
	by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.16 2002/04/15 17:47:00 root Exp $) with SMTP id g3FMSsw11675
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 22:28:54 GMT
Received: from fmsmsx28.fm.intel.com ([132.233.42.28])
 by fmsmsxvs040.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002041515262413641
 ; Mon, 15 Apr 2002 15:26:24 -0700
Received: by fmsmsx28.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2KDV7BKD>; Mon, 15 Apr 2002 15:27:00 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F8B@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Vivek Kashyap'" <kashyapv@us.ibm.com>, adube@windows.microsoft.com
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Mon, 15 Apr 2002 15:26:54 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

> From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> Sent: Monday, April 15, 2002 1:25 PM
> To: adube@windows.microsoft.com
> Cc: kashyapv@us.ibm.com; ipoverib@ietf.org
> Subject: Re: [Ipoverib] IPoIB multicast forwarding
> 
> 
> 
> > 
> > Hi Vivek,
> > 
> > After reading through your response, I realized that there 
> is one more
> > case to consider: 
> > 
> > Sender does a check for IB Mcast Group...check succeeds because a
> > receiver has created the group.
> > 
> > Now, the receiver decides to leave the group and the IB MGroup gets
> > deleted.
> > 
> > The Sender needs detect this and revert back to sending the 
> multicast
> > packets to the All-router Address.
> > 
> > I think in the scheme we are discussing, the sender will 
> always have to
> > check for the existence of the IB Group just because of the 
> edge cases.
> 
>         I have made some assumptions on IB deletion. Let us assume
>         that the sender has joined the group (I am in the group that
>         believes for IB to work right all have to join - no-join-send
>         does not work universally).
> 
>         The receiver will not delete the group just leave. The
>         sender's 'check' really is 'join'. The sender is a member of
>         the group - the group cannot really be deleted until all
>         members leave. The sender can age its entries of the groups it
>         has joined and after certain idle time 'leave' the group.
> 
>         Also, if the receiver goes through the process, it will first
>         have sent a report to the router. THe router is also a member
>         of the group.
> 
>         We can also have it that the last guy leaving the group delete
>         it. IF someone joins it just as we are deleting it then the
>         deletion should fail if the membership count is greater than
>         one.

Vivek,

What mechanism do you propose to use that will let a node know, when it
leaves, that it is the last one in the group and thus responsible for
deleting it?

Roy

> 
>         So, if the sender has joined the group, I don't think that it
>         will have to check the group everytime it sends a packet.
> 
> 
> Vivek
> 
> > 
> > Aditya Dube
> > 
> > -----Original Message-----
> > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
> > Sent: Monday, April 15, 2002 11:39 AM
> > To: Aditya Dube
> > Cc: ipoverib@ietf.org
> > Subject: Re: [Ipoverib] IPoIB multicast forwarding
> > 
> > > 
> > > 
> > > Vivek
> > > 
> > > As the sender will only send packets to the all-router MGID after
> > doing
> > > the check, how will a receiver who creates the MGroup on the same
> > subnet
> > > as the sender ever receive the multicast packets that it 
> should get?
> > 
> > 
> >         The suggested solution is as follows. The basic idea is
> >         certainly workable. Hopefully, after some discussion we can
> >         refine it to cover all the issues that crop up.
> > 
> > 
> > 	Sender:
> > 		got IP multicast packet to send
> > 			determine the corresponding IB group
> > 			if already 'IB_joined' or 'recorded'
> > 				IB_send to it
> > 
> > 			else if IB group exists
> > 				IB_join/record MGID locally
> > 				IB_send to it
> > 			else
> > 				
> > 				IB_send to all-routers MGID
> > 
> > 	
> > 	Receiver:
> > 		want to receive IP multicast beloging to an IP group 
> > 			check for corresponding IB group
> > 			if IB group exists 
> > 				IB_join it
> > 				IB_send report to group
> > 			else
> > 				IB_create IB group
> > 				IB_send report to all-routers MGID
> > 				(IB_send report to group)
> > 
> > 	
> > 	Router:
> > 		always receive on broadcast/all-hosts and all-routers
> > 		if IGMP/MLD indicates new group
> > 			IB_join corresponding IB group
> > 		if routing protocol indicates new group
> > 			IB_create corresponding IB group 
> > 
> > 
> >         Now, a sender must check for the IB group everytime it is
> > sending
> >         to an IB group it doesn't know about (no prior 
> IB_join/record
> >         with it). But once the group exists and the sender has a
> >         packet to send, it will detect the presence and join the
> >         group. From then on it will always send to that MGID. The
> >         receiver will get the packets since it (or another receiver)
> >         created the IB group.
> > 
> >         In other situation where the router determines that 
> there are
> >         external receivers it creates the group too. The sender will
> >         therefore find this and join the group.
> > 
> >         The sender needs to continuously check for the IB group only
> >         in case is where the router doesn't need to forward 
> the packet
> >         and there are no receivers in the local subnet. I 
> don't think
> >         this will be too often. Is this case of concern ?
> > 
> > 
> > 
> > Vivek
> > 
> > <snip>
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 15 18:57:10 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA09052
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 18:57:10 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id SAA20243
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 18:57:13 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA20198;
	Mon, 15 Apr 2002 18:53:27 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA20169
	for <ipoverib@ns.ietf.org>; Mon, 15 Apr 2002 18:53:25 -0400 (EDT)
Received: from nwkea-mail-1.sun.com (nwkea-mail-1.sun.com [192.18.42.13])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA09023
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 18:53:22 -0400 (EDT)
Received: from phys-ha10nwka.ebay.sun.com ([129.150.144.210])
	by nwkea-mail-1.sun.com (8.9.3+Sun/8.9.3) with ESMTP id PAA01940;
	Mon, 15 Apr 2002 15:52:42 -0700 (PDT)
Received: from endeavour (endeavour [129.150.143.50])
	by phys-ha10nwka.ebay.sun.com (8.8.8+Sun/8.8.8/ENSMAIL,v2.2) with SMTP id PAA12151;
	Mon, 15 Apr 2002 15:52:41 -0700 (PDT)
Message-Id: <200204152252.PAA12151@phys-ha10nwka.ebay.sun.com>
Date: Mon, 15 Apr 2002 15:52:39 -0700 (PDT)
From: Ajoy C Siddabathuni <Ajoy.Siddabathuni@sun.com>
Reply-To: Ajoy C Siddabathuni <Ajoy.Siddabathuni@sun.com>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
To: Ajoy.Siddabathuni@sun.com, kashyapv@us.ibm.com
Cc: yaronh@voltaire.com, Jerry.Chu@eng.sun.com, ipoverib@ietf.org,
        krause@cup.hp.com
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: kvkYLCe+4hY8QGvWuXNDrA==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.3.4p_5 SunOS 5.7 sun4u sparc 
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

 

>It takes another 4 bytes and provides no new information. The issues
>you indicate above are equally addressed by a 'type' field.

Ofcourse it provides no new information - if you are running IPoIB
in vacum. It is much cleaner than putting a 4 byte header with reserved
fields. Also expect (in a few years) that there will be multi-protocol 
routers that will be sitting at the edges of an IB-Ethernet-osi-apple-network.
They would definetly like to handle packets in a std way - especially when
cp'ing pkts from one kind of "link" to another. They dont need to handle
cases such as - "oh by the way, if this link was an IPoIB link..I needed to
strip the top 4bytes...and not interpret the next 16 bits before I look at
the eth-type".

>
>'No header case' is disallowed as per WG consensus. Otherwise, the
>4-byte ethertype field is equally viable as the 8 byte LLC/SNAP field.
>It has the advantage of being 4 bytes smaller.

I would'nt consider "4 bytes smaller" as an advantage. Especially if you are
working with IB header lengths upto 68 bytes. And then if you are working with
TCP/IPv4/v6 add upto 60+.

>
>How about RFC2734 ? There is nothing standard about this.
Well - time to learn from another's mistakes.

Here is my summary -

 - Having no-header at all will give you alignment. No ethtype to mux on.
   But you could have always figured out all IP'ish types from the IP-hdrtype
   field.
 
 - Having 4byte "non-std" header will also give you alignment. Gives you
   all the info you need. But yes, there are certain non-std elements in
   the header that implementations would have to be aware of  - especially
   if they were porting an existing stack.
   
 - Having 8byte std-header will be standards conforming for future multi-prot-
   col enviroments. Ofcourse, retains alignment.

~Ajoy


>> 
>> Frame formats -
>> 
>> a) If LLC/SNAP was used then you automatically get a 32bit-word alignment
>>    for the IP-Datagram.
>>    
>>    The frame would look something like --
>>    
>>    LRH (8bytes)
>>    GRH (40 bytes)*
>>    BTH (12 bytes)
>>    DETH (8 bytes)
>>                LLC       OUI     EthType
>>    LLC/SNAP <AA-AA-03><00-00-00><xx-xx>   (8bytes)
>>    IPv4/IPv6 Datagram
>>    ICRC
>>    VCRC
>>    
>> b) Embedding just the  ethtype - the frame would look like -- also gives
>>    you the alignment - but is a non-std way of doing things (with reserved
>>    fields etc in the format).
>> 
>>    LRH (8bytes)
>>    GRH (40 bytes)*
>>    BTH (12 bytes)
>>    DETH (8 bytes)
>>    
>>     Rsvd   EthType
>>    <??-??><xx-xx>  (4bytes)
>>    IPv4/IPv6 Datagram
>>    ICRC
>>    VCRC
>> ----
>> 
>> My opinion on this is that if EthType is ever supported, it should be
>> supported using the std mechanism (a) to be consistent with how other
>> IPoverXXX protocols have been defined in the past. (See RFCs -ATM: 2225,
>> 1577, 1483 SMDS:1209 FR:1294)
>
>How about RFC2734 ? There is nothing standard about this.
>
>[ As another example one can even use a 2 byte header if it works out
>for the hardware -i.e. if one can DMA to a 2 byte boundary, the IP
>header will land at a 4 byte boundary. ]
>
>Vivek
>
>> 
>> If the WG feels that in the foreseeable future, only IP will ever be
>> supported, then yes there is no need for the Ethtype - ie. leave it alone,
>> you can still get the IP datagram 32bit aligned right after DETH. On the
>> other hand, if the WG feels that we do it the way our ancestors have done it
>> there is enough argument already in place for supporting llc/snap hdrs.
>> 
>> ~Ajoy
>> 
>> 
>> >X-Sender: krause@esalpha2.cup.hp.com
>> >Date: Mon, 15 Apr 2002 10:20:49 -0700
>> >To: Yaron Haviv <yaronh@voltaire.com>, "'H.K. Jerry Chu'" 
>> <Jerry.Chu@eng.sun.com>, "'ipoverib@ietf.org'" <ipoverib@ietf.org>, 
>> "'Ajoy.Siddabathuni@Sun.COM'" <Ajoy.Siddabathuni@Sun.COM>
>> >From: Michael Krause <krause@cup.hp.com>
>> >Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
>> >Mime-Version: 1.0
>> >X-Mailman-Version: 1.0
>> >List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
>> >X-BeenThere: ipoverib@ietf.org
>> >
>> >At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
>> >>Jerry,
>> >>
>> >>We must stop expending the definition somewhere so we get the spec out
>> >>This is IPoIB we are talking about, there is no need for LLC/SNAP, and we
>> >>just need the Ethertype
>> >
>> >The consensus is that people want to have the full range of Ethertype 
>> >values supported.  As such, this needs to be specified independent of the 
>> >IP over IB specification so that everyone may use these values without 
>> >requiring them to implement IP over IB.
>> >
>> >Mike
>> >
>> >
>> >_______________________________________________
>> >IPoverIB mailing list
>> >IPoverIB@ietf.org
>> >https://www1.ietf.org/mailman/listinfo/ipoverib
>> 
>> 
>> _______________________________________________
>> IPoverIB mailing list
>> IPoverIB@ietf.org
>> https://www1.ietf.org/mailman/listinfo/ipoverib
>> 
>
>
>-- 
>Vivek Kashyap
>Linux Technology Center, IBM
>kashyapv@us.ibm.com 
>vivk@us.ibm.com 
>503 578 3422 (o)


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 15 19:28:34 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA09415
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 19:28:34 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA21325
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 19:28:34 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA21266;
	Mon, 15 Apr 2002 19:23:51 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA21232
	for <ipoverib@ns.ietf.org>; Mon, 15 Apr 2002 19:23:49 -0400 (EDT)
Received: from palrel13.hp.com (palrel13.hp.com [156.153.255.238])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA09356
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 19:23:45 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel13.hp.com (Postfix) with ESMTP
	id B8C7C400723; Mon, 15 Apr 2002 16:23:16 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id QAA08139;
	Mon, 15 Apr 2002 16:23:16 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020415160750.02a73a10@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Mon, 15 Apr 2002 16:22:50 -0700
To: Ajoy C Siddabathuni <Ajoy.Siddabathuni@sun.com>
From: Michael Krause <krause@cup.hp.com>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
Cc: ipoverib@ietf.org
In-Reply-To: <200204152252.PAA12151@phys-ha10nwka.ebay.sun.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 03:52 PM 4/15/2002 -0700, Ajoy C Siddabathuni wrote:
>
>
> >It takes another 4 bytes and provides no new information. The issues
> >you indicate above are equally addressed by a 'type' field.
>
>Ofcourse it provides no new information - if you are running IPoIB
>in vacum. It is much cleaner than putting a 4 byte header with reserved
>fields. Also expect (in a few years) that there will be multi-protocol
>routers that will be sitting at the edges of an IB-Ethernet-osi-apple-network.
>They would definetly like to handle packets in a std way - especially when
>cp'ing pkts from one kind of "link" to another. They dont need to handle
>cases such as - "oh by the way, if this link was an IPoIB link..I needed to
>strip the top 4bytes...and not interpret the next 16 bits before I look at
>the eth-type".

Amazing to see the charter of this workgroup be so permuted to solve all 
multi-protocol problems that we now should be encumbered with even more 
overhead.  Small packet protocol efficiency is absurd when you consider 
there is now at least 2-3x protocol overhead just to move a TCP SYN/FIN/ACK 
packet through the fabric.

> >'No header case' is disallowed as per WG consensus. Otherwise, the
> >4-byte ethertype field is equally viable as the 8 byte LLC/SNAP field.
> >It has the advantage of being 4 bytes smaller.
>
>I would'nt consider "4 bytes smaller" as an advantage. Especially if you are
>working with IB header lengths upto 68 bytes. And then if you are working with
>TCP/IPv4/v6 add upto 60+.

This translates into 1-2 cache lines of overhead per packet moved through 
the fabric (function of the cache line size). In general, we'd like to not 
touch cache lines whenever possible since this impacts performance.

> >How about RFC2734 ? There is nothing standard about this.
>Well - time to learn from another's mistakes.
>
>Here is my summary -
>
>  - Having no-header at all will give you alignment. No ethtype to mux on.
>    But you could have always figured out all IP'ish types from the IP-hdrtype
>    field.
>
>  - Having 4byte "non-std" header will also give you alignment. Gives you
>    all the info you need. But yes, there are certain non-std elements in
>    the header that implementations would have to be aware of  - especially
>    if they were porting an existing stack.
>
>  - Having 8byte std-header will be standards conforming for future 
> multi-prot-
>    col enviroments. Ofcourse, retains alignment.

Continuing along this line of specification development makes one wonder 
how much performance people were expecting to get out of their solutions?

Going back to the original BOF when someone questioned why don't you just 
add in Ethernet TCAs and simplify the entire IP router problem by running 
something like RNDIS or an equivalent open solution and many noted this was 
a reasonable approach.  Why that same approach cannot be used now using a 
UD instead of a connected transport service to solve the perceived 
scalability problem is no longer viewed as reasonable makes little 
sense.  Anyone added up the cost of building / deploying these 
multiprotocol routers to determine whether all of this added complexity 
(IMO, unnecessary especially w.r.t. multicast and multi-protocol approaches 
to date) provides a reasonable cost / benefit trade-off?

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 15 19:36:36 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA09485
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 19:36:36 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA22046
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 19:36:39 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA21977;
	Mon, 15 Apr 2002 19:31:46 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA21918
	for <ipoverib@ns.ietf.org>; Mon, 15 Apr 2002 19:31:42 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA09469
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 19:31:38 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3FNVeLQ120732;
	Mon, 15 Apr 2002 19:31:40 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3FNVd2165348;
	Mon, 15 Apr 2002 19:31:39 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g3FNVaM00946;
	Mon, 15 Apr 2002 16:31:36 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3FNVYc27721;
	Mon, 15 Apr 2002 16:31:34 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204152331.g3FNVYc27721@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: roy.k.larsen@intel.com (Larsen, Roy K)
Date: Mon, 15 Apr 2002 16:31:33 -0700 (PDT)
Cc: kashyapv@us.ibm.com ('Vivek Kashyap'), adube@windows.microsoft.com,
        ipoverib@ietf.org
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4F8B@ORSMSX108> from "Larsen, Roy K" at Apr 15, 2002 02:26:54 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
> > Sent: Monday, April 15, 2002 1:25 PM
> > To: adube@windows.microsoft.com
> > Cc: kashyapv@us.ibm.com; ipoverib@ietf.org
> > Subject: Re: [Ipoverib] IPoIB multicast forwarding
> > 
> > 
> > 
> > > 
> > > Hi Vivek,
> > > 
> > > After reading through your response, I realized that there 
> > is one more
> > > case to consider: 
> > > 
> > > Sender does a check for IB Mcast Group...check succeeds because a
> > > receiver has created the group.
> > > 
> > > Now, the receiver decides to leave the group and the IB MGroup gets
> > > deleted.
> > > 
> > > The Sender needs detect this and revert back to sending the 
> > multicast
> > > packets to the All-router Address.
> > > 
> > > I think in the scheme we are discussing, the sender will 
> > always have to
> > > check for the existence of the IB Group just because of the 
> > edge cases.
> > 
> >         I have made some assumptions on IB deletion. Let us assume
> >         that the sender has joined the group (I am in the group that
> >         believes for IB to work right all have to join - no-join-send
> >         does not work universally).
> > 
> >         The receiver will not delete the group just leave. The
> >         sender's 'check' really is 'join'. The sender is a member of
> >         the group - the group cannot really be deleted until all
> >         members leave. The sender can age its entries of the groups it
> >         has joined and after certain idle time 'leave' the group.
> > 
> >         Also, if the receiver goes through the process, it will first
> >         have sent a report to the router. THe router is also a member
> >         of the group.
> > 
> >         We can also have it that the last guy leaving the group delete
> >         it. IF someone joins it just as we are deleting it then the
> >         deletion should fail if the membership count is greater than
> >         one.
> 
> Vivek,
> 
> What mechanism do you propose to use that will let a node know, when it
> leaves, that it is the last one in the group and thus responsible for
> deleting it?

        There are a few possible solutions; all requiring IB
        clarifications. Any alternate solutions are welcome.

	a) No delete is done by the IPoIB interfaces, it is an SM issue

        This I took from the IB architecture document in chapter 3. It
        talks of the SM deleting unused groups. ( Now, there are a few
        who say that this is not going to be the case -- in which case
        a different solution is needed.)

        The receiver leaves when it no longer wants to receive. The
        router when there are no more receivers. The sender after some
        idle time.


        b) Any entity wanting to leave an IB group attempts to delete
        it. If there are members in the group then the deletion should
        fail. The last one will automatically delete the group.

	[Depends on delete semantics]


        c) Leave is fine. No need to bother about delete.
	
        let the group entries stay around in the database. They don't
        bother anyone. The admin can periodically, once in a few
        months may be, delete unused ones. The entries will get reused
        if others join. The multicast tree is affected by joins and
        leaves and not by deletes.

	d) {Don't know about this }

        The setup is such that if the last member leaves the group the
        group is automatically removed.

Vivek

> 
> Roy
> 
> > 
> >         So, if the sender has joined the group, I don't think that it
> >         will have to check the group everytime it sends a packet.
> > 
> > 
> > Vivek
> > 
> > > 
> > > Aditya Dube
> > > 
> > > -----Original Message-----
> > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
> > > Sent: Monday, April 15, 2002 11:39 AM
> > > To: Aditya Dube
> > > Cc: ipoverib@ietf.org
> > > Subject: Re: [Ipoverib] IPoIB multicast forwarding
> > > 
> > > > 
> > > > 
> > > > Vivek
> > > > 
> > > > As the sender will only send packets to the all-router MGID after
> > > doing
> > > > the check, how will a receiver who creates the MGroup on the same
> > > subnet
> > > > as the sender ever receive the multicast packets that it 
> > should get?
> > > 
> > > 
> > >         The suggested solution is as follows. The basic idea is
> > >         certainly workable. Hopefully, after some discussion we can
> > >         refine it to cover all the issues that crop up.
> > > 
> > > 
> > > 	Sender:
> > > 		got IP multicast packet to send
> > > 			determine the corresponding IB group
> > > 			if already 'IB_joined' or 'recorded'
> > > 				IB_send to it
> > > 
> > > 			else if IB group exists
> > > 				IB_join/record MGID locally
> > > 				IB_send to it
> > > 			else
> > > 				
> > > 				IB_send to all-routers MGID
> > > 
> > > 	
> > > 	Receiver:
> > > 		want to receive IP multicast beloging to an IP group 
> > > 			check for corresponding IB group
> > > 			if IB group exists 
> > > 				IB_join it
> > > 				IB_send report to group
> > > 			else
> > > 				IB_create IB group
> > > 				IB_send report to all-routers MGID
> > > 				(IB_send report to group)
> > > 
> > > 	
> > > 	Router:
> > > 		always receive on broadcast/all-hosts and all-routers
> > > 		if IGMP/MLD indicates new group
> > > 			IB_join corresponding IB group
> > > 		if routing protocol indicates new group
> > > 			IB_create corresponding IB group 
> > > 
> > > 
> > >         Now, a sender must check for the IB group everytime it is
> > > sending
> > >         to an IB group it doesn't know about (no prior 
> > IB_join/record
> > >         with it). But once the group exists and the sender has a
> > >         packet to send, it will detect the presence and join the
> > >         group. From then on it will always send to that MGID. The
> > >         receiver will get the packets since it (or another receiver)
> > >         created the IB group.
> > > 
> > >         In other situation where the router determines that 
> > there are
> > >         external receivers it creates the group too. The sender will
> > >         therefore find this and join the group.
> > > 
> > >         The sender needs to continuously check for the IB group only
> > >         in case is where the router doesn't need to forward 
> > the packet
> > >         and there are no receivers in the local subnet. I 
> > don't think
> > >         this will be too often. Is this case of concern ?
> > > 
> > > 
> > > 
> > > Vivek
> > > 
> > > <snip>
> > > 
> > 
> > 
> > -- 
> > Vivek Kashyap
> > Linux Technology Center, IBM
> > kashyapv@us.ibm.com 
> > vivk@us.ibm.com 
> > 503 578 3422 (o)
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 15 21:02:25 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA10363
	for <ipoverib-archive@odin.ietf.org>; Mon, 15 Apr 2002 21:02:25 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id VAA25655
	for ipoverib-archive@odin.ietf.org; Mon, 15 Apr 2002 21:02:27 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id UAA24994;
	Mon, 15 Apr 2002 20:58:15 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id UAA24961
	for <ipoverib@ns.ietf.org>; Mon, 15 Apr 2002 20:58:13 -0400 (EDT)
Received: from palrel13.hp.com (palrel13.hp.com [156.153.255.238])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA10298
	for <ipoverib@ietf.org>; Mon, 15 Apr 2002 20:58:10 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel13.hp.com (Postfix) with ESMTP
	id 12BDF400569; Mon, 15 Apr 2002 17:57:40 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id RAA24043;
	Mon, 15 Apr 2002 17:57:40 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020415174755.02b15630@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Mon, 15 Apr 2002 17:57:34 -0700
To: Vivek Kashyap <kashyapv@us.ibm.com>
From: Michael Krause <krause@cup.hp.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
Cc: adube@windows.microsoft.com, ipoverib@ietf.org
In-Reply-To: <200204152331.g3FNVYc27721@eng4.beaverton.ibm.com>
References: <D9223EB959A5D511A98F00508B68C20C06BC4F8B@ORSMSX108>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 04:31 PM 4/15/2002 -0700, Vivek Kashyap wrote:
>         a) No delete is done by the IPoIB interfaces, it is an SM issue
>
>         This I took from the IB architecture document in chapter 3. It
>         talks of the SM deleting unused groups. ( Now, there are a few
>         who say that this is not going to be the case -- in which case
>         a different solution is needed.)
>
>         The receiver leaves when it no longer wants to receive. The
>         router when there are no more receivers. The sender after some
>         idle time.

The specification states that a multicast management entity not the SM is 
responsible for multicast group management.  This entity implement IB 
multicast group management and could also be responsible for resolving IP 
multicast addressing to IB multicast groups.  It may also be responsible 
for informing / updating participating IB routers of any changes (and vice 
versa for the IB routers informing the entity of changes within its view of 
the router cloud).  This entity could also be used to provide a similar 
service for CAs that act as multi-protocol router endnodes.

>         b) Any entity wanting to leave an IB group attempts to delete
>         it. If there are members in the group then the deletion should
>         fail. The last one will automatically delete the group.
>
>         [Depends on delete semantics]

If an endnode wants to leave an IB multicast group, it informs the 
management entity of this event and the management entity working with the 
SM removes this endnode from the IB multicast tree.  If the entity 
determines that there are no longer any endnodes participating in the 
multicast group, it may delete the entire group from the fabric.  This does 
not require the endnode to delete the group and deal with any failure 
semantics.  An endnode should simply delete itself and let the management 
entity deal with the problem.

>         c) Leave is fine. No need to bother about delete.
>
>         let the group entries stay around in the database. They don't
>         bother anyone. The admin can periodically, once in a few
>         months may be, delete unused ones. The entries will get reused
>         if others join. The multicast tree is affected by joins and
>         leaves and not by deletes.

Delete should be performed automatically by the management entity; this is 
a policy decision that has no bearing on the IP over IB specification.

>         d) {Don't know about this }
>
>         The setup is such that if the last member leaves the group the
>         group is automatically removed.

This is how IB specifies it should work.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Tue Apr 16 04:33:39 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA26407
	for <ipoverib-archive@odin.ietf.org>; Tue, 16 Apr 2002 04:33:39 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id EAA28109
	for ipoverib-archive@odin.ietf.org; Tue, 16 Apr 2002 04:33:41 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id EAA27423;
	Tue, 16 Apr 2002 04:26:06 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id EAA27388
	for <ipoverib@ns.ietf.org>; Tue, 16 Apr 2002 04:26:03 -0400 (EDT)
Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA26334
	for <ipoverib@ietf.org>; Tue, 16 Apr 2002 04:26:00 -0400 (EDT)
Received: from westrelay01.boulder.ibm.com (westrelay01.boulder.ibm.com [9.17.194.22])
	by e31.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3G8NQKB034146;
	Tue, 16 Apr 2002 04:23:26 -0400
Received: from d03nm036.boulder.ibm.com (avpilot.boulder.ibm.com [9.17.203.135])
	by westrelay01.boulder.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3G8NBt38986;
	Tue, 16 Apr 2002 02:23:11 -0600
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
To: Ajoy C Siddabathuni <Ajoy.Siddabathuni@sun.com>
Cc: Ajoy.Siddabathuni@sun.com, ipoverib@ietf.org, ipoverib-admin@ietf.org,
        Jerry.Chu@eng.sun.com, krause@cup.hp.com, viv@beaverton.ibm.com,
        yaronh@voltaire.com
X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000
Message-ID: <OFFDD5EB99.C7171EBB-ON88256B9D.002DA959@boulder.ibm.com>
From: "Vivek Kashyap" <vivk@us.ibm.com>
Date: Tue, 16 Apr 2002 01:21:16 -0700
X-MIMETrack: Serialize by Router on D03NM036/03/M/IBM(Release 5.0.9a |January 7, 2002) at
 04/16/2002 02:23:11 AM
MIME-Version: 1.0
Content-type: text/plain; charset=us-ascii
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org


To me the reasoning is very simple.

 The intelligence we need is the protocol type of the data stream.

No header: we get the intelligence from the QPN and/or from the first 4
bits of the payload.  We get the necessary intelligence without adding any
additional signal. I would say this is the best option.

2 byte header: Signal is 2 bytes. Intelligence is carried by both the
bytes. If all DMA or system architecture can be considered to be able to
copy the payload to a  4*n+2 byte boundary then this would be the next best
option.

4 byte header:Signal is 4 bytes but intelligence is only in 2
bytes(ethertype). Only half of the signal provides any information.

LLC/SNAP: Signal is 8 bytes but intelligence is only in 2 bytes. So we get
25% return. We have added another 4 bytes with no increase in intelligence.

Hence a '4-byte ethertype' is better than the 8 byte LLC/SNAP.

Also, I don't think you are right in dismissing non-LLC/SNAP use. LLC/SNAP
is not the favored mode in general - just look at 802.3 vs. DIX ethernet.

Vivek
--
Vivek Kashyap
Linux Technology Center, IBM
vivk@us.ibm.com
kashyapv@us.ibm.com
Ph: 503 578 3422 T/L: 775 3422



                                                                                                                                   
                      Ajoy C                                                                                                       
                      Siddabathuni             To:       Ajoy.Siddabathuni@sun.com, viv@beaverton.ibm.com                          
                      <Ajoy.Siddabathun        cc:       yaronh@voltaire.com, Jerry.Chu@eng.sun.com, ipoverib@ietf.org,            
                      i@sun.com>                krause@cup.hp.com                                                                  
                      Sent by:                 Subject:  Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4            
                      ipoverib-admin@ie                                                                                            
                      tf.org                                                                                                       
                                                                                                                                   
                                                                                                                                   
                      04/15/2002 03:52                                                                                             
                      PM                                                                                                           
                      Please respond to                                                                                            
                      Ajoy C                                                                                                       
                      Siddabathuni                                                                                                 
                                                                                                                                   
                                                                                                                                   





>It takes another 4 bytes and provides no new information. The issues
>you indicate above are equally addressed by a 'type' field.

Ofcourse it provides no new information - if you are running IPoIB
in vacum. It is much cleaner than putting a 4 byte header with reserved
fields. Also expect (in a few years) that there will be multi-protocol
routers that will be sitting at the edges of an
IB-Ethernet-osi-apple-network.
They would definetly like to handle packets in a std way - especially when
cp'ing pkts from one kind of "link" to another. They dont need to handle
cases such as - "oh by the way, if this link was an IPoIB link..I needed to
strip the top 4bytes...and not interpret the next 16 bits before I look at
the eth-type".

>
>'No header case' is disallowed as per WG consensus. Otherwise, the
>4-byte ethertype field is equally viable as the 8 byte LLC/SNAP field.
>It has the advantage of being 4 bytes smaller.

I would'nt consider "4 bytes smaller" as an advantage. Especially if you
are
working with IB header lengths upto 68 bytes. And then if you are working
with
TCP/IPv4/v6 add upto 60+.

>
>How about RFC2734 ? There is nothing standard about this.
Well - time to learn from another's mistakes.

Here is my summary -

 - Having no-header at all will give you alignment. No ethtype to mux on.
   But you could have always figured out all IP'ish types from the
IP-hdrtype
   field.

 - Having 4byte "non-std" header will also give you alignment. Gives you
   all the info you need. But yes, there are certain non-std elements in
   the header that implementations would have to be aware of  - especially
   if they were porting an existing stack.

 - Having 8byte std-header will be standards conforming for future
multi-prot-
   col enviroments. Ofcourse, retains alignment.

~Ajoy


>>
>> Frame formats -
>>
>> a) If LLC/SNAP was used then you automatically get a 32bit-word
alignment
>>    for the IP-Datagram.
>>
>>    The frame would look something like --
>>
>>    LRH (8bytes)
>>    GRH (40 bytes)*
>>    BTH (12 bytes)
>>    DETH (8 bytes)
>>                LLC       OUI     EthType
>>    LLC/SNAP <AA-AA-03><00-00-00><xx-xx>   (8bytes)
>>    IPv4/IPv6 Datagram
>>    ICRC
>>    VCRC
>>
>> b) Embedding just the  ethtype - the frame would look like -- also gives
>>    you the alignment - but is a non-std way of doing things (with
reserved
>>    fields etc in the format).
>>
>>    LRH (8bytes)
>>    GRH (40 bytes)*
>>    BTH (12 bytes)
>>    DETH (8 bytes)
>>
>>     Rsvd   EthType
>>    <??-??><xx-xx>  (4bytes)
>>    IPv4/IPv6 Datagram
>>    ICRC
>>    VCRC
>> ----
>>
>> My opinion on this is that if EthType is ever supported, it should be
>> supported using the std mechanism (a) to be consistent with how other
>> IPoverXXX protocols have been defined in the past. (See RFCs -ATM: 2225,
>> 1577, 1483 SMDS:1209 FR:1294)
>
>How about RFC2734 ? There is nothing standard about this.
>
>[ As another example one can even use a 2 byte header if it works out
>for the hardware -i.e. if one can DMA to a 2 byte boundary, the IP
>header will land at a 4 byte boundary. ]
>
>Vivek
>
>>
>> If the WG feels that in the foreseeable future, only IP will ever be
>> supported, then yes there is no need for the Ethtype - ie. leave it
alone,
>> you can still get the IP datagram 32bit aligned right after DETH. On the
>> other hand, if the WG feels that we do it the way our ancestors have
done it
>> there is enough argument already in place for supporting llc/snap hdrs.
>>
>> ~Ajoy
>>
>>
>> >X-Sender: krause@esalpha2.cup.hp.com
>> >Date: Mon, 15 Apr 2002 10:20:49 -0700
>> >To: Yaron Haviv <yaronh@voltaire.com>, "'H.K. Jerry Chu'"
>> <Jerry.Chu@eng.sun.com>, "'ipoverib@ietf.org'" <ipoverib@ietf.org>,
>> "'Ajoy.Siddabathuni@Sun.COM'" <Ajoy.Siddabathuni@Sun.COM>
>> >From: Michael Krause <krause@cup.hp.com>
>> >Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
>> >Mime-Version: 1.0
>> >X-Mailman-Version: 1.0
>> >List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
>> >X-BeenThere: ipoverib@ietf.org
>> >
>> >At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
>> >>Jerry,
>> >>
>> >>We must stop expending the definition somewhere so we get the spec out
>> >>This is IPoIB we are talking about, there is no need for LLC/SNAP, and
we
>> >>just need the Ethertype
>> >
>> >The consensus is that people want to have the full range of Ethertype
>> >values supported.  As such, this needs to be specified independent of
the
>> >IP over IB specification so that everyone may use these values without
>> >requiring them to implement IP over IB.
>> >
>> >Mike
>> >
>> >
>> >_______________________________________________
>> >IPoverIB mailing list
>> >IPoverIB@ietf.org
>> >https://www1.ietf.org/mailman/listinfo/ipoverib
>>
>>
>> _______________________________________________
>> IPoverIB mailing list
>> IPoverIB@ietf.org
>> https://www1.ietf.org/mailman/listinfo/ipoverib
>>
>
>
>--
>Vivek Kashyap
>Linux Technology Center, IBM
>kashyapv@us.ibm.com
>vivk@us.ibm.com
>503 578 3422 (o)


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib





_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Tue Apr 16 14:05:50 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA27450
	for <ipoverib-archive@odin.ietf.org>; Tue, 16 Apr 2002 14:05:49 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA04213
	for ipoverib-archive@odin.ietf.org; Tue, 16 Apr 2002 14:05:51 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA03594;
	Tue, 16 Apr 2002 14:01:33 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA03553
	for <ipoverib@ns.ietf.org>; Tue, 16 Apr 2002 14:01:29 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA27077
	for <ipoverib@ietf.org>; Tue, 16 Apr 2002 14:01:26 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3GHwsLQ069492;
	Tue, 16 Apr 2002 13:58:54 -0400
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3GHwka230374;
	Tue, 16 Apr 2002 13:58:46 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g3GHwjX08257;
	Tue, 16 Apr 2002 10:58:45 -0700
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3GHwjd11622;
	Tue, 16 Apr 2002 10:58:45 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204161758.g3GHwjd11622@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: krause@cup.hp.com (Michael Krause)
Date: Tue, 16 Apr 2002 10:58:43 -0700 (PDT)
Cc: kashyapv@us.ibm.com (Vivek Kashyap), adube@windows.microsoft.com,
        ipoverib@ietf.org
In-Reply-To: <5.1.0.14.2.20020415174755.02b15630@esalpha2.cup.hp.com> from "Michael Krause" at Apr 15, 2002 04:57:34 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

Great! So the deletion of the group is not a difficult issue at all.
The IPoIB nodes only need to leave the group.

        The receiver leaves when it doesn't need to receive any more.
        Router when there are no receivers and it knows it doesn't
        need to forward. sender after some idle time i.e. no activity
        on the group. My suggestion is to have a minimum timeout of 2
        minutes - avoids group join/leave happening too often.

Vivek

> 
> At 04:31 PM 4/15/2002 -0700, Vivek Kashyap wrote:
> >         a) No delete is done by the IPoIB interfaces, it is an SM issue
> >
> >         This I took from the IB architecture document in chapter 3. It
> >         talks of the SM deleting unused groups. ( Now, there are a few
> >         who say that this is not going to be the case -- in which case
> >         a different solution is needed.)
> >
> >         The receiver leaves when it no longer wants to receive. The
> >         router when there are no more receivers. The sender after some
> >         idle time.
> 
> The specification states that a multicast management entity not the SM is 
> responsible for multicast group management.  This entity implement IB 
> multicast group management and could also be responsible for resolving IP 
> multicast addressing to IB multicast groups.  It may also be responsible 
> for informing / updating participating IB routers of any changes (and vice 
> versa for the IB routers informing the entity of changes within its view of 
> the router cloud).  This entity could also be used to provide a similar 
> service for CAs that act as multi-protocol router endnodes.
> 
> >         b) Any entity wanting to leave an IB group attempts to delete
> >         it. If there are members in the group then the deletion should
> >         fail. The last one will automatically delete the group.
> >
> >         [Depends on delete semantics]
> 
> If an endnode wants to leave an IB multicast group, it informs the 
> management entity of this event and the management entity working with the 
> SM removes this endnode from the IB multicast tree.  If the entity 
> determines that there are no longer any endnodes participating in the 
> multicast group, it may delete the entire group from the fabric.  This does 
> not require the endnode to delete the group and deal with any failure 
> semantics.  An endnode should simply delete itself and let the management 
> entity deal with the problem.
> 
> >         c) Leave is fine. No need to bother about delete.
> >
> >         let the group entries stay around in the database. They don't
> >         bother anyone. The admin can periodically, once in a few
> >         months may be, delete unused ones. The entries will get reused
> >         if others join. The multicast tree is affected by joins and
> >         leaves and not by deletes.
> 
> Delete should be performed automatically by the management entity; this is 
> a policy decision that has no bearing on the IP over IB specification.
> 
> >         d) {Don't know about this }
> >
> >         The setup is such that if the last member leaves the group the
> >         group is automatically removed.
> 
> This is how IB specifies it should work.
> 
> Mike
> 
> 
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Tue Apr 16 15:01:36 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04385
	for <ipoverib-archive@odin.ietf.org>; Tue, 16 Apr 2002 15:01:36 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA07623
	for ipoverib-archive@odin.ietf.org; Tue, 16 Apr 2002 15:01:38 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA07051;
	Tue, 16 Apr 2002 14:57:29 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA07019
	for <ipoverib@ns.ietf.org>; Tue, 16 Apr 2002 14:57:26 -0400 (EDT)
Received: from mail4.microsoft.com (mail4.microsoft.com [131.107.3.122])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA03667
	for <ipoverib@ietf.org>; Tue, 16 Apr 2002 14:57:23 -0400 (EDT)
Received: from inet-vrs-04.redmond.corp.microsoft.com ([157.54.8.154]) by mail4.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Tue, 16 Apr 2002 11:56:44 -0700
Received: from 157.54.8.155 by inet-vrs-04.redmond.corp.microsoft.com (InterScan E-Mail VirusWall NT); Tue, 16 Apr 2002 11:56:55 -0700
Received: from red-imc-01.redmond.corp.microsoft.com ([157.54.9.102]) by inet-hub-04.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Tue, 16 Apr 2002 11:56:54 -0700
Received: from win-imc-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.39]) by red-imc-01.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Tue, 16 Apr 2002 11:56:54 -0700
Received: from win-msg-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.52]) by win-imc-01.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(6.0.3604.0);
	 Tue, 16 Apr 2002 11:53:06 -0700
X-MimeOLE: Produced By Microsoft Exchange V6.0.6177.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: multipart/mixed;
	boundary="------------InterScan_NT_MIME_Boundary"
Date: Tue, 16 Apr 2002 11:53:06 -0700
Message-ID: <2E33960095B58E40A4D3345AB9F65EC102450E35@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Thread-Topic: MIB's ifPhysAddress needs to be updated
Thread-Index: AcHleHF5VOwX03mxSyaapzdpBLRC9g==
From: "Aditya Dube" <adube@windows.microsoft.com>
To: <ipoverib@ietf.org>
X-OriginalArrivalTime: 16 Apr 2002 18:53:06.0326 (UTC) FILETIME=[F1C65360:01C1E577]
Subject: [Ipoverib] MIB's ifPhysAddress needs to be updated
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

This is a multi-part message in MIME format.

--------------InterScan_NT_MIME_Boundary
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C1E577.F19F4F03"

------_=_NextPart_001_01C1E577.F19F4F03
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable


In the MIB draft, the ifPhysAddress is specified as the LID. This is out
of sync with the IpOverIb draft. Can the ifPhysAddress in the MIB be
changed to match  IPoverIB's physical address, i.e. the GID + QPN?

http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.txt

Aditya Dube


------_=_NextPart_001_01C1E577.F19F4F03
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.0.6177.0">
<TITLE>MIB's ifPhysAddress needs to be updated</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->
<BR>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">In =
the MIB</FONT><FONT SIZE=3D2 FACE=3D"Arial"> draft, the ifPhysAddress is =
specified as the LID.</FONT> <FONT SIZE=3D2 FACE=3D"Arial">This is out =
of sync with the IpOverIb draft. Can the =
ifPhysAddress</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT SIZE=3D2 FACE=3D"Arial">in the =
MIB</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT =
SIZE=3D2 FACE=3D"Arial">be changed to match</FONT>&nbsp;<FONT SIZE=3D2 =
FACE=3D"Arial"> IPoverIB</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">&#8217;</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">s</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"> <FONT SIZE=3D2 =
FACE=3D"Arial">physical</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial"></FONT> <FONT SIZE=3D2 =
FACE=3D"Arial">address</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">, i.e</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">.</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial"> the GID + =
QPN?</FONT></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial"><A =
HREF=3D"http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.=
txt">http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.txt=
</A></FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><I><SPAN LANG=3D"en-us"></SPAN></I><A NAME=3D""><I><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#800000" =
SIZE=3D2 FACE=3D"Arial">Aditya Dube</FONT></SPAN></I></A><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

<P ALIGN=3DLEFT><SPAN LANG=3D"en-us"></SPAN></P>

</BODY>
</HTML>
------_=_NextPart_001_01C1E577.F19F4F03--

--------------InterScan_NT_MIME_Boundary--


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Tue Apr 16 15:23:13 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA07165
	for <ipoverib-archive@odin.ietf.org>; Tue, 16 Apr 2002 15:23:08 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA08961
	for ipoverib-archive@odin.ietf.org; Tue, 16 Apr 2002 15:23:10 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA08640;
	Tue, 16 Apr 2002 15:18:49 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA08604
	for <ipoverib@ns.ietf.org>; Tue, 16 Apr 2002 15:18:46 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA06650
	for <ipoverib@ietf.org>; Tue, 16 Apr 2002 15:18:30 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3GJIXLQ018816;
	Tue, 16 Apr 2002 15:18:33 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3GJITa99820;
	Tue, 16 Apr 2002 15:18:29 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g3GJIQM03934;
	Tue, 16 Apr 2002 12:18:26 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3GJIOh23861;
	Tue, 16 Apr 2002 12:18:24 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204161918.g3GJIOh23861@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
To: adube@windows.microsoft.com (Aditya Dube)
Date: Tue, 16 Apr 2002 12:18:23 -0700 (PDT)
Cc: ipoverib@ietf.org
In-Reply-To: <2E33960095B58E40A4D3345AB9F65EC102450E35@win-msg-01.wingroup.windeploy.ntdev.microsoft.com> from "Aditya Dube" at Apr 16, 2002 10:53:06 AM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

The same question was addressed last october/november.
See the message:

http://www1.ietf.org/mail-archive/working-groups/ipoverib/current/msg00087.html

Vivek



> 
> This is a multi-part message in MIME format.
> 
> --------------InterScan_NT_MIME_Boundary
> Content-Type: multipart/alternative;
> 	boundary="----_=_NextPart_001_01C1E577.F19F4F03"
> 
> ------_=_NextPart_001_01C1E577.F19F4F03
> Content-Type: text/plain;
> 	charset="us-ascii"
> Content-Transfer-Encoding: quoted-printable
> 
> 
> In the MIB draft, the ifPhysAddress is specified as the LID. This is out
> of sync with the IpOverIb draft. Can the ifPhysAddress in the MIB be
> changed to match  IPoverIB's physical address, i.e. the GID + QPN?
> 
> http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.txt
> 
> Aditya Dube
> 
> 

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Tue Apr 16 19:08:32 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA02227
	for <ipoverib-archive@odin.ietf.org>; Tue, 16 Apr 2002 19:08:32 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA19910
	for ipoverib-archive@odin.ietf.org; Tue, 16 Apr 2002 19:08:34 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA19856;
	Tue, 16 Apr 2002 19:04:30 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA19822
	for <ipoverib@ns.ietf.org>; Tue, 16 Apr 2002 19:04:26 -0400 (EDT)
Received: from strahm.net (adsl-67-115-29-109.dsl.sndg02.pacbell.net [67.115.29.109])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA01825
	for <ipoverib@ietf.org>; Tue, 16 Apr 2002 19:04:22 -0400 (EDT)
Received: (from bill@localhost)
	by strahm.net (8.11.6/8.11.6) id g3GN2MF87024;
	Tue, 16 Apr 2002 16:02:22 -0700 (PDT)
	(envelope-from bill)
Date: Tue, 16 Apr 2002 16:02:19 -0700
From: Bill Strahm <bill@strahm.net>
To: Vivek Kashyap <kashyapv@us.ibm.com>
Cc: Michael Krause <krause@cup.hp.com>, adube@windows.microsoft.com,
        ipoverib@ietf.org
Subject: Re: [Ipoverib] IPoIB multicast forwarding
Message-ID: <20020416160219.B86906@homegate.strahm.net>
References: <5.1.0.14.2.20020415174755.02b15630@esalpha2.cup.hp.com> <200204161758.g3GHwjd11622@eng4.beaverton.ibm.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <200204161758.g3GHwjd11622@eng4.beaverton.ibm.com>; from kashyapv@us.ibm.com on Tue, Apr 16, 2002 at 10:58:43AM -0700
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

I don't think it is that easy... How will the router know to leave ? It can't
be when there are no receivers, because it has to forward any traffic from
the senders to other locations on the network...  This is especially true for
flood/prune algorithms (see PIM-DM)

I'm also not sure that this would work in an intermitant sender environment.

I may have a sender that only wants to multicast trigger events, lets say
an SNMP TRAP message that happens once in a blue moon...  This means that
there is the overhead of group creating and deletion with every send (and
you still have the problem with the router not being on the line)

I am very tempted to do some deep thinking about having ALL multicast packets
go to a default multicast traffic IB address, and have the IP portion echo
the packets back to the IB subnet on the correct IB MC Group if there are 
konwn subscribers (through IGMP etc) and forward on to other ports depending
on the IP multicast forwarding tree...

This means that all locally sent MC traffic goes onto the fabric twice, once
to get to the router on a "All multicast" traffic channel, and once to the
correct multicast channel...

Does IB provide a mechanism to allow the Router to detect multicast group
creations... if it does, then we can use the existance of the group rather
than needing IGMP, I don't think this has been defined in IBTA fully however

Bill
On Tue, Apr 16, 2002 at 10:58:43AM -0700, Vivek Kashyap wrote:
> Great! So the deletion of the group is not a difficult issue at all.
> The IPoIB nodes only need to leave the group.
> 
>         The receiver leaves when it doesn't need to receive any more.
>         Router when there are no receivers and it knows it doesn't
>         need to forward. sender after some idle time i.e. no activity
>         on the group. My suggestion is to have a minimum timeout of 2
>         minutes - avoids group join/leave happening too often.
> 
> Vivek
> 
> > 
> > At 04:31 PM 4/15/2002 -0700, Vivek Kashyap wrote:
> > >         a) No delete is done by the IPoIB interfaces, it is an SM issue
> > >
> > >         This I took from the IB architecture document in chapter 3. It
> > >         talks of the SM deleting unused groups. ( Now, there are a few
> > >         who say that this is not going to be the case -- in which case
> > >         a different solution is needed.)
> > >
> > >         The receiver leaves when it no longer wants to receive. The
> > >         router when there are no more receivers. The sender after some
> > >         idle time.
> > 
> > The specification states that a multicast management entity not the SM is 
> > responsible for multicast group management.  This entity implement IB 
> > multicast group management and could also be responsible for resolving IP 
> > multicast addressing to IB multicast groups.  It may also be responsible 
> > for informing / updating participating IB routers of any changes (and vice 
> > versa for the IB routers informing the entity of changes within its view of 
> > the router cloud).  This entity could also be used to provide a similar 
> > service for CAs that act as multi-protocol router endnodes.
> > 
> > >         b) Any entity wanting to leave an IB group attempts to delete
> > >         it. If there are members in the group then the deletion should
> > >         fail. The last one will automatically delete the group.
> > >
> > >         [Depends on delete semantics]
> > 
> > If an endnode wants to leave an IB multicast group, it informs the 
> > management entity of this event and the management entity working with the 
> > SM removes this endnode from the IB multicast tree.  If the entity 
> > determines that there are no longer any endnodes participating in the 
> > multicast group, it may delete the entire group from the fabric.  This does 
> > not require the endnode to delete the group and deal with any failure 
> > semantics.  An endnode should simply delete itself and let the management 
> > entity deal with the problem.
> > 
> > >         c) Leave is fine. No need to bother about delete.
> > >
> > >         let the group entries stay around in the database. They don't
> > >         bother anyone. The admin can periodically, once in a few
> > >         months may be, delete unused ones. The entries will get reused
> > >         if others join. The multicast tree is affected by joins and
> > >         leaves and not by deletes.
> > 
> > Delete should be performed automatically by the management entity; this is 
> > a policy decision that has no bearing on the IP over IB specification.
> > 
> > >         d) {Don't know about this }
> > >
> > >         The setup is such that if the last member leaves the group the
> > >         group is automatically removed.
> > 
> > This is how IB specifies it should work.
> > 
> > Mike
> > 
> > 
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Tue Apr 16 19:44:38 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA06123
	for <ipoverib-archive@odin.ietf.org>; Tue, 16 Apr 2002 19:44:38 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA21109
	for ipoverib-archive@odin.ietf.org; Tue, 16 Apr 2002 19:44:41 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA21014;
	Tue, 16 Apr 2002 19:39:38 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA20981
	for <ipoverib@ns.ietf.org>; Tue, 16 Apr 2002 19:39:35 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA05501
	for <ipoverib@ietf.org>; Tue, 16 Apr 2002 19:39:31 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3GNdWLQ010806;
	Tue, 16 Apr 2002 19:39:32 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3GNdVa97692;
	Tue, 16 Apr 2002 19:39:32 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g3GNdSM10096;
	Tue, 16 Apr 2002 16:39:29 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3GNdQu29329;
	Tue, 16 Apr 2002 16:39:26 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204162339.g3GNdQu29329@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: bill@strahm.net (Bill Strahm)
Date: Tue, 16 Apr 2002 16:39:25 -0700 (PDT)
Cc: kashyapv@us.ibm.com (Vivek Kashyap), krause@cup.hp.com (Michael Krause),
        adube@windows.microsoft.com, ipoverib@ietf.org
In-Reply-To: <20020416160219.B86906@homegate.strahm.net> from "Bill Strahm" at Apr 16, 2002 03:02:19 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> I don't think it is that easy... How will the router know to leave ? It can't
> be when there are no receivers, because it has to forward any traffic from
> the senders to other locations on the network...  This is especially true for
> flood/prune algorithms (see PIM-DM)

        That is not a problem. As I noted earlier a router will join
        the groups on getting a notification that there are receivers.
        When no receivers are present it can choose to leave. 
	
        If it is forwarding the packets then it creates the IB group
        and when it thinks it no longer needs to forward the packets,
        it leaves the group.

> 
> I'm also not sure that this would work in an intermitant sender environment.
> 
> I may have a sender that only wants to multicast trigger events, lets say
> an SNMP TRAP message that happens once in a blue moon...  This means that
> there is the overhead of group creating and deletion with every send (and
> you still have the problem with the router not being on the line)

        If it is once in a blue-moon the overhead also occurs once in
        a blue-moon; that should not be too much of trouble.

        In general you join the group when you need to send the snmp
        trap. If you join the group and you know that you need to stay
        with this group you can choose not to leave the IB group at
        all. I have suggested the sender leaving the group after some
        idle time - the delay is suggested to be a minimum of 2
        minutes to avoid the create/delete operation to occur too
        often. You can always set the timer to be longer.

> 
> I am very tempted to do some deep thinking about having ALL multicast packets
> go to a default multicast traffic IB address, and have the IP portion echo
> the packets back to the IB subnet on the correct IB MC Group if there are 
> konwn subscribers (through IGMP etc) and forward on to other ports depending
> on the IP multicast forwarding tree...
> 
> This means that all locally sent MC traffic goes onto the fabric twice, once
> to get to the router on a "All multicast" traffic channel, and once to the
> correct multicast channel...

        So you are also asking the router to join the IB group based
        on IGMP information. So, in that case why not just let the
        router join the group, and have the sender send directly to
        the IB group? I don't think the load on the sender is all that
        much that the router(which router? if there are multiple)
        needs to be loaded with the task of resending the packet.

        Deliberately sending duplicate multicast packets on the fabric
        does not seem like a good idea.

> 
> Does IB provide a mechanism to allow the Router to detect multicast group
> creations... if it does, then we can use the existance of the group rather
> than needing IGMP, I don't think this has been defined in IBTA fully however

        Believe me - I would have used an existing IB method if I
        could find one. There is not one and hence I have suggested
        the solution I have.

        I have already received a request, and it is a very valid one,
        to put the overall algorithm for senders/routers/receivers
        together in one draft and post it here. I'll do that in the
        next couple of days. That way we can evaluate the overall
        solution in one go and see what is missing.

Vivek

> 
> Bill
> On Tue, Apr 16, 2002 at 10:58:43AM -0700, Vivek Kashyap wrote:
> > Great! So the deletion of the group is not a difficult issue at all.
> > The IPoIB nodes only need to leave the group.
> > 
> >         The receiver leaves when it doesn't need to receive any more.
> >         Router when there are no receivers and it knows it doesn't
> >         need to forward. sender after some idle time i.e. no activity
> >         on the group. My suggestion is to have a minimum timeout of 2
> >         minutes - avoids group join/leave happening too often.
> > 
> > Vivek
> > 
> > > 
> > > At 04:31 PM 4/15/2002 -0700, Vivek Kashyap wrote:
> > > >         a) No delete is done by the IPoIB interfaces, it is an SM issue
> > > >
> > > >         This I took from the IB architecture document in chapter 3. It
> > > >         talks of the SM deleting unused groups. ( Now, there are a few
> > > >         who say that this is not going to be the case -- in which case
> > > >         a different solution is needed.)
> > > >
> > > >         The receiver leaves when it no longer wants to receive. The
> > > >         router when there are no more receivers. The sender after some
> > > >         idle time.
> > > 
> > > The specification states that a multicast management entity not the SM is 
> > > responsible for multicast group management.  This entity implement IB 
> > > multicast group management and could also be responsible for resolving IP 
> > > multicast addressing to IB multicast groups.  It may also be responsible 
> > > for informing / updating participating IB routers of any changes (and vice 
> > > versa for the IB routers informing the entity of changes within its view of 
> > > the router cloud).  This entity could also be used to provide a similar 
> > > service for CAs that act as multi-protocol router endnodes.
> > > 
> > > >         b) Any entity wanting to leave an IB group attempts to delete
> > > >         it. If there are members in the group then the deletion should
> > > >         fail. The last one will automatically delete the group.
> > > >
> > > >         [Depends on delete semantics]
> > > 
> > > If an endnode wants to leave an IB multicast group, it informs the 
> > > management entity of this event and the management entity working with the 
> > > SM removes this endnode from the IB multicast tree.  If the entity 
> > > determines that there are no longer any endnodes participating in the 
> > > multicast group, it may delete the entire group from the fabric.  This does 
> > > not require the endnode to delete the group and deal with any failure 
> > > semantics.  An endnode should simply delete itself and let the management 
> > > entity deal with the problem.
> > > 
> > > >         c) Leave is fine. No need to bother about delete.
> > > >
> > > >         let the group entries stay around in the database. They don't
> > > >         bother anyone. The admin can periodically, once in a few
> > > >         months may be, delete unused ones. The entries will get reused
> > > >         if others join. The multicast tree is affected by joins and
> > > >         leaves and not by deletes.
> > > 
> > > Delete should be performed automatically by the management entity; this is 
> > > a policy decision that has no bearing on the IP over IB specification.
> > > 
> > > >         d) {Don't know about this }
> > > >
> > > >         The setup is such that if the last member leaves the group the
> > > >         group is automatically removed.
> > > 
> > > This is how IB specifies it should work.
> > > 
> > > Mike
> > > 
> > > 
> > > 
> > 
> > 
> > -- 
> > Vivek Kashyap
> > Linux Technology Center, IBM
> > kashyapv@us.ibm.com 
> > vivk@us.ibm.com 
> > 503 578 3422 (o)
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Tue Apr 16 19:59:25 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA07623
	for <ipoverib-archive@odin.ietf.org>; Tue, 16 Apr 2002 19:59:25 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA21447
	for ipoverib-archive@odin.ietf.org; Tue, 16 Apr 2002 19:59:28 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA21398;
	Tue, 16 Apr 2002 19:55:24 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA21350
	for <ipoverib@ns.ietf.org>; Tue, 16 Apr 2002 19:55:21 -0400 (EDT)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA07095
	for <ipoverib@ietf.org>; Tue, 16 Apr 2002 19:55:16 -0400 (EDT)
Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.39 2002/04/15 17:47:23 root Exp $) with ESMTP id g3GNsb410262
	for <ipoverib@ietf.org>; Tue, 16 Apr 2002 23:54:37 GMT
Received: from fmsmsxvs042.fm.intel.com (fmsmsxv042-1.fm.intel.com [132.233.48.110])
	by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.16 2002/04/15 17:47:00 root Exp $) with SMTP id g3GNrXO14859
	for <ipoverib@ietf.org>; Tue, 16 Apr 2002 23:53:33 GMT
Received: from fmsmsx26.fm.intel.com ([132.233.42.26])
 by fmsmsxvs042.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002041616590313140
 ; Tue, 16 Apr 2002 16:59:03 -0700
Received: by fmsmsx26.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2HL02RPR>; Tue, 16 Apr 2002 16:55:01 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F92@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Bill Strahm'" <bill@strahm.net>, Vivek Kashyap <kashyapv@us.ibm.com>
Cc: Michael Krause <krause@cup.hp.com>, adube@windows.microsoft.com,
        ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Tue, 16 Apr 2002 16:54:54 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Given all the uncertainty of multicast group semantics and some of the
algorithms we're coming up with for presumed correctness, I still lean
toward multicast = broadcast.  It is guaranteed to be correct and straight
forward to implement.  We can always come back to it once we get some
experience with how multicast is going to be used in the data center and
time for the IBTA to fix/clarify group semantics.  The way it stands now,
the SM implementations would seem to have a lot of flexibility which will
lead to interoperability problems.  IBTA needs to fix this before we can
reliably use this in IPoIB.

Roy

-----Original Message-----
From: Bill Strahm [mailto:bill@strahm.net]
Sent: Tuesday, April 16, 2002 4:02 PM
To: Vivek Kashyap
Cc: Michael Krause; adube@windows.microsoft.com; ipoverib@ietf.org
Subject: Re: [Ipoverib] IPoIB multicast forwarding


I don't think it is that easy... How will the router know to leave ? It
can't
be when there are no receivers, because it has to forward any traffic from
the senders to other locations on the network...  This is especially true
for
flood/prune algorithms (see PIM-DM)

I'm also not sure that this would work in an intermitant sender environment.

I may have a sender that only wants to multicast trigger events, lets say
an SNMP TRAP message that happens once in a blue moon...  This means that
there is the overhead of group creating and deletion with every send (and
you still have the problem with the router not being on the line)

I am very tempted to do some deep thinking about having ALL multicast
packets
go to a default multicast traffic IB address, and have the IP portion echo
the packets back to the IB subnet on the correct IB MC Group if there are 
konwn subscribers (through IGMP etc) and forward on to other ports depending
on the IP multicast forwarding tree...

This means that all locally sent MC traffic goes onto the fabric twice, once
to get to the router on a "All multicast" traffic channel, and once to the
correct multicast channel...

Does IB provide a mechanism to allow the Router to detect multicast group
creations... if it does, then we can use the existance of the group rather
than needing IGMP, I don't think this has been defined in IBTA fully however

Bill
On Tue, Apr 16, 2002 at 10:58:43AM -0700, Vivek Kashyap wrote:
> Great! So the deletion of the group is not a difficult issue at all.
> The IPoIB nodes only need to leave the group.
> 
>         The receiver leaves when it doesn't need to receive any more.
>         Router when there are no receivers and it knows it doesn't
>         need to forward. sender after some idle time i.e. no activity
>         on the group. My suggestion is to have a minimum timeout of 2
>         minutes - avoids group join/leave happening too often.
> 
> Vivek
> 
> > 
> > At 04:31 PM 4/15/2002 -0700, Vivek Kashyap wrote:
> > >         a) No delete is done by the IPoIB interfaces, it is an SM
issue
> > >
> > >         This I took from the IB architecture document in chapter 3. It
> > >         talks of the SM deleting unused groups. ( Now, there are a few
> > >         who say that this is not going to be the case -- in which case
> > >         a different solution is needed.)
> > >
> > >         The receiver leaves when it no longer wants to receive. The
> > >         router when there are no more receivers. The sender after some
> > >         idle time.
> > 
> > The specification states that a multicast management entity not the SM
is 
> > responsible for multicast group management.  This entity implement IB 
> > multicast group management and could also be responsible for resolving
IP 
> > multicast addressing to IB multicast groups.  It may also be responsible

> > for informing / updating participating IB routers of any changes (and
vice 
> > versa for the IB routers informing the entity of changes within its view
of 
> > the router cloud).  This entity could also be used to provide a similar 
> > service for CAs that act as multi-protocol router endnodes.
> > 
> > >         b) Any entity wanting to leave an IB group attempts to delete
> > >         it. If there are members in the group then the deletion should
> > >         fail. The last one will automatically delete the group.
> > >
> > >         [Depends on delete semantics]
> > 
> > If an endnode wants to leave an IB multicast group, it informs the 
> > management entity of this event and the management entity working with
the 
> > SM removes this endnode from the IB multicast tree.  If the entity 
> > determines that there are no longer any endnodes participating in the 
> > multicast group, it may delete the entire group from the fabric.  This
does 
> > not require the endnode to delete the group and deal with any failure 
> > semantics.  An endnode should simply delete itself and let the
management 
> > entity deal with the problem.
> > 
> > >         c) Leave is fine. No need to bother about delete.
> > >
> > >         let the group entries stay around in the database. They don't
> > >         bother anyone. The admin can periodically, once in a few
> > >         months may be, delete unused ones. The entries will get reused
> > >         if others join. The multicast tree is affected by joins and
> > >         leaves and not by deletes.
> > 
> > Delete should be performed automatically by the management entity; this
is 
> > a policy decision that has no bearing on the IP over IB specification.
> > 
> > >         d) {Don't know about this }
> > >
> > >         The setup is such that if the last member leaves the group the
> > >         group is automatically removed.
> > 
> > This is how IB specifies it should work.
> > 
> > Mike
> > 
> > 
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr 16 21:01:47 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA14562
	for <ipoverib-archive@odin.ietf.org>; Tue, 16 Apr 2002 21:01:47 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id VAA25012
	for ipoverib-archive@odin.ietf.org; Tue, 16 Apr 2002 21:01:50 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id UAA24234;
	Tue, 16 Apr 2002 20:52:57 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id UAA24162
	for <ipoverib@optimus.ietf.org>; Tue, 16 Apr 2002 20:52:52 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA13570
	for <ipoverib@ietf.org>; Tue, 16 Apr 2002 20:52:48 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3H0qoLQ044280;
	Tue, 16 Apr 2002 20:52:50 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3H0qna122640;
	Tue, 16 Apr 2002 20:52:49 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g3H0qkM11524;
	Tue, 16 Apr 2002 17:52:46 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3H0qh500508;
	Tue, 16 Apr 2002 17:52:43 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204170052.g3H0qh500508@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
To: roy.k.larsen@intel.com (Larsen, Roy K)
Date: Tue, 16 Apr 2002 17:52:42 -0700 (PDT)
Cc: bill@strahm.net ('Bill Strahm'), kashyapv@us.ibm.com (Vivek Kashyap),
        krause@cup.hp.com (Michael Krause), adube@windows.microsoft.com,
        ipoverib@ietf.org
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4F92@ORSMSX108> from "Larsen, Roy K" at Apr 16, 2002 03:54:54 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> Given all the uncertainty of multicast group semantics and some of the
> algorithms we're coming up with for presumed correctness, I still lean

        What is not 'correct' in the solution that I have presented? I
        feel that the solution hangs together pretty well. I'll
        present it in a consolidated form in the next few days. Let us
        then see if there is anything missing.

> toward multicast = broadcast.  It is guaranteed to be correct and straight

        As we discussed earlier in this thread, the broadcast option
        is not optimal and short changes InfiniBand.

> forward to implement.  We can always come back to it once we get some
> experience with how multicast is going to be used in the data center and
> time for the IBTA to fix/clarify group semantics.  The way it stands now,
> the SM implementations would seem to have a lot of flexibility which will
> lead to interoperability problems.  IBTA needs to fix this before we can
> reliably use this in IPoIB.

        The solutions presented are in line with the specification as
        published by IBTA. As far as I know the basic assumptions
        specified in IBA version 1.0a are not being overridden in IBTA
        even if they are working on rounding off some of the
        unresolved issues of IB multicast. 

Vivek

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr 16 22:51:49 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA25860
	for <ipoverib-archive@odin.ietf.org>; Tue, 16 Apr 2002 22:51:48 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id WAA29073
	for ipoverib-archive@odin.ietf.org; Tue, 16 Apr 2002 22:51:53 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id WAA28788;
	Tue, 16 Apr 2002 22:42:15 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id WAA28734
	for <ipoverib@optimus.ietf.org>; Tue, 16 Apr 2002 22:42:11 -0400 (EDT)
Received: from pheriche.sun.com (pheriche.sun.com [192.18.98.34])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA25077;
	Tue, 16 Apr 2002 22:42:01 -0400 (EDT)
Received: from phys-ha10nwka.ebay.sun.com ([129.150.153.210])
	by pheriche.sun.com (8.9.3+Sun/8.9.3) with ESMTP id UAA14235;
	Tue, 16 Apr 2002 20:41:50 -0600 (MDT)
Received: from endeavour (endeavour [129.150.143.50])
	by phys-ha10nwka.ebay.sun.com (8.8.8+Sun/8.8.8/ENSMAIL,v2.2) with SMTP id TAA04430;
	Tue, 16 Apr 2002 19:41:39 -0700 (PDT)
Message-Id: <200204170241.TAA04430@phys-ha10nwka.ebay.sun.com>
Date: Tue, 16 Apr 2002 19:41:30 -0700 (PDT)
From: Ajoy C Siddabathuni <Ajoy.Siddabathuni@sun.com>
Reply-To: Ajoy C Siddabathuni <Ajoy.Siddabathuni@sun.com>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
To: Ajoy.Siddabathuni@sun.com, vivk@us.ibm.com
Cc: Ajoy.Siddabathuni@sun.com, ipoverib@ietf.org, ipoverib-admin@ietf.org,
        Jerry.Chu@eng.sun.com, krause@cup.hp.com, viv@beaverton.ibm.com,
        yaronh@voltaire.com
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: n7OvsKCF6jjg2dOxjlzYkQ==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.3.4p_5 SunOS 5.7 sun4u sparc 
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org


>4 byte header:Signal is 4 bytes but intelligence is only in 2
>bytes(ethertype). Only half of the signal provides any information.
>
>LLC/SNAP: Signal is 8 bytes but intelligence is only in 2 bytes. So we get
>25% return. We have added another 4 bytes with no increase in intelligence.
>
>Hence a '4-byte ethertype' is better than the 8 byte LLC/SNAP.

The conclusion drawn here does not consider all the discussion that was put forth
in the last msg of the thread.

If you consider what would be extra over head of 8 bytes verses 4 bytes aganist
an MTU of 2k. It is negligible or nothing.

I think one must get outside the "try to save some bytes here and there" line
of thinking when defining wire protocols and make sure that you dont have to
write another rfc next year to obsolete this one. All those RFCs that defined the
LLC/SNAP headers in them were not mis-guided, would you think? You would'nt endup
with an IB header of upto 68 bytes for just UD if saving bytes from the header
was the goal. Recall that at wire speeds of 2-10Gbps these should not matter.
Things like alignment, cache-line sizes, receive queue depths and page-sizes are
the bigger fishes to fry at those rates.

~Ajoy

>
>Also, I don't think you are right in dismissing non-LLC/SNAP use. LLC/SNAP
>is not the favored mode in general - just look at 802.3 vs. DIX ethernet.



>
>
>                                                                                                                                   
>                      Ajoy C                                                                                                       
>                      Siddabathuni             To:       
Ajoy.Siddabathuni@sun.com, viv@beaverton.ibm.com                          
>                      <Ajoy.Siddabathun        cc:       yaronh@voltaire.com, 
Jerry.Chu@eng.sun.com, ipoverib@ietf.org,            
>                      i@sun.com>                krause@cup.hp.com                                                                  
>                      Sent by:                 Subject:  Re: [Ipoverib] 
consensus call on EtherType, min MTU and DHCPv4            
>                      ipoverib-admin@ie                                                                                            
>                      tf.org                                                                                                       
>                                                                                                                                   
>                                                                                                                                   
>                      04/15/2002 03:52                                                                                             
>                      PM                                                                                                           
>                      Please respond to                                                                                            
>                      Ajoy C                                                                                                       
>                      Siddabathuni                                                                                                 
>                                                                                                                                   
>                                                                                                                                   
>
>
>
>
>
>>It takes another 4 bytes and provides no new information. The issues
>>you indicate above are equally addressed by a 'type' field.
>
>Ofcourse it provides no new information - if you are running IPoIB
>in vacum. It is much cleaner than putting a 4 byte header with reserved
>fields. Also expect (in a few years) that there will be multi-protocol
>routers that will be sitting at the edges of an
>IB-Ethernet-osi-apple-network.
>They would definetly like to handle packets in a std way - especially when
>cp'ing pkts from one kind of "link" to another. They dont need to handle
>cases such as - "oh by the way, if this link was an IPoIB link..I needed to
>strip the top 4bytes...and not interpret the next 16 bits before I look at
>the eth-type".
>
>>
>>'No header case' is disallowed as per WG consensus. Otherwise, the
>>4-byte ethertype field is equally viable as the 8 byte LLC/SNAP field.
>>It has the advantage of being 4 bytes smaller.
>
>I would'nt consider "4 bytes smaller" as an advantage. Especially if you
>are
>working with IB header lengths upto 68 bytes. And then if you are working
>with
>TCP/IPv4/v6 add upto 60+.
>
>>
>>How about RFC2734 ? There is nothing standard about this.
>Well - time to learn from another's mistakes.
>
>Here is my summary -
>
> - Having no-header at all will give you alignment. No ethtype to mux on.
>   But you could have always figured out all IP'ish types from the
>IP-hdrtype
>   field.
>
> - Having 4byte "non-std" header will also give you alignment. Gives you
>   all the info you need. But yes, there are certain non-std elements in
>   the header that implementations would have to be aware of  - especially
>   if they were porting an existing stack.
>
> - Having 8byte std-header will be standards conforming for future
>multi-prot-
>   col enviroments. Ofcourse, retains alignment.
>
>~Ajoy
>
>
>>>
>>> Frame formats -
>>>
>>> a) If LLC/SNAP was used then you automatically get a 32bit-word
>alignment
>>>    for the IP-Datagram.
>>>
>>>    The frame would look something like --
>>>
>>>    LRH (8bytes)
>>>    GRH (40 bytes)*
>>>    BTH (12 bytes)
>>>    DETH (8 bytes)
>>>                LLC       OUI     EthType
>>>    LLC/SNAP <AA-AA-03><00-00-00><xx-xx>   (8bytes)
>>>    IPv4/IPv6 Datagram
>>>    ICRC
>>>    VCRC
>>>
>>> b) Embedding just the  ethtype - the frame would look like -- also gives
>>>    you the alignment - but is a non-std way of doing things (with
>reserved
>>>    fields etc in the format).
>>>
>>>    LRH (8bytes)
>>>    GRH (40 bytes)*
>>>    BTH (12 bytes)
>>>    DETH (8 bytes)
>>>
>>>     Rsvd   EthType
>>>    <??-??><xx-xx>  (4bytes)
>>>    IPv4/IPv6 Datagram
>>>    ICRC
>>>    VCRC
>>> ----
>>>
>>> My opinion on this is that if EthType is ever supported, it should be
>>> supported using the std mechanism (a) to be consistent with how other
>>> IPoverXXX protocols have been defined in the past. (See RFCs -ATM: 2225,
>>> 1577, 1483 SMDS:1209 FR:1294)
>>
>>How about RFC2734 ? There is nothing standard about this.
>>
>>[ As another example one can even use a 2 byte header if it works out
>>for the hardware -i.e. if one can DMA to a 2 byte boundary, the IP
>>header will land at a 4 byte boundary. ]
>>
>>Vivek
>>
>>>
>>> If the WG feels that in the foreseeable future, only IP will ever be
>>> supported, then yes there is no need for the Ethtype - ie. leave it
>alone,
>>> you can still get the IP datagram 32bit aligned right after DETH. On the
>>> other hand, if the WG feels that we do it the way our ancestors have
>done it
>>> there is enough argument already in place for supporting llc/snap hdrs.
>>>
>>> ~Ajoy
>>>
>>>
>>> >X-Sender: krause@esalpha2.cup.hp.com
>>> >Date: Mon, 15 Apr 2002 10:20:49 -0700
>>> >To: Yaron Haviv <yaronh@voltaire.com>, "'H.K. Jerry Chu'"
>>> <Jerry.Chu@eng.sun.com>, "'ipoverib@ietf.org'" <ipoverib@ietf.org>,
>>> "'Ajoy.Siddabathuni@Sun.COM'" <Ajoy.Siddabathuni@Sun.COM>
>>> >From: Michael Krause <krause@cup.hp.com>
>>> >Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
>>> >Mime-Version: 1.0
>>> >X-Mailman-Version: 1.0
>>> >List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
>>> >X-BeenThere: ipoverib@ietf.org
>>> >
>>> >At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
>>> >>Jerry,
>>> >>
>>> >>We must stop expending the definition somewhere so we get the spec out
>>> >>This is IPoIB we are talking about, there is no need for LLC/SNAP, and
>we
>>> >>just need the Ethertype
>>> >
>>> >The consensus is that people want to have the full range of Ethertype
>>> >values supported.  As such, this needs to be specified independent of
>the
>>> >IP over IB specification so that everyone may use these values without
>>> >requiring them to implement IP over IB.
>>> >
>>> >Mike
>>> >
>>> >
>>> >_______________________________________________
>>> >IPoverIB mailing list
>>> >IPoverIB@ietf.org
>>> >https://www1.ietf.org/mailman/listinfo/ipoverib
>>>
>>>
>>> _______________________________________________
>>> IPoverIB mailing list
>>> IPoverIB@ietf.org
>>> https://www1.ietf.org/mailman/listinfo/ipoverib
>>>
>>
>>
>>--
>>Vivek Kashyap
>>Linux Technology Center, IBM
>>kashyapv@us.ibm.com
>>vivk@us.ibm.com
>>503 578 3422 (o)
>
>
>_______________________________________________
>IPoverIB mailing list
>IPoverIB@ietf.org
>https://www1.ietf.org/mailman/listinfo/ipoverib
>
>
>
>
>
>_______________________________________________
>IPoverIB mailing list
>IPoverIB@ietf.org
>https://www1.ietf.org/mailman/listinfo/ipoverib


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr 16 22:56:47 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA26322
	for <ipoverib-archive@odin.ietf.org>; Tue, 16 Apr 2002 22:56:47 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id WAA29243
	for ipoverib-archive@odin.ietf.org; Tue, 16 Apr 2002 22:56:49 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id WAA29138;
	Tue, 16 Apr 2002 22:52:39 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id WAA29104
	for <ipoverib@optimus.ietf.org>; Tue, 16 Apr 2002 22:52:36 -0400 (EDT)
Received: from palrel12.hp.com (palrel12.hp.com [156.153.255.237])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA25947
	for <ipoverib@ietf.org>; Tue, 16 Apr 2002 22:52:31 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel12.hp.com (Postfix) with ESMTP
	id 3294FE0049E; Tue, 16 Apr 2002 19:52:04 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id TAA04182;
	Tue, 16 Apr 2002 19:52:03 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020416132025.02b57630@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Tue, 16 Apr 2002 13:23:11 -0700
To: Vivek Kashyap <kashyapv@us.ibm.com>
From: Michael Krause <krause@cup.hp.com>
Subject: Re: [Ipoverib] IPoIB multicast forwarding
Cc: adube@windows.microsoft.com, ipoverib@ietf.org
In-Reply-To: <200204161758.g3GHwjd11622@eng4.beaverton.ibm.com>
References: <5.1.0.14.2.20020415174755.02b15630@esalpha2.cup.hp.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 10:58 AM 4/16/2002 -0700, Vivek Kashyap wrote:
>Great! So the deletion of the group is not a difficult issue at all.
>The IPoIB nodes only need to leave the group.
>
>         The receiver leaves when it doesn't need to receive any more.
>         Router when there are no receivers and it knows it doesn't
>         need to forward. sender after some idle time i.e. no activity
>         on the group. My suggestion is to have a minimum timeout of 2
>         minutes - avoids group join/leave happening too often.

I don't believe this is an IP over IB issue - the policy is implemented 
within the management entity which is an IB focused agent not an IP 
agent.  A CA acting as a multiprotocol router could continue to forward 
packets to the IB switches which would simply drop the packets as there is 
no place to route.  An agent providing performance monitoring would note 
the drops and determine whether they should be terminate or not.  It would 
contact the multicast entity who would inform the CA that the following IB 
multicast group no longer exists and that CA-specific software would then 
update its "routing" tables to no longer forward packets into the IB subnet.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 17 01:59:16 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA02649
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 01:59:11 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id BAA17000
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 01:59:13 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id BAA16839;
	Wed, 17 Apr 2002 01:51:31 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id BAA16808
	for <ipoverib@optimus.ietf.org>; Wed, 17 Apr 2002 01:51:29 -0400 (EDT)
Received: from strahm.net (adsl-67-115-29-109.dsl.sndg02.pacbell.net [67.115.29.109])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA02612
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 01:51:25 -0400 (EDT)
Received: from bonfire (adsl-67-115-29-106.dsl.sndg02.pacbell.net [67.115.29.106])
	by strahm.net (8.11.6/8.11.6) with SMTP id g3H5qK494371;
	Tue, 16 Apr 2002 22:52:20 -0700 (PDT)
	(envelope-from bill@strahm.net)
From: "Bill Strahm" <bill@strahm.net>
To: "Ajoy C Siddabathuni" <Ajoy.Siddabathuni@sun.com>
Cc: <ipoverib@ietf.org>
Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
Date: Tue, 16 Apr 2002 22:51:08 -0700
Message-ID: <HHEDIJGOEFLPCPNILMPACEFLCIAA.bill@strahm.net>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
In-Reply-To: <200204170241.TAA04430@phys-ha10nwka.ebay.sun.com>
Importance: Normal
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

I don't agree with the consideration that "Its just 4 bytes out of
thousands"

Like the politicians say "A million here, A million there, next thing you
know we are talking about real money"

Every byte matters.  If a byte doesn't add value why have it...  I am
comfortable with a 4 byte Ethertype because that maintains DWORD alignment,
and is not horrible... I get concerned that people who won the 4 byte
debate, now want to start talking about 8 bytes... Where next 16, 32 ???

I don't see a full LLC/SNAP header adding any value, so why have it then ?

-----Original Message-----
From: Ajoy C Siddabathuni [mailto:Ajoy.Siddabathuni@sun.com]
Sent: Tuesday, April 16, 2002 7:42 PM
To: Ajoy.Siddabathuni@sun.com; vivk@us.ibm.com
Cc: Ajoy.Siddabathuni@sun.com; ipoverib@ietf.org;
ipoverib-admin@ietf.org; Jerry.Chu@eng.sun.com; krause@cup.hp.com;
viv@beaverton.ibm.com; yaronh@voltaire.com
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4



>4 byte header:Signal is 4 bytes but intelligence is only in 2
>bytes(ethertype). Only half of the signal provides any information.
>
>LLC/SNAP: Signal is 8 bytes but intelligence is only in 2 bytes. So we get
>25% return. We have added another 4 bytes with no increase in intelligence.
>
>Hence a '4-byte ethertype' is better than the 8 byte LLC/SNAP.

The conclusion drawn here does not consider all the discussion that was put
forth
in the last msg of the thread.

If you consider what would be extra over head of 8 bytes verses 4 bytes
aganist
an MTU of 2k. It is negligible or nothing.

I think one must get outside the "try to save some bytes here and there"
line
of thinking when defining wire protocols and make sure that you dont have to
write another rfc next year to obsolete this one. All those RFCs that
defined the
LLC/SNAP headers in them were not mis-guided, would you think? You would'nt
endup
with an IB header of upto 68 bytes for just UD if saving bytes from the
header
was the goal. Recall that at wire speeds of 2-10Gbps these should not
matter.
Things like alignment, cache-line sizes, receive queue depths and page-sizes
are
the bigger fishes to fry at those rates.

~Ajoy

>
>Also, I don't think you are right in dismissing non-LLC/SNAP use. LLC/SNAP
>is not the favored mode in general - just look at 802.3 vs. DIX ethernet.



>
>
>
>                      Ajoy C
>                      Siddabathuni             To:
Ajoy.Siddabathuni@sun.com, viv@beaverton.ibm.com
>                      <Ajoy.Siddabathun        cc:
yaronh@voltaire.com,
Jerry.Chu@eng.sun.com, ipoverib@ietf.org,
>                      i@sun.com>                krause@cup.hp.com
>                      Sent by:                 Subject:  Re: [Ipoverib]
consensus call on EtherType, min MTU and DHCPv4
>                      ipoverib-admin@ie
>                      tf.org
>
>
>                      04/15/2002 03:52
>                      PM
>                      Please respond to

>                      Ajoy C
>                      Siddabathuni
>
>
>
>
>
>
>
>>It takes another 4 bytes and provides no new information. The issues
>>you indicate above are equally addressed by a 'type' field.
>
>Ofcourse it provides no new information - if you are running IPoIB
>in vacum. It is much cleaner than putting a 4 byte header with reserved
>fields. Also expect (in a few years) that there will be multi-protocol
>routers that will be sitting at the edges of an
>IB-Ethernet-osi-apple-network.
>They would definetly like to handle packets in a std way - especially when
>cp'ing pkts from one kind of "link" to another. They dont need to handle
>cases such as - "oh by the way, if this link was an IPoIB link..I needed to
>strip the top 4bytes...and not interpret the next 16 bits before I look at
>the eth-type".
>
>>
>>'No header case' is disallowed as per WG consensus. Otherwise, the
>>4-byte ethertype field is equally viable as the 8 byte LLC/SNAP field.
>>It has the advantage of being 4 bytes smaller.
>
>I would'nt consider "4 bytes smaller" as an advantage. Especially if you
>are
>working with IB header lengths upto 68 bytes. And then if you are working
>with
>TCP/IPv4/v6 add upto 60+.
>
>>
>>How about RFC2734 ? There is nothing standard about this.
>Well - time to learn from another's mistakes.
>
>Here is my summary -
>
> - Having no-header at all will give you alignment. No ethtype to mux on.
>   But you could have always figured out all IP'ish types from the
>IP-hdrtype
>   field.
>
> - Having 4byte "non-std" header will also give you alignment. Gives you
>   all the info you need. But yes, there are certain non-std elements in
>   the header that implementations would have to be aware of  - especially
>   if they were porting an existing stack.
>
> - Having 8byte std-header will be standards conforming for future
>multi-prot-
>   col enviroments. Ofcourse, retains alignment.
>
>~Ajoy
>
>
>>>
>>> Frame formats -
>>>
>>> a) If LLC/SNAP was used then you automatically get a 32bit-word
>alignment
>>>    for the IP-Datagram.
>>>
>>>    The frame would look something like --
>>>
>>>    LRH (8bytes)
>>>    GRH (40 bytes)*
>>>    BTH (12 bytes)
>>>    DETH (8 bytes)
>>>                LLC       OUI     EthType
>>>    LLC/SNAP <AA-AA-03><00-00-00><xx-xx>   (8bytes)
>>>    IPv4/IPv6 Datagram
>>>    ICRC
>>>    VCRC
>>>
>>> b) Embedding just the  ethtype - the frame would look like -- also gives
>>>    you the alignment - but is a non-std way of doing things (with
>reserved
>>>    fields etc in the format).
>>>
>>>    LRH (8bytes)
>>>    GRH (40 bytes)*
>>>    BTH (12 bytes)
>>>    DETH (8 bytes)
>>>
>>>     Rsvd   EthType
>>>    <??-??><xx-xx>  (4bytes)
>>>    IPv4/IPv6 Datagram
>>>    ICRC
>>>    VCRC
>>> ----
>>>
>>> My opinion on this is that if EthType is ever supported, it should be
>>> supported using the std mechanism (a) to be consistent with how other
>>> IPoverXXX protocols have been defined in the past. (See RFCs -ATM: 2225,
>>> 1577, 1483 SMDS:1209 FR:1294)
>>
>>How about RFC2734 ? There is nothing standard about this.
>>
>>[ As another example one can even use a 2 byte header if it works out
>>for the hardware -i.e. if one can DMA to a 2 byte boundary, the IP
>>header will land at a 4 byte boundary. ]
>>
>>Vivek
>>
>>>
>>> If the WG feels that in the foreseeable future, only IP will ever be
>>> supported, then yes there is no need for the Ethtype - ie. leave it
>alone,
>>> you can still get the IP datagram 32bit aligned right after DETH. On the
>>> other hand, if the WG feels that we do it the way our ancestors have
>done it
>>> there is enough argument already in place for supporting llc/snap hdrs.
>>>
>>> ~Ajoy
>>>
>>>
>>> >X-Sender: krause@esalpha2.cup.hp.com
>>> >Date: Mon, 15 Apr 2002 10:20:49 -0700
>>> >To: Yaron Haviv <yaronh@voltaire.com>, "'H.K. Jerry Chu'"
>>> <Jerry.Chu@eng.sun.com>, "'ipoverib@ietf.org'" <ipoverib@ietf.org>,
>>> "'Ajoy.Siddabathuni@Sun.COM'" <Ajoy.Siddabathuni@Sun.COM>
>>> >From: Michael Krause <krause@cup.hp.com>
>>> >Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
>>> >Mime-Version: 1.0
>>> >X-Mailman-Version: 1.0
>>> >List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
>>> >X-BeenThere: ipoverib@ietf.org
>>> >
>>> >At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
>>> >>Jerry,
>>> >>
>>> >>We must stop expending the definition somewhere so we get the spec out
>>> >>This is IPoIB we are talking about, there is no need for LLC/SNAP, and
>we
>>> >>just need the Ethertype
>>> >
>>> >The consensus is that people want to have the full range of Ethertype
>>> >values supported.  As such, this needs to be specified independent of
>the
>>> >IP over IB specification so that everyone may use these values without
>>> >requiring them to implement IP over IB.
>>> >
>>> >Mike
>>> >
>>> >
>>> >_______________________________________________
>>> >IPoverIB mailing list
>>> >IPoverIB@ietf.org
>>> >https://www1.ietf.org/mailman/listinfo/ipoverib
>>>
>>>
>>> _______________________________________________
>>> IPoverIB mailing list
>>> IPoverIB@ietf.org
>>> https://www1.ietf.org/mailman/listinfo/ipoverib
>>>
>>
>>
>>--
>>Vivek Kashyap
>>Linux Technology Center, IBM
>>kashyapv@us.ibm.com
>>vivk@us.ibm.com
>>503 578 3422 (o)
>
>
>_______________________________________________
>IPoverIB mailing list
>IPoverIB@ietf.org
>https://www1.ietf.org/mailman/listinfo/ipoverib
>
>
>
>
>
>_______________________________________________
>IPoverIB mailing list
>IPoverIB@ietf.org
>https://www1.ietf.org/mailman/listinfo/ipoverib





_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 17 02:56:34 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA11554
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 02:56:30 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id CAA19713
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 02:56:32 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id CAA19419;
	Wed, 17 Apr 2002 02:49:47 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id CAA19356
	for <ipoverib@optimus.ietf.org>; Wed, 17 Apr 2002 02:49:44 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA11464
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 02:49:40 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3H6nhLQ142036;
	Wed, 17 Apr 2002 02:49:43 -0400
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3H6nga101734;
	Wed, 17 Apr 2002 02:49:42 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g3H6neX30333;
	Tue, 16 Apr 2002 23:49:40 -0700
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3H6ndf05005;
	Tue, 16 Apr 2002 23:49:39 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204170649.g3H6ndf05005@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
To: Ajoy.Siddabathuni@sun.com
Date: Tue, 16 Apr 2002 23:49:38 -0700 (PDT)
Cc: vivk@us.ibm.com, ipoverib@ietf.org, ipoverib-admin%iipoverib@us.ibm.com
In-Reply-To: <200204170241.TAA04430@phys-ha10nwka.ebay.sun.com> from "Ajoy C Siddabathuni" at Apr 16, 2002 06:41:30 PM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit


> 
> 
> >4 byte header:Signal is 4 bytes but intelligence is only in 2
> >bytes(ethertype). Only half of the signal provides any information.
> >
> >LLC/SNAP: Signal is 8 bytes but intelligence is only in 2 bytes. So we get
> >25% return. We have added another 4 bytes with no increase in intelligence.
> >
> >Hence a '4-byte ethertype' is better than the 8 byte LLC/SNAP.
> 
> The conclusion drawn here does not consider all the discussion that was put forth
> in the last msg of the thread.

	See below.

> 
> If you consider what would be extra over head of 8 bytes verses 4 bytes aganist
> an MTU of 2k. It is negligible or nothing.

        I have suggested that there is no need to have an overhead
        when it provides no intelligence. Whether the overhead is
        negligible or not, it should have a reason to exist.

> 
> I think one must get outside the "try to save some bytes here and there" line
> of thinking when defining wire protocols and make sure that you dont have to
> write another rfc next year to obsolete this one. All those RFCs that defined the

	And why will it become obsolete ? 

> LLC/SNAP headers in them were not mis-guided, would you think? You would'nt endup

        Oh c'mon! you are the one who belittled the most recent RFC in
        the list: RFC2734. You exhorted us to "learn from others'
        mistakes" regarding this RFC. My comment was that just because
        some RFCs used LLC/SNAP, it does not make it a standard but
        just one way of doing things.

        And I do think that LLC/SNAP is useless - the 3 bytes of LLC
        provide nothing. Then you have 3 bytes of OUI before getting
        to the type field. Those 6 bytes might as well not exist.


> with an IB header of upto 68 bytes for just UD if saving bytes from the header
> was the goal. Recall that at wire speeds of 2-10Gbps these should not matter.

        Just because one can afford to be lax doesn't mean one should
        be. What good is deliberately adding bytes that provide
        nothing? 

> Things like alignment, cache-line sizes, receive queue depths and page-sizes are
> the bigger fishes to fry at those rates.

        Good point. And how are all these helped by adding unnecessary
        overhead? 

	 
> 
> ~Ajoy
> 
> >
> >Also, I don't think you are right in dismissing non-LLC/SNAP use. LLC/SNAP
> >is not the favored mode in general - just look at 802.3 vs. DIX ethernet.

        You mentioned working with multiprotocols in 'edge routers' -
        what about the ubiquitous DIX ethernet ? No header at all,
        what of LLC/SNAP! You mentioned ATM - as far as I know it
        suggests that the encapsulation format is negotiable. So, you
        can work with or without LLC/SNAP. LLC/SNAP doesn't seem to be
        the common encapsulation format.

        I, for one, need a lot more convincing before I can support
        the idea of LLC/SNAP being the common encapsulation in the
        future.  

Vivek

> 
> 
> 
> >
> >
> >                                                                                                                                   
> >                      Ajoy C                                                                                                       
> >                      Siddabathuni             To:       
> Ajoy.Siddabathuni@sun.com, viv@beaverton.ibm.com                          
> >                      <Ajoy.Siddabathun        cc:       yaronh@voltaire.com, 
> Jerry.Chu@eng.sun.com, ipoverib@ietf.org,            
> >                      i@sun.com>                krause@cup.hp.com                                                                  
> >                      Sent by:                 Subject:  Re: [Ipoverib] 
> consensus call on EtherType, min MTU and DHCPv4            
> >                      ipoverib-admin@ie                                                                                            
> >                      tf.org                                                                                                       
> >                                                                                                                                   
> >                                                                                                                                   
> >                      04/15/2002 03:52                                                                                             
> >                      PM                                                                                                           
> >                      Please respond to                                                                                            
> >                      Ajoy C                                                                                                       
> >                      Siddabathuni                                                                                                 
> >                                                                                                                                   
> >                                                                                                                                   
> >
> >
> >
> >
> >
> >>It takes another 4 bytes and provides no new information. The issues
> >>you indicate above are equally addressed by a 'type' field.
> >
> >Ofcourse it provides no new information - if you are running IPoIB
> >in vacum. It is much cleaner than putting a 4 byte header with reserved
> >fields. Also expect (in a few years) that there will be multi-protocol
> >routers that will be sitting at the edges of an
> >IB-Ethernet-osi-apple-network.
> >They would definetly like to handle packets in a std way - especially when
> >cp'ing pkts from one kind of "link" to another. They dont need to handle
> >cases such as - "oh by the way, if this link was an IPoIB link..I needed to
> >strip the top 4bytes...and not interpret the next 16 bits before I look at
> >the eth-type".
> >
> >>
> >>'No header case' is disallowed as per WG consensus. Otherwise, the
> >>4-byte ethertype field is equally viable as the 8 byte LLC/SNAP field.
> >>It has the advantage of being 4 bytes smaller.
> >
> >I would'nt consider "4 bytes smaller" as an advantage. Especially if you
> >are
> >working with IB header lengths upto 68 bytes. And then if you are working
> >with
> >TCP/IPv4/v6 add upto 60+.
> >
> >>
> >>How about RFC2734 ? There is nothing standard about this.
> >Well - time to learn from another's mistakes.
> >
> >Here is my summary -
> >
> > - Having no-header at all will give you alignment. No ethtype to mux on.
> >   But you could have always figured out all IP'ish types from the
> >IP-hdrtype
> >   field.
> >
> > - Having 4byte "non-std" header will also give you alignment. Gives you
> >   all the info you need. But yes, there are certain non-std elements in
> >   the header that implementations would have to be aware of  - especially
> >   if they were porting an existing stack.
> >
> > - Having 8byte std-header will be standards conforming for future
> >multi-prot-
> >   col enviroments. Ofcourse, retains alignment.
> >
> >~Ajoy
> >
> >
> >>>
> >>> Frame formats -
> >>>
> >>> a) If LLC/SNAP was used then you automatically get a 32bit-word
> >alignment
> >>>    for the IP-Datagram.
> >>>
> >>>    The frame would look something like --
> >>>
> >>>    LRH (8bytes)
> >>>    GRH (40 bytes)*
> >>>    BTH (12 bytes)
> >>>    DETH (8 bytes)
> >>>                LLC       OUI     EthType
> >>>    LLC/SNAP <AA-AA-03><00-00-00><xx-xx>   (8bytes)
> >>>    IPv4/IPv6 Datagram
> >>>    ICRC
> >>>    VCRC
> >>>
> >>> b) Embedding just the  ethtype - the frame would look like -- also gives
> >>>    you the alignment - but is a non-std way of doing things (with
> >reserved
> >>>    fields etc in the format).
> >>>
> >>>    LRH (8bytes)
> >>>    GRH (40 bytes)*
> >>>    BTH (12 bytes)
> >>>    DETH (8 bytes)
> >>>
> >>>     Rsvd   EthType
> >>>    <??-??><xx-xx>  (4bytes)
> >>>    IPv4/IPv6 Datagram
> >>>    ICRC
> >>>    VCRC
> >>> ----
> >>>
> >>> My opinion on this is that if EthType is ever supported, it should be
> >>> supported using the std mechanism (a) to be consistent with how other
> >>> IPoverXXX protocols have been defined in the past. (See RFCs -ATM: 2225,
> >>> 1577, 1483 SMDS:1209 FR:1294)
> >>
> >>How about RFC2734 ? There is nothing standard about this.
> >>
> >>[ As another example one can even use a 2 byte header if it works out
> >>for the hardware -i.e. if one can DMA to a 2 byte boundary, the IP
> >>header will land at a 4 byte boundary. ]
> >>
> >>Vivek
> >>
> >>>
> >>> If the WG feels that in the foreseeable future, only IP will ever be
> >>> supported, then yes there is no need for the Ethtype - ie. leave it
> >alone,
> >>> you can still get the IP datagram 32bit aligned right after DETH. On the
> >>> other hand, if the WG feels that we do it the way our ancestors have
> >done it
> >>> there is enough argument already in place for supporting llc/snap hdrs.
> >>>
> >>> ~Ajoy
> >>>
> >>>
> >>> >X-Sender: krause@esalpha2.cup.hp.com
> >>> >Date: Mon, 15 Apr 2002 10:20:49 -0700
> >>> >To: Yaron Haviv <yaronh@voltaire.com>, "'H.K. Jerry Chu'"
> >>> <Jerry.Chu@eng.sun.com>, "'ipoverib@ietf.org'" <ipoverib@ietf.org>,
> >>> "'Ajoy.Siddabathuni@Sun.COM'" <Ajoy.Siddabathuni@Sun.COM>
> >>> >From: Michael Krause <krause@cup.hp.com>
> >>> >Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
> >>> >Mime-Version: 1.0
> >>> >X-Mailman-Version: 1.0
> >>> >List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
> >>> >X-BeenThere: ipoverib@ietf.org
> >>> >
> >>> >At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
> >>> >>Jerry,
> >>> >>
> >>> >>We must stop expending the definition somewhere so we get the spec out
> >>> >>This is IPoIB we are talking about, there is no need for LLC/SNAP, and
> >we
> >>> >>just need the Ethertype
> >>> >
> >>> >The consensus is that people want to have the full range of Ethertype
> >>> >values supported.  As such, this needs to be specified independent of
> >the
> >>> >IP over IB specification so that everyone may use these values without
> >>> >requiring them to implement IP over IB.
> >>> >
> >>> >Mike
> >>> >
> >>> >
> >>> >_______________________________________________
> >>> >IPoverIB mailing list
> >>> >IPoverIB@ietf.org
> >>> >https://www1.ietf.org/mailman/listinfo/ipoverib
> >>>
> >>>
> >>> _______________________________________________
> >>> IPoverIB mailing list
> >>> IPoverIB@ietf.org
> >>> https://www1.ietf.org/mailman/listinfo/ipoverib
> >>>
> >>
> >>
> >>--
> >>Vivek Kashyap
> >>Linux Technology Center, IBM
> >>kashyapv@us.ibm.com
> >>vivk@us.ibm.com
> >>503 578 3422 (o)
> >
> >
> >_______________________________________________
> >IPoverIB mailing list
> >IPoverIB@ietf.org
> >https://www1.ietf.org/mailman/listinfo/ipoverib
> >
> >
> >
> >
> >
> >_______________________________________________
> >IPoverIB mailing list
> >IPoverIB@ietf.org
> >https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)
-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 17 03:14:18 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA11690
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 03:14:17 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id DAA20825
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 03:14:20 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id DAA20626;
	Wed, 17 Apr 2002 03:07:28 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA13321
	for <ipoverib@ns.ietf.org>; Tue, 16 Apr 2002 09:19:26 -0400 (EDT)
Received: from CNRI.Reston.VA.US (localhost [127.0.0.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA01799;
	Tue, 16 Apr 2002 09:19:22 -0400 (EDT)
Message-Id: <200204161319.JAA01799@ietf.org>
Mime-Version: 1.0
Content-Type: Multipart/Mixed; Boundary="NextPart"
To: IETF-Announce: ;
Cc: ipoverib@ietf.org
From: Internet-Drafts@ietf.org
Reply-to: Internet-Drafts@ietf.org
Date: Tue, 16 Apr 2002 09:19:21 -0400
Subject: [Ipoverib] I-D ACTION:draft-ietf-ipoib-link-multicast-01.txt
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the IP over InfiniBand Working Group of the IETF.

	Title		: IP link and multicast over InfiniBand networks
	Author(s)	: J. Chu, V. Kashyap
	Filename	: draft-ietf-ipoib-link-multicast-01.txt
	Pages		: 12
	Date		: 15-Apr-02
	
This document specifies a method for setting up IP subnets and
multicast services over InfiniBand(TM) networks. Discussions in this
document are applicable to both IPv4 and IPv6, unless explicitly
specified. A separate document will cover unicast and encapsulation
of IP datagrams over InfiniBand networks.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-ipoib-link-multicast-01.txt

To remove yourself from the IETF Announcement list, send a message to 
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-ipoib-link-multicast-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt


Internet-Drafts can also be obtained by e-mail.

Send a message to:
	mailserv@ietf.org.
In the body type:
	"FILE /internet-drafts/draft-ietf-ipoib-link-multicast-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.
		
		
Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="mailserv@ietf.org"

Content-Type: text/plain
Content-ID:	<20020415112524.I-D@ietf.org>

ENCODING mime
FILE /internet-drafts/draft-ietf-ipoib-link-multicast-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-ipoib-link-multicast-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<20020415112524.I-D@ietf.org>

--OtherAccess--

--NextPart--




_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 17 04:35:32 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA12582
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 04:35:31 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id EAA24959
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 04:35:34 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id EAA24319;
	Wed, 17 Apr 2002 04:28:34 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id EAA24287
	for <ipoverib@optimus.ietf.org>; Wed, 17 Apr 2002 04:28:32 -0400 (EDT)
Received: from mxout2.netvision.net.il (mxout2.netvision.net.il [194.90.9.21])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA12510
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 04:28:28 -0400 (EDT)
Received: from vaio505he ([139.92.208.198]) by mxout2.netvision.net.il
 (iPlanet Messaging Server 5.2 (built Feb 21 2002))
 with ESMTPA id <0GUP00JD4E6F2C@mxout2.netvision.net.il> for ipoverib@ietf.org;
 Wed, 17 Apr 2002 11:27:59 +0300 (IDT)
Date: Wed, 17 Apr 2002 10:28:32 +0200
From: Yaron Haviv <yaronh@voltaire.com>
Subject: RE: [Ipoverib] IPoIB multicast forwarding
In-reply-to: <D9223EB959A5D511A98F00508B68C20C06BC4F92@ORSMSX108>
To: "'Larsen, Roy K'" <roy.k.larsen@intel.com>,
        "'Bill Strahm'" <bill@strahm.net>,
        "'Vivek Kashyap'" <kashyapv@us.ibm.com>
Cc: "'Michael Krause'" <krause@cup.hp.com>, adube@windows.microsoft.com,
        ipoverib@ietf.org
Reply-to: yaronh@voltaire.com
Message-id: <002001c1e5e9$dccf5340$c6d05c8b@vaio505he>
MIME-version: 1.0
X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4807.1700
X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2910.0)
Content-type: text/plain; charset=iso-8859-1
Content-transfer-encoding: 7BIT
Importance: Normal
X-Priority: 3 (Normal)
X-MSMail-priority: Normal
Content-Transfer-Encoding: 7BIT
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7BIT

Roy, in general I agree with you, since we have to have something running
this year
However we need to have some flexibility for future expansion

Another problem we haven't discussed is that Multicast LID's number is
limited, and in each CA it is a different number
Not to mention that the MLID's are used for other things beside IPoIB
This might create a problem where you can fail when trying to join a group,
or configure the CA
In Ethernet a NIC can just pass all the multicast traffic to the stack if it
needs more groups

We need to enable both the sender and receiver to use either broadcast or
multicast in an interoperable manner
The problem occurs when the sender sends to a multicast group and the
receiver listens only on the broadcast group
Since it failed joining the multicast group, or it even didn't try doing so
We need to have a mechanism telling the sender it can't use multicast for
that group, and it should use broadcast

This could be achieved is several ways:
- Manually: forcing all sender traffic to use broadcast (used initially,
answers Roy's request)
- Limiting the number of IPoIB MCGroups at the SM (where zero is also a
valid value), Whenever we exceed this number we can use broadcast for the
rest
- Any thing you can also think of

This approach also address the router issues, as well as the leave issues
(by deleting the group at the SM)

Example: limiting number of IPoIB MCGroups, Allocated MLID's=3
(Initially we can set the number of allocated MLID's to zero, and get all
broadcasted, in future tune the Multicast to use a non zero
This way we can both maintain interoperability with future implementations,
and solve the MLID limitation)

Receiver:
	Try to create/join the group
		SM: If less & eq 3 return info, if more than 3 return fail
		(if MLID's=zero fail all the time, or if manually configured avoid doing
the SM join request)
	If failed use broadcast else use mcast

Sender
	Want to send to mcast and not already a member
	Try to join, if failed use broadcast
		** to solve the issue where the multicast is created later and we want to
use it we can:
 1. Check with the SM periodically or every few packets
 2. Or register for change notifications on the MCGroup

This way we can accommodate both approaches, and help the early adopters
come up with implementations
Even before the IB MngWG and SM vendors finish defining and implementing the
mcast mechanisms
Otherwise I agree with Roy we wont manage to have a working solution this
year

Yaron

-----Original Message-----
From: ipoverib-admin@ietf.org [mailto:ipoverib-admin@ietf.org]On Behalf Of
Larsen, Roy K
Sent: Wednesday, April 17, 2002 1:55 AM
To: 'Bill Strahm'; Vivek Kashyap
Cc: Michael Krause; adube@windows.microsoft.com; ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast forwarding

Given all the uncertainty of multicast group semantics and some of the
algorithms we're coming up with for presumed correctness, I still lean
toward multicast = broadcast.  It is guaranteed to be correct and straight
forward to implement.  We can always come back to it once we get some
experience with how multicast is going to be used in the data center and
time for the IBTA to fix/clarify group semantics.  The way it stands now,
the SM implementations would seem to have a lot of flexibility which will
lead to interoperability problems.  IBTA needs to fix this before we can
reliably use this in IPoIB.

Roy

-----Original Message-----
From: Bill Strahm [mailto:bill@strahm.net]
Sent: Tuesday, April 16, 2002 4:02 PM
To: Vivek Kashyap
Cc: Michael Krause; adube@windows.microsoft.com; ipoverib@ietf.org
Subject: Re: [Ipoverib] IPoIB multicast forwarding


I don't think it is that easy... How will the router know to leave ? It
can't
be when there are no receivers, because it has to forward any traffic from
the senders to other locations on the network...  This is especially true
for
flood/prune algorithms (see PIM-DM)

I'm also not sure that this would work in an intermitant sender environment.

I may have a sender that only wants to multicast trigger events, lets say
an SNMP TRAP message that happens once in a blue moon...  This means that
there is the overhead of group creating and deletion with every send (and
you still have the problem with the router not being on the line)

I am very tempted to do some deep thinking about having ALL multicast
packets
go to a default multicast traffic IB address, and have the IP portion echo
the packets back to the IB subnet on the correct IB MC Group if there are
konwn subscribers (through IGMP etc) and forward on to other ports depending
on the IP multicast forwarding tree...

This means that all locally sent MC traffic goes onto the fabric twice, once
to get to the router on a "All multicast" traffic channel, and once to the
correct multicast channel...

Does IB provide a mechanism to allow the Router to detect multicast group
creations... if it does, then we can use the existance of the group rather
than needing IGMP, I don't think this has been defined in IBTA fully however

Bill
On Tue, Apr 16, 2002 at 10:58:43AM -0700, Vivek Kashyap wrote:
> Great! So the deletion of the group is not a difficult issue at all.
> The IPoIB nodes only need to leave the group.
>
>         The receiver leaves when it doesn't need to receive any more.
>         Router when there are no receivers and it knows it doesn't
>         need to forward. sender after some idle time i.e. no activity
>         on the group. My suggestion is to have a minimum timeout of 2
>         minutes - avoids group join/leave happening too often.
>
> Vivek
>
> >
> > At 04:31 PM 4/15/2002 -0700, Vivek Kashyap wrote:
> > >         a) No delete is done by the IPoIB interfaces, it is an SM
issue
> > >
> > >         This I took from the IB architecture document in chapter 3. It
> > >         talks of the SM deleting unused groups. ( Now, there are a few
> > >         who say that this is not going to be the case -- in which case
> > >         a different solution is needed.)
> > >
> > >         The receiver leaves when it no longer wants to receive. The
> > >         router when there are no more receivers. The sender after some
> > >         idle time.
> >
> > The specification states that a multicast management entity not the SM
is
> > responsible for multicast group management.  This entity implement IB
> > multicast group management and could also be responsible for resolving
IP
> > multicast addressing to IB multicast groups.  It may also be responsible

> > for informing / updating participating IB routers of any changes (and
vice
> > versa for the IB routers informing the entity of changes within its view
of
> > the router cloud).  This entity could also be used to provide a similar
> > service for CAs that act as multi-protocol router endnodes.
> >
> > >         b) Any entity wanting to leave an IB group attempts to delete
> > >         it. If there are members in the group then the deletion should
> > >         fail. The last one will automatically delete the group.
> > >
> > >         [Depends on delete semantics]
> >
> > If an endnode wants to leave an IB multicast group, it informs the
> > management entity of this event and the management entity working with
the
> > SM removes this endnode from the IB multicast tree.  If the entity
> > determines that there are no longer any endnodes participating in the
> > multicast group, it may delete the entire group from the fabric.  This
does
> > not require the endnode to delete the group and deal with any failure
> > semantics.  An endnode should simply delete itself and let the
management
> > entity deal with the problem.
> >
> > >         c) Leave is fine. No need to bother about delete.
> > >
> > >         let the group entries stay around in the database. They don't
> > >         bother anyone. The admin can periodically, once in a few
> > >         months may be, delete unused ones. The entries will get reused
> > >         if others join. The multicast tree is affected by joins and
> > >         leaves and not by deletes.
> >
> > Delete should be performed automatically by the management entity; this
is
> > a policy decision that has no bearing on the IP over IB specification.
> >
> > >         d) {Don't know about this }
> > >
> > >         The setup is such that if the last member leaves the group the
> > >         group is automatically removed.
> >
> > This is how IB specifies it should work.
> >
> > Mike
> >
> >
> >
>
>
> --
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com
> vivk@us.ibm.com
> 503 578 3422 (o)
>
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 17 09:36:06 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA17138
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 09:36:05 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id JAA09611
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 09:36:09 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA08819;
	Wed, 17 Apr 2002 09:28:49 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA08783
	for <ipoverib@optimus.ietf.org>; Wed, 17 Apr 2002 09:28:45 -0400 (EDT)
Received: from palrel10.hp.com (palrel10.hp.com [156.153.255.245])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA16947
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 09:28:41 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel10.hp.com (Postfix) with ESMTP
	id 1A81AC00F09; Wed, 17 Apr 2002 06:28:00 -0700 (PDT)
Received: from mkrause.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id GAA11856;
	Wed, 17 Apr 2002 06:27:59 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020417062349.00a8c138@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Wed, 17 Apr 2002 06:27:55 -0700
To: "Ajoy C Siddabathuni" <Ajoy.Siddabathuni@sun.com>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
Cc: <ipoverib@ietf.org>
In-Reply-To: <HHEDIJGOEFLPCPNILMPACEFLCIAA.bill@strahm.net>
References: <200204170241.TAA04430@phys-ha10nwka.ebay.sun.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 10:51 PM 4/16/2002 -0700, Bill Strahm wrote:
>I don't agree with the consideration that "Its just 4 bytes out of
>thousands"
>
>Like the politicians say "A million here, A million there, next thing you
>know we are talking about real money"
>
>Every byte matters.  If a byte doesn't add value why have it...  I am
>comfortable with a 4 byte Ethertype because that maintains DWORD alignment,
>and is not horrible... I get concerned that people who won the 4 byte
>debate, now want to start talking about 8 bytes... Where next 16, 32 ???
>
>I don't see a full LLC/SNAP header adding any value, so why have it then ?

2/3 of IP packets are less than 256 bytes (1/3 are less than 50 bytes) of 
payload so this continual adding of, IMO, unnecessary overhead creates more 
wire inefficiency for the bulk of traffic and increases the chance of 
crossing into another cache line for various workloads which directly 
impacts the memory controllers, increases cache pollution, etc.  The whole 
argument of link bandwidth is so high so this isn't a problem is 
irrelevant.  It isn't link bandwidth that is the problem within any system 
- it is memory bandwidth that is and will remain potentially for all time 
the primary bottleneck.   Any new communication technology that does not 
focus on memory bandwidth as one of its primary concerns is not a well 
conceived and potentially not a well used technology.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 17 10:28:38 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA20182
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 10:28:38 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id KAA13406
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 10:28:42 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA12316;
	Wed, 17 Apr 2002 10:14:05 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA12283
	for <ipoverib@optimus.ietf.org>; Wed, 17 Apr 2002 10:14:03 -0400 (EDT)
Received: from palrel12.hp.com (palrel12.hp.com [156.153.255.237])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA19079
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 10:13:58 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel12.hp.com (Postfix) with ESMTP id B8989E0058F
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 07:13:31 -0700 (PDT)
Received: from mkrause.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id HAA18808
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 07:13:31 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020417070720.02a367a0@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Wed, 17 Apr 2002 07:13:27 -0700
To: ipoverib@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast forwarding
In-Reply-To: <002001c1e5e9$dccf5340$c6d05c8b@vaio505he>
References: <D9223EB959A5D511A98F00508B68C20C06BC4F92@ORSMSX108>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 10:28 AM 4/17/2002 +0200, Yaron Haviv wrote:
>Roy, in general I agree with you, since we have to have something running
>this year However we need to have some flexibility for future expansion

The IBTA is trying to address this problem as we speak which will allow one 
to ship product this year....something a bit similar to what I described in 
a previous e-mail is being proposed and this could be used to simplify this 
whole discussion and eliminate this ambiguity being debated here.  One does 
not need to go into a broadcast solution.  One does not need to flip 
between known multicast group to all routers.  One only needs to obtain / 
track the multicast group identifiers that one is using and leave the rest 
of these problems to IB to resolve.

Again, a multiprotocol router is nothing more than an IB CA (nominally a 
TCA but could be a HCA) that joins an IB multicast group just like a normal 
IB endnode.  It is the one responsible for interpreting the layer 3 headers 
and determining how to forward the packet.  It is the one responsible for 
layer 3 multicast group management not IB and not IP over IB in particular.

If IP over IB were to focus on how to join an IB multicast group and how to 
encapsulate IP multicast addresses within IB MGIDs, then this work would be 
greatly accelerated.   Leave the IP router issues to their appropriate 
workgroups which I do not believe is this workgroup in particular.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Wed Apr 17 14:13:32 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA29998
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 14:13:32 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA05449
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 14:13:34 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA05088;
	Wed, 17 Apr 2002 14:07:52 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA05054
	for <ipoverib@optimus.ietf.org>; Wed, 17 Apr 2002 14:07:50 -0400 (EDT)
Received: from mail.brocade.com (sj6-b.brocade.com [63.121.140.220])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA29805
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 14:07:47 -0400 (EDT)
Received: from hq-ex-c2.brocade.com (hq-ex-c2 [192.168.126.35])
	by mail.brocade.com (8.8.8+Sun/8.8.8) with ESMTP id LAA12442;
	Wed, 17 Apr 2002 11:02:57 -0700 (PDT)
Received: by hq-ex-c2.brocade.com with Internet Mail Service (5.5.2653.19)
	id <H8FDPRB0>; Wed, 17 Apr 2002 11:02:56 -0700
Message-ID: <B6AB380934B5F0488974238446762E4A84850F@hq-ex-5>
From: Brian Forbes <bforbes@brocade.com>
To: "'Bill Strahm'" <bill@strahm.net>, Vivek Kashyap <kashyapv@us.ibm.com>
Cc: Michael Krause <krause@cup.hp.com>, adube@windows.microsoft.com,
        ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast forwarding
Date: Wed, 17 Apr 2002 11:02:54 -0700
X-Mailer: Internet Mail Service (5.5.2653.19)
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Bill, you are correct that router interaction with MC groups has not been
fully defined.  I think it's safe to assume that anything that is useful can
be done, but you can't assume that the IB and IP semantics will/can be
identical.

Brian Forbes
Brocade Communications 

-----Original Message-----
From: Bill Strahm [mailto:bill@strahm.net]
Sent: Tuesday, April 16, 2002 4:02 PM
To: Vivek Kashyap
Cc: Michael Krause; adube@windows.microsoft.com; ipoverib@ietf.org
Subject: Re: [Ipoverib] IPoIB multicast forwarding


I don't think it is that easy... How will the router know to leave ? It
can't
be when there are no receivers, because it has to forward any traffic from
the senders to other locations on the network...  This is especially true
for
flood/prune algorithms (see PIM-DM)

I'm also not sure that this would work in an intermitant sender environment.

I may have a sender that only wants to multicast trigger events, lets say
an SNMP TRAP message that happens once in a blue moon...  This means that
there is the overhead of group creating and deletion with every send (and
you still have the problem with the router not being on the line)

I am very tempted to do some deep thinking about having ALL multicast
packets
go to a default multicast traffic IB address, and have the IP portion echo
the packets back to the IB subnet on the correct IB MC Group if there are 
konwn subscribers (through IGMP etc) and forward on to other ports depending
on the IP multicast forwarding tree...

This means that all locally sent MC traffic goes onto the fabric twice, once
to get to the router on a "All multicast" traffic channel, and once to the
correct multicast channel...

Does IB provide a mechanism to allow the Router to detect multicast group
creations... if it does, then we can use the existance of the group rather
than needing IGMP, I don't think this has been defined in IBTA fully however

Bill
On Tue, Apr 16, 2002 at 10:58:43AM -0700, Vivek Kashyap wrote:
> Great! So the deletion of the group is not a difficult issue at all.
> The IPoIB nodes only need to leave the group.
> 
>         The receiver leaves when it doesn't need to receive any more.
>         Router when there are no receivers and it knows it doesn't
>         need to forward. sender after some idle time i.e. no activity
>         on the group. My suggestion is to have a minimum timeout of 2
>         minutes - avoids group join/leave happening too often.
> 
> Vivek
> 
> > 
> > At 04:31 PM 4/15/2002 -0700, Vivek Kashyap wrote:
> > >         a) No delete is done by the IPoIB interfaces, it is an SM
issue
> > >
> > >         This I took from the IB architecture document in chapter 3. It
> > >         talks of the SM deleting unused groups. ( Now, there are a few
> > >         who say that this is not going to be the case -- in which case
> > >         a different solution is needed.)
> > >
> > >         The receiver leaves when it no longer wants to receive. The
> > >         router when there are no more receivers. The sender after some
> > >         idle time.
> > 
> > The specification states that a multicast management entity not the SM
is 
> > responsible for multicast group management.  This entity implement IB 
> > multicast group management and could also be responsible for resolving
IP 
> > multicast addressing to IB multicast groups.  It may also be responsible

> > for informing / updating participating IB routers of any changes (and
vice 
> > versa for the IB routers informing the entity of changes within its view
of 
> > the router cloud).  This entity could also be used to provide a similar 
> > service for CAs that act as multi-protocol router endnodes.
> > 
> > >         b) Any entity wanting to leave an IB group attempts to delete
> > >         it. If there are members in the group then the deletion should
> > >         fail. The last one will automatically delete the group.
> > >
> > >         [Depends on delete semantics]
> > 
> > If an endnode wants to leave an IB multicast group, it informs the 
> > management entity of this event and the management entity working with
the 
> > SM removes this endnode from the IB multicast tree.  If the entity 
> > determines that there are no longer any endnodes participating in the 
> > multicast group, it may delete the entire group from the fabric.  This
does 
> > not require the endnode to delete the group and deal with any failure 
> > semantics.  An endnode should simply delete itself and let the
management 
> > entity deal with the problem.
> > 
> > >         c) Leave is fine. No need to bother about delete.
> > >
> > >         let the group entries stay around in the database. They don't
> > >         bother anyone. The admin can periodically, once in a few
> > >         months may be, delete unused ones. The entries will get reused
> > >         if others join. The multicast tree is affected by joins and
> > >         leaves and not by deletes.
> > 
> > Delete should be performed automatically by the management entity; this
is 
> > a policy decision that has no bearing on the IP over IB specification.
> > 
> > >         d) {Don't know about this }
> > >
> > >         The setup is such that if the last member leaves the group the
> > >         group is automatically removed.
> > 
> > This is how IB specifies it should work.
> > 
> > Mike
> > 
> > 
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Wed Apr 17 14:27:49 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA00533
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 14:27:48 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA06169
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 14:27:51 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA05927;
	Wed, 17 Apr 2002 14:23:34 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA05894
	for <ipoverib@ns.ietf.org>; Wed, 17 Apr 2002 14:23:32 -0400 (EDT)
Received: from mail4.microsoft.com (mail4.microsoft.com [131.107.3.122])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA00322
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 14:23:28 -0400 (EDT)
Received: from inet-vrs-04.redmond.corp.microsoft.com ([157.54.8.154]) by mail4.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Wed, 17 Apr 2002 11:22:48 -0700
Received: from 157.54.5.25 by inet-vrs-04.redmond.corp.microsoft.com (InterScan E-Mail VirusWall NT); Wed, 17 Apr 2002 11:23:00 -0700
Received: from red-imc-02.redmond.corp.microsoft.com ([157.54.9.107]) by inet-hub-03.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Wed, 17 Apr 2002 11:23:00 -0700
Received: from win-imc-02.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.84]) by red-imc-02.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Wed, 17 Apr 2002 11:22:49 -0700
Received: from win-msg-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.52]) by win-imc-02.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(6.0.3590.0);
	 Wed, 17 Apr 2002 11:19:25 -0700
X-MimeOLE: Produced By Microsoft Exchange V6.0.6177.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Subject: RE: [Ipoverib] MIB's ifPhysAddress needs to be updated
Date: Wed, 17 Apr 2002 11:19:24 -0700
Message-ID: <2E33960095B58E40A4D3345AB9F65EC102450E3C@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Thread-Topic: [Ipoverib] MIB's ifPhysAddress needs to be updated
Thread-Index: AcHlewYNeauTrWzFTEWjO+fFwa7x1QAv3Rrw
From: "Aditya Dube" <adube@windows.microsoft.com>
To: <ipoverib@ietf.org>
X-OriginalArrivalTime: 17 Apr 2002 18:19:25.0062 (UTC) FILETIME=[676BAA60:01C1E63C]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id OAA05895
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 8bit


As the Lid is an assigned address rather than a burned in address, as in
Ethernet, the chances of duplicate ifphysaddr's in a network spanning
multiple IB subnets are a lot higher. We should reduce this chance as
much as possible.

The ifphysaddr is also not in sync with what IpOverIB has decided to use
as the link address in ARPs.

IMHO the two, ifphysaddr and the link address is an arp packet, should
match.

Aditya Dube

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
Sent: Tuesday, April 16, 2002 12:18 PM
To: Aditya Dube
Cc: ipoverib@ietf.org
Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated

The same question was addressed last october/november.
See the message:

http://www1.ietf.org/mail-archive/working-groups/ipoverib/current/msg000
87.html

Vivek



> 
> This is a multi-part message in MIME format.
> 
> --------------InterScan_NT_MIME_Boundary
> Content-Type: multipart/alternative;
> 	boundary="----_=_NextPart_001_01C1E577.F19F4F03"
> 
> ------_=_NextPart_001_01C1E577.F19F4F03
> Content-Type: text/plain;
> 	charset="us-ascii"
> Content-Transfer-Encoding: quoted-printable
> 
> 
> In the MIB draft, the ifPhysAddress is specified as the LID. This is
out
> of sync with the IpOverIb draft. Can the ifPhysAddress in the MIB be
> changed to match  IPoverIB's physical address, i.e. the GID + QPN?
> 
> http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.txt
> 
> Aditya Dube
> 
> 

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Wed Apr 17 14:37:01 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA00935
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 14:37:00 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA07105
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 14:37:03 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA06970;
	Wed, 17 Apr 2002 14:32:51 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA06937
	for <ipoverib@ns.ietf.org>; Wed, 17 Apr 2002 14:32:49 -0400 (EDT)
Received: from strahm.net (adsl-67-115-29-109.dsl.sndg02.pacbell.net [67.115.29.109])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA00718
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 14:32:45 -0400 (EDT)
Received: (from bill@localhost)
	by strahm.net (8.11.6/8.11.6) id g3HIYLB95837;
	Wed, 17 Apr 2002 11:34:21 -0700 (PDT)
	(envelope-from bill)
Date: Wed, 17 Apr 2002 11:34:21 -0700
From: Bill Strahm <bill@strahm.net>
To: Aditya Dube <adube@windows.microsoft.com>
Cc: ipoverib@ietf.org
Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
Message-ID: <20020417113421.A95811@homegate.strahm.net>
References: <2E33960095B58E40A4D3345AB9F65EC102450E3C@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <2E33960095B58E40A4D3345AB9F65EC102450E3C@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>; from adube@windows.microsoft.com on Wed, Apr 17, 2002 at 11:19:24AM -0700
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

The problem is that ifPhysAddr covers more than just IP, it is just
the layer 2 wire representation of address.  If you can show me an
IB format that includes GID+LID in a packet header, I might conceed
defeat here.

The address duplicating across multiple subnets is a red herring.  You
can assign identical Ethernet MAC addresses, so long as they are on
differing subnets as well.  

From RFC 2863...
            "The interface's address at its protocol sub-layer.  For
            example, for an 802.x interface, this object normally
            contains a MAC address.  The interface's media-specific MIB
            must define the bit and byte ordering and the format of the
            value of this object.  For interfaces which do not have such
            an address (e.g., a serial line), this object should contain
            an octet string of zero length."
Note that there is no mention of any Layer 3 semantics.  

We can bring the topic up for discussion, I don't know which of the MIB
authors are currently active right now though.

Bill

On Wed, Apr 17, 2002 at 11:19:24AM -0700, Aditya Dube wrote:
> 
> As the Lid is an assigned address rather than a burned in address, as in
> Ethernet, the chances of duplicate ifphysaddr's in a network spanning
> multiple IB subnets are a lot higher. We should reduce this chance as
> much as possible.
> 
> The ifphysaddr is also not in sync with what IpOverIB has decided to use
> as the link address in ARPs.
> 
> IMHO the two, ifphysaddr and the link address is an arp packet, should
> match.
> 
> Aditya Dube
> 
> -----Original Message-----
> From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
> Sent: Tuesday, April 16, 2002 12:18 PM
> To: Aditya Dube
> Cc: ipoverib@ietf.org
> Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> 
> The same question was addressed last october/november.
> See the message:
> 
> http://www1.ietf.org/mail-archive/working-groups/ipoverib/current/msg000
> 87.html
> 
> Vivek
> 
> 
> 
> > 
> > This is a multi-part message in MIME format.
> > 
> > --------------InterScan_NT_MIME_Boundary
> > Content-Type: multipart/alternative;
> > 	boundary="----_=_NextPart_001_01C1E577.F19F4F03"
> > 
> > ------_=_NextPart_001_01C1E577.F19F4F03
> > Content-Type: text/plain;
> > 	charset="us-ascii"
> > Content-Transfer-Encoding: quoted-printable
> > 
> > 
> > In the MIB draft, the ifPhysAddress is specified as the LID. This is
> out
> > of sync with the IpOverIb draft. Can the ifPhysAddress in the MIB be
> > changed to match  IPoverIB's physical address, i.e. the GID + QPN?
> > 
> > http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.txt
> > 
> > Aditya Dube
> > 
> > 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Wed Apr 17 14:41:29 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01170
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 14:41:28 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA07425
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 14:41:31 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA07152;
	Wed, 17 Apr 2002 14:37:45 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA07118
	for <ipoverib@ns.ietf.org>; Wed, 17 Apr 2002 14:37:42 -0400 (EDT)
Received: from nwkea-mail-1.sun.com (nwkea-mail-1.sun.com [192.18.42.13])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA00980
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 14:37:38 -0400 (EDT)
Received: from phys-ha10nwka.ebay.sun.com ([129.150.140.210])
	by nwkea-mail-1.sun.com (8.9.3+Sun/8.9.3) with ESMTP id LAA21030
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 11:37:11 -0700 (PDT)
Received: from endeavour (endeavour [129.150.143.50])
	by phys-ha10nwka.ebay.sun.com (8.8.8+Sun/8.8.8/ENSMAIL,v2.2) with SMTP id LAA14960
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 11:37:10 -0700 (PDT)
Message-Id: <200204171837.LAA14960@phys-ha10nwka.ebay.sun.com>
Date: Wed, 17 Apr 2002 11:36:58 -0700 (PDT)
From: Ajoy C Siddabathuni <Ajoy.Siddabathuni@Sun.COM>
Reply-To: Ajoy C Siddabathuni <Ajoy.Siddabathuni@Sun.COM>
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
To: ipoverib@ietf.org
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: DEqHlANATTJUvV/LcGatCw==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.3.4p_5 SunOS 5.7 sun4u sparc 
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

[Resend:attempt-2]

On Tue, 16 Apr 2002, Vivek Kashyap wrote:

> > I think one must get outside the "try to save some bytes here and there" line
> > of thinking when defining wire protocols and make sure that you dont have to
> > write another rfc next year to obsolete this one. All those RFCs that defined 
the
> 
> 	And why will it become obsolete ? 

Well, if the router vendors come along and discover that it is so
inefficient to convert frames from one format to another - they
might just write a new one.

> 
> > LLC/SNAP headers in them were not mis-guided, would you think? You would'nt 
endup
> 
>         Oh c'mon! you are the one who belittled the most recent RFC in
>         the list: RFC2734. You exhorted us to "learn from others'
>         mistakes" regarding this RFC. My comment was that just because
>         some RFCs used LLC/SNAP, it does not make it a standard but
>         just one way of doing things.

It is a std. when most protocols defined so far chose to do it in that
"one way of doing things" - what more is needed to make a std.
There are already  'n' different standards out there - my  point was to
stick with the one with the highest prevalence and not invent a new one.
I think the WG is also required to look and understand existing  IP encap-
sulation protocols that are in wide use before going ahead and defing a
new format/protocol. I am sure the WG will need to refer back and forth
to the those RFCs when we get to the IPoverIB(RC) - UD is a nobrainer
relative to the complexities of RC.

Most IPoverXXX protocols defined so far went by the book to accomodate
that format. There may be 1 or 2 exceptions, I just dont want us to go
down the path of those exceptions.

> 
>         And I do think that LLC/SNAP is useless - the 3 bytes of LLC
>         provide nothing. Then you have 3 bytes of OUI before getting
>         to the type field. Those 6 bytes might as well not exist.

My point has been that it is, yes, useless when you are only concerned
about IPoverIB and not even consider how this meshes with a multi-protocol
enviroment.

You might come around and tell me, we will deal with it when it becomes
a problem. That is the time for the new rfc.

> 
> 
> > with an IB header of upto 68 bytes for just UD if saving bytes from the 
header
> > was the goal. Recall that at wire speeds of 2-10Gbps these should not matter.
> 
>         Just because one can afford to be lax doesn't mean one should
>         be. What good is deliberately adding bytes that provide
>         nothing? 

IBA already provides a Raw-Datagram format with a RWH, it seems like we
made one full circle to get there - why not just use raw-datagrams  and
get it over with it.

> > >is not the favored mode in general - just look at 802.3 vs. DIX ethernet.
> 
>         You mentioned working with multiprotocols in 'edge routers' -
>         what about the ubiquitous DIX ethernet ? No header at all,
>         what of LLC/SNAP! You mentioned ATM - as far as I know it
>         suggests that the encapsulation format is negotiable. So, you
>         can work with or without LLC/SNAP. LLC/SNAP doesn't seem to be
>         the common encapsulation format.

Infact, LLC/SNAP is the default encapsulation format for Classical IP over
ATM. Also see the other RFCs that I pointed out. This had a lot to do
with how all edge devices such as routers and bridges like to see commonly
understood formats in thier internal memory structures. Beyond that I am
not allowed to speak, the RFCs can speak for themselves.

~Ajoy

> 
>         I, for one, need a lot more convincing before I can support
>         the idea of LLC/SNAP being the common encapsulation in the
>         future.  
> 
> Vivek
> 
> > 
> > 
> > 
> > >
> > >
> > >                                                                                                                                   
> > >                      Ajoy C                                                                                                       
> > >                      Siddabathuni             To:       
> > Ajoy.Siddabathuni@sun.com, viv@beaverton.ibm.com                          
> > >                      <Ajoy.Siddabathun        cc:       
yaronh@voltaire.com, 
> > Jerry.Chu@eng.sun.com, ipoverib@ietf.org,            
> > >                      i@sun.com>                krause@cup.hp.com                                                                  
> > >                      Sent by:                 Subject:  Re: [Ipoverib] 
> > consensus call on EtherType, min MTU and DHCPv4            
> > >                      ipoverib-admin@ie                                                                                            
> > >                      tf.org                                                                                                       
> > >                                                                                                                                   
> > >                                                                                                                                   
> > >                      04/15/2002 03:52                                                                                             
> > >                      PM                                                                                                           
> > >                      Please respond to                                                                                            
> > >                      Ajoy C                                                                                                       
> > >                      Siddabathuni                                                                                                 
> > >                                                                                                                                   
> > >                                                                                                                                   
> > >
> > >
> > >
> > >
> > >
> > >>It takes another 4 bytes and provides no new information. The issues
> > >>you indicate above are equally addressed by a 'type' field.
> > >
> > >Ofcourse it provides no new information - if you are running IPoIB
> > >in vacum. It is much cleaner than putting a 4 byte header with reserved
> > >fields. Also expect (in a few years) that there will be multi-protocol
> > >routers that will be sitting at the edges of an
> > >IB-Ethernet-osi-apple-network.
> > >They would definetly like to handle packets in a std way - especially when
> > >cp'ing pkts from one kind of "link" to another. They dont need to handle
> > >cases such as - "oh by the way, if this link was an IPoIB link..I needed to
> > >strip the top 4bytes...and not interpret the next 16 bits before I look at
> > >the eth-type".
> > >
> > >>
> > >>'No header case' is disallowed as per WG consensus. Otherwise, the
> > >>4-byte ethertype field is equally viable as the 8 byte LLC/SNAP field.
> > >>It has the advantage of being 4 bytes smaller.
> > >
> > >I would'nt consider "4 bytes smaller" as an advantage. Especially if you
> > >are
> > >working with IB header lengths upto 68 bytes. And then if you are working
> > >with
> > >TCP/IPv4/v6 add upto 60+.
> > >
> > >>
> > >>How about RFC2734 ? There is nothing standard about this.
> > >Well - time to learn from another's mistakes.
> > >
> > >Here is my summary -
> > >
> > > - Having no-header at all will give you alignment. No ethtype to mux on.
> > >   But you could have always figured out all IP'ish types from the
> > >IP-hdrtype
> > >   field.
> > >
> > > - Having 4byte "non-std" header will also give you alignment. Gives you
> > >   all the info you need. But yes, there are certain non-std elements in
> > >   the header that implementations would have to be aware of  - especially
> > >   if they were porting an existing stack.
> > >
> > > - Having 8byte std-header will be standards conforming for future
> > >multi-prot-
> > >   col enviroments. Ofcourse, retains alignment.
> > >
> > >~Ajoy
> > >
> > >
> > >>>
> > >>> Frame formats -
> > >>>
> > >>> a) If LLC/SNAP was used then you automatically get a 32bit-word
> > >alignment
> > >>>    for the IP-Datagram.
> > >>>
> > >>>    The frame would look something like --
> > >>>
> > >>>    LRH (8bytes)
> > >>>    GRH (40 bytes)*
> > >>>    BTH (12 bytes)
> > >>>    DETH (8 bytes)
> > >>>                LLC       OUI     EthType
> > >>>    LLC/SNAP <AA-AA-03><00-00-00><xx-xx>   (8bytes)
> > >>>    IPv4/IPv6 Datagram
> > >>>    ICRC
> > >>>    VCRC
> > >>>
> > >>> b) Embedding just the  ethtype - the frame would look like -- also gives
> > >>>    you the alignment - but is a non-std way of doing things (with
> > >reserved
> > >>>    fields etc in the format).
> > >>>
> > >>>    LRH (8bytes)
> > >>>    GRH (40 bytes)*
> > >>>    BTH (12 bytes)
> > >>>    DETH (8 bytes)
> > >>>
> > >>>     Rsvd   EthType
> > >>>    <??-??><xx-xx>  (4bytes)
> > >>>    IPv4/IPv6 Datagram
> > >>>    ICRC
> > >>>    VCRC
> > >>> ----
> > >>>
> > >>> My opinion on this is that if EthType is ever supported, it should be
> > >>> supported using the std mechanism (a) to be consistent with how other
> > >>> IPoverXXX protocols have been defined in the past. (See RFCs -ATM: 2225,
> > >>> 1577, 1483 SMDS:1209 FR:1294)
> > >>
> > >>How about RFC2734 ? There is nothing standard about this.
> > >>
> > >>[ As another example one can even use a 2 byte header if it works out
> > >>for the hardware -i.e. if one can DMA to a 2 byte boundary, the IP
> > >>header will land at a 4 byte boundary. ]
> > >>
> > >>Vivek
> > >>
> > >>>
> > >>> If the WG feels that in the foreseeable future, only IP will ever be
> > >>> supported, then yes there is no need for the Ethtype - ie. leave it
> > >alone,
> > >>> you can still get the IP datagram 32bit aligned right after DETH. On the
> > >>> other hand, if the WG feels that we do it the way our ancestors have
> > >done it
> > >>> there is enough argument already in place for supporting llc/snap hdrs.
> > >>>
> > >>> ~Ajoy
> > >>>
> > >>>
> > >>> >X-Sender: krause@esalpha2.cup.hp.com
> > >>> >Date: Mon, 15 Apr 2002 10:20:49 -0700
> > >>> >To: Yaron Haviv <yaronh@voltaire.com>, "'H.K. Jerry Chu'"
> > >>> <Jerry.Chu@eng.sun.com>, "'ipoverib@ietf.org'" <ipoverib@ietf.org>,
> > >>> "'Ajoy.Siddabathuni@Sun.COM'" <Ajoy.Siddabathuni@Sun.COM>
> > >>> >From: Michael Krause <krause@cup.hp.com>
> > >>> >Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
> > >>> >Mime-Version: 1.0
> > >>> >X-Mailman-Version: 1.0
> > >>> >List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
> > >>> >X-BeenThere: ipoverib@ietf.org
> > >>> >
> > >>> >At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
> > >>> >>Jerry,
> > >>> >>
> > >>> >>We must stop expending the definition somewhere so we get the spec out
> > >>> >>This is IPoIB we are talking about, there is no need for LLC/SNAP, and
> > >we
> > >>> >>just need the Ethertype
> > >>> >
> > >>> >The consensus is that people want to have the full range of Ethertype
> > >>> >values supported.  As such, this needs to be specified independent of
> > >the
> > >>> >IP over IB specification so that everyone may use these values without
> > >>> >requiring them to implement IP over IB.
> > >>> >
> > >>> >Mike
> > >>> >
> > >>> >
> > >>> >_______________________________________________
> > >>> >IPoverIB mailing list
> > >>> >IPoverIB@ietf.org
> > >>> >https://www1.ietf.org/mailman/listinfo/ipoverib
> > >>>
> > >>>
> > >>> _______________________________________________
> > >>> IPoverIB mailing list
> > >>> IPoverIB@ietf.org
> > >>> https://www1.ietf.org/mailman/listinfo/ipoverib
> > >>>
> > >>
> > >>
> > >>--
> > >>Vivek Kashyap
> > >>Linux Technology Center, IBM
> > >>kashyapv@us.ibm.com
> > >>vivk@us.ibm.com
> > >>503 578 3422 (o)
> > >
> > >
> > >_______________________________________________
> > >IPoverIB mailing list
> > >IPoverIB@ietf.org
> > >https://www1.ietf.org/mailman/listinfo/ipoverib
> > >
> > >
> > >
> > >
> > >
> > >_______________________________________________
> > >IPoverIB mailing list
> > >IPoverIB@ietf.org
> > >https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Wed Apr 17 14:48:47 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01380
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 14:48:45 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA08011
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 14:48:48 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA07719;
	Wed, 17 Apr 2002 14:44:41 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA07683
	for <ipoverib@ns.ietf.org>; Wed, 17 Apr 2002 14:44:38 -0400 (EDT)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01263
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 14:44:34 -0400 (EDT)
Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.39 2002/04/15 17:47:23 root Exp $) with ESMTP id g3HIhth04462
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 18:43:55 GMT
Received: from fmsmsxvs042.fm.intel.com (fmsmsxv042-1.fm.intel.com [132.233.48.110])
	by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.16 2002/04/15 17:47:00 root Exp $) with SMTP id g3HIh6e05681
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 18:43:06 GMT
Received: from fmsmsx29.FM.INTEL.COM ([132.233.42.29])
 by fmsmsxvs042.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002041711483812937
 ; Wed, 17 Apr 2002 11:48:38 -0700
Received: by fmsmsx29.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2LKB79JL>; Wed, 17 Apr 2002 11:44:35 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4F9A@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Bill Strahm'" <bill@strahm.net>,
        Aditya Dube <adube@windows.microsoft.com>
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] MIB's ifPhysAddress needs to be updated
Date: Wed, 17 Apr 2002 11:44:32 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

But it's being define in the IPoIB working group.  Why isn't it legitimate
to say it should support IPoIB hardware address semantics?

Roy

-----Original Message-----
From: Bill Strahm [mailto:bill@strahm.net]
Sent: Wednesday, April 17, 2002 11:34 AM
To: Aditya Dube
Cc: ipoverib@ietf.org
Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated


The problem is that ifPhysAddr covers more than just IP, it is just
the layer 2 wire representation of address.  If you can show me an
IB format that includes GID+LID in a packet header, I might conceed
defeat here.

The address duplicating across multiple subnets is a red herring.  You
can assign identical Ethernet MAC addresses, so long as they are on
differing subnets as well.  

From RFC 2863...
            "The interface's address at its protocol sub-layer.  For
            example, for an 802.x interface, this object normally
            contains a MAC address.  The interface's media-specific MIB
            must define the bit and byte ordering and the format of the
            value of this object.  For interfaces which do not have such
            an address (e.g., a serial line), this object should contain
            an octet string of zero length."
Note that there is no mention of any Layer 3 semantics.  

We can bring the topic up for discussion, I don't know which of the MIB
authors are currently active right now though.

Bill

On Wed, Apr 17, 2002 at 11:19:24AM -0700, Aditya Dube wrote:
> 
> As the Lid is an assigned address rather than a burned in address, as in
> Ethernet, the chances of duplicate ifphysaddr's in a network spanning
> multiple IB subnets are a lot higher. We should reduce this chance as
> much as possible.
> 
> The ifphysaddr is also not in sync with what IpOverIB has decided to use
> as the link address in ARPs.
> 
> IMHO the two, ifphysaddr and the link address is an arp packet, should
> match.
> 
> Aditya Dube
> 
> -----Original Message-----
> From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
> Sent: Tuesday, April 16, 2002 12:18 PM
> To: Aditya Dube
> Cc: ipoverib@ietf.org
> Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> 
> The same question was addressed last october/november.
> See the message:
> 
> http://www1.ietf.org/mail-archive/working-groups/ipoverib/current/msg000
> 87.html
> 
> Vivek
> 
> 
> 
> > 
> > This is a multi-part message in MIME format.
> > 
> > --------------InterScan_NT_MIME_Boundary
> > Content-Type: multipart/alternative;
> > 	boundary="----_=_NextPart_001_01C1E577.F19F4F03"
> > 
> > ------_=_NextPart_001_01C1E577.F19F4F03
> > Content-Type: text/plain;
> > 	charset="us-ascii"
> > Content-Transfer-Encoding: quoted-printable
> > 
> > 
> > In the MIB draft, the ifPhysAddress is specified as the LID. This is
> out
> > of sync with the IpOverIb draft. Can the ifPhysAddress in the MIB be
> > changed to match  IPoverIB's physical address, i.e. the GID + QPN?
> > 
> > http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.txt
> > 
> > Aditya Dube
> > 
> > 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Wed Apr 17 15:19:49 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA02683
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 15:19:48 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA09741
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 15:19:51 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA09623;
	Wed, 17 Apr 2002 15:12:58 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA09549
	for <ipoverib@ns.ietf.org>; Wed, 17 Apr 2002 15:12:54 -0400 (EDT)
Received: from mxout1.netvision.net.il (mxout1.netvision.net.il [194.90.9.20])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA02461
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 15:12:50 -0400 (EDT)
Received: from vaio505he ([139.92.208.156]) by mxout1.netvision.net.il
 (iPlanet Messaging Server 5.1 (built Sep  5 2001))
 with ESMTPA id <0GUQ00F6U80DWU@mxout1.netvision.net.il> for ipoverib@ietf.org;
 Wed, 17 Apr 2002 22:12:16 +0300 (IDT)
Date: Wed, 17 Apr 2002 21:13:15 +0200
From: Yaron Haviv <yaronh@voltaire.com>
Subject: RE: [Ipoverib] IPoIB multicast forwarding
In-reply-to: <5.1.0.14.2.20020417070720.02a367a0@esalpha2.cup.hp.com>
To: "'Michael Krause'" <krause@cup.hp.com>, ipoverib@ietf.org
Reply-to: yaronh@voltaire.com
Message-id: <000f01c1e643$edf4b840$0100007f@vaio505he>
MIME-version: 1.0
X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4807.1700
X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2910.0)
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Importance: Normal
X-Priority: 3 (Normal)
X-MSMail-priority: Normal
Content-Transfer-Encoding: 7BIT
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7BIT

Mike,

My email did not refer to IP or IB routers specifically, you didn't refer to
the problem I mentioned in the email
What happens if you run out of MLID resources, or fail to join?
Not to mention that I don't see how SM guys finish dealing with all the MC
side issues soon this leaves us with no IPoIB this year

If you mention IB routers, going back to your suggestion,
Lets say the sender sends the packet to the mcast group,
How can you make sure that the IB router (not an IP router) joins that group
before the sender issued the send ?
Do you want the sender to request a join, than the SM make sure that the
routers joined, only than the SM returns the response ?
Can you shed some light on how you see a complete sender operation ?

Yaron

-----Original Message-----
From: ipoverib-admin@ietf.org [mailto:ipoverib-admin@ietf.org]On Behalf Of
Michael Krause
Sent: Wednesday, April 17, 2002 4:13 PM
To: ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast forwarding

At 10:28 AM 4/17/2002 +0200, Yaron Haviv wrote:
>Roy, in general I agree with you, since we have to have something running
>this year However we need to have some flexibility for future expansion

The IBTA is trying to address this problem as we speak which will allow one
to ship product this year....something a bit similar to what I described in
a previous e-mail is being proposed and this could be used to simplify this
whole discussion and eliminate this ambiguity being debated here.  One does
not need to go into a broadcast solution.  One does not need to flip
between known multicast group to all routers.  One only needs to obtain /
track the multicast group identifiers that one is using and leave the rest
of these problems to IB to resolve.

Again, a multiprotocol router is nothing more than an IB CA (nominally a
TCA but could be a HCA) that joins an IB multicast group just like a normal
IB endnode.  It is the one responsible for interpreting the layer 3 headers
and determining how to forward the packet.  It is the one responsible for
layer 3 multicast group management not IB and not IP over IB in particular.

If IP over IB were to focus on how to join an IB multicast group and how to
encapsulate IP multicast addresses within IB MGIDs, then this work would be
greatly accelerated.   Leave the IP router issues to their appropriate
workgroups which I do not believe is this workgroup in particular.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Wed Apr 17 15:54:42 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04490
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 15:54:42 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA12711
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 15:54:45 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA12369;
	Wed, 17 Apr 2002 15:49:10 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA12337
	for <ipoverib@ns.ietf.org>; Wed, 17 Apr 2002 15:49:08 -0400 (EDT)
Received: from palrel13.hp.com (palrel13.hp.com [156.153.255.238])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04114
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 15:49:04 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel13.hp.com (Postfix) with ESMTP id 3C3CA400CF9
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 12:48:34 -0700 (PDT)
Received: from mkrause.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id MAA21721
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 12:48:32 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020417124030.02a4e278@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Wed, 17 Apr 2002 12:48:20 -0700
To: ipoverib@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast forwarding
In-Reply-To: <000f01c1e643$edf4b840$0100007f@vaio505he>
References: <5.1.0.14.2.20020417070720.02a367a0@esalpha2.cup.hp.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 09:13 PM 4/17/2002 +0200, Yaron Haviv wrote:
>Mike,
>
>My email did not refer to IP or IB routers specifically, you didn't refer to
>the problem I mentioned in the email
>What happens if you run out of MLID resources, or fail to join?

Either free up resources or fail to join. If people are good in their 
policies for assigning MLIDs then this will not be a problem for all 
reasonable implementations within switches (I've reviewed nearly every 
design in flight and their associated resources).

>Not to mention that I don't see how SM guys finish dealing with all the MC
>side issues soon this leaves us with no IPoIB this year

It does not prevent shipping this year at all.

>If you mention IB routers, going back to your suggestion,
>Lets say the sender sends the packet to the mcast group,
>How can you make sure that the IB router (not an IP router) joins that group
>before the sender issued the send ?

Multicast is unreliable by definition so the race condition is not an 
issue.  If the application cares about always getting through then this an 
application issue not an IB issue (no different than IP today).

>Do you want the sender to request a join, than the SM make sure that the
>routers joined, only than the SM returns the response ?

No. It requests to join and the SM configures the endnode so that its 
packets can be routed through the switch and life proceeds.  If the router 
is up, then it is already part of the tree and receives the packet should 
it make it there.  If the router is not part of the tree, then it does not 
receive anything.  Multicast group membership is simply unknown so why are 
we trying to make this so hard to know that all parties are up and accessible.

>Can you shed some light on how you see a complete sender operation ?

(1) Join multicast group (send only or send/receive)
   - Obtain appropriate IB parameters to fill out a UD packet

(2) Fill out a UD packet as appropriate with headers and data

(3) Inject packet into fabric

(4) Repeat as needed


Some time in the future, if sender does not care about multicast any 
longer, leave the multicast group.

If endnodes join (independent of whether they are multi-protocol capable or 
not), add them into the multicast tree so they may receive packets if they 
have requested a send/receive join capability else allow them to send 
packets as desired.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Wed Apr 17 16:39:18 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06691
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 16:39:17 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA15929
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 16:39:21 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA15820;
	Wed, 17 Apr 2002 16:35:32 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA15784
	for <ipoverib@ns.ietf.org>; Wed, 17 Apr 2002 16:35:28 -0400 (EDT)
Received: from mxout2.netvision.net.il (mxout2.netvision.net.il [194.90.9.21])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06530
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 16:35:23 -0400 (EDT)
Received: from vaio505he ([139.92.208.136]) by mxout2.netvision.net.il
 (iPlanet Messaging Server 5.2 (built Feb 21 2002))
 with ESMTPA id <0GUQ00L3NBU0UL@mxout2.netvision.net.il> for ipoverib@ietf.org;
 Wed, 17 Apr 2002 23:34:50 +0300 (IDT)
Date: Wed, 17 Apr 2002 22:35:43 +0200
From: Yaron Haviv <yaronh@voltaire.com>
Subject: RE: [Ipoverib] IPoIB multicast forwarding
In-reply-to: <5.1.0.14.2.20020417124030.02a4e278@esalpha2.cup.hp.com>
To: "'Michael Krause'" <krause@cup.hp.com>, ipoverib@ietf.org
Reply-to: yaronh@voltaire.com
Message-id: <000001c1e64f$73077d00$88d05c8b@vaio505he>
MIME-version: 1.0
X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4807.1700
X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2910.0)
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7BIT
Importance: Normal
X-Priority: 3 (Normal)
X-MSMail-priority: Normal
Content-Transfer-Encoding: 7BIT
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7BIT

Mike,

How does the IB or IP router know that it needs to listen on a specific
group (in the send case) ?
The sender joined/created the group, how does the router know about it ?
Is the SM going to configure it like it does to switches ?
What happens until everything is fully configured in the fabric, do we wait
? do we drop ?

Why not allow the sender to broadcast as an optional method ?
We can also use your method, and it will probably prevail in the future (not
now)
It will certainly help many of us to build our products sooner than later

Yaron

-----Original Message-----
From: ipoverib-admin@ietf.org [mailto:ipoverib-admin@ietf.org]On Behalf Of
Michael Krause
Sent: Wednesday, April 17, 2002 9:48 PM
To: ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast forwarding

At 09:13 PM 4/17/2002 +0200, Yaron Haviv wrote:
>Mike,
>
>My email did not refer to IP or IB routers specifically, you didn't refer
to
>the problem I mentioned in the email
>What happens if you run out of MLID resources, or fail to join?

Either free up resources or fail to join. If people are good in their
policies for assigning MLIDs then this will not be a problem for all
reasonable implementations within switches (I've reviewed nearly every
design in flight and their associated resources).

>Not to mention that I don't see how SM guys finish dealing with all the MC
>side issues soon this leaves us with no IPoIB this year

It does not prevent shipping this year at all.

>If you mention IB routers, going back to your suggestion,
>Lets say the sender sends the packet to the mcast group,
>How can you make sure that the IB router (not an IP router) joins that
group
>before the sender issued the send ?

Multicast is unreliable by definition so the race condition is not an
issue.  If the application cares about always getting through then this an
application issue not an IB issue (no different than IP today).

>Do you want the sender to request a join, than the SM make sure that the
>routers joined, only than the SM returns the response ?

No. It requests to join and the SM configures the endnode so that its
packets can be routed through the switch and life proceeds.  If the router
is up, then it is already part of the tree and receives the packet should
it make it there.  If the router is not part of the tree, then it does not
receive anything.  Multicast group membership is simply unknown so why are
we trying to make this so hard to know that all parties are up and
accessible.

>Can you shed some light on how you see a complete sender operation ?

(1) Join multicast group (send only or send/receive)
   - Obtain appropriate IB parameters to fill out a UD packet

(2) Fill out a UD packet as appropriate with headers and data

(3) Inject packet into fabric

(4) Repeat as needed


Some time in the future, if sender does not care about multicast any
longer, leave the multicast group.

If endnodes join (independent of whether they are multi-protocol capable or
not), add them into the multicast tree so they may receive packets if they
have requested a send/receive join capability else allow them to send
packets as desired.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Wed Apr 17 18:54:18 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA10265
	for <ipoverib-archive@odin.ietf.org>; Wed, 17 Apr 2002 18:54:18 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id SAA24599
	for ipoverib-archive@odin.ietf.org; Wed, 17 Apr 2002 18:54:20 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA24343;
	Wed, 17 Apr 2002 18:50:19 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA24307
	for <ipoverib@ns.ietf.org>; Wed, 17 Apr 2002 18:50:16 -0400 (EDT)
Received: from palrel10.hp.com (palrel10.hp.com [156.153.255.245])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA10200
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 18:50:13 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel10.hp.com (Postfix) with ESMTP id 80EC1C00BA5
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 15:49:45 -0700 (PDT)
Received: from mkrause.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id PAA22453
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 15:49:44 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020417154410.02a6ed68@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Wed, 17 Apr 2002 15:49:15 -0700
To: ipoverib@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast forwarding
In-Reply-To: <000001c1e64f$73077d00$88d05c8b@vaio505he>
References: <5.1.0.14.2.20020417124030.02a4e278@esalpha2.cup.hp.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 10:35 PM 4/17/2002 +0200, Yaron Haviv wrote:
>Mike,
>
>How does the IB or IP router know that it needs to listen on a specific
>group (in the send case) ?

It is informed of when there is a change in the multicast groups within the 
subnet it is attached to just like one does for other change notifications.

>The sender joined/created the group, how does the router know about it ?

See above.

>Is the SM going to configure it like it does to switches ?

An IB router is treated as a leaf node from a SM perspective (same would 
apply to an IP router via a CA). As such, the SM does not configure it but 
the leaf switch it is attached to.  The IB router protocol deals with cross 
subnet routing and thus there is no SM involvement in such actions.

>What happens until everything is fully configured in the fabric, do we wait
>? do we drop ?

If a multicast packet cannot make forward progress, the CA (IB router or 
IP) can drop it if desired.

>Why not allow the sender to broadcast as an optional method ?

Unnecessary complexity, too many interactions for update purposes, 
unnecessary performance loss, etc.

>We can also use your method, and it will probably prevail in the future (not
>now) It will certainly help many of us to build our products sooner than later

It can be used today via a management entity providing such multicast group 
management services (something not defined today within the IBTA) or one 
can wait until the IBTA releases a normative method.  I don't think the 
wait is a major issue and the IBTA will solve the update problem thereby 
eliminating having to specify such complexity within the IP over IB 
work.  It will also eliminate some interoperability problems because there 
is a single method to solve this problem and not two which is completely 
unnecessary.

Mike



_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Thu Apr 18 01:54:22 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA18228
	for <ipoverib-archive@odin.ietf.org>; Thu, 18 Apr 2002 01:54:18 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id BAA25943
	for ipoverib-archive@odin.ietf.org; Thu, 18 Apr 2002 01:54:21 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id BAA25726;
	Thu, 18 Apr 2002 01:50:41 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA16625
	for <ipoverib@optimus.ietf.org>; Wed, 17 Apr 2002 12:02:44 -0400 (EDT)
Received: from nwkea-mail-1.sun.com (nwkea-mail-1.sun.com [192.18.42.13])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA24510
	for <ipoverib@ietf.org>; Wed, 17 Apr 2002 12:02:40 -0400 (EDT)
Received: from ebaymail2.EBay.Sun.COM ([129.150.111.20])
	by nwkea-mail-1.sun.com (8.9.3+Sun/8.9.3) with ESMTP id JAA14388;
	Wed, 17 Apr 2002 09:02:04 -0700 (PDT)
Received: from atmsw-22.ebay.sun.com (atmsw-22.EBay.Sun.COM [192.168.203.122])
	by ebaymail2.EBay.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1) with ESMTP id JAA11710;
	Wed, 17 Apr 2002 09:02:04 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by atmsw-22.ebay.sun.com (8.9.3+Sun/8.9.3) with ESMTP id IAA02235;
	Wed, 17 Apr 2002 08:59:14 -0700 (PDT)
Date: Wed, 17 Apr 2002 08:59:14 -0700 (PDT)
From: Ajoy Siddabathuni <ajoy@ebay.sun.com>
X-Sender: ajoy@atmsw-22
To: Vivek Kashyap <kashyapv@us.ibm.com>
cc: Ajoy.Siddabathuni@sun.com, vivk@us.ibm.com, ipoverib@ietf.org,
        ipoverib-admin%iipoverib@us.ibm.com
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
In-Reply-To: <200204170649.g3H6ndf05005@eng4.beaverton.ibm.com>
Message-ID: <Pine.GSO.4.05.10204170755430.2226-100000@atmsw-22>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org



On Tue, 16 Apr 2002, Vivek Kashyap wrote:

> > I think one must get outside the "try to save some bytes here and there" line
> > of thinking when defining wire protocols and make sure that you dont have to
> > write another rfc next year to obsolete this one. All those RFCs that defined the
> 
> 	And why will it become obsolete ? 

Well, if the router vendors come along and discover that it is so
inefficient to convert frames from one format to another - they
might just write a new one.

> 
> > LLC/SNAP headers in them were not mis-guided, would you think? You would'nt endup
> 
>         Oh c'mon! you are the one who belittled the most recent RFC in
>         the list: RFC2734. You exhorted us to "learn from others'
>         mistakes" regarding this RFC. My comment was that just because
>         some RFCs used LLC/SNAP, it does not make it a standard but
>         just one way of doing things.

It is a std. when most protocols defined so far chose to do it in that
"one way of doing things" - what more is needed to make a std.
There are already  'n' different standards out there - my  point was to
stick with the one with the highest prevalence and not invent a new one.
I think the WG is also required to look and understand existing  IP encap-
sulation protocols that are in wide use before going ahead and defing a
new format/protocol. I am sure the WG will need to refer back and forth
to the those RFCs when we get to the IPoverIB(RC) - UD is a nobrainer
relative to the complexities of RC.

Most IPoverXXX protocols defined so far went by the book to accomodate
that format. There may be 1 or 2 exceptions, I just dont want us to go
down the path of those exceptions.

> 
>         And I do think that LLC/SNAP is useless - the 3 bytes of LLC
>         provide nothing. Then you have 3 bytes of OUI before getting
>         to the type field. Those 6 bytes might as well not exist.

My point has been that it is, yes, useless when you are only concerned
about IPoverIB and not even consider how this meshes with a multi-protocol
enviroment.

You might come around and tell me, we will deal with it when it becomes
a problem. That is the time for the new rfc.

> 
> 
> > with an IB header of upto 68 bytes for just UD if saving bytes from the header
> > was the goal. Recall that at wire speeds of 2-10Gbps these should not matter.
> 
>         Just because one can afford to be lax doesn't mean one should
>         be. What good is deliberately adding bytes that provide
>         nothing? 

IBA already provides a Raw-Datagram format with a RWH, it seems like we
made one full circle to get there - why not just use raw-datagrams  and
get it over with it.

> > >is not the favored mode in general - just look at 802.3 vs. DIX ethernet.
> 
>         You mentioned working with multiprotocols in 'edge routers' -
>         what about the ubiquitous DIX ethernet ? No header at all,
>         what of LLC/SNAP! You mentioned ATM - as far as I know it
>         suggests that the encapsulation format is negotiable. So, you
>         can work with or without LLC/SNAP. LLC/SNAP doesn't seem to be
>         the common encapsulation format.

Infact, LLC/SNAP is the default encapsulation format for Classical IP over
ATM. Also see the other RFCs that I pointed out. This had a lot to do
with how all edge devices such as routers and bridges like to see commonly
understood formats in thier internal memory structures. Beyond that I am
not allowed to speak, the RFCs can speak for themselves.

~Ajoy

> 
>         I, for one, need a lot more convincing before I can support
>         the idea of LLC/SNAP being the common encapsulation in the
>         future.  
> 
> Vivek
> 
> > 
> > 
> > 
> > >
> > >
> > >                                                                                                                                   
> > >                      Ajoy C                                                                                                       
> > >                      Siddabathuni             To:       
> > Ajoy.Siddabathuni@sun.com, viv@beaverton.ibm.com                          
> > >                      <Ajoy.Siddabathun        cc:       yaronh@voltaire.com, 
> > Jerry.Chu@eng.sun.com, ipoverib@ietf.org,            
> > >                      i@sun.com>                krause@cup.hp.com                                                                  
> > >                      Sent by:                 Subject:  Re: [Ipoverib] 
> > consensus call on EtherType, min MTU and DHCPv4            
> > >                      ipoverib-admin@ie                                                                                            
> > >                      tf.org                                                                                                       
> > >                                                                                                                                   
> > >                                                                                                                                   
> > >                      04/15/2002 03:52                                                                                             
> > >                      PM                                                                                                           
> > >                      Please respond to                                                                                            
> > >                      Ajoy C                                                                                                       
> > >                      Siddabathuni                                                                                                 
> > >                                                                                                                                   
> > >                                                                                                                                   
> > >
> > >
> > >
> > >
> > >
> > >>It takes another 4 bytes and provides no new information. The issues
> > >>you indicate above are equally addressed by a 'type' field.
> > >
> > >Ofcourse it provides no new information - if you are running IPoIB
> > >in vacum. It is much cleaner than putting a 4 byte header with reserved
> > >fields. Also expect (in a few years) that there will be multi-protocol
> > >routers that will be sitting at the edges of an
> > >IB-Ethernet-osi-apple-network.
> > >They would definetly like to handle packets in a std way - especially when
> > >cp'ing pkts from one kind of "link" to another. They dont need to handle
> > >cases such as - "oh by the way, if this link was an IPoIB link..I needed to
> > >strip the top 4bytes...and not interpret the next 16 bits before I look at
> > >the eth-type".
> > >
> > >>
> > >>'No header case' is disallowed as per WG consensus. Otherwise, the
> > >>4-byte ethertype field is equally viable as the 8 byte LLC/SNAP field.
> > >>It has the advantage of being 4 bytes smaller.
> > >
> > >I would'nt consider "4 bytes smaller" as an advantage. Especially if you
> > >are
> > >working with IB header lengths upto 68 bytes. And then if you are working
> > >with
> > >TCP/IPv4/v6 add upto 60+.
> > >
> > >>
> > >>How about RFC2734 ? There is nothing standard about this.
> > >Well - time to learn from another's mistakes.
> > >
> > >Here is my summary -
> > >
> > > - Having no-header at all will give you alignment. No ethtype to mux on.
> > >   But you could have always figured out all IP'ish types from the
> > >IP-hdrtype
> > >   field.
> > >
> > > - Having 4byte "non-std" header will also give you alignment. Gives you
> > >   all the info you need. But yes, there are certain non-std elements in
> > >   the header that implementations would have to be aware of  - especially
> > >   if they were porting an existing stack.
> > >
> > > - Having 8byte std-header will be standards conforming for future
> > >multi-prot-
> > >   col enviroments. Ofcourse, retains alignment.
> > >
> > >~Ajoy
> > >
> > >
> > >>>
> > >>> Frame formats -
> > >>>
> > >>> a) If LLC/SNAP was used then you automatically get a 32bit-word
> > >alignment
> > >>>    for the IP-Datagram.
> > >>>
> > >>>    The frame would look something like --
> > >>>
> > >>>    LRH (8bytes)
> > >>>    GRH (40 bytes)*
> > >>>    BTH (12 bytes)
> > >>>    DETH (8 bytes)
> > >>>                LLC       OUI     EthType
> > >>>    LLC/SNAP <AA-AA-03><00-00-00><xx-xx>   (8bytes)
> > >>>    IPv4/IPv6 Datagram
> > >>>    ICRC
> > >>>    VCRC
> > >>>
> > >>> b) Embedding just the  ethtype - the frame would look like -- also gives
> > >>>    you the alignment - but is a non-std way of doing things (with
> > >reserved
> > >>>    fields etc in the format).
> > >>>
> > >>>    LRH (8bytes)
> > >>>    GRH (40 bytes)*
> > >>>    BTH (12 bytes)
> > >>>    DETH (8 bytes)
> > >>>
> > >>>     Rsvd   EthType
> > >>>    <??-??><xx-xx>  (4bytes)
> > >>>    IPv4/IPv6 Datagram
> > >>>    ICRC
> > >>>    VCRC
> > >>> ----
> > >>>
> > >>> My opinion on this is that if EthType is ever supported, it should be
> > >>> supported using the std mechanism (a) to be consistent with how other
> > >>> IPoverXXX protocols have been defined in the past. (See RFCs -ATM: 2225,
> > >>> 1577, 1483 SMDS:1209 FR:1294)
> > >>
> > >>How about RFC2734 ? There is nothing standard about this.
> > >>
> > >>[ As another example one can even use a 2 byte header if it works out
> > >>for the hardware -i.e. if one can DMA to a 2 byte boundary, the IP
> > >>header will land at a 4 byte boundary. ]
> > >>
> > >>Vivek
> > >>
> > >>>
> > >>> If the WG feels that in the foreseeable future, only IP will ever be
> > >>> supported, then yes there is no need for the Ethtype - ie. leave it
> > >alone,
> > >>> you can still get the IP datagram 32bit aligned right after DETH. On the
> > >>> other hand, if the WG feels that we do it the way our ancestors have
> > >done it
> > >>> there is enough argument already in place for supporting llc/snap hdrs.
> > >>>
> > >>> ~Ajoy
> > >>>
> > >>>
> > >>> >X-Sender: krause@esalpha2.cup.hp.com
> > >>> >Date: Mon, 15 Apr 2002 10:20:49 -0700
> > >>> >To: Yaron Haviv <yaronh@voltaire.com>, "'H.K. Jerry Chu'"
> > >>> <Jerry.Chu@eng.sun.com>, "'ipoverib@ietf.org'" <ipoverib@ietf.org>,
> > >>> "'Ajoy.Siddabathuni@Sun.COM'" <Ajoy.Siddabathuni@Sun.COM>
> > >>> >From: Michael Krause <krause@cup.hp.com>
> > >>> >Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
> > >>> >Mime-Version: 1.0
> > >>> >X-Mailman-Version: 1.0
> > >>> >List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
> > >>> >X-BeenThere: ipoverib@ietf.org
> > >>> >
> > >>> >At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
> > >>> >>Jerry,
> > >>> >>
> > >>> >>We must stop expending the definition somewhere so we get the spec out
> > >>> >>This is IPoIB we are talking about, there is no need for LLC/SNAP, and
> > >we
> > >>> >>just need the Ethertype
> > >>> >
> > >>> >The consensus is that people want to have the full range of Ethertype
> > >>> >values supported.  As such, this needs to be specified independent of
> > >the
> > >>> >IP over IB specification so that everyone may use these values without
> > >>> >requiring them to implement IP over IB.
> > >>> >
> > >>> >Mike
> > >>> >
> > >>> >
> > >>> >_______________________________________________
> > >>> >IPoverIB mailing list
> > >>> >IPoverIB@ietf.org
> > >>> >https://www1.ietf.org/mailman/listinfo/ipoverib
> > >>>
> > >>>
> > >>> _______________________________________________
> > >>> IPoverIB mailing list
> > >>> IPoverIB@ietf.org
> > >>> https://www1.ietf.org/mailman/listinfo/ipoverib
> > >>>
> > >>
> > >>
> > >>--
> > >>Vivek Kashyap
> > >>Linux Technology Center, IBM
> > >>kashyapv@us.ibm.com
> > >>vivk@us.ibm.com
> > >>503 578 3422 (o)
> > >
> > >
> > >_______________________________________________
> > >IPoverIB mailing list
> > >IPoverIB@ietf.org
> > >https://www1.ietf.org/mailman/listinfo/ipoverib
> > >
> > >
> > >
> > >
> > >
> > >_______________________________________________
> > >IPoverIB mailing list
> > >IPoverIB@ietf.org
> > >https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> kashyapv@us.ibm.com 
> vivk@us.ibm.com 
> 503 578 3422 (o)
> 



_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr 18 14:20:07 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA18121
	for <ipoverib-archive@odin.ietf.org>; Thu, 18 Apr 2002 14:20:07 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA22390
	for ipoverib-archive@odin.ietf.org; Thu, 18 Apr 2002 14:20:10 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA22131;
	Thu, 18 Apr 2002 14:15:50 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA22102
	for <ipoverib@optimus.ietf.org>; Thu, 18 Apr 2002 14:15:48 -0400 (EDT)
Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA17838
	for <ipoverib@ietf.org>; Thu, 18 Apr 2002 14:15:44 -0400 (EDT)
Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.17.194.24])
	by e31.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3IIFjKB104540
	for <ipoverib@ietf.org>; Thu, 18 Apr 2002 14:15:45 -0400
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by westrelay03.boulder.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3IIFZ789618
	for <ipoverib@ietf.org>; Thu, 18 Apr 2002 12:15:35 -0600
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g3IIFYX10318
	for <ipoverib@ietf.org>; Thu, 18 Apr 2002 11:15:34 -0700
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3IIFXQ04048
	for ipoverib@ietf.org; Thu, 18 Apr 2002 11:15:33 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204181815.g3IIFXQ04048@eng4.beaverton.ibm.com>
To: ipoverib@ietf.org
Date: Thu, 18 Apr 2002 11:15:32 -0700 (PDT)
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Subject: [Ipoverib] IPoIB multicast
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

Aditya,

Below is the overall algorithm since you had asked me to submit it in
one writeup. However, in IBTA we are working on some ideas (I
participated in one such meeting today morning) that will make all
this a lot simpler.

The following is assuming IB multicast as defined at present.
There is no notification in IB that can be sent to IB nodes on group
creation or deletion. As per Mike's mail the IB group gets deleted
when the last member leaves.

Note: In this discussion I am referring to IP routers, receivers and
senders. 

The idea below is to let the router(an IP router) know that a group
has been created (by the receivers) or have the routers create the
groups when they need to forward. This is accomplished by the routers
acting on the IP group creation reports(IGMP/MLD) or the routing
protocol updates.

A sender can send at any time. It therefore attempts to join the group
when it has some packet ot send and retries if it cannot join the
first time.

The receivers leave the IB groups when they are not interested in the
associated IP group. The routers leave when they leave the associated
IP group or determine that they no longer have to forward the
associated IP group packets.

Senders leave the groups when the group has not been used for some
time. 

	Receiver:

        IP receivers must join a specific local interface and send a
        report to the router.

	
		IP_ADD_MEMBERSHIP called
			check for corresponding IB group
			if IB group exists 
				IB_join it
				IB_send report to group
			else
				IB_create IB group
                                IB_send report to all-routers MGID or
                                could send it to all-hosts group

		IP_DROP_MEMBERSHIP called
			(Send leave report to router )
			IB_leave the IB group
			
	
	Router:
		always receive on broadcast/all-hosts and all-routers
		if IGMP/MLD indicates new group
			IB_join corresponding IB group
		if routing protocol indicates new group
			IB_create corresponding IB group 
		if report informs of leave
			IB_leave corresponding IB group
		if routing protocol indicates no need to forward
			IB_leave corresponding IB Group


	Sender:
                IP senders can send at any time. They don't have to
                join at an interface or report to an IP router.

		got IP multicast packet to send
			map to the corresponding IB group
			if already 'IB_joined' the IB_group
				IB_send to it
				reset timer

			else if IB group exists
				IB_join the group
				start a timer
				IB_send to it
			else
				
				IB_send to all-routers MGID ....(c)

		if timer goes off
			IB_leave the IB group


        We can modify (c) above to 'send to all-hosts MGID'. That way
        the timer can be set to a longer value and the sender
        algorithm can become:

		got IP multicast packet to send
			map to the coresponding IB group
			if already IB_joined
				IB_send to it
				reset timer
			else 
				IB_send to all-hosts MGID
				if no timer associated
					create a timer

		if  timer goes off
			if IB_joined
				IB_leave the IB group
			else
				try to IB_join
				restart timer
			
Vivek
				
-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr 18 17:22:00 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA23334
	for <ipoverib-archive@odin.ietf.org>; Thu, 18 Apr 2002 17:22:00 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id RAA04344
	for ipoverib-archive@odin.ietf.org; Thu, 18 Apr 2002 17:22:04 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA04133;
	Thu, 18 Apr 2002 17:17:47 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA04099
	for <ipoverib@optimus.ietf.org>; Thu, 18 Apr 2002 17:17:45 -0400 (EDT)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA23268
	for <ipoverib@ietf.org>; Thu, 18 Apr 2002 17:17:39 -0400 (EDT)
Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.39 2002/04/15 17:47:23 root Exp $) with ESMTP id g3ILGqb12293
	for <ipoverib@ietf.org>; Thu, 18 Apr 2002 21:16:52 GMT
Received: from fmsmsxvs041.fm.intel.com (fmsmsxvs041.fm.intel.com [132.233.42.126])
	by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.16 2002/04/15 17:47:00 root Exp $) with SMTP id g3ILG0Q15162
	for <ipoverib@ietf.org>; Thu, 18 Apr 2002 21:16:03 GMT
Received: from FMSMSX017.fm.intel.com ([132.233.42.196])
 by fmsmsxvs041.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002041814175623337
 for <ipoverib@ietf.org>; Thu, 18 Apr 2002 14:17:56 -0700
Received: by fmsmsx017.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <28FPDJB4>; Thu, 18 Apr 2002 14:17:31 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4FA0@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast
Date: Thu, 18 Apr 2002 14:17:29 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

I'm reluctant to comment on the multicast proposal given that multicast
semantics and services are likely to be changed by the IBTA.  However, I do
have some questions on IGMP that would apply in any case and would
appreciate any clarifications from those familiar with IP routing protocols.

As I understand it, when an application listens on a multicast address, the
IP stack asked the associated link driver to add a multicast group (i.e.
SIOCADDMULTI) and the IGMP protocol on that node will send a report
addressed to the newly added multicast group.  IP routers have set their
network interfaces to receive all multicast traffic so they have the
opportunity to receive the report.  But, since multicast is unreliable, what
happens if the report does not reach one or more routers.  How do they
recover from this situation?

Roy

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr 18 17:36:00 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA23667
	for <ipoverib-archive@odin.ietf.org>; Thu, 18 Apr 2002 17:36:00 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id RAA05639
	for ipoverib-archive@odin.ietf.org; Thu, 18 Apr 2002 17:36:05 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA05207;
	Thu, 18 Apr 2002 17:31:08 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA05170
	for <ipoverib@optimus.ietf.org>; Thu, 18 Apr 2002 17:31:05 -0400 (EDT)
Received: from palrel10.hp.com (palrel10.hp.com [156.153.255.245])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA23529
	for <ipoverib@ietf.org>; Thu, 18 Apr 2002 17:31:00 -0400 (EDT)
Received: from hpindda.cup.hp.com (moose-95.cup.hp.com [15.13.95.92])
	by palrel10.hp.com (Postfix) with ESMTP id AB81DC004B5
	for <ipoverib@ietf.org>; Thu, 18 Apr 2002 14:30:33 -0700 (PDT)
Received: from VRAO72425.cup.hp.com (pal2nai168195.nsr.hp.com [15.244.168.195])
	by hpindda.cup.hp.com (8.9.3 (PHNE_18546)/8.9.3 SMKit7.02) with ESMTP id OAA04669
	for <ipoverib@ietf.org>; Thu, 18 Apr 2002 14:30:32 -0700 (PDT)
Message-Id: <5.0.2.1.0.20020418142855.01b53b18@moose.cup.hp.com>
X-Sender: vandana@moose.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.0.2
Date: Thu, 18 Apr 2002 14:30:30 -0700
To: ipoverib@ietf.org
From: Vandana Rao <vandana@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4FA0@ORSMSX108>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Routers have the option of querying nodes for all associated multicast 
addresses associated with its interfaces.
-Vandana

At 02:17 PM 4/18/2002 -0700, Larsen, Roy K wrote:
>I'm reluctant to comment on the multicast proposal given that multicast
>semantics and services are likely to be changed by the IBTA.  However, I do
>have some questions on IGMP that would apply in any case and would
>appreciate any clarifications from those familiar with IP routing protocols.
>
>As I understand it, when an application listens on a multicast address, the
>IP stack asked the associated link driver to add a multicast group (i.e.
>SIOCADDMULTI) and the IGMP protocol on that node will send a report
>addressed to the newly added multicast group.  IP routers have set their
>network interfaces to receive all multicast traffic so they have the
>opportunity to receive the report.  But, since multicast is unreliable, what
>happens if the report does not reach one or more routers.  How do they
>recover from this situation?
>
>Roy
>
>_______________________________________________
>IPoverIB mailing list
>IPoverIB@ietf.org
>https://www1.ietf.org/mailman/listinfo/ipoverib


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 19 14:01:20 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA12244
	for <ipoverib-archive@odin.ietf.org>; Fri, 19 Apr 2002 14:01:20 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA00869
	for ipoverib-archive@odin.ietf.org; Fri, 19 Apr 2002 14:01:18 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id NAA00261;
	Fri, 19 Apr 2002 13:55:45 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id NAA00226
	for <ipoverib@ns.ietf.org>; Fri, 19 Apr 2002 13:55:43 -0400 (EDT)
Received: from mail1.microsoft.com (mail1.microsoft.com [131.107.3.125])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA11163
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 13:55:40 -0400 (EDT)
Received: from inet-vrs-01.redmond.corp.microsoft.com ([157.54.8.27]) by mail1.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Fri, 19 Apr 2002 10:54:28 -0700
Received: from 157.54.5.25 by inet-vrs-01.redmond.corp.microsoft.com (InterScan E-Mail VirusWall NT); Fri, 19 Apr 2002 10:55:10 -0700
Received: from red-imc-01.redmond.corp.microsoft.com ([157.54.9.102]) by inet-hub-03.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Fri, 19 Apr 2002 10:54:26 -0700
Received: from win-imc-02.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.84]) by red-imc-01.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Fri, 19 Apr 2002 10:54:28 -0700
Received: from win-msg-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.52]) by win-imc-02.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(6.0.3590.0);
	 Fri, 19 Apr 2002 10:51:33 -0700
X-MimeOLE: Produced By Microsoft Exchange V6.0.6177.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Subject: RE: [Ipoverib] IPoIB multicast
Date: Fri, 19 Apr 2002 10:51:32 -0700
Message-ID: <2E33960095B58E40A4D3345AB9F65EC102450E4A@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Thread-Topic: [Ipoverib] IPoIB multicast
Thread-Index: AcHnBJ5Cs6wQnrSCTte4m0a1imAPggAr+ScgAAUFLvA=
From: "Aditya Dube" <adube@windows.microsoft.com>
To: <ipoverib@ietf.org>
X-OriginalArrivalTime: 19 Apr 2002 17:51:33.0515 (UTC) FILETIME=[D7ED55B0:01C1E7CA]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id NAA00227
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 8bit


Vivek

Thanks for sending this out. This definitely makes it much easier for me
to analyze the proposed solution.


The Router algorithm has a problem:

		if report informs of leave
			IB_leave corresponding IB group

The First leave will cause the Router to leave the group regardless of
whether there are other nodes that have done an IB_Join.


The other thing that I have recently learnt is that Profile B hardware
requires a node to support a minimum of 16 multicast groups. Each join
is therefore a valuable resource.

Aditya Dube

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
Sent: Thursday, April 18, 2002 11:16 AM
To: ipoverib@ietf.org
Subject: [Ipoverib] IPoIB multicast

Aditya,

Below is the overall algorithm since you had asked me to submit it in
one writeup. However, in IBTA we are working on some ideas (I
participated in one such meeting today morning) that will make all
this a lot simpler.

The following is assuming IB multicast as defined at present.
There is no notification in IB that can be sent to IB nodes on group
creation or deletion. As per Mike's mail the IB group gets deleted
when the last member leaves.

Note: In this discussion I am referring to IP routers, receivers and
senders. 

The idea below is to let the router(an IP router) know that a group
has been created (by the receivers) or have the routers create the
groups when they need to forward. This is accomplished by the routers
acting on the IP group creation reports(IGMP/MLD) or the routing
protocol updates.

A sender can send at any time. It therefore attempts to join the group
when it has some packet ot send and retries if it cannot join the
first time.

The receivers leave the IB groups when they are not interested in the
associated IP group. The routers leave when they leave the associated
IP group or determine that they no longer have to forward the
associated IP group packets.

Senders leave the groups when the group has not been used for some
time. 

	Receiver:

        IP receivers must join a specific local interface and send a
        report to the router.

	
		IP_ADD_MEMBERSHIP called
			check for corresponding IB group
			if IB group exists 
				IB_join it
				IB_send report to group
			else
				IB_create IB group
                                IB_send report to all-routers MGID or
                                could send it to all-hosts group

		IP_DROP_MEMBERSHIP called
			(Send leave report to router )
			IB_leave the IB group
			
	
	Router:
		always receive on broadcast/all-hosts and all-routers
		if IGMP/MLD indicates new group
			IB_join corresponding IB group
		if routing protocol indicates new group
			IB_create corresponding IB group 
		if report informs of leave
			IB_leave corresponding IB group
		if routing protocol indicates no need to forward
			IB_leave corresponding IB Group


	Sender:
                IP senders can send at any time. They don't have to
                join at an interface or report to an IP router.

		got IP multicast packet to send
			map to the corresponding IB group
			if already 'IB_joined' the IB_group
				IB_send to it
				reset timer

			else if IB group exists
				IB_join the group
				start a timer
				IB_send to it
			else
				
				IB_send to all-routers MGID ....(c)

		if timer goes off
			IB_leave the IB group


        We can modify (c) above to 'send to all-hosts MGID'. That way
        the timer can be set to a longer value and the sender
        algorithm can become:

		got IP multicast packet to send
			map to the coresponding IB group
			if already IB_joined
				IB_send to it
				reset timer
			else 
				IB_send to all-hosts MGID
				if no timer associated
					create a timer

		if  timer goes off
			if IB_joined
				IB_leave the IB group
			else
				try to IB_join
				restart timer
			
Vivek
				
-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 19 14:32:18 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA15433
	for <ipoverib-archive@odin.ietf.org>; Fri, 19 Apr 2002 14:32:16 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA02628
	for ipoverib-archive@odin.ietf.org; Fri, 19 Apr 2002 14:32:18 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA02044;
	Fri, 19 Apr 2002 14:28:19 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA02009
	for <ipoverib@ns.ietf.org>; Fri, 19 Apr 2002 14:28:15 -0400 (EDT)
Received: from palrel13.hp.com (palrel13.hp.com [156.153.255.238])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA14894
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 14:28:10 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel13.hp.com (Postfix) with ESMTP id 25B6C4009A6
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 11:26:24 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id LAA29863
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 11:26:23 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020419111524.02a6be98@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Fri, 19 Apr 2002 11:23:47 -0700
To: <ipoverib@ietf.org>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast
In-Reply-To: <2E33960095B58E40A4D3345AB9F65EC102450E4A@win-msg-01.wingro
 up.windeploy.ntdev.microsoft.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 10:51 AM 4/19/2002 -0700, Aditya Dube wrote:

>Vivek
>
>Thanks for sending this out. This definitely makes it much easier for me
>to analyze the proposed solution.
>
>
>The Router algorithm has a problem:
>
>                 if report informs of leave
>                         IB_leave corresponding IB group
>
>The First leave will cause the Router to leave the group regardless of
>whether there are other nodes that have done an IB_Join.

IB routers leave based on policy.  IB CA acting as a multi-protocol routers 
leave based on policy.  There is nothing here that is normative that 
requires it to leave upon a lower-water mark being hit, e.g. a gross-level 
timer could be used to delay a leave until one is sure the resource will 
not be used for a significant period of time.

>The other thing that I have recently learnt is that Profile B hardware
>requires a node to support a minimum of 16 multicast groups. Each join
>is therefore a valuable resource.

Profile B is not a requirement for anyone to implement to - no one that I'm 
aware is really following any of the recommendations in this profile.  I 
will state that most are implementing a larger number than 16 groups.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 19 14:43:32 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA17001
	for <ipoverib-archive@odin.ietf.org>; Fri, 19 Apr 2002 14:43:27 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA03035
	for ipoverib-archive@odin.ietf.org; Fri, 19 Apr 2002 14:43:29 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA02977;
	Fri, 19 Apr 2002 14:39:16 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA02927
	for <ipoverib@ns.ietf.org>; Fri, 19 Apr 2002 14:39:13 -0400 (EDT)
Received: from e32.co.us.ibm.com (e32.co.us.ibm.com [32.97.110.130])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA15827;
	Fri, 19 Apr 2002 14:39:07 -0400 (EDT)
Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.194.23])
	by e32.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3JId6K5048852;
	Fri, 19 Apr 2002 14:39:06 -0400
Received: from d03nm036.boulder.ibm.com (d03nm036.boulder.ibm.com [9.17.194.36])
	by westrelay02.boulder.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3JIbmT215658;
	Fri, 19 Apr 2002 12:37:48 -0600
Subject: RE: [Ipoverib] IPoIB multicast
To: "Aditya Dube" <adube@windows.microsoft.com>
Cc: ipoverib@ietf.org, ipoverib-admin@ietf.org
X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000
Message-ID: <OF002C44FC.0E17765A-ON88256BA0.0062FC68@boulder.ibm.com>
From: "Vivek Kashyap" <vivk@us.ibm.com>
Date: Fri, 19 Apr 2002 11:35:59 -0700
X-MIMETrack: Serialize by Router on D03NM036/03/M/IBM(Release 5.0.9a |January 7, 2002) at
 04/19/2002 12:37:48 PM
MIME-Version: 1.0
Content-type: text/plain; charset=us-ascii
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org




Vivek

Thanks for sending this out. This definitely makes it much easier for me
to analyze the proposed solution.


The Router algorithm has a problem:

                         if report informs of leave
                                     IB_leave corresponding IB group

The First leave will cause the Router to leave the group regardless of
whether there are other nodes that have done an IB_Join.

<VK>
Yes, I was careless in stating this. I certainly didn't mean that one
leaves at the first report but was listing conditionals for leaving. I am
not looking into it anymore because of the changes being contemplated in
IBTA; and it also the reason for the delete case not being fully fleshed
out. I guess if we allow the router to leave the IB group (due to no
receivers and no need to forward) we also need to ensure that the senders
leave the groups. It works ok with my earlier comment on check for group
before every send(that could be quite inefficient though). Now one would
need to check for group existence periodically or have the router set a
timer when there are no receivers or need to forward on an MGID and leave
after some idle time.

The algorithms provide an outline that we can revisit (along with the
corrections as discussed above) and see how the conditions are met with the
IBA changes.

<VK>

The other thing that I have recently learnt is that Profile B hardware
requires a node to support a minimum of 16 multicast groups. Each join
is therefore a valuable resource.

<VK>
      Isn't profile B just a list of characteristics that are needed. As
far as I know it doesn't reflect the actual products in the market. Do you
know what is the number advertised by various vendors ?

      If it is number of MLIDs - one can always overload them. If it is
number of MGIDs at the CA then we need to consider the issue - it is not
limited to IPoIB multicast though. It is an IB multicast issue. We might
have to just say, use adequately provisioned CAs just like we state that
one must use multicast aware CAs or CAs which support a minimum of 2K MTU
etc.


Vivek
<VK>




Aditya Dube

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
Sent: Thursday, April 18, 2002 11:16 AM
To: ipoverib@ietf.org
Subject: [Ipoverib] IPoIB multicast

Aditya,

Below is the overall algorithm since you had asked me to submit it in
one writeup. However, in IBTA we are working on some ideas (I
participated in one such meeting today morning) that will make all
this a lot simpler.

The following is assuming IB multicast as defined at present.
There is no notification in IB that can be sent to IB nodes on group
creation or deletion. As per Mike's mail the IB group gets deleted
when the last member leaves.

Note: In this discussion I am referring to IP routers, receivers and
senders.

The idea below is to let the router(an IP router) know that a group
has been created (by the receivers) or have the routers create the
groups when they need to forward. This is accomplished by the routers
acting on the IP group creation reports(IGMP/MLD) or the routing
protocol updates.

A sender can send at any time. It therefore attempts to join the group
when it has some packet ot send and retries if it cannot join the
first time.

The receivers leave the IB groups when they are not interested in the
associated IP group. The routers leave when they leave the associated
IP group or determine that they no longer have to forward the
associated IP group packets.

Senders leave the groups when the group has not been used for some
time.

             Receiver:

        IP receivers must join a specific local interface and send a
        report to the router.


                         IP_ADD_MEMBERSHIP called
                                     check for corresponding IB group
                                     if IB group exists
                                                 IB_join it
                                                 IB_send report to group
                                     else
                                                 IB_create IB group
                                IB_send report to all-routers MGID or
                                could send it to all-hosts group

                         IP_DROP_MEMBERSHIP called
                                     (Send leave report to router )
                                     IB_leave the IB group


             Router:
                         always receive on broadcast/all-hosts and
all-routers
                         if IGMP/MLD indicates new group
                                     IB_join corresponding IB group
                         if routing protocol indicates new group
                                     IB_create corresponding IB group
                         if report informs of leave
                                     IB_leave corresponding IB group
                         if routing protocol indicates no need to forward
                                     IB_leave corresponding IB Group


             Sender:
                IP senders can send at any time. They don't have to
                join at an interface or report to an IP router.

                         got IP multicast packet to send
                                     map to the corresponding IB group
                                     if already 'IB_joined' the IB_group
                                                 IB_send to it
                                                 reset timer

                                     else if IB group exists
                                                 IB_join the group
                                                 start a timer
                                                 IB_send to it
                                     else

                                                 IB_send to all-routers
MGID ....(c)

                         if timer goes off
                                     IB_leave the IB group


        We can modify (c) above to 'send to all-hosts MGID'. That way
        the timer can be set to a longer value and the sender
        algorithm can become:

                         got IP multicast packet to send
                                     map to the coresponding IB group
                                     if already IB_joined
                                                 IB_send to it
                                                 reset timer
                                     else
                                                 IB_send to all-hosts MGID
                                                 if no timer associated
                                                             create a timer

                         if  timer goes off
                                     if IB_joined
                                                 IB_leave the IB group
                                     else
                                                 try to IB_join
                                                 restart timer

Vivek

--
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com
vivk@us.ibm.com
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib





_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 19 15:12:31 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA20604
	for <ipoverib-archive@odin.ietf.org>; Fri, 19 Apr 2002 15:12:26 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA04435
	for ipoverib-archive@odin.ietf.org; Fri, 19 Apr 2002 15:12:28 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA04268;
	Fri, 19 Apr 2002 15:06:00 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA04232
	for <ipoverib@ns.ietf.org>; Fri, 19 Apr 2002 15:05:57 -0400 (EDT)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA19604
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 15:05:46 -0400 (EDT)
Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.39 2002/04/15 17:47:23 root Exp $) with ESMTP id g3JJ54o04621
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 19:05:04 GMT
Received: from fmsmsxvs042.fm.intel.com (fmsmsxv042-1.fm.intel.com [132.233.48.110])
	by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.16 2002/04/15 17:47:00 root Exp $) with SMTP id g3JJ7fo06555
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 19:07:41 GMT
Received: from fmsmsx29.FM.INTEL.COM ([132.233.42.29])
 by fmsmsxvs042.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002041912095203476
 ; Fri, 19 Apr 2002 12:09:52 -0700
Received: by fmsmsx29.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2LKB0BJ9>; Fri, 19 Apr 2002 12:05:46 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4FA6@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'H.K. Jerry Chu'" <Jerry.Chu@eng.sun.com>,
        "'Bill Strahm'" <bill@strahm.net>
Cc: ipoverib@ietf.org, "'Vivek Kashyap'" <vivk@us.ibm.com>
Subject: RE: [Ipoverib] IPoIB multicast
Date: Fri, 19 Apr 2002 12:05:44 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Jerry and Bill,

As Vivek alluded to when he published his latest thinking on multicast and
again here, the IBTA is currently considering making some significant
changes to multicast semantics and facilities.  I echo Vivek's sentiments on
not taking the multicast design any further until the IBTA makes public
their multicast updates.  Otherwise, we are going to have to revisit many of
these issues or worse, accept non-optimal solutions because no one wants to
rehash this again.  Therefore, I am requesting that this topic be tabled
until we have a stable IBA multicast service to design against.

Roy

-----Original Message-----
From: Vivek Kashyap [mailto:vivk@us.ibm.com]
Sent: Friday, April 19, 2002 11:36 AM
To: Aditya Dube
Cc: ipoverib@ietf.org; ipoverib-admin@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast





Vivek

Thanks for sending this out. This definitely makes it much easier for me
to analyze the proposed solution.


The Router algorithm has a problem:

                         if report informs of leave
                                     IB_leave corresponding IB group

The First leave will cause the Router to leave the group regardless of
whether there are other nodes that have done an IB_Join.

<VK>
Yes, I was careless in stating this. I certainly didn't mean that one
leaves at the first report but was listing conditionals for leaving. I am
not looking into it anymore because of the changes being contemplated in
IBTA; and it also the reason for the delete case not being fully fleshed
out. I guess if we allow the router to leave the IB group (due to no
receivers and no need to forward) we also need to ensure that the senders
leave the groups. It works ok with my earlier comment on check for group
before every send(that could be quite inefficient though). Now one would
need to check for group existence periodically or have the router set a
timer when there are no receivers or need to forward on an MGID and leave
after some idle time.

The algorithms provide an outline that we can revisit (along with the
corrections as discussed above) and see how the conditions are met with the
IBA changes.

<VK>

The other thing that I have recently learnt is that Profile B hardware
requires a node to support a minimum of 16 multicast groups. Each join
is therefore a valuable resource.

<VK>
      Isn't profile B just a list of characteristics that are needed. As
far as I know it doesn't reflect the actual products in the market. Do you
know what is the number advertised by various vendors ?

      If it is number of MLIDs - one can always overload them. If it is
number of MGIDs at the CA then we need to consider the issue - it is not
limited to IPoIB multicast though. It is an IB multicast issue. We might
have to just say, use adequately provisioned CAs just like we state that
one must use multicast aware CAs or CAs which support a minimum of 2K MTU
etc.


Vivek
<VK>




Aditya Dube

-----Original Message-----
From: Vivek Kashyap [mailto:kashyapv@us.ibm.com]
Sent: Thursday, April 18, 2002 11:16 AM
To: ipoverib@ietf.org
Subject: [Ipoverib] IPoIB multicast

Aditya,

Below is the overall algorithm since you had asked me to submit it in
one writeup. However, in IBTA we are working on some ideas (I
participated in one such meeting today morning) that will make all
this a lot simpler.

The following is assuming IB multicast as defined at present.
There is no notification in IB that can be sent to IB nodes on group
creation or deletion. As per Mike's mail the IB group gets deleted
when the last member leaves.

Note: In this discussion I am referring to IP routers, receivers and
senders.

The idea below is to let the router(an IP router) know that a group
has been created (by the receivers) or have the routers create the
groups when they need to forward. This is accomplished by the routers
acting on the IP group creation reports(IGMP/MLD) or the routing
protocol updates.

A sender can send at any time. It therefore attempts to join the group
when it has some packet ot send and retries if it cannot join the
first time.

The receivers leave the IB groups when they are not interested in the
associated IP group. The routers leave when they leave the associated
IP group or determine that they no longer have to forward the
associated IP group packets.

Senders leave the groups when the group has not been used for some
time.

             Receiver:

        IP receivers must join a specific local interface and send a
        report to the router.


                         IP_ADD_MEMBERSHIP called
                                     check for corresponding IB group
                                     if IB group exists
                                                 IB_join it
                                                 IB_send report to group
                                     else
                                                 IB_create IB group
                                IB_send report to all-routers MGID or
                                could send it to all-hosts group

                         IP_DROP_MEMBERSHIP called
                                     (Send leave report to router )
                                     IB_leave the IB group


             Router:
                         always receive on broadcast/all-hosts and
all-routers
                         if IGMP/MLD indicates new group
                                     IB_join corresponding IB group
                         if routing protocol indicates new group
                                     IB_create corresponding IB group
                         if report informs of leave
                                     IB_leave corresponding IB group
                         if routing protocol indicates no need to forward
                                     IB_leave corresponding IB Group


             Sender:
                IP senders can send at any time. They don't have to
                join at an interface or report to an IP router.

                         got IP multicast packet to send
                                     map to the corresponding IB group
                                     if already 'IB_joined' the IB_group
                                                 IB_send to it
                                                 reset timer

                                     else if IB group exists
                                                 IB_join the group
                                                 start a timer
                                                 IB_send to it
                                     else

                                                 IB_send to all-routers
MGID ....(c)

                         if timer goes off
                                     IB_leave the IB group


        We can modify (c) above to 'send to all-hosts MGID'. That way
        the timer can be set to a longer value and the sender
        algorithm can become:

                         got IP multicast packet to send
                                     map to the coresponding IB group
                                     if already IB_joined
                                                 IB_send to it
                                                 reset timer
                                     else
                                                 IB_send to all-hosts MGID
                                                 if no timer associated
                                                             create a timer

                         if  timer goes off
                                     if IB_joined
                                                 IB_leave the IB group
                                     else
                                                 try to IB_join
                                                 restart timer

Vivek

--
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com
vivk@us.ibm.com
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib





_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 19 15:36:33 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA23719
	for <ipoverib-archive@odin.ietf.org>; Fri, 19 Apr 2002 15:36:33 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA05887
	for ipoverib-archive@odin.ietf.org; Fri, 19 Apr 2002 15:36:35 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA05683;
	Fri, 19 Apr 2002 15:31:51 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA05647
	for <ipoverib@ns.ietf.org>; Fri, 19 Apr 2002 15:31:48 -0400 (EDT)
Received: from palrel11.hp.com (palrel11.hp.com [156.153.255.246])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA23185
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 15:31:45 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel11.hp.com (Postfix) with ESMTP
	id 2B969600D71; Sat, 20 Apr 2002 20:49:12 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id MAA11276;
	Fri, 19 Apr 2002 12:31:12 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020419122857.02a95a98@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Fri, 19 Apr 2002 12:31:08 -0700
To: "Larsen, Roy K" <roy.k.larsen@intel.com>,
        "'H.K. Jerry Chu'" <Jerry.Chu@eng.sun.com>,
        "'Bill Strahm'" <bill@strahm.net>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast
Cc: ipoverib@ietf.org
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4FA6@ORSMSX108>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 12:05 PM 4/19/2002 -0700, Larsen, Roy K wrote:
>Jerry and Bill,
>
>As Vivek alluded to when he published his latest thinking on multicast and
>again here, the IBTA is currently considering making some significant
>changes to multicast semantics and facilities.  I echo Vivek's sentiments on
>not taking the multicast design any further until the IBTA makes public
>their multicast updates.  Otherwise, we are going to have to revisit many of
>these issues or worse, accept non-optimal solutions because no one wants to
>rehash this again.  Therefore, I am requesting that this topic be tabled
>until we have a stable IBA multicast service to design against.

You have a good idea based on what I wrote and Vivek wrote to understand 
that there will be minimal impact to IP over IB specifications and this can 
go forward.  All that needs to be done is just describe the IP multicast 
address encapsulation and the work is essentially done.  The IP router 
interactions are also a non-issue.  Keep in mind that the IP over IB 
implementation is what is impacted by any of these IB changes and not the 
IP over IB protocol so let's finish encapsulation for unicast and multicast 
and be done with it.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 19 16:28:08 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA29285
	for <ipoverib-archive@odin.ietf.org>; Fri, 19 Apr 2002 16:28:08 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA08200
	for ipoverib-archive@odin.ietf.org; Fri, 19 Apr 2002 16:28:10 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA08044;
	Fri, 19 Apr 2002 16:23:26 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA08008
	for <ipoverib@ns.ietf.org>; Fri, 19 Apr 2002 16:23:23 -0400 (EDT)
Received: from hermes.fm.intel.com (fmr01.intel.com [192.55.52.18])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA28732
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 16:23:00 -0400 (EDT)
Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37])
	by hermes.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.39 2002/04/15 17:47:23 root Exp $) with ESMTP id g3JKO6x19695
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 20:24:06 GMT
Received: from fmsmsxvs042.fm.intel.com (fmsmsxv042-1.fm.intel.com [132.233.48.110])
	by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.16 2002/04/15 17:47:00 root Exp $) with SMTP id g3JKLOC13708
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 20:21:29 GMT
Received: from fmsmsx26.fm.intel.com ([132.233.42.26])
 by fmsmsxvs042.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002041913270013097
 ; Fri, 19 Apr 2002 13:27:00 -0700
Received: by fmsmsx26.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2HL0L1H2>; Fri, 19 Apr 2002 13:22:55 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4FA7@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Michael Krause'" <krause@cup.hp.com>,
        "'H.K. Jerry Chu'"<Jerry.Chu@eng.sun.com>,
        "'Bill Strahm'" <bill@strahm.net>
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast
Date: Fri, 19 Apr 2002 13:22:45 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Mike,

I would be thrilled if a correct and interoperable IPoIB multicast
implementation is as simple as having receivers join the associated group
when needed and leave when done, and senders join using the appropriate IBA
method for senders, send their frames to the requested address, and leave as
deemed appropriate.  Unfortunately, due to my shallow knowledge of IP
routing issues and protocols, I've been drawn into the believe that it is
not that simple.  If we can get consensus that it is, you are absolutely
right.  I don't think we have that consensus so it just seems futile to have
a discussion where the majority of the participants are in the dark about
pending IBA multicast changes.

Roy 
 

-----Original Message-----
From: Michael Krause [mailto:krause@cup.hp.com]
Sent: Friday, April 19, 2002 12:31 PM
To: Larsen, Roy K; 'H.K. Jerry Chu'; 'Bill Strahm'
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast


At 12:05 PM 4/19/2002 -0700, Larsen, Roy K wrote:
>Jerry and Bill,
>
>As Vivek alluded to when he published his latest thinking on multicast and
>again here, the IBTA is currently considering making some significant
>changes to multicast semantics and facilities.  I echo Vivek's sentiments
on
>not taking the multicast design any further until the IBTA makes public
>their multicast updates.  Otherwise, we are going to have to revisit many
of
>these issues or worse, accept non-optimal solutions because no one wants to
>rehash this again.  Therefore, I am requesting that this topic be tabled
>until we have a stable IBA multicast service to design against.

You have a good idea based on what I wrote and Vivek wrote to understand 
that there will be minimal impact to IP over IB specifications and this can 
go forward.  All that needs to be done is just describe the IP multicast 
address encapsulation and the work is essentially done.  The IP router 
interactions are also a non-issue.  Keep in mind that the IP over IB 
implementation is what is impacted by any of these IB changes and not the 
IP over IB protocol so let's finish encapsulation for unicast and multicast 
and be done with it.

Mike

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 19 17:04:06 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA03411
	for <ipoverib-archive@odin.ietf.org>; Fri, 19 Apr 2002 17:04:06 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id RAA10634
	for ipoverib-archive@odin.ietf.org; Fri, 19 Apr 2002 17:04:08 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA09343;
	Fri, 19 Apr 2002 16:54:06 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA09313
	for <ipoverib@ns.ietf.org>; Fri, 19 Apr 2002 16:54:04 -0400 (EDT)
Received: from palrel13.hp.com (palrel13.hp.com [156.153.255.238])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA02109
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 16:54:00 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel13.hp.com (Postfix) with ESMTP
	id C9BD7400AAB; Fri, 19 Apr 2002 13:53:32 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id NAA00176;
	Fri, 19 Apr 2002 13:53:32 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020419134904.02ad3e60@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Fri, 19 Apr 2002 13:50:41 -0700
To: "Larsen, Roy K" <roy.k.larsen@intel.com>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast
Cc: ipoverib@ietf.org
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C06BC4FA7@ORSMSX108>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 01:22 PM 4/19/2002 -0700, Larsen, Roy K wrote:
>Mike,
>
>I would be thrilled if a correct and interoperable IPoIB multicast
>implementation is as simple as having receivers join the associated group
>when needed and leave when done, and senders join using the appropriate IBA
>method for senders, send their frames to the requested address, and leave as
>deemed appropriate.

This is all that is needed and should be applicable to IP over IB.

>Unfortunately, due to my shallow knowledge of IP
>routing issues and protocols, I've been drawn into the believe that it is
>not that simple.  If we can get consensus that it is, you are absolutely
>right.  I don't think we have that consensus so it just seems futile to have
>a discussion where the majority of the participants are in the dark about
>pending IBA multicast changes.

The IB changes should result in this simple approach being all that IP over 
IB needs to deal with and since the IB portions are really a LLP issue from 
a spec point of view, there is no reason to get completion of the IP over 
IB spec at this time.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 19 17:17:38 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA04711
	for <ipoverib-archive@odin.ietf.org>; Fri, 19 Apr 2002 17:17:38 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id RAA11134
	for ipoverib-archive@odin.ietf.org; Fri, 19 Apr 2002 17:17:41 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA10880;
	Fri, 19 Apr 2002 17:07:34 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA10813
	for <ipoverib@ns.ietf.org>; Fri, 19 Apr 2002 17:07:30 -0400 (EDT)
Received: from mxout1.netvision.net.il (mxout1.netvision.net.il [194.90.9.20])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA03741
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 17:07:26 -0400 (EDT)
Received: from vaio505he ([139.92.208.158]) by mxout1.netvision.net.il
 (iPlanet Messaging Server 5.1 (built Sep  5 2001))
 with ESMTPA id <0GUU0056K2ND1R@mxout1.netvision.net.il> for ipoverib@ietf.org;
 Sat, 20 Apr 2002 00:06:51 +0300 (IDT)
Date: Fri, 19 Apr 2002 23:07:54 +0200
From: Yaron Haviv <yaronh@voltaire.com>
Subject: RE: [Ipoverib] IPoIB multicast
In-reply-to: <5.1.0.14.2.20020419111524.02a6be98@esalpha2.cup.hp.com>
To: "'Michael Krause'" <krause@cup.hp.com>, ipoverib@ietf.org
Reply-to: yaronh@voltaire.com
Message-id: <002401c1e7e6$46d345f0$9ed05c8b@vaio505he>
MIME-version: 1.0
X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4807.1700
X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2910.0)
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Importance: Normal
X-Priority: 3 (Normal)
X-MSMail-priority: Normal
Content-Transfer-Encoding: 7BIT
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7BIT

For my knowledge at least one major HCA vendor supports only 16
So this issue is real, not to mention that each IPoIB partition consumes one
of these groups

This should be addressed some how, maybe even by having multiple MGID's use
the same MLID

YARON

-----Original Message-----
From: ipoverib-admin@ietf.org [mailto:ipoverib-admin@ietf.org]On Behalf Of
Michael Krause
Sent: Friday, April 19, 2002 8:24 PM
To: ipoverib@ietf.org
Subject: RE: [Ipoverib] IPoIB multicast

At 10:51 AM 4/19/2002 -0700, Aditya Dube wrote:

>Vivek
>
>Thanks for sending this out. This definitely makes it much easier for me
>to analyze the proposed solution.
>
>
>The Router algorithm has a problem:
>
>                 if report informs of leave
>                         IB_leave corresponding IB group
>
>The First leave will cause the Router to leave the group regardless of
>whether there are other nodes that have done an IB_Join.

IB routers leave based on policy.  IB CA acting as a multi-protocol routers
leave based on policy.  There is nothing here that is normative that
requires it to leave upon a lower-water mark being hit, e.g. a gross-level
timer could be used to delay a leave until one is sure the resource will
not be used for a significant period of time.

>The other thing that I have recently learnt is that Profile B hardware
>requires a node to support a minimum of 16 multicast groups. Each join
>is therefore a valuable resource.

Profile B is not a requirement for anyone to implement to - no one that I'm
aware is really following any of the recommendations in this profile.  I
will state that most are implementing a larger number than 16 groups.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 19 18:53:02 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA14815
	for <ipoverib-archive@odin.ietf.org>; Fri, 19 Apr 2002 18:53:02 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id SAA15327
	for ipoverib-archive@odin.ietf.org; Fri, 19 Apr 2002 18:53:04 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA15050;
	Fri, 19 Apr 2002 18:43:06 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id SAA15016
	for <ipoverib@ns.ietf.org>; Fri, 19 Apr 2002 18:43:04 -0400 (EDT)
Received: from palrel13.hp.com (palrel13.hp.com [156.153.255.238])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA13796
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 18:42:59 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel13.hp.com (Postfix) with ESMTP id 9E9D6400286
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 15:42:32 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id PAA19273
	for <ipoverib@ietf.org>; Fri, 19 Apr 2002 15:42:32 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020419153253.02ad7c48@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Fri, 19 Apr 2002 15:34:07 -0700
To: ipoverib@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast
In-Reply-To: <002401c1e7e6$46d345f0$9ed05c8b@vaio505he>
References: <5.1.0.14.2.20020419111524.02a6be98@esalpha2.cup.hp.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 11:07 PM 4/19/2002 +0200, Yaron Haviv wrote:
>For my knowledge at least one major HCA vendor supports only 16
>So this issue is real, not to mention that each IPoIB partition consumes one
>of these groups

Some early implementations may have such limitations but most 
implementations are increasing their resources in this regard by quite a 
bit so I don't think 16 is a magic number.

>This should be addressed some how, maybe even by having multiple MGID's use
>the same MLID

This is supported already and is strictly a LLP issue not an IP over IB issue.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr 22 04:49:14 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA19409
	for <ipoverib-archive@odin.ietf.org>; Mon, 22 Apr 2002 04:49:09 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id EAA29054
	for ipoverib-archive@odin.ietf.org; Mon, 22 Apr 2002 04:49:13 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id EAA28745;
	Mon, 22 Apr 2002 04:45:04 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id EAA28716
	for <ipoverib@optimus.ietf.org>; Mon, 22 Apr 2002 04:45:02 -0400 (EDT)
Received: from patan.sun.com (patan.Sun.COM [192.18.98.43])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA19324
	for <ipoverib@ietf.org>; Mon, 22 Apr 2002 04:44:58 -0400 (EDT)
Received: from sunmail1.Sun.COM ([129.145.1.2])
	by patan.sun.com (8.9.3+Sun/8.9.3) with ESMTP id CAA05330;
	Mon, 22 Apr 2002 02:44:34 -0600 (MDT)
Received: from jurassic.eng.sun.com (jurassic-17-a.Eng.Sun.COM [129.146.17.55])
	by sunmail1.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1-Sun.COM.mod.2) with ESMTP id BAA22301;
	Mon, 22 Apr 2002 01:44:45 -0700 (PDT)
Received: from sweethome (dsl-194-138.Eng.Sun.COM [129.146.194.138])
	by jurassic.eng.sun.com (8.12.2+Sun/8.12.2) with SMTP id g3M8iVJe018562;
	Mon, 22 Apr 2002 01:44:33 -0700 (PDT)
Message-Id: <200204220844.g3M8iVJe018562@jurassic.eng.sun.com>
Date: Mon, 22 Apr 2002 01:45:53 -0700 (PDT)
From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Reply-To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Subject: RE: [Ipoverib] IPoIB multicast
To: roy.k.larsen@intel.com, bill@strahm.net, krause@cup.hp.com
Cc: ipoverib@ietf.org
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: jgHNH7faWBAbc2jR0OIC4Q==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.2 SunOS 5.8 sun4u sparc 
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

>At 12:05 PM 4/19/2002 -0700, Larsen, Roy K wrote:
>>Jerry and Bill,
>>
>>As Vivek alluded to when he published his latest thinking on multicast and
>>again here, the IBTA is currently considering making some significant
>>changes to multicast semantics and facilities.  I echo Vivek's sentiments on
>>not taking the multicast design any further until the IBTA makes public
>>their multicast updates.

These multicast related issues have been brought to IBTA by a number
of us (including both me and Vivek) for some time, and have been
actively debated upon over there. Since it was Vivek who initiated
this round of discussion here before IBTA closes on their part of
the debate, I certainly have no problem if Vivek and you want to
suspend the discussion. But remember IBTA also looks to IETF for
input on a sensible mutlicast desgin. So the discussion here is
not completely wasted.

>>Otherwise, we are going to have to revisit many of
>>these issues or worse, accept non-optimal solutions because no one wants to
>>rehash this again.  Therefore, I am requesting that this topic be tabled
>>until we have a stable IBA multicast service to design against.
>
>You have a good idea based on what I wrote and Vivek wrote to understand 
>that there will be minimal impact to IP over IB specifications and this can 
>go forward.  All that needs to be done is just describe the IP multicast 
>address encapsulation and the work is essentially done.  The IP router 
>interactions are also a non-issue.  Keep in mind that the IP over IB 
>implementation is what is impacted by any of these IB changes and not the 
>IP over IB protocol so let's finish encapsulation for unicast and multicast 
>and be done with it.

Disagreed. These multicast issues, if not resolved, will affect the
proper function of IP multicast over IB. They are not mere
implementation issues, but they are interoperability concerns.

Let me attempt to summarize the differences between IB multicast
(as defined in 1.0) and IP multicast that cause all the grieves.
Similar discussions have also been documented in the link and
multicast draft.

Note that sometimes when I say "IP multicast" I really mean
"IP multicast over Ethernet". I use the two interchangably because,
although IP is supposed to be link agnostic, Ethernet has been the
dominant link type, and has exerted a great deal of influence over
IP design. The reality is that in order for other link types to
work seamlessly with IP, the easiest way often is to emulate the
model used in Ethernet.

The two main differences between IP/Ethernet and IB multicast are

1. In Ethernet multicast groups don't need to be created before
use, or at least this is the illusion made to the hosts by the
Ether switches. An IP multicast sender conveniently dump pkts
into the corresponding Ethernet multicast address without worrying
whether such address "exists" or not. All the right things happen
automagically behind the switches.

But in IB, a MC group must be created before use. This is not a
biggie for a multicast receiver, as a group can be created by
the first join, and deleted by the last member leaving. But it
causes some grief for the multicast sender, as IP doesn't require
a sender to be a member. Anyone can freely send MC pkts to any
MC group. In that case, should an IB MC group be created for
send only? If yes, how does one determine when to delete a
"send-only" MC group, since IP doesn't give any hint to the
interface driver when a sender is done sending MC pkts? (For
receivers there is explict join/leave call.)

The proposal I made in the link-multicast draft when the IB MC
group corresponding to the target IP MC address doesn't already
exist, is to have the interface driver forward the pkt to the all-
router MC group. If the MC group comes into existence later
(i.e. someone joins as a member), the driver needs to be notified
so it can forward pkts back to the right MC group. This requires
some support from SA. I believe a proposal has been made recently
in IBTA to add this support to SM/SA.

2. Ethernet NICs can be programmed into promiscuous multicast
mode when the NIC will send a copy of every MC pkt to IP.
IGMP/MLD uses this feature conveniently to pick up a copy of
every MC pkt to track MC membership, and to route MC pkts
correctly to other subnets.

Since IB doesn't support a promiscuous multicast mode. The
workaround I proposed is to have the interface driver make a copy
of every MC pkt and send it to the all-router MC group. This is
pretty straightfoward with a small performance penalty. But is
still much more efficient than resorting to broadcast. There is
a small problem in case the router itself also joins a MC group
and will then receive two copies. This case can be adressed with
some more check in the interface driver of a router but since
the case is rare, I'm not sure it's worth the effort and additional
complexity.

Other alternative includes Vivek's first proposal to snoop IGMP/MLD
pkts in the interface driver. This requires driver to understand
IGMP/MLD, and will make it very difficult to maintain (with newer
version of IGMPv1,2,3.../MLD).

Vivek's recent proposal fixed the above problem by moving the work
from the driver to IGMP/MLD themselves. Unfortunately IMO the
chance of requesting link specific changes in a link-independent
module like IGMP is not very good.

Perhaps if IBTA can supply us an equivalent of a promiscuous
multicast mode without bending IBA backwards too much that would
be ideal.  I've heard some discussion along that line is going on.
Let's see what comes out of it.

Jerry

>
>Mike
>


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 22 09:40:46 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA25419
	for <ipoverib-archive@odin.ietf.org>; Mon, 22 Apr 2002 09:40:46 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id JAA14976
	for ipoverib-archive@odin.ietf.org; Mon, 22 Apr 2002 09:40:49 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA14728;
	Mon, 22 Apr 2002 09:35:28 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA14696
	for <ipoverib@ns.ietf.org>; Mon, 22 Apr 2002 09:35:25 -0400 (EDT)
Received: from palrel12.hp.com (palrel12.hp.com [156.153.255.237])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA25238
	for <ipoverib@ietf.org>; Mon, 22 Apr 2002 09:35:22 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel12.hp.com (Postfix) with ESMTP
	id 65E93E00558; Mon, 22 Apr 2002 06:34:52 -0700 (PDT)
Received: from mkrause.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id GAA17115;
	Mon, 22 Apr 2002 06:34:51 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020422062257.02afeeb0@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Mon, 22 Apr 2002 06:34:18 -0700
To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [Ipoverib] IPoIB multicast
Cc: ipoverib@ietf.org
In-Reply-To: <200204220844.g3M8iVJe018562@jurassic.eng.sun.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 01:45 AM 4/22/2002 -0700, H.K. Jerry Chu wrote:
>The proposal I made in the link-multicast draft when the IB MC
>group corresponding to the target IP MC address doesn't already
>exist, is to have the interface driver forward the pkt to the all-
>router MC group. If the MC group comes into existence later
>(i.e. someone joins as a member), the driver needs to be notified
>so it can forward pkts back to the right MC group. This requires
>some support from SA. I believe a proposal has been made recently
>in IBTA to add this support to SM/SA.

To make it quite clear - one cannot send on IB without providing a MLID 
that has been configured into switches.  It does not matter whether this is 
for an all routers or a specific multicast group - this level of 
configuration must be performed.  Hence, trying to target an all routers 
multicast group provides zero benefit since the same work needs to be 
performed for both types so why not do it right the first time.  This is 
why I am strongly opposed to the use of such a technique and view this as 
unnecessary complexity.

I've stated the simplest algorithm:

- Join the IB multicast group
- Get the MLID configured
- Send the packet
- If any one is listening, they will receive it
- If they are not listening at the time of the send, then they will get a 
subsequent send but no attempt should be made to turn this into a reliable 
service as suggested in some e-mails.
- Group changes are tracked by the SA
- The SA will inform a group when it is no longer available

The new IBTA method will be a form of async notification service that will 
allow endnodes to detect when groups are added / deleted which is something 
a couple of us have proposed for quite some time and is now finding 
traction.  If the workgroup focuses on the join process and recognizes that 
a notification method will exist (which is entirely an IB issue with an 
implementation's LLP masking any of this complexity - this is clearly not 
an IP over IB issue), then we can complete this set of specifications.

>2. Ethernet NICs can be programmed into promiscuous multicast
>mode when the NIC will send a copy of every MC pkt to IP.
>IGMP/MLD uses this feature conveniently to pick up a copy of
>every MC pkt to track MC membership, and to route MC pkts
>correctly to other subnets.
>
>Since IB doesn't support a promiscuous multicast mode. The
>workaround I proposed is to have the interface driver make a copy
>of every MC pkt and send it to the all-router MC group. This is
>pretty straightfoward with a small performance penalty. But is
>still much more efficient than resorting to broadcast. There is
>a small problem in case the router itself also joins a MC group
>and will then receive two copies. This case can be adressed with
>some more check in the interface driver of a router but since
>the case is rare, I'm not sure it's worth the effort and additional
>complexity.

IB can be configured to see all multicast packets within a given partition 
- this requires one to join all multicast groups.  This is a policy 
decision.  The fact that it is different than Ethernet is irrelevant quite 
frankly.  This is an IP over IB not IP over Ethernet workgroup.  What you 
are describing is in many cases really no different than the class of 
problems associated with LANE.

>Other alternative includes Vivek's first proposal to snoop IGMP/MLD
>pkts in the interface driver. This requires driver to understand
>IGMP/MLD, and will make it very difficult to maintain (with newer
>version of IGMPv1,2,3.../MLD).
>
>Vivek's recent proposal fixed the above problem by moving the work
>from the driver to IGMP/MLD themselves. Unfortunately IMO the
>chance of requesting link specific changes in a link-independent
>module like IGMP is not very good.

There is no reason to add this complexity at all which is why I continue to 
oppose this line of logic.

>Perhaps if IBTA can supply us an equivalent of a promiscuous
>multicast mode without bending IBA backwards too much that would
>be ideal.  I've heard some discussion along that line is going on.
>Let's see what comes out of it.

One has an promiscuous solution by employing the notification method and 
performing a join operation for all multicast groups associated with a 
particular partition.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 22 14:20:48 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA06730
	for <ipoverib-archive@odin.ietf.org>; Mon, 22 Apr 2002 14:20:48 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA06594
	for ipoverib-archive@odin.ietf.org; Mon, 22 Apr 2002 14:20:51 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA06089;
	Mon, 22 Apr 2002 14:16:44 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA06056
	for <ipoverib@ns.ietf.org>; Mon, 22 Apr 2002 14:16:35 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA06501
	for <ipoverib@ietf.org>; Mon, 22 Apr 2002 14:16:31 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3MIDxuE025382;
	Mon, 22 Apr 2002 14:13:59 -0400
Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3MIDsc145174;
	Mon, 22 Apr 2002 14:13:54 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id g3MIGlW11714;
	Mon, 22 Apr 2002 11:16:47 -0700
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3MIDmp18644;
	Mon, 22 Apr 2002 11:13:48 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204221813.g3MIDmp18644@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast
To: Jerry.Chu@eng.sun.com
Date: Mon, 22 Apr 2002 11:13:47 -0700 (PDT)
Cc: roy.k.larsen@intel.com, bill@strahm.net, krause@cup.hp.com,
        ipoverib@ietf.org
In-Reply-To: <200204220844.g3M8iVJe018562@jurassic.eng.sun.com> from "H.K. Jerry Chu" at Apr 22, 2002 12:45:53 AM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> >At 12:05 PM 4/19/2002 -0700, Larsen, Roy K wrote:
> >>Jerry and Bill,
> >>
> >>As Vivek alluded to when he published his latest thinking on multicast and
> >>again here, the IBTA is currently considering making some significant
> >>changes to multicast semantics and facilities.  I echo Vivek's sentiments on
> >>not taking the multicast design any further until the IBTA makes public
> >>their multicast updates.
> 
> These multicast related issues have been brought to IBTA by a number
> of us (including both me and Vivek) for some time, and have been
> actively debated upon over there. Since it was Vivek who initiated
> this round of discussion here before IBTA closes on their part of
> the debate, I certainly have no problem if Vivek and you want to
> suspend the discussion. But remember IBTA also looks to IETF for
> input on a sensible mutlicast desgin. So the discussion here is
> not completely wasted.

  
        I did not ask for the debate to be closed. Roy mistook my
        message. 
  
<snip>

> The two main differences between IP/Ethernet and IB multicast are
> 
> 1. In Ethernet multicast groups don't need to be created before
> use, or at least this is the illusion made to the hosts by the
> Ether switches. An IP multicast sender conveniently dump pkts
> into the corresponding Ethernet multicast address without worrying
> whether such address "exists" or not. All the right things happen
> automagically behind the switches.
> 
> But in IB, a MC group must be created before use. This is not a
> biggie for a multicast receiver, as a group can be created by
> the first join, and deleted by the last member leaving. But it
> causes some grief for the multicast sender, as IP doesn't require
> a sender to be a member. Anyone can freely send MC pkts to any

        IP still is not involved. All the issues of group
        existence/join or not etc. exist in the driver layer. The
        groups in question are the IB level groups.

> MC group. In that case, should an IB MC group be created for
> send only? If yes, how does one determine when to delete a
> "send-only" MC group, since IP doesn't give any hint to the
> interface driver when a sender is done sending MC pkts? (For
> receivers there is explict join/leave call.)

        Also, no one is informed beforehand of any intention to
        transmit multicast packets. For receive there is the IGMP/MLD
        report mechanism.

        That is the reason, in the algorithm that I presented, to
        leave the creation with the receivers and use an existing
        mechanism (IGMP/MLD update) to trigger router's discovery of
        the groups. Senders don't create groups and are asked to use
        timers. 

> 
> The proposal I made in the link-multicast draft when the IB MC
> group corresponding to the target IP MC address doesn't already
> exist, is to have the interface driver forward the pkt to the all-
> router MC group. If the MC group comes into existence later
> (i.e. someone joins as a member), the driver needs to be notified
> so it can forward pkts back to the right MC group. This requires
> some support from SA. I believe a proposal has been made recently
> in IBTA to add this support to SM/SA.

        That won't gel with 1.0 spec and we didn't provide any method
        in link-multicast draft for the necessary notification. Now
        that IBTA is moving to modify the IBTA setup we can certainly
        make use of such a mechanism.

> 
> 2. Ethernet NICs can be programmed into promiscuous multicast
> mode when the NIC will send a copy of every MC pkt to IP.
> IGMP/MLD uses this feature conveniently to pick up a copy of
> every MC pkt to track MC membership, and to route MC pkts
> correctly to other subnets.
> 
> Since IB doesn't support a promiscuous multicast mode. The
> workaround I proposed is to have the interface driver make a copy
> of every MC pkt and send it to the all-router MC group. This is
> pretty straightfoward with a small performance penalty. But is
> still much more efficient than resorting to broadcast. There is
> a small problem in case the router itself also joins a MC group
> and will then receive two copies. This case can be adressed with
> some more check in the interface driver of a router but since
> the case is rare, I'm not sure it's worth the effort and additional
> complexity.

	One of the issues I have with this proposal is that it 
        deliberately introduces duplicates in the fabric. The solution
        also doesn't provide the method by which a sender gets to know
        the MGIDs nor does it say how the router comes to know of the
        new IB multicast Groups.

> 
> Other alternative includes Vivek's first proposal to snoop IGMP/MLD
> pkts in the interface driver. This requires driver to understand
> IGMP/MLD, and will make it very difficult to maintain (with newer
> version of IGMPv1,2,3.../MLD).
> 
> Vivek's recent proposal fixed the above problem by moving the work
> from the driver to IGMP/MLD themselves. Unfortunately IMO the
> chance of requesting link specific changes in a link-independent
> module like IGMP is not very good.

        I disagree here. I checked with the routing area AD before
        continuing with the discussion in IPoIB WG. Given the current
        IBA multicast infrastructure this is a good solution.

        Now, if IBTA does bring out the right mechanism we won't have
        to do this.

> 
> Perhaps if IBTA can supply us an equivalent of a promiscuous
> multicast mode without bending IBA backwards too much that would
> be ideal.  I've heard some discussion along that line is going on.
> Let's see what comes out of it.

        I am not sure how IB architecture can accomodate a promiscuous
        mode - it will require including the router(s) interfaces in
        every multicast tree setup. But other solutions are being
        discussed in IBTA that hopefully will address IPoIB WG's
        concerns. 


Vivek

> 
> Jerry
> 
> >
> >Mike
> >
> 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 22 14:33:19 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA07416
	for <ipoverib-archive@odin.ietf.org>; Mon, 22 Apr 2002 14:33:19 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA07957
	for ipoverib-archive@odin.ietf.org; Mon, 22 Apr 2002 14:33:22 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAB07316;
	Mon, 22 Apr 2002 14:29:36 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA07282
	for <ipoverib@ns.ietf.org>; Mon, 22 Apr 2002 14:29:34 -0400 (EDT)
Received: from palrel11.hp.com (palrel11.hp.com [156.153.255.246])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA07216
	for <ipoverib@ietf.org>; Mon, 22 Apr 2002 14:29:30 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel11.hp.com (Postfix) with ESMTP
	id 4EDD16002B6; Thu, 25 Apr 2002 15:50:35 -0700 (PDT)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id LAA03726;
	Mon, 22 Apr 2002 11:29:02 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020422112632.02bdaf60@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Mon, 22 Apr 2002 11:28:58 -0700
To: Vivek Kashyap <kashyapv@us.ibm.com>
From: Michael Krause <krause@cup.hp.com>
Subject: Re: [Ipoverib] IPoIB multicast
Cc: ipoverib@ietf.org
In-Reply-To: <200204221813.g3MIDmp18644@eng4.beaverton.ibm.com>
References: <200204220844.g3M8iVJe018562@jurassic.eng.sun.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

At 11:13 AM 4/22/2002 -0700, Vivek Kashyap wrote:

>         IP still is not involved. All the issues of group
>         existence/join or not etc. exist in the driver layer. The
>         groups in question are the IB level groups.

Agreed.


> > MC group. In that case, should an IB MC group be created for
> > send only? If yes, how does one determine when to delete a
> > "send-only" MC group, since IP doesn't give any hint to the
> > interface driver when a sender is done sending MC pkts? (For
> > receivers there is explict join/leave call.)
>
>         Also, no one is informed beforehand of any intention to
>         transmit multicast packets. For receive there is the IGMP/MLD
>         report mechanism.
>
>         That is the reason, in the algorithm that I presented, to
>         leave the creation with the receivers and use an existing
>         mechanism (IGMP/MLD update) to trigger router's discovery of
>         the groups. Senders don't create groups and are asked to use
>         timers.

We don't need this involvement as it is the IB LLP driver implementation 
that deals with this and not IP.


>         One of the issues I have with this proposal is that it
>         deliberately introduces duplicates in the fabric. The solution
>         also doesn't provide the method by which a sender gets to know
>         the MGIDs nor does it say how the router comes to know of the
>         new IB multicast Groups.

These are the exact issues I raised a couple of weeks ago and am strongly 
opposed to pursuing such a strategy.


> >
> > Other alternative includes Vivek's first proposal to snoop IGMP/MLD
> > pkts in the interface driver. This requires driver to understand
> > IGMP/MLD, and will make it very difficult to maintain (with newer
> > version of IGMPv1,2,3.../MLD).
> >
> > Vivek's recent proposal fixed the above problem by moving the work
> > from the driver to IGMP/MLD themselves. Unfortunately IMO the
> > chance of requesting link specific changes in a link-independent
> > module like IGMP is not very good.
>
>         I disagree here. I checked with the routing area AD before
>         continuing with the discussion in IPoIB WG. Given the current
>         IBA multicast infrastructure this is a good solution.
>
>         Now, if IBTA does bring out the right mechanism we won't have
>         to do this.

Exactly.


> >
> > Perhaps if IBTA can supply us an equivalent of a promiscuous
> > multicast mode without bending IBA backwards too much that would
> > be ideal.  I've heard some discussion along that line is going on.
> > Let's see what comes out of it.
>
>         I am not sure how IB architecture can accomodate a promiscuous
>         mode - it will require including the router(s) interfaces in
>         every multicast tree setup. But other solutions are being
>         discussed in IBTA that hopefully will address IPoIB WG's
>         concerns.

See my e-mail.  Any CA can join all multicast groups for a given partition 
(akin to a specified VLAN) and thus can operate in promiscuous mode.  This 
is all hidden within the IB LLP driver implementation.

Mike


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 22 14:37:46 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA07666
	for <ipoverib-archive@odin.ietf.org>; Mon, 22 Apr 2002 14:37:45 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA08391
	for ipoverib-archive@odin.ietf.org; Mon, 22 Apr 2002 14:37:49 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA08009;
	Mon, 22 Apr 2002 14:33:46 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA07977
	for <ipoverib@ns.ietf.org>; Mon, 22 Apr 2002 14:33:44 -0400 (EDT)
Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA07441
	for <ipoverib@ietf.org>; Mon, 22 Apr 2002 14:33:41 -0400 (EDT)
Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.us.ibm.com [9.37.3.210])
	by e21.nc.us.ibm.com (8.12.2/8.12.2) with ESMTP id g3MIUruE038150;
	Mon, 22 Apr 2002 14:30:53 -0400
Received: from gateway.beaverton.ibm.com (gateway.beaverton.ibm.com [138.95.180.1])
	by southrelay03.raleigh.ibm.com (8.11.1m3/NCO/VER6.1) with ESMTP id g3MIUnc40010;
	Mon, 22 Apr 2002 14:30:50 -0400
Received: from eng4.beaverton.ibm.com (eng4.beaverton.ibm.com [138.95.7.64])
	by gateway.beaverton.ibm.com (8.10.0.Beta10/8.11.6) with ESMTP id g3MIUkM22625;
	Mon, 22 Apr 2002 11:30:46 -0700 (PDT)
Received: (from viv@localhost)
	by eng4.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id g3MIUhb19150;
	Mon, 22 Apr 2002 11:30:43 -0700 (PDT)
From: Vivek Kashyap <kashyapv@us.ibm.com>
Message-Id: <200204221830.g3MIUhb19150@eng4.beaverton.ibm.com>
Subject: Re: [Ipoverib] IPoIB multicast
To: krause@cup.hp.com (Michael Krause)
Date: Mon, 22 Apr 2002 11:30:42 -0700 (PDT)
Cc: Jerry.Chu@eng.sun.com (H.K. Jerry Chu), ipoverib@ietf.org
In-Reply-To: <5.1.0.14.2.20020422062257.02afeeb0@esalpha2.cup.hp.com> from "Michael Krause" at Apr 22, 2002 05:34:18 AM PST
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 7bit

> 
> At 01:45 AM 4/22/2002 -0700, H.K. Jerry Chu wrote:
> >The proposal I made in the link-multicast draft when the IB MC
> >group corresponding to the target IP MC address doesn't already
> >exist, is to have the interface driver forward the pkt to the all-
> >router MC group. If the MC group comes into existence later
> >(i.e. someone joins as a member), the driver needs to be notified
> >so it can forward pkts back to the right MC group. This requires
> >some support from SA. I believe a proposal has been made recently
> >in IBTA to add this support to SM/SA.
> 
> To make it quite clear - one cannot send on IB without providing a MLID 
> that has been configured into switches.  It does not matter whether this is 
> for an all routers or a specific multicast group - this level of 
> configuration must be performed.  Hence, trying to target an all routers 
> multicast group provides zero benefit since the same work needs to be 
> performed for both types so why not do it right the first time.  This is 
> why I am strongly opposed to the use of such a technique and view this as 
> unnecessary complexity.
> 
> I've stated the simplest algorithm:
> 

This is pretty straightforward alright -

	IP level send => IB level (create?) join/send 

	IP level receive(join) => IB level (create?)/join/send/receive

        But it leaves some questiosn - when do we create ? when does a
        sender leave ?

> - Join the IB multicast group

        Only if it exists. We need to define a method for its
        creation. Based on the new IBTA mechanims we will have to
        decide whether anyone can create the IB group or we leave it
        to the receivers and routers.


> - Get the MLID configured
> - Send the packet
> - If any one is listening, they will receive it
> - If they are not listening at the time of the send, then they will get a 
> subsequent send but no attempt should be made to turn this into a reliable 
> service as suggested in some e-mails.

        They need to join first. Secondly, do we allow the group to
        stay around - we still need to suggest a timer (for some idle
        time) at which a sender should leave the group.


> - Group changes are tracked by the SA
> - The SA will inform a group when it is no longer available

        Inform whom ? Do we need to define a method that lets the SA
        know about the nodes to be informed or is it implicit in some
        way? We need to answer these questions in the IBA though.

> 
> The new IBTA method will be a form of async notification service that will 
> allow endnodes to detect when groups are added / deleted which is something 
> a couple of us have proposed for quite some time and is now finding 
> traction.  If the workgroup focuses on the join process and recognizes that 
> a notification method will exist (which is entirely an IB issue with an 
> implementation's LLP masking any of this complexity - this is clearly not 
> an IP over IB issue), then we can complete this set of specifications.
> 
> >2. Ethernet NICs can be programmed into promiscuous multicast
> >mode when the NIC will send a copy of every MC pkt to IP.
> >IGMP/MLD uses this feature conveniently to pick up a copy of
> >every MC pkt to track MC membership, and to route MC pkts
> >correctly to other subnets.
> >
> >Since IB doesn't support a promiscuous multicast mode. The
> >workaround I proposed is to have the interface driver make a copy
> >of every MC pkt and send it to the all-router MC group. This is
> >pretty straightfoward with a small performance penalty. But is
> >still much more efficient than resorting to broadcast. There is
> >a small problem in case the router itself also joins a MC group
> >and will then receive two copies. This case can be adressed with
> >some more check in the interface driver of a router but since
> >the case is rare, I'm not sure it's worth the effort and additional
> >complexity.
> 
> IB can be configured to see all multicast packets within a given partition 
> - this requires one to join all multicast groups.  This is a policy 
> decision.  The fact that it is different than Ethernet is irrelevant quite 
> frankly.  This is an IP over IB not IP over Ethernet workgroup.  What you 
> are describing is in many cases really no different than the class of 
> problems associated with LANE.

        I agree with Mike here - this is IPoIB and irrelevant to IPoE.
        We need to ensure that the IP interface doesn't have to change
        but don't have to ensure that the IB driver looks the same as
        ethernet driver.

> 
> >Other alternative includes Vivek's first proposal to snoop IGMP/MLD
> >pkts in the interface driver. This requires driver to understand
> >IGMP/MLD, and will make it very difficult to maintain (with newer
> >version of IGMPv1,2,3.../MLD).
> >
> >Vivek's recent proposal fixed the above problem by moving the work
> >from the driver to IGMP/MLD themselves. Unfortunately IMO the
> >chance of requesting link specific changes in a link-independent
> >module like IGMP is not very good.
> 
> There is no reason to add this complexity at all which is why I continue to 
> oppose this line of logic.

        Only if the new IBTA changes are brought in. Without any IBTA
        changes there is no alternative other than use broadcast
        always or the dual-transmit of multicast packets suggestion.

> 
> >Perhaps if IBTA can supply us an equivalent of a promiscuous
> >multicast mode without bending IBA backwards too much that would
> >be ideal.  I've heard some discussion along that line is going on.
> >Let's see what comes out of it.
> 
> One has an promiscuous solution by employing the notification method and 
> performing a join operation for all multicast groups associated with a 
> particular partition.
> 
> Mike
> 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 


-- 
Vivek Kashyap
Linux Technology Center, IBM
kashyapv@us.ibm.com 
vivk@us.ibm.com 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Mon Apr 22 14:43:51 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA07889
	for <ipoverib-archive@odin.ietf.org>; Mon, 22 Apr 2002 14:43:51 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA08833
	for ipoverib-archive@odin.ietf.org; Mon, 22 Apr 2002 14:43:54 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA08539;
	Mon, 22 Apr 2002 14:40:20 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA08509
	for <ipoverib@ns.ietf.org>; Mon, 22 Apr 2002 14:40:18 -0400 (EDT)
Received: from kathmandu.sun.com (kathmandu.sun.com [192.18.98.36])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA07764
	for <ipoverib@ietf.org>; Mon, 22 Apr 2002 14:40:09 -0400 (EDT)
Received: from sunmail1.Sun.COM ([129.145.1.2])
	by kathmandu.sun.com (8.9.3+Sun/8.9.3) with ESMTP id MAA22495;
	Mon, 22 Apr 2002 12:40:00 -0600 (MDT)
Received: from jurassic.eng.sun.com (jurassic.Eng.Sun.COM [129.146.85.31])
	by sunmail1.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1-Sun.COM.mod.2) with ESMTP id LAA05745;
	Mon, 22 Apr 2002 11:40:09 -0700 (PDT)
Received: from taipei (taipei.Eng.Sun.COM [129.146.85.178])
	by jurassic.eng.sun.com (8.12.2+Sun/8.12.2) with SMTP id g3MIdwJe023157;
	Mon, 22 Apr 2002 11:39:58 -0700 (PDT)
Message-Id: <200204221839.g3MIdwJe023157@jurassic.eng.sun.com>
Date: Mon, 22 Apr 2002 11:39:32 -0700 (PDT)
From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Reply-To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Subject: RE: [Ipoverib] IPoIB multicast
To: krause@cup.hp.com
Cc: ipoverib@ietf.org
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: M/HAalaWB9Y2IAxxqYeYOw==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc 
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

>At 01:45 AM 4/22/2002 -0700, H.K. Jerry Chu wrote:
>>The proposal I made in the link-multicast draft when the IB MC
>>group corresponding to the target IP MC address doesn't already
>>exist, is to have the interface driver forward the pkt to the all-
>>router MC group. If the MC group comes into existence later
>>(i.e. someone joins as a member), the driver needs to be notified
>>so it can forward pkts back to the right MC group. This requires
>>some support from SA. I believe a proposal has been made recently
>>in IBTA to add this support to SM/SA.
>
>To make it quite clear - one cannot send on IB without providing a MLID 
>that has been configured into switches.  It does not matter whether this is
>for an all routers or a specific multicast group - this level of 
>configuration must be performed.

I'm confused. What part of my proposal implies sending to a non-existing
MGID/MLID? The all-router MC group is required by IP and thus will
always exist (with a valid MLID).

>Hence, trying to target an all routers 
>multicast group provides zero benefit since the same work needs to be 
>performed for both types so why not do it right the first time.

I think you're assuming the only way a MLID will get configured correctly
for a sender to use is to require the sender to also be a member. In that
case yes the above algorithm degenerates itself into broadcast if every
potential MC sender must also become a member of the all-router MC group.
It defeats the purpose of the latter.

But I have also heard different opinions regarding the join-to-send
issue. My original proposal was based on the current 1.0 spec. that
doesn't require a MC sender to join before send.

Has IBTA come to a consensus on the join-to-send issue yet?

>This is 
>why I am strongly opposed to the use of such a technique and view this as 
>unnecessary complexity.
>
>I've stated the simplest algorithm:
>
>- Join the IB multicast group
>- Get the MLID configured
>- Send the packet
>- If any one is listening, they will receive it
>- If they are not listening at the time of the send, then they will get a 
>subsequent send but no attempt should be made to turn this into a reliable 
>service as suggested in some e-mails.

Agreed, as long as whatever race condition there may be doesn't turn
into a persistent one.

>- Group changes are tracked by the SA
>- The SA will inform a group when it is no longer available
>
>The new IBTA method will be a form of async notification service that will 
>allow endnodes to detect when groups are added / deleted which is something 
>a couple of us have proposed for quite some time and is now finding 
>traction.  If the workgroup focuses on the join process and recognizes that 
>a notification method will exist (which is entirely an IB issue with an 
>implementation's LLP masking any of this complexity - this is clearly not 
>an IP over IB issue), then we can complete this set of specifications.

If IBA can successfully hide the complexity of IB MC group management
from the IP stack so that to the latter, every link multicast group
exist and can be readily used (sorta like Ethernet), then I agree we don't
have to muck with IP stack and IPoIB spec. But this is a big IF. I don't
believe this is the case in the 1.0a of IBA spec. We need a clear
statement from IBTA on the direction it will take.

>
>>2. Ethernet NICs can be programmed into promiscuous multicast
>>mode when the NIC will send a copy of every MC pkt to IP.
>>IGMP/MLD uses this feature conveniently to pick up a copy of
>>every MC pkt to track MC membership, and to route MC pkts
>>correctly to other subnets.
>>
>>Since IB doesn't support a promiscuous multicast mode. The
>>workaround I proposed is to have the interface driver make a copy
>>of every MC pkt and send it to the all-router MC group. This is
>>pretty straightfoward with a small performance penalty. But is
>>still much more efficient than resorting to broadcast. There is
>>a small problem in case the router itself also joins a MC group
>>and will then receive two copies. This case can be adressed with
>>some more check in the interface driver of a router but since
>>the case is rare, I'm not sure it's worth the effort and additional
>>complexity.
>
>IB can be configured to see all multicast packets within a given partition 
>- this requires one to join all multicast groups.  This is a policy 
>decision.  The fact that it is different than Ethernet is irrelevant quite 
>frankly.  This is an IP over IB not IP over Ethernet workgroup.

If one can use the existing IBA spec. to emulate promisucous MC mode
from any CA attachment point, then we've been wasting time solving
a non-existing problem. I don't think that has been the case.

Like I said in my previous mail, like it or not, much of the original
IP design was done on Ethernet so was influenced by it. Both IGMP and
MLD are supposed to be link agnostic. But both assume a promisuous MC
mode, which exists on Ethernet but not on many other link types. This
is just reality.

>What you 
>are describing is in many cases really no different than the class of 
>problems associated with LANE.
>
>>Other alternative includes Vivek's first proposal to snoop IGMP/MLD
>>pkts in the interface driver. This requires driver to understand
>>IGMP/MLD, and will make it very difficult to maintain (with newer
>>version of IGMPv1,2,3.../MLD).
>>
>>Vivek's recent proposal fixed the above problem by moving the work
>>from the driver to IGMP/MLD themselves. Unfortunately IMO the
>>chance of requesting link specific changes in a link-independent
>>module like IGMP is not very good.
>
>There is no reason to add this complexity at all which is why I continue to 
>oppose this line of logic.

Agreed.

>
>>Perhaps if IBTA can supply us an equivalent of a promiscuous
>>multicast mode without bending IBA backwards too much that would
>>be ideal.  I've heard some discussion along that line is going on.
>>Let's see what comes out of it.
>
>One has an promiscuous solution by employing the notification method and 
>performing a join operation for all multicast groups associated with a 
>particular partition.

Good. I'll be looking forward to a concrete solution by IBTA then.

Jerry

>
>Mike
>


_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr 25 12:43:36 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA26890
	for <ipoverib-archive@odin.ietf.org>; Thu, 25 Apr 2002 12:43:36 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id MAA21477
	for ipoverib-archive@odin.ietf.org; Thu, 25 Apr 2002 12:43:38 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA21360;
	Thu, 25 Apr 2002 12:41:41 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA21328
	for <ipoverib@optimus.ietf.org>; Thu, 25 Apr 2002 12:41:39 -0400 (EDT)
Received: from mail2.microsoft.com (mail2.microsoft.com [131.107.3.124])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA26828
	for <ipoverib@ietf.org>; Thu, 25 Apr 2002 12:41:36 -0400 (EDT)
Received: from INET-VRS-02.redmond.corp.microsoft.com ([157.54.8.110]) by mail2.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Thu, 25 Apr 2002 09:41:07 -0700
Received: from 157.54.6.197 by INET-VRS-02.redmond.corp.microsoft.com (InterScan E-Mail VirusWall NT); Thu, 25 Apr 2002 09:41:07 -0700
Received: from red-imc-04.redmond.corp.microsoft.com ([157.54.2.168]) by inet-hub-06.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Thu, 25 Apr 2002 09:41:06 -0700
Received: from win-imc-02.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.84]) by red-imc-04.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.2966);
	 Thu, 25 Apr 2002 09:41:06 -0700
Received: from win-msg-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.52]) by win-imc-02.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(6.0.3590.0);
	 Thu, 25 Apr 2002 09:37:24 -0700
X-MimeOLE: Produced By Microsoft Exchange V6.0.6177.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Subject: RE: [Ipoverib] MIB's ifPhysAddress needs to be updated
Date: Thu, 25 Apr 2002 09:37:24 -0700
Message-ID: <2E33960095B58E40A4D3345AB9F65EC102450E5E@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Thread-Topic: [Ipoverib] MIB's ifPhysAddress needs to be updated
Thread-Index: AcHmPzNqIX37cFIyEdawQQBQi2jYqAAAFyngAY189OA=
From: "Aditya Dube" <adube@windows.microsoft.com>
To: <ipoverib@ietf.org>
X-OriginalArrivalTime: 25 Apr 2002 16:37:24.0764 (UTC) FILETIME=[7ABE4DC0:01C1EC77]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id MAA21329
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 8bit

IMO, the IpOverIB's notion of a physical link layer address and what the
MIB reports as the MAC address should be the same. 

If the MIB draft is to be amended to comply with IPoverIB, what is the
procedure for it to happen?

Aditya Dube

-----Original Message-----
From: Larsen, Roy K [mailto:roy.k.larsen@intel.com] 
Sent: Wednesday, April 17, 2002 11:45 AM
To: 'Bill Strahm'; Aditya Dube
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] MIB's ifPhysAddress needs to be updated

But it's being define in the IPoIB working group.  Why isn't it
legitimate to say it should support IPoIB hardware address semantics?

Roy

-----Original Message-----
From: Bill Strahm [mailto:bill@strahm.net]
Sent: Wednesday, April 17, 2002 11:34 AM
To: Aditya Dube
Cc: ipoverib@ietf.org
Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated


The problem is that ifPhysAddr covers more than just IP, it is just
the layer 2 wire representation of address.  If you can show me an
IB format that includes GID+LID in a packet header, I might conceed
defeat here.

The address duplicating across multiple subnets is a red herring.  You
can assign identical Ethernet MAC addresses, so long as they are on
differing subnets as well.  

From RFC 2863...
            "The interface's address at its protocol sub-layer.  For
            example, for an 802.x interface, this object normally
            contains a MAC address.  The interface's media-specific MIB
            must define the bit and byte ordering and the format of the
            value of this object.  For interfaces which do not have such
            an address (e.g., a serial line), this object should contain
            an octet string of zero length."
Note that there is no mention of any Layer 3 semantics.  

We can bring the topic up for discussion, I don't know which of the MIB
authors are currently active right now though.

Bill

On Wed, Apr 17, 2002 at 11:19:24AM -0700, Aditya Dube wrote:
> 
> As the Lid is an assigned address rather than a burned in address, as
in
> Ethernet, the chances of duplicate ifphysaddr's in a network spanning
> multiple IB subnets are a lot higher. We should reduce this chance as
> much as possible.
> 
> The ifphysaddr is also not in sync with what IpOverIB has decided to
use
> as the link address in ARPs.
> 
> IMHO the two, ifphysaddr and the link address is an arp packet, should
> match.
> 
> Aditya Dube
> 
> -----Original Message-----
> From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
> Sent: Tuesday, April 16, 2002 12:18 PM
> To: Aditya Dube
> Cc: ipoverib@ietf.org
> Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> 
> The same question was addressed last october/november.
> See the message:
> 
>
http://www1.ietf.org/mail-archive/working-groups/ipoverib/current/msg000
> 87.html
> 
> Vivek
> 
> 
> 
> > 
> > This is a multi-part message in MIME format.
> > 
> > --------------InterScan_NT_MIME_Boundary
> > Content-Type: multipart/alternative;
> > 	boundary="----_=_NextPart_001_01C1E577.F19F4F03"
> > 
> > ------_=_NextPart_001_01C1E577.F19F4F03
> > Content-Type: text/plain;
> > 	charset="us-ascii"
> > Content-Transfer-Encoding: quoted-printable
> > 
> > 
> > In the MIB draft, the ifPhysAddress is specified as the LID. This is
> out
> > of sync with the IpOverIb draft. Can the ifPhysAddress in the MIB be
> > changed to match  IPoverIB's physical address, i.e. the GID + QPN?
> > 
> > http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.txt
> > 
> > Aditya Dube
> > 
> > 
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Thu Apr 25 12:50:56 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27175
	for <ipoverib-archive@odin.ietf.org>; Thu, 25 Apr 2002 12:50:51 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id MAA21900
	for ipoverib-archive@odin.ietf.org; Thu, 25 Apr 2002 12:50:53 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA21798;
	Thu, 25 Apr 2002 12:50:20 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA21762
	for <ipoverib@optimus.ietf.org>; Thu, 25 Apr 2002 12:50:17 -0400 (EDT)
Received: from strahm.net (adsl-67-115-29-109.dsl.sndg02.pacbell.net [67.115.29.109])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27142
	for <ipoverib@ietf.org>; Thu, 25 Apr 2002 12:50:09 -0400 (EDT)
Received: (from bill@localhost)
	by strahm.net (8.11.6/8.11.6) id g3PGxrb55423;
	Thu, 25 Apr 2002 09:59:53 -0700 (PDT)
	(envelope-from bill)
Date: Thu, 25 Apr 2002 09:59:52 -0700
From: Bill Strahm <bill@strahm.net>
To: Aditya Dube <adube@windows.microsoft.com>
Cc: ipoverib@ietf.org
Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
Message-ID: <20020425095952.A55407@homegate.strahm.net>
References: <2E33960095B58E40A4D3345AB9F65EC102450E5E@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <2E33960095B58E40A4D3345AB9F65EC102450E5E@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>; from adube@windows.microsoft.com on Thu, Apr 25, 2002 at 09:37:24AM -0700
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

You are making a subtle mistake... The MIB is an Infiniband layer 2 MIB, not 
an IP over IB Layer 2 MIB...

The procedure is the same for any draft in the group, get group concensus
on the change, and then the author will update the draft accordingly...

Bill
On Thu, Apr 25, 2002 at 09:37:24AM -0700, Aditya Dube wrote:
> IMO, the IpOverIB's notion of a physical link layer address and what the
> MIB reports as the MAC address should be the same. 
> 
> If the MIB draft is to be amended to comply with IPoverIB, what is the
> procedure for it to happen?
> 
> Aditya Dube
> 
> -----Original Message-----
> From: Larsen, Roy K [mailto:roy.k.larsen@intel.com] 
> Sent: Wednesday, April 17, 2002 11:45 AM
> To: 'Bill Strahm'; Aditya Dube
> Cc: ipoverib@ietf.org
> Subject: RE: [Ipoverib] MIB's ifPhysAddress needs to be updated
> 
> But it's being define in the IPoIB working group.  Why isn't it
> legitimate to say it should support IPoIB hardware address semantics?
> 
> Roy
> 
> -----Original Message-----
> From: Bill Strahm [mailto:bill@strahm.net]
> Sent: Wednesday, April 17, 2002 11:34 AM
> To: Aditya Dube
> Cc: ipoverib@ietf.org
> Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> 
> 
> The problem is that ifPhysAddr covers more than just IP, it is just
> the layer 2 wire representation of address.  If you can show me an
> IB format that includes GID+LID in a packet header, I might conceed
> defeat here.
> 
> The address duplicating across multiple subnets is a red herring.  You
> can assign identical Ethernet MAC addresses, so long as they are on
> differing subnets as well.  
> 
> >From RFC 2863...
>             "The interface's address at its protocol sub-layer.  For
>             example, for an 802.x interface, this object normally
>             contains a MAC address.  The interface's media-specific MIB
>             must define the bit and byte ordering and the format of the
>             value of this object.  For interfaces which do not have such
>             an address (e.g., a serial line), this object should contain
>             an octet string of zero length."
> Note that there is no mention of any Layer 3 semantics.  
> 
> We can bring the topic up for discussion, I don't know which of the MIB
> authors are currently active right now though.
> 
> Bill
> 
> On Wed, Apr 17, 2002 at 11:19:24AM -0700, Aditya Dube wrote:
> > 
> > As the Lid is an assigned address rather than a burned in address, as
> in
> > Ethernet, the chances of duplicate ifphysaddr's in a network spanning
> > multiple IB subnets are a lot higher. We should reduce this chance as
> > much as possible.
> > 
> > The ifphysaddr is also not in sync with what IpOverIB has decided to
> use
> > as the link address in ARPs.
> > 
> > IMHO the two, ifphysaddr and the link address is an arp packet, should
> > match.
> > 
> > Aditya Dube
> > 
> > -----Original Message-----
> > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
> > Sent: Tuesday, April 16, 2002 12:18 PM
> > To: Aditya Dube
> > Cc: ipoverib@ietf.org
> > Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> > 
> > The same question was addressed last october/november.
> > See the message:
> > 
> >
> http://www1.ietf.org/mail-archive/working-groups/ipoverib/current/msg000
> > 87.html
> > 
> > Vivek
> > 
> > 
> > 
> > > 
> > > This is a multi-part message in MIME format.
> > > 
> > > --------------InterScan_NT_MIME_Boundary
> > > Content-Type: multipart/alternative;
> > > 	boundary="----_=_NextPart_001_01C1E577.F19F4F03"
> > > 
> > > ------_=_NextPart_001_01C1E577.F19F4F03
> > > Content-Type: text/plain;
> > > 	charset="us-ascii"
> > > Content-Transfer-Encoding: quoted-printable
> > > 
> > > 
> > > In the MIB draft, the ifPhysAddress is specified as the LID. This is
> > out
> > > of sync with the IpOverIb draft. Can the ifPhysAddress in the MIB be
> > > changed to match  IPoverIB's physical address, i.e. the GID + QPN?
> > > 
> > > http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.txt
> > > 
> > > Aditya Dube
> > > 
> > > 
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 26 17:44:53 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA16209
	for <ipoverib-archive@odin.ietf.org>; Fri, 26 Apr 2002 17:44:53 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id RAA26308
	for ipoverib-archive@odin.ietf.org; Fri, 26 Apr 2002 17:44:57 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA26228;
	Fri, 26 Apr 2002 17:43:42 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA26196
	for <ipoverib@ns.ietf.org>; Fri, 26 Apr 2002 17:43:35 -0400 (EDT)
Received: from mail2.microsoft.com (mail2.microsoft.com [131.107.3.124])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA16061
	for <ipoverib@ietf.org>; Fri, 26 Apr 2002 17:43:30 -0400 (EDT)
Received: from INET-VRS-02.redmond.corp.microsoft.com ([157.54.8.110]) by mail2.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Fri, 26 Apr 2002 14:43:03 -0700
Received: from 157.54.6.150 by INET-VRS-02.redmond.corp.microsoft.com (InterScan E-Mail VirusWall NT); Fri, 26 Apr 2002 14:43:03 -0700
Received: from red-imc-04.redmond.corp.microsoft.com ([157.54.2.168]) by inet-hub-05.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.4905);
	 Fri, 26 Apr 2002 14:43:02 -0700
Received: from win-imc-02.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.84]) by red-imc-04.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.2966);
	 Fri, 26 Apr 2002 14:43:02 -0700
Received: from win-msg-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.52]) by win-imc-02.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(6.0.3590.0);
	 Fri, 26 Apr 2002 14:39:19 -0700
X-MimeOLE: Produced By Microsoft Exchange V6.0.6177.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Subject: RE: [Ipoverib] MIB's ifPhysAddress needs to be updated
Date: Fri, 26 Apr 2002 14:39:18 -0700
Message-ID: <2E33960095B58E40A4D3345AB9F65EC106D1273F@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Thread-Topic: [Ipoverib] MIB's ifPhysAddress needs to be updated
Thread-Index: AcHseL0YgW73M4OXSQaNYdxu6xhNwgA5qUjA
From: "Aditya Dube" <adube@windows.microsoft.com>
To: "Bill Strahm" <bill@strahm.net>
Cc: <ipoverib@ietf.org>
X-OriginalArrivalTime: 26 Apr 2002 21:39:19.0102 (UTC) FILETIME=[D224ADE0:01C1ED6A]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id RAA26197
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 8bit

Bill

The MIB document predates the decision to use the GID + QPN as the
physical address.

In the current scheme, the entity reported as the physical address by
the encapsulation layer is inconsistent with what one gets using the
MIB.

Aditya Dube

-----Original Message-----
From: Bill Strahm [mailto:bill@strahm.net] 
Sent: Thursday, April 25, 2002 10:00 AM
To: Aditya Dube
Cc: ipoverib@ietf.org
Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated

You are making a subtle mistake... The MIB is an Infiniband layer 2 MIB,
not 
an IP over IB Layer 2 MIB...

The procedure is the same for any draft in the group, get group
concensus
on the change, and then the author will update the draft accordingly...

Bill
On Thu, Apr 25, 2002 at 09:37:24AM -0700, Aditya Dube wrote:
> IMO, the IpOverIB's notion of a physical link layer address and what
the
> MIB reports as the MAC address should be the same. 
> 
> If the MIB draft is to be amended to comply with IPoverIB, what is the
> procedure for it to happen?
> 
> Aditya Dube
> 
> -----Original Message-----
> From: Larsen, Roy K [mailto:roy.k.larsen@intel.com] 
> Sent: Wednesday, April 17, 2002 11:45 AM
> To: 'Bill Strahm'; Aditya Dube
> Cc: ipoverib@ietf.org
> Subject: RE: [Ipoverib] MIB's ifPhysAddress needs to be updated
> 
> But it's being define in the IPoIB working group.  Why isn't it
> legitimate to say it should support IPoIB hardware address semantics?
> 
> Roy
> 
> -----Original Message-----
> From: Bill Strahm [mailto:bill@strahm.net]
> Sent: Wednesday, April 17, 2002 11:34 AM
> To: Aditya Dube
> Cc: ipoverib@ietf.org
> Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> 
> 
> The problem is that ifPhysAddr covers more than just IP, it is just
> the layer 2 wire representation of address.  If you can show me an
> IB format that includes GID+LID in a packet header, I might conceed
> defeat here.
> 
> The address duplicating across multiple subnets is a red herring.  You
> can assign identical Ethernet MAC addresses, so long as they are on
> differing subnets as well.  
> 
> >From RFC 2863...
>             "The interface's address at its protocol sub-layer.  For
>             example, for an 802.x interface, this object normally
>             contains a MAC address.  The interface's media-specific
MIB
>             must define the bit and byte ordering and the format of
the
>             value of this object.  For interfaces which do not have
such
>             an address (e.g., a serial line), this object should
contain
>             an octet string of zero length."
> Note that there is no mention of any Layer 3 semantics.  
> 
> We can bring the topic up for discussion, I don't know which of the
MIB
> authors are currently active right now though.
> 
> Bill
> 
> On Wed, Apr 17, 2002 at 11:19:24AM -0700, Aditya Dube wrote:
> > 
> > As the Lid is an assigned address rather than a burned in address,
as
> in
> > Ethernet, the chances of duplicate ifphysaddr's in a network
spanning
> > multiple IB subnets are a lot higher. We should reduce this chance
as
> > much as possible.
> > 
> > The ifphysaddr is also not in sync with what IpOverIB has decided to
> use
> > as the link address in ARPs.
> > 
> > IMHO the two, ifphysaddr and the link address is an arp packet,
should
> > match.
> > 
> > Aditya Dube
> > 
> > -----Original Message-----
> > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
> > Sent: Tuesday, April 16, 2002 12:18 PM
> > To: Aditya Dube
> > Cc: ipoverib@ietf.org
> > Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> > 
> > The same question was addressed last october/november.
> > See the message:
> > 
> >
>
http://www1.ietf.org/mail-archive/working-groups/ipoverib/current/msg000
> > 87.html
> > 
> > Vivek
> > 
> > 
> > 
> > > 
> > > This is a multi-part message in MIME format.
> > > 
> > > --------------InterScan_NT_MIME_Boundary
> > > Content-Type: multipart/alternative;
> > > 	boundary="----_=_NextPart_001_01C1E577.F19F4F03"
> > > 
> > > ------_=_NextPart_001_01C1E577.F19F4F03
> > > Content-Type: text/plain;
> > > 	charset="us-ascii"
> > > Content-Transfer-Encoding: quoted-printable
> > > 
> > > 
> > > In the MIB draft, the ifPhysAddress is specified as the LID. This
is
> > out
> > > of sync with the IpOverIb draft. Can the ifPhysAddress in the MIB
be
> > > changed to match  IPoverIB's physical address, i.e. the GID + QPN?
> > > 
> > >
http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.txt
> > > 
> > > Aditya Dube
> > > 
> > > 
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> _______________________________________________
> IPoverIB mailing list
> IPoverIB@ietf.org
> https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 26 19:05:04 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA25110
	for <ipoverib-archive@odin.ietf.org>; Fri, 26 Apr 2002 19:05:04 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA00624
	for ipoverib-archive@odin.ietf.org; Fri, 26 Apr 2002 19:05:06 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA00543;
	Fri, 26 Apr 2002 19:02:59 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA00507
	for <ipoverib@ns.ietf.org>; Fri, 26 Apr 2002 19:02:57 -0400 (EDT)
Received: from strahm.net (adsl-67-115-29-109.dsl.sndg02.pacbell.net [67.115.29.109])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA24931
	for <ipoverib@ietf.org>; Fri, 26 Apr 2002 19:02:53 -0400 (EDT)
Received: (from bill@localhost)
	by strahm.net (8.11.6/8.11.6) id g3QNE3T58527;
	Fri, 26 Apr 2002 16:14:03 -0700 (PDT)
	(envelope-from bill)
Date: Fri, 26 Apr 2002 16:14:00 -0700
From: Bill Strahm <bill@strahm.net>
To: Aditya Dube <adube@windows.microsoft.com>
Cc: Bill Strahm <bill@strahm.net>, ipoverib@ietf.org
Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
Message-ID: <20020426161400.A58479@homegate.strahm.net>
References: <2E33960095B58E40A4D3345AB9F65EC106D1273F@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <2E33960095B58E40A4D3345AB9F65EC106D1273F@win-msg-01.wingroup.windeploy.ntdev.microsoft.com>; from adube@windows.microsoft.com on Fri, Apr 26, 2002 at 02:39:18PM -0700
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Again, an interesting problem,

What would you consider ifInOctets to be if the Physical address is GID + QPN.

Now think about that answer in an environment that is both IP and IB native,
would each QPN now be an interface in the ifTable ???

The if MIB is designed to handle the Layer 2 properties of the interface 
without regard to what is running on top of it.  Rather than simple Ethernet
where IP runs directly on top of Ethernets Layer 2 interface, in Infiniband
we have IP running on top of Infinibans Layer 3 UD interface.  I hate to tell
you but I will fight adding QPN to the ifPhysAddress tooth and nail becasue
it does not belong there (in my humble non-chair role)  If you can explain how
to get other layer 2 protocols on top of Infiniband without incurring 64K
interfaces, I will listen.

Bill
On Fri, Apr 26, 2002 at 02:39:18PM -0700, Aditya Dube wrote:
> Bill
> 
> The MIB document predates the decision to use the GID + QPN as the
> physical address.
> 
> In the current scheme, the entity reported as the physical address by
> the encapsulation layer is inconsistent with what one gets using the
> MIB.
> 
> Aditya Dube
> 
> -----Original Message-----
> From: Bill Strahm [mailto:bill@strahm.net] 
> Sent: Thursday, April 25, 2002 10:00 AM
> To: Aditya Dube
> Cc: ipoverib@ietf.org
> Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> 
> You are making a subtle mistake... The MIB is an Infiniband layer 2 MIB,
> not 
> an IP over IB Layer 2 MIB...
> 
> The procedure is the same for any draft in the group, get group
> concensus
> on the change, and then the author will update the draft accordingly...
> 
> Bill
> On Thu, Apr 25, 2002 at 09:37:24AM -0700, Aditya Dube wrote:
> > IMO, the IpOverIB's notion of a physical link layer address and what
> the
> > MIB reports as the MAC address should be the same. 
> > 
> > If the MIB draft is to be amended to comply with IPoverIB, what is the
> > procedure for it to happen?
> > 
> > Aditya Dube
> > 
> > -----Original Message-----
> > From: Larsen, Roy K [mailto:roy.k.larsen@intel.com] 
> > Sent: Wednesday, April 17, 2002 11:45 AM
> > To: 'Bill Strahm'; Aditya Dube
> > Cc: ipoverib@ietf.org
> > Subject: RE: [Ipoverib] MIB's ifPhysAddress needs to be updated
> > 
> > But it's being define in the IPoIB working group.  Why isn't it
> > legitimate to say it should support IPoIB hardware address semantics?
> > 
> > Roy
> > 
> > -----Original Message-----
> > From: Bill Strahm [mailto:bill@strahm.net]
> > Sent: Wednesday, April 17, 2002 11:34 AM
> > To: Aditya Dube
> > Cc: ipoverib@ietf.org
> > Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> > 
> > 
> > The problem is that ifPhysAddr covers more than just IP, it is just
> > the layer 2 wire representation of address.  If you can show me an
> > IB format that includes GID+LID in a packet header, I might conceed
> > defeat here.
> > 
> > The address duplicating across multiple subnets is a red herring.  You
> > can assign identical Ethernet MAC addresses, so long as they are on
> > differing subnets as well.  
> > 
> > >From RFC 2863...
> >             "The interface's address at its protocol sub-layer.  For
> >             example, for an 802.x interface, this object normally
> >             contains a MAC address.  The interface's media-specific
> MIB
> >             must define the bit and byte ordering and the format of
> the
> >             value of this object.  For interfaces which do not have
> such
> >             an address (e.g., a serial line), this object should
> contain
> >             an octet string of zero length."
> > Note that there is no mention of any Layer 3 semantics.  
> > 
> > We can bring the topic up for discussion, I don't know which of the
> MIB
> > authors are currently active right now though.
> > 
> > Bill
> > 
> > On Wed, Apr 17, 2002 at 11:19:24AM -0700, Aditya Dube wrote:
> > > 
> > > As the Lid is an assigned address rather than a burned in address,
> as
> > in
> > > Ethernet, the chances of duplicate ifphysaddr's in a network
> spanning
> > > multiple IB subnets are a lot higher. We should reduce this chance
> as
> > > much as possible.
> > > 
> > > The ifphysaddr is also not in sync with what IpOverIB has decided to
> > use
> > > as the link address in ARPs.
> > > 
> > > IMHO the two, ifphysaddr and the link address is an arp packet,
> should
> > > match.
> > > 
> > > Aditya Dube
> > > 
> > > -----Original Message-----
> > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
> > > Sent: Tuesday, April 16, 2002 12:18 PM
> > > To: Aditya Dube
> > > Cc: ipoverib@ietf.org
> > > Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> > > 
> > > The same question was addressed last october/november.
> > > See the message:
> > > 
> > >
> >
> http://www1.ietf.org/mail-archive/working-groups/ipoverib/current/msg000
> > > 87.html
> > > 
> > > Vivek
> > > 
> > > 
> > > 
> > > > 
> > > > This is a multi-part message in MIME format.
> > > > 
> > > > --------------InterScan_NT_MIME_Boundary
> > > > Content-Type: multipart/alternative;
> > > > 	boundary="----_=_NextPart_001_01C1E577.F19F4F03"
> > > > 
> > > > ------_=_NextPart_001_01C1E577.F19F4F03
> > > > Content-Type: text/plain;
> > > > 	charset="us-ascii"
> > > > Content-Transfer-Encoding: quoted-printable
> > > > 
> > > > 
> > > > In the MIB draft, the ifPhysAddress is specified as the LID. This
> is
> > > out
> > > > of sync with the IpOverIb draft. Can the ifPhysAddress in the MIB
> be
> > > > changed to match  IPoverIB's physical address, i.e. the GID + QPN?
> > > > 
> > > >
> http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.txt
> > > > 
> > > > Aditya Dube
> > > > 
> > > > 
> > > 
> > > _______________________________________________
> > > IPoverIB mailing list
> > > IPoverIB@ietf.org
> > > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@ns.ietf.org  Fri Apr 26 21:05:59 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA07225
	for <ipoverib-archive@odin.ietf.org>; Fri, 26 Apr 2002 21:05:59 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id VAA05112
	for ipoverib-archive@odin.ietf.org; Fri, 26 Apr 2002 21:06:02 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id VAA05099;
	Fri, 26 Apr 2002 21:05:34 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id VAA05066
	for <ipoverib@ns.ietf.org>; Fri, 26 Apr 2002 21:05:32 -0400 (EDT)
Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA07175
	for <ipoverib@ietf.org>; Fri, 26 Apr 2002 21:05:28 -0400 (EDT)
Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37])
	by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.42 2002/04/26 23:25:13 root Exp $) with ESMTP id g3R14du24975
	for <ipoverib@ietf.org>; Sat, 27 Apr 2002 01:04:40 GMT
Received: from fmsmsxvs040.fm.intel.com (fmsmsxv040-1.fm.intel.com [132.233.48.108])
	by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.17 2002/04/27 00:24:04 root Exp $) with SMTP id g3R13rR13657
	for <ipoverib@ietf.org>; Sat, 27 Apr 2002 01:03:53 GMT
Received: from fmsmsx28.fm.intel.com ([132.233.42.28])
 by fmsmsxvs040.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002042618045329485
 ; Fri, 26 Apr 2002 18:04:53 -0700
Received: by fmsmsx28.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <2KDW25H7>; Fri, 26 Apr 2002 18:05:30 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C06BC4FBD@ORSMSX108>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Bill Strahm'" <bill@strahm.net>,
        Aditya Dube <adube@windows.microsoft.com>
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] MIB's ifPhysAddress needs to be updated
Date: Fri, 26 Apr 2002 18:05:28 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org

Bill,

If the MIB is IPoIB agnostic, what is it doing in the IPoIB working group?
If it doesn't need to jive with the information returned from things like
ifconfig(8), arp(8), and netstat(8), why does this group care about this MIB
at all?

Roy

-----Original Message-----
From: Bill Strahm [mailto:bill@strahm.net]
Sent: Friday, April 26, 2002 4:14 PM
To: Aditya Dube
Cc: Bill Strahm; ipoverib@ietf.org
Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated


Again, an interesting problem,

What would you consider ifInOctets to be if the Physical address is GID +
QPN.

Now think about that answer in an environment that is both IP and IB native,
would each QPN now be an interface in the ifTable ???

The if MIB is designed to handle the Layer 2 properties of the interface 
without regard to what is running on top of it.  Rather than simple Ethernet
where IP runs directly on top of Ethernets Layer 2 interface, in Infiniband
we have IP running on top of Infinibans Layer 3 UD interface.  I hate to
tell
you but I will fight adding QPN to the ifPhysAddress tooth and nail becasue
it does not belong there (in my humble non-chair role)  If you can explain
how
to get other layer 2 protocols on top of Infiniband without incurring 64K
interfaces, I will listen.

Bill
On Fri, Apr 26, 2002 at 02:39:18PM -0700, Aditya Dube wrote:
> Bill
> 
> The MIB document predates the decision to use the GID + QPN as the
> physical address.
> 
> In the current scheme, the entity reported as the physical address by
> the encapsulation layer is inconsistent with what one gets using the
> MIB.
> 
> Aditya Dube
> 
> -----Original Message-----
> From: Bill Strahm [mailto:bill@strahm.net] 
> Sent: Thursday, April 25, 2002 10:00 AM
> To: Aditya Dube
> Cc: ipoverib@ietf.org
> Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> 
> You are making a subtle mistake... The MIB is an Infiniband layer 2 MIB,
> not 
> an IP over IB Layer 2 MIB...
> 
> The procedure is the same for any draft in the group, get group
> concensus
> on the change, and then the author will update the draft accordingly...
> 
> Bill
> On Thu, Apr 25, 2002 at 09:37:24AM -0700, Aditya Dube wrote:
> > IMO, the IpOverIB's notion of a physical link layer address and what
> the
> > MIB reports as the MAC address should be the same. 
> > 
> > If the MIB draft is to be amended to comply with IPoverIB, what is the
> > procedure for it to happen?
> > 
> > Aditya Dube
> > 
> > -----Original Message-----
> > From: Larsen, Roy K [mailto:roy.k.larsen@intel.com] 
> > Sent: Wednesday, April 17, 2002 11:45 AM
> > To: 'Bill Strahm'; Aditya Dube
> > Cc: ipoverib@ietf.org
> > Subject: RE: [Ipoverib] MIB's ifPhysAddress needs to be updated
> > 
> > But it's being define in the IPoIB working group.  Why isn't it
> > legitimate to say it should support IPoIB hardware address semantics?
> > 
> > Roy
> > 
> > -----Original Message-----
> > From: Bill Strahm [mailto:bill@strahm.net]
> > Sent: Wednesday, April 17, 2002 11:34 AM
> > To: Aditya Dube
> > Cc: ipoverib@ietf.org
> > Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> > 
> > 
> > The problem is that ifPhysAddr covers more than just IP, it is just
> > the layer 2 wire representation of address.  If you can show me an
> > IB format that includes GID+LID in a packet header, I might conceed
> > defeat here.
> > 
> > The address duplicating across multiple subnets is a red herring.  You
> > can assign identical Ethernet MAC addresses, so long as they are on
> > differing subnets as well.  
> > 
> > >From RFC 2863...
> >             "The interface's address at its protocol sub-layer.  For
> >             example, for an 802.x interface, this object normally
> >             contains a MAC address.  The interface's media-specific
> MIB
> >             must define the bit and byte ordering and the format of
> the
> >             value of this object.  For interfaces which do not have
> such
> >             an address (e.g., a serial line), this object should
> contain
> >             an octet string of zero length."
> > Note that there is no mention of any Layer 3 semantics.  
> > 
> > We can bring the topic up for discussion, I don't know which of the
> MIB
> > authors are currently active right now though.
> > 
> > Bill
> > 
> > On Wed, Apr 17, 2002 at 11:19:24AM -0700, Aditya Dube wrote:
> > > 
> > > As the Lid is an assigned address rather than a burned in address,
> as
> > in
> > > Ethernet, the chances of duplicate ifphysaddr's in a network
> spanning
> > > multiple IB subnets are a lot higher. We should reduce this chance
> as
> > > much as possible.
> > > 
> > > The ifphysaddr is also not in sync with what IpOverIB has decided to
> > use
> > > as the link address in ARPs.
> > > 
> > > IMHO the two, ifphysaddr and the link address is an arp packet,
> should
> > > match.
> > > 
> > > Aditya Dube
> > > 
> > > -----Original Message-----
> > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
> > > Sent: Tuesday, April 16, 2002 12:18 PM
> > > To: Aditya Dube
> > > Cc: ipoverib@ietf.org
> > > Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> > > 
> > > The same question was addressed last october/november.
> > > See the message:
> > > 
> > >
> >
> http://www1.ietf.org/mail-archive/working-groups/ipoverib/current/msg000
> > > 87.html
> > > 
> > > Vivek
> > > 
> > > 
> > > 
> > > > 
> > > > This is a multi-part message in MIME format.
> > > > 
> > > > --------------InterScan_NT_MIME_Boundary
> > > > Content-Type: multipart/alternative;
> > > > 	boundary="----_=_NextPart_001_01C1E577.F19F4F03"
> > > > 
> > > > ------_=_NextPart_001_01C1E577.F19F4F03
> > > > Content-Type: text/plain;
> > > > 	charset="us-ascii"
> > > > Content-Transfer-Encoding: quoted-printable
> > > > 
> > > > 
> > > > In the MIB draft, the ifPhysAddress is specified as the LID. This
> is
> > > out
> > > > of sync with the IpOverIb draft. Can the ifPhysAddress in the MIB
> be
> > > > changed to match  IPoverIB's physical address, i.e. the GID + QPN?
> > > > 
> > > >
> http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.txt
> > > > 
> > > > Aditya Dube
> > > > 
> > > > 
> > > 
> > > _______________________________________________
> > > IPoverIB mailing list
> > > IPoverIB@ietf.org
> > > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Mon Apr 29 09:50:28 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA07949
	for <ipoverib-archive@odin.ietf.org>; Mon, 29 Apr 2002 09:50:28 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id JAA22047
	for ipoverib-archive@odin.ietf.org; Mon, 29 Apr 2002 09:50:28 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA22005;
	Mon, 29 Apr 2002 09:49:41 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA21973
	for <ipoverib@optimus.ietf.org>; Mon, 29 Apr 2002 09:49:37 -0400 (EDT)
Received: from voyager.ops.infiniswitch.com (webo.infiniswitch.com [63.122.241.162])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA07906
	for <ipoverib@ietf.org>; Mon, 29 Apr 2002 09:49:33 -0400 (EDT)
X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: RE: [Ipoverib] Proposed Update for IPOIB Interface MIB
Date: Mon, 29 Apr 2002 09:49:36 -0400
Message-ID: <FF6CF34C346E724596A2FB5CAAB2C497A09171@voyager.ops.infiniswitch.com>
Thread-Topic: [Ipoverib] MIB's ifPhysAddress needs to be updated
Thread-Index: AcHth6oGouXxMPkPQ1iPWt0CmUfseAB/Lr5w
From: "Harnedy, Sean" <sharnedy@infiniswitch.com>
To: "Larsen, Roy K" <roy.k.larsen@intel.com>, "Bill Strahm" <bill@strahm.net>,
        "Aditya Dube" <adube@windows.microsoft.com>
Cc: <ipoverib@ietf.org>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id JAA21974
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 8bit

The current IPOIB Interfaces MIB (Definitions of Managed Objects
for InfiniBand Interfaces Types: draft-ietf-ipoib-ibif-mib-01.txt)
defines the Infiniband interface to appear as a layer 2 interface
for IP (IPOIB). The Interfaces MIB definition of ifPhysAddr in this 
MIB equates an InfiniBand port with an interface. Because it is a 
port, the LID was chosen as the best value to reflect the "ifPhysAddr"
as the address that goes over the wire.

The GID+QPN information is important and related to the port. I
propose adding a table of IPOIB Link Layer Addresses that is 
associated with each InfiniBand interface. This is necesary because
the relationship of GID+QPN can be 1:n for any given port. 

Perhaps, something like this:

ibIpoibLinkLayerAddrTable OBJECT-TYPE
   SYNTAX      SEQUENCE OF IbIpoibLinkLayerAddrEntry
   MAX-ACCESS  not-accessible
   STATUS      current
   DESCRIPTION
      "This table contains information about all IPOIB link
       layer addresses associated with this InfiniBand interface."
   ::= { ibIfObjects n }

ibIpoibLinkLayerAddrEntry OBJECT-TYPE
   SYNTAX      IbIpoibLinkLayerAddrEntry
   MAX-ACCESS  not-accessible
   STATUS      current
   DESCRIPTION
      "Information about specific IPOIB link layer addresses 
       associated with this InfiniBand interface."
   ::= { ibIpoibLinkLayerAddrTable 1 }
   
IbIpoibLinkLayerAddrEntry ::= SEQUENCE {
   ibIpoibLinkLayerIndex     INTEGER,
   ibIpoibInterfaceID        OCTETSTRING(SIZE(4)),
   ibIpoibGlobalIdentifier   OCTETSTRING(SIZE(16))
   }

ibIpoibLinkLayerIndex  OBJECT-TYPE
   SYNTAX      INTEGER (1..n))
   MAX-ACCESS  read-only
   STATUS      current
   DESCRIPTION
      "Index of the GID+QPN IPOIB Link Layer addresses
       in the table."
   ::= { ibIpoibLinkLayerEntry 1 }

ibIpoibInterfaceID  OBJECT-TYPE        
   SYNTAX   OCTETSTRING (SIZE(4)),
   MAX-ACCESS  read-only
   STATUS      current
   DESCRIPTION
      "The Interface ID. This object contains 1 octet 
       for reserved (must be 0) and 3 octets for the
       queue pair number (QPN)."
   ::= { ibIpoibLinkLayerEntry 2 }

ibIpoibGlobalIdentifier  OBJECT-TYPE   
   SYNTAX   OCTETSTRING (SIZE(16))
   MAX-ACCESS  read-only
   STATUS      current
   DESCRIPTION
      "The Global Identifier (GID)."
   ::= { ibIpoibLinkLayerEntry 3 }

This will allow the statistics and counters currently defined in the
MIB to be unchanged and add the flexibility of defining the additional
IPOIB Link layer address definitions.

Sean

-----Original Message-----
From: Larsen, Roy K [mailto:roy.k.larsen@intel.com]
Sent: Friday, April 26, 2002 9:05 PM
To: 'Bill Strahm'; Aditya Dube
Cc: ipoverib@ietf.org
Subject: RE: [Ipoverib] MIB's ifPhysAddress needs to be updated


Bill,

If the MIB is IPoIB agnostic, what is it doing in the IPoIB working group?
If it doesn't need to jive with the information returned from things like
ifconfig(8), arp(8), and netstat(8), why does this group care about this MIB
at all?

Roy

-----Original Message-----
From: Bill Strahm [mailto:bill@strahm.net]
Sent: Friday, April 26, 2002 4:14 PM
To: Aditya Dube
Cc: Bill Strahm; ipoverib@ietf.org
Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated


Again, an interesting problem,

What would you consider ifInOctets to be if the Physical address is GID +
QPN.

Now think about that answer in an environment that is both IP and IB native,
would each QPN now be an interface in the ifTable ???

The if MIB is designed to handle the Layer 2 properties of the interface 
without regard to what is running on top of it.  Rather than simple Ethernet
where IP runs directly on top of Ethernets Layer 2 interface, in Infiniband
we have IP running on top of Infinibans Layer 3 UD interface.  I hate to
tell
you but I will fight adding QPN to the ifPhysAddress tooth and nail becasue
it does not belong there (in my humble non-chair role)  If you can explain
how
to get other layer 2 protocols on top of Infiniband without incurring 64K
interfaces, I will listen.

Bill
On Fri, Apr 26, 2002 at 02:39:18PM -0700, Aditya Dube wrote:
> Bill
> 
> The MIB document predates the decision to use the GID + QPN as the
> physical address.
> 
> In the current scheme, the entity reported as the physical address by
> the encapsulation layer is inconsistent with what one gets using the
> MIB.
> 
> Aditya Dube
> 
> -----Original Message-----
> From: Bill Strahm [mailto:bill@strahm.net] 
> Sent: Thursday, April 25, 2002 10:00 AM
> To: Aditya Dube
> Cc: ipoverib@ietf.org
> Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> 
> You are making a subtle mistake... The MIB is an Infiniband layer 2 MIB,
> not 
> an IP over IB Layer 2 MIB...
> 
> The procedure is the same for any draft in the group, get group
> concensus
> on the change, and then the author will update the draft accordingly...
> 
> Bill
> On Thu, Apr 25, 2002 at 09:37:24AM -0700, Aditya Dube wrote:
> > IMO, the IpOverIB's notion of a physical link layer address and what
> the
> > MIB reports as the MAC address should be the same. 
> > 
> > If the MIB draft is to be amended to comply with IPoverIB, what is the
> > procedure for it to happen?
> > 
> > Aditya Dube
> > 
> > -----Original Message-----
> > From: Larsen, Roy K [mailto:roy.k.larsen@intel.com] 
> > Sent: Wednesday, April 17, 2002 11:45 AM
> > To: 'Bill Strahm'; Aditya Dube
> > Cc: ipoverib@ietf.org
> > Subject: RE: [Ipoverib] MIB's ifPhysAddress needs to be updated
> > 
> > But it's being define in the IPoIB working group.  Why isn't it
> > legitimate to say it should support IPoIB hardware address semantics?
> > 
> > Roy
> > 
> > -----Original Message-----
> > From: Bill Strahm [mailto:bill@strahm.net]
> > Sent: Wednesday, April 17, 2002 11:34 AM
> > To: Aditya Dube
> > Cc: ipoverib@ietf.org
> > Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> > 
> > 
> > The problem is that ifPhysAddr covers more than just IP, it is just
> > the layer 2 wire representation of address.  If you can show me an
> > IB format that includes GID+LID in a packet header, I might conceed
> > defeat here.
> > 
> > The address duplicating across multiple subnets is a red herring.  You
> > can assign identical Ethernet MAC addresses, so long as they are on
> > differing subnets as well.  
> > 
> > >From RFC 2863...
> >             "The interface's address at its protocol sub-layer.  For
> >             example, for an 802.x interface, this object normally
> >             contains a MAC address.  The interface's media-specific
> MIB
> >             must define the bit and byte ordering and the format of
> the
> >             value of this object.  For interfaces which do not have
> such
> >             an address (e.g., a serial line), this object should
> contain
> >             an octet string of zero length."
> > Note that there is no mention of any Layer 3 semantics.  
> > 
> > We can bring the topic up for discussion, I don't know which of the
> MIB
> > authors are currently active right now though.
> > 
> > Bill
> > 
> > On Wed, Apr 17, 2002 at 11:19:24AM -0700, Aditya Dube wrote:
> > > 
> > > As the Lid is an assigned address rather than a burned in address,
> as
> > in
> > > Ethernet, the chances of duplicate ifphysaddr's in a network
> spanning
> > > multiple IB subnets are a lot higher. We should reduce this chance
> as
> > > much as possible.
> > > 
> > > The ifphysaddr is also not in sync with what IpOverIB has decided to
> > use
> > > as the link address in ARPs.
> > > 
> > > IMHO the two, ifphysaddr and the link address is an arp packet,
> should
> > > match.
> > > 
> > > Aditya Dube
> > > 
> > > -----Original Message-----
> > > From: Vivek Kashyap [mailto:kashyapv@us.ibm.com] 
> > > Sent: Tuesday, April 16, 2002 12:18 PM
> > > To: Aditya Dube
> > > Cc: ipoverib@ietf.org
> > > Subject: Re: [Ipoverib] MIB's ifPhysAddress needs to be updated
> > > 
> > > The same question was addressed last october/november.
> > > See the message:
> > > 
> > >
> >
> http://www1.ietf.org/mail-archive/working-groups/ipoverib/current/msg000
> > > 87.html
> > > 
> > > Vivek
> > > 
> > > 
> > > 
> > > > 
> > > > This is a multi-part message in MIME format.
> > > > 
> > > > --------------InterScan_NT_MIME_Boundary
> > > > Content-Type: multipart/alternative;
> > > > 	boundary="----_=_NextPart_001_01C1E577.F19F4F03"
> > > > 
> > > > ------_=_NextPart_001_01C1E577.F19F4F03
> > > > Content-Type: text/plain;
> > > > 	charset="us-ascii"
> > > > Content-Transfer-Encoding: quoted-printable
> > > > 
> > > > 
> > > > In the MIB draft, the ifPhysAddress is specified as the LID. This
> is
> > > out
> > > > of sync with the IpOverIb draft. Can the ifPhysAddress in the MIB
> be
> > > > changed to match  IPoverIB's physical address, i.e. the GID + QPN?
> > > > 
> > > >
> http://www.ietf.org/internet-drafts/draft-ietf-ipoib-ibif-mib-01.txt
> > > > 
> > > > Aditya Dube
> > > > 
> > > > 
> > > 
> > > _______________________________________________
> > > IPoverIB mailing list
> > > IPoverIB@ietf.org
> > > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > IPoverIB@ietf.org
> > https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



From daemon@optimus.ietf.org  Tue Apr 30 16:16:58 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA07497
	for <ipoverib-archive@odin.ietf.org>; Tue, 30 Apr 2002 16:16:58 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA22928
	for ipoverib-archive@odin.ietf.org; Tue, 30 Apr 2002 16:17:01 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA22823;
	Tue, 30 Apr 2002 16:16:01 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA22790
	for <ipoverib@optimus.ietf.org>; Tue, 30 Apr 2002 16:15:58 -0400 (EDT)
Received: from voyager.ops.infiniswitch.com (webo.infiniswitch.com [63.122.241.162])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA07416
	for <ipoverib@ietf.org>; Tue, 30 Apr 2002 16:15:54 -0400 (EDT)
X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Date: Tue, 30 Apr 2002 16:15:52 -0400
Message-ID: <FF6CF34C346E724596A2FB5CAAB2C497663DE0@voyager.ops.infiniswitch.com>
Thread-Topic: IPOIB Link Layer Address Table MIB Definition
Thread-Index: AcHwg9Pzc0IZFZYbRmeHqSbI6TaMrA==
From: "Harnedy, Sean" <sharnedy@infiniswitch.com>
To: <ipoverib@ietf.org>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id QAA22791
Subject: [Ipoverib] IPOIB Link Layer Address Table MIB Definition
Sender: ipoverib-admin@ietf.org
Errors-To: ipoverib-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
X-BeenThere: ipoverib@ietf.org
Content-Transfer-Encoding: 8bit

I propose the addition of the IPOIB Link Layer Address Table to the
InfiniBand Interfaces MIB to define the IPOIB Client Identifiers
that can be associated with each InfiniBand interface (port). I have
created a textual convention to define the 20 octet definition that
is consistent with the other IPOIB encapsulation and architecture
documents. The table allows for GID+different QPNs and different GIDs+
QPN that may be configured for each IPOIB interface at the InfiniBand
port. The index for the table is the IfIndex (e.g. InfiniBand port
number) and the entry index number. I chose 65535 as the upper limit
for the number of Client Identifiers associated with each port, although
there may be a more appropriate number defined in the IBTA spec (i.e,
number of GIDs associated with a port * QPN ?).

Sean

--
-- IPOIB Client Identifier Textual Convention
--

   IbIpoibClientIdentifier ::= TEXTUAL CONVENTION
      STATUS   current
      DESCRIPTION
         "The IPOIB Client Identifier uniquely identifies an IPOIB
          link layer address associated with an InfiniBand port. It
          comprises 3 fields.

            1. reserved
            2. Queue Pair Number field (QPN)
            3. Global Identifier (GID)

       +................+................+................+...............+
       |20: (reserved)  |19: QPN[23-16]  |18: QPN[15-8]   |17: QPN[7-0]   |
       +................+................+................+...............+
       |16: GID[127-120]|15: GID[119-112]|14: GID[111-104]|13: GID[103-96]|
       +................+................+................+...............+
       |12: GID[95-88]  |11: GID[87-80]  |10: GID[79-72]  |9:  GID[71-64] |
       +................+................+................+...............+
       |8:  GID[63-56]  |7:  GID[55-48]  |6:  GID[47-40]  |5:  GID[39-32] |
       +................+................+................+...............+
       |4:  GID[31-24]  |3:  GID[23-16]  |2:  GID[15-8]   |1:  GID[7-0]   |
       +................+................+................+...............+
                       
         The reserved field is octet 20. It is reserved for future use.
         These bits MUST be set to zero.
 
         The Queue Pair Number field is a 3 octet field (octets 17, 18 & 19)
         that identifies the destination queue pair. Note: the reserved field
         and the QPN field are collectively referred to as the interface-id.
         If an IPOIB interface has only 1 GID associated with it, the 
         inteface-id may MAY contain all zeroes.

         The Global Identifier field is a 16 octet field (octets 1 thru 16)
         that is formed by the combination of the IB subnet prefix and the 
         port's GUID be included. It is unique in the InfiniBand fabric. 
         Note: An IPOIB interface may have more than 1 GID associated with 
         it."
      SYNTAX OCTET STRING (SIZE (20))

--
-- IPOIB Link Layer Address Table
--

ibIpoibLinkLayerAddrTable OBJECT-TYPE
   SYNTAX      SEQUENCE OF IbIpoibLinkLayerAddrEntry
   MAX-ACCESS  not-accessible
   STATUS      current
   DESCRIPTION
      "This table contains information about all IPOIB link
       layer addresses associated with this InfiniBand interface
       (port)."
   ::= { ibIfObjects 3 }

ibIpoibLinkLayerAddrEntry OBJECT-TYPE
   SYNTAX      IbIpoibLinkLayerAddrEntry
   MAX-ACCESS  not-accessible
   STATUS      current
   DESCRIPTION
      "Information about specific IPOIB link layer addresses 
       associated with this InfiniBand interface (port)."
   INDEX { ibIfLinkLayerIfIndex, ibIpoibLinkLayerIndex }
   ::= { ibIpoibLinkLayerAddrTable 1 }
   
IbIpoibLinkLayerAddrEntry ::= SEQUENCE {
   ibIpoibLinkLayerIfIndex   InterfaceIndex,
   ibIpoibLinkLayerIndex     INTEGER,
   ibIpoibLinkLayerAddr      IbIpoibClientIdentifier
   }

ibIpoibLinkLayerIfIndex  OBJECT-TYPE
   SYNTAX      InterfaceIndex
   MAX-ACCESS  read-only
   STATUS      current
   DESCRIPTION
      "The IfIndex of the IPOIB InfiniBand interface (port)."
   ::= { ibIpoibLinkLayerEntry 1 }

ibIpoibLinkLayerIndex  OBJECT-TYPE
   SYNTAX      INTEGER (1..65535))
   MAX-ACCESS  read-only
   STATUS      current
   DESCRIPTION
      "Index of the IPOIB Link Layer address in the table."
   ::= { ibIpoibLinkLayerEntry 2 }

ibIpoibLinkLayerAddr  OBJECT-TYPE        
   SYNTAX      IbIpoibClientIdentifier
   MAX-ACCESS  read-only
   STATUS      current
   DESCRIPTION
      "A Client Identifier for this interface (port). An implementation
       may associate multiple IPOIB interfaces on the same port. It is 
       up to the implementation to ensure a unique Client Identifier 
       when multiple IPoIB interfaces are defined over the same port and
       same GID. A unique, invariant interface-id value must be included
       in addition to the GID within the Client Identifier definition to 
       achieve this. Note: a port may also be associated with multiple
       GIDs. Therefore, multiple IPoIB interfaces may exist on the same 
       port while using a different GID from among the GIDs associated 
       with the port. In either case, the IbIpoibLinkLayerAddr must be
       unique." 
    ::= { ibIpoibLinkLayerEntry 3 }

_______________________________________________
IPoverIB mailing list
IPoverIB@ietf.org
https://www1.ietf.org/mailman/listinfo/ipoverib



