From narten@us.ibm.com Fri Sep 30 17:01:29 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A290221F88B7; Fri, 30 Sep 2011 17:01:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.503 X-Spam-Level: X-Spam-Status: No, score=-106.503 tagged_above=-999 required=5 tests=[AWL=0.096, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id axeYqsjEUE4D; Fri, 30 Sep 2011 17:01:29 -0700 (PDT) Received: from e6.ny.us.ibm.com (e6.ny.us.ibm.com [32.97.182.146]) by ietfa.amsl.com (Postfix) with ESMTP id EBC3C21F8888; Fri, 30 Sep 2011 17:01:28 -0700 (PDT) Received: from d01relay02.pok.ibm.com (d01relay02.pok.ibm.com [9.56.227.234]) by e6.ny.us.ibm.com (8.14.4/8.13.1) with ESMTP id p8UNe9tS008905; Fri, 30 Sep 2011 19:40:09 -0400 Received: from d01av03.pok.ibm.com (d01av03.pok.ibm.com [9.56.224.217]) by d01relay02.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id p9103a1v451934; Fri, 30 Sep 2011 20:03:51 -0400 Received: from d01av03.pok.ibm.com (loopback [127.0.0.1]) by d01av03.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id p9103aj3026775; Fri, 30 Sep 2011 21:03:36 -0300 Received: from cichlid.raleigh.ibm.com (sig-9-65-211-53.mts.ibm.com [9.65.211.53]) by d01av03.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id p9103ZOZ026758 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 30 Sep 2011 21:03:35 -0300 Received: from cichlid.raleigh.ibm.com (cichlid.raleigh.ibm.com [127.0.0.1]) by cichlid.raleigh.ibm.com (8.14.4/8.12.5) with ESMTP id p9103Y8e011423; Fri, 30 Sep 2011 20:03:34 -0400 Message-Id: <201110010003.p9103Y8e011423@cichlid.raleigh.ibm.com> Date: Fri, 30 Sep 2011 20:03:34 -0400 From: Thomas Narten To: undisclosed-recipients:; X-Mailman-Approved-At: Sat, 01 Oct 2011 12:17:58 -0700 Subject: [armd] Call for Participation: Using IP Overlays to provide L2 Virtualization X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Oct 2011 00:01:29 -0000 ------- Blind-Carbon-Copy To: ietf@ietf.org cc: nvo3@ietf.org Subject: Call for Participation: Using IP Overlays to provide L2 Virtualization In-reply-to: <20110929214502.9DB6621F8E8E@ietfa.amsl.com> References: <20110929214502.9DB6621F8E8E@ietfa.amsl.com> Comments: In-reply-to IETF Secretariat message dated "Thu, 29 Sep 2011 14:45:02 -0700." Date: Fri, 30 Sep 2011 20:03:34 -0400 From: Thomas Narten A new mailing list has been set up to explore possible IETF work in the area of providing L2 network virtualization service over an L3 (IP) overlay network. As background, there are a number of drafts that relate to this area, including: http://tools.ietf.org/html/draft-mahalingam-dutt-dcops-vxlan-00 http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-00 http://tools.ietf.org/html/draft-wkumari-dcops-l3-vmmobility-00 I've put together a first-cut at a problem statement that focuses on the issues and potential work areas, without getting into solution specifics. See: http://tools.ietf.org/html/draft-narten-nvo3-overlay-problem-statement-00.txt There have also been some related vendor announcements and presentations as well (these are ones I happen to know of, there are surely others). For example: http://blogs.cisco.com/datacenter/introducing-vxlan/ http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9902/white_paper_c11-685115.html http://blogs.vmware.com/console/2011/08/towards-virtualized-networking-for-the-cloud.html http://channel9.msdn.com/Events/BUILD/BUILD2011/SAC-442T The list is called nvo3, for "Network Virtualization Over L3", aka N-Vee-Oh-3 (we'll see how well that acronym sticks...). I've put in a formal request to hold a BOF in Taipai, so we can explore whether it makes sense to form a WG in this area. Subscription information for the mailing list can be found at List address: nvo3@ietf.org Archive: http://www.ietf.org/mail-archive/web/nvo3/current/maillist.html To subscribe: https://www.ietf.org/mailman/listinfo/nvo3 I look forward to your participation! Thomas ------- End of Blind-Carbon-Copy From linda.dunbar@huawei.com Tue Oct 4 07:46:01 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DD37B21F8C6D for ; Tue, 4 Oct 2011 07:46:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.39 X-Spam-Level: X-Spam-Status: No, score=-6.39 tagged_above=-999 required=5 tests=[AWL=0.209, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fCgjY2Pjf9A5 for ; Tue, 4 Oct 2011 07:46:00 -0700 (PDT) Received: from usaga04-in.huawei.com (usaga04-in.huawei.com [206.16.17.180]) by ietfa.amsl.com (Postfix) with ESMTP id B8B8021F8C61 for ; Tue, 4 Oct 2011 07:46:00 -0700 (PDT) Received: from huawei.com (usaga04-in [172.18.4.101]) by usaga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LSJ006A8QHSIZ@usaga04-in.huawei.com> for armd@ietf.org; Tue, 04 Oct 2011 09:49:05 -0500 (CDT) Received: from dfweml202-edg.china.huawei.com ([172.18.4.104]) by usaga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LSJ00DHFQHP1L@usaga04-in.huawei.com> for armd@ietf.org; Tue, 04 Oct 2011 09:49:04 -0500 (CDT) Received: from DFWEML401-HUB.china.huawei.com (10.193.5.101) by dfweml202-edg.china.huawei.com (172.18.9.108) with Microsoft SMTP Server (TLS) id 14.1.270.1; Tue, 04 Oct 2011 07:49:03 -0700 Received: from DFWEML506-MBX.china.huawei.com ([10.124.31.111]) by DFWEML401-HUB.china.huawei.com ([fe80::f07f:889f:78ef:8df3%13]) with mapi id 14.01.0270.001; Tue, 04 Oct 2011 07:48:51 -0700 Date: Tue, 04 Oct 2011 14:48:51 +0000 From: Linda Dunbar X-Originating-IP: [10.192.11.155] To: Thomas Narten , Murari Sridharan Message-id: <4A95BA014132FF49AE685FAB4B9F17F61209A7F3@dfweml506-mbx> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-language: en-US Content-transfer-encoding: 7BIT Accept-Language: en-US Thread-topic: [armd] Call for Participation: Using IP Overlays to provide L2 Virtualization Thread-index: AQHMgG9fv8wbYy1Gj0mzduupMOD9aZVrNwhAgAEQccA= X-MS-Has-Attach: X-MS-TNEF-Correlator: Cc: "armd@ietf.org" Subject: Re: [armd] Call for Participation: Using IP Overlays to provide L2 Virtualization X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 14:46:02 -0000 Thomas and Murari, I agree that "Overlay push some of the L2 scaling concerns" in your draft. But this statement should not be listed under the sub-section of ARMD. Overlay is to hide VMs' addresses from network interior nodes. Address Resolution is to map VMs' IP addresses to physical addresses. Applications (or VMs) within Data center need to communicate with peers outside data center. Regardless of overlay is deployed within the data center or not, the Gateway routers still need to resolve physical addresses for all VMs which have external communications. Therefore, Address resolution and Overlay are orthogonal to each other. Overlay can only make address resolution more complex, because Gateway router not only needs to resolve physical address for each VM's IP, it also needs to resolve the overlay edge address for each VM's IP. Do you view it differently? I would like to know your view. Thanks, Linda --------------------------------------- 6. Related Work 6.1. ARMD ARMD is chartered to look at data center scaling issues with a focus on address resolution. ARMD is currently chartered to develop a problem statement and is not currently developing solutions. While an overlay-based approach may address some of the "pain points" that have been raised in ARMD (e.g., better support for multitenancy), an overlay approach may also push some of the L2 scaling concerns (e.g., excessive flooding) to the IP level (flooding via IP multicast). Analysis will be needed to understand the scaling trade offs of an overlay based approach compared with existing approaches. On the other hand, existing IP-based approaches such as proxy ARP may help mitigate some concerns. > -----Original Message----- > From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of > Thomas Narten > Sent: Friday, September 30, 2011 7:04 PM > Subject: [armd] Call for Participation: Using IP Overlays to provide L2 > Virtualization > > ------- Blind-Carbon-Copy > > To: ietf@ietf.org > cc: nvo3@ietf.org > Subject: Call for Participation: Using IP Overlays to provide L2 > Virtualization > In-reply-to: <20110929214502.9DB6621F8E8E@ietfa.amsl.com> > References: <20110929214502.9DB6621F8E8E@ietfa.amsl.com> > Comments: In-reply-to IETF Secretariat > message dated "Thu, 29 Sep 2011 14:45:02 -0700." > Date: Fri, 30 Sep 2011 20:03:34 -0400 > From: Thomas Narten > > A new mailing list has been set up to explore possible IETF work in > the area of providing L2 network virtualization service over an L3 > (IP) overlay network. > > As background, there are a number of drafts that relate to this area, > including: > > http://tools.ietf.org/html/draft-mahalingam-dutt-dcops-vxlan-00 > http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-00 > http://tools.ietf.org/html/draft-wkumari-dcops-l3-vmmobility-00 > > I've put together a first-cut at a problem statement that focuses on > the issues and potential work areas, without getting into solution > specifics. See: > > http://tools.ietf.org/html/draft-narten-nvo3-overlay-problem- > statement-00.txt > > There have also been some related vendor announcements and > presentations as well (these are ones I happen to know of, there are > surely others). For example: > > http://blogs.cisco.com/datacenter/introducing-vxlan/ > > > http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9902/white > _paper_c11-685115.html > > http://blogs.vmware.com/console/2011/08/towards-virtualized- > networking-for-the-cloud.html > > http://channel9.msdn.com/Events/BUILD/BUILD2011/SAC-442T > > The list is called nvo3, for "Network Virtualization Over L3", aka > N-Vee-Oh-3 (we'll see how well that acronym sticks...). > > I've put in a formal request to hold a BOF in Taipai, so we can > explore whether it makes sense to form a WG in this area. > > Subscription information for the mailing list can be found at > > List address: nvo3@ietf.org > Archive: http://www.ietf.org/mail- > archive/web/nvo3/current/maillist.html > To subscribe: https://www.ietf.org/mailman/listinfo/nvo3 > > I look forward to your participation! > > Thomas > > ------- End of Blind-Carbon-Copy > _______________________________________________ > armd mailing list > armd@ietf.org > https://www.ietf.org/mailman/listinfo/armd From narten@us.ibm.com Tue Oct 4 09:06:43 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 53BDD21F8DA1 for ; Tue, 4 Oct 2011 09:06:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -105.916 X-Spam-Level: X-Spam-Status: No, score=-105.916 tagged_above=-999 required=5 tests=[AWL=0.683, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 37E8Ac2pSdBx for ; Tue, 4 Oct 2011 09:06:42 -0700 (PDT) Received: from e36.co.us.ibm.com (e36.co.us.ibm.com [32.97.110.154]) by ietfa.amsl.com (Postfix) with ESMTP id BFE8321F8DA0 for ; Tue, 4 Oct 2011 09:06:42 -0700 (PDT) Received: from d03relay03.boulder.ibm.com (d03relay03.boulder.ibm.com [9.17.195.228]) by e36.co.us.ibm.com (8.14.4/8.13.1) with ESMTP id p94G2vqW022974 for ; Tue, 4 Oct 2011 10:02:57 -0600 Received: from d03av06.boulder.ibm.com (d03av06.boulder.ibm.com [9.17.195.245]) by d03relay03.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id p94G9Vqe162798 for ; Tue, 4 Oct 2011 10:09:35 -0600 Received: from d03av06.boulder.ibm.com (loopback [127.0.0.1]) by d03av06.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id p94G9Vbk020378 for ; Tue, 4 Oct 2011 10:09:31 -0600 Received: from cichlid.raleigh.ibm.com (sig-9-49-158-243.mts.ibm.com [9.49.158.243]) by d03av06.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id p94G9U78020319 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 4 Oct 2011 10:09:31 -0600 Received: from cichlid.raleigh.ibm.com (cichlid.raleigh.ibm.com [127.0.0.1]) by cichlid.raleigh.ibm.com (8.14.4/8.12.5) with ESMTP id p94G9SVh019333; Tue, 4 Oct 2011 12:09:29 -0400 Message-Id: <201110041609.p94G9SVh019333@cichlid.raleigh.ibm.com> To: Linda Dunbar In-reply-to: <4A95BA014132FF49AE685FAB4B9F17F61209A7F3@dfweml506-mbx> References: <4A95BA014132FF49AE685FAB4B9F17F61209A7F3@dfweml506-mbx> Comments: In-reply-to Linda Dunbar message dated "Tue, 04 Oct 2011 14:48:51 -0000." Date: Tue, 04 Oct 2011 12:09:28 -0400 From: Thomas Narten Cc: "armd@ietf.org" Subject: Re: [armd] Call for Participation: Using IP Overlays to provide L2 Virtualization X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 16:06:43 -0000 Hi Linda. > I agree that "Overlay push some of the L2 scaling concerns" in your > draft. But this statement should not be listed under the sub-section > of ARMD. I will tweak this in the next version. > Overlay is to hide VMs' addresses from network interior > nodes. Agreed. > Address Resolution is to map VMs' IP addresses to physical > addresses. Yes, but. Address resolution is also a sort of generic term and can mean more than this. Anytime you have to map an "upper layer" address to a "lower layer" address, you need address resolution or address mapping. We usually think of ARP/ND doing address resolution between an IP address and a physical (ethernet) address. But, with overlays, you will have multiple layers doing address mapping. In the overlay context, you will have the VM's IP address, which needs to be mapped into the infrastructure IP address where that VM current resides (and where packets can be tunneled to). That is something that is handled at the overlay level. Separately, at some point, a router/gateway will need to map that infrastructure IP address into a physical address (just as is done today). I assume this is what you mean by "address resolution". The "mapping function" that the problem statement draft uses is a form of Address Resolution. > Applications (or VMs) within Data center need to communicate with > peers outside data center. Regardless of overlay is deployed within > the data center or not, the Gateway routers still need to resolve > physical addresses for all VMs which have external communications. I'm not sure I understand this fully (see above). There are two IP addresses in use (private within the overlay and infrastructure). Which IP addresses do you mean above? > Therefore, Address resolution and Overlay are orthogonal to each > other. If you mean address resolution as in the ARP/ND sense vs. "mappings" in the overlay sense, I agree. > Overlay can only make address resolution more complex, > because Gateway router not only needs to resolve physical address > for each VM's IP, it also needs to resolve the overlay edge address > for each VM's IP. Yes. If the gateway is at the edge of the overlay, it will need to encap/decap packets and perform address mappings. Separately, it will need to do address resolution for infrastructure addresses. Thomas From vishwas.ietf@gmail.com Wed Oct 5 08:07:17 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5D93621F8BFB for ; Wed, 5 Oct 2011 08:07:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.28 X-Spam-Level: X-Spam-Status: No, score=-3.28 tagged_above=-999 required=5 tests=[AWL=0.318, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cX6pmNpDY1u8 for ; Wed, 5 Oct 2011 08:07:17 -0700 (PDT) Received: from mail-qy0-f179.google.com (mail-qy0-f179.google.com [209.85.216.179]) by ietfa.amsl.com (Postfix) with ESMTP id CAFBD21F8BF0 for ; Wed, 5 Oct 2011 08:07:16 -0700 (PDT) Received: by qyk33 with SMTP id 33so1483880qyk.10 for ; Wed, 05 Oct 2011 08:10:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=LphnsZVoPpZ9Uk9D+lU2NUp2AKEnfuAhqifkBXiA3/M=; b=BkC3YExZImhqXZDOXrG9KgTR6TECcn59Y7po4lWJs4aQfUR/1hhzaF/Tt17NoidBkE lyjKs7/vBRdx8THuQMwN/gBMlgiNzKtzZdfNEwMd+OdUdaffrM9ReFW1jtM0EsBtu/0A JB3GWJv40jFeti8tebd3dsMrYtiV07+PX+0q4= MIME-Version: 1.0 Received: by 10.229.196.233 with SMTP id eh41mr2050789qcb.187.1317827424600; Wed, 05 Oct 2011 08:10:24 -0700 (PDT) Received: by 10.229.91.131 with HTTP; Wed, 5 Oct 2011 08:10:24 -0700 (PDT) Date: Wed, 5 Oct 2011 08:10:24 -0700 Message-ID: From: Vishwas Manral To: armd@ietf.org Content-Type: multipart/alternative; boundary=0016369cffb484c3d004ae8e9b4a Subject: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 15:07:17 -0000 --0016369cffb484c3d004ae8e9b4a Content-Type: text/plain; charset=ISO-8859-1 Hi, I was thinking of multi-tenancy and the way to achieve > 4094 without any changes in any layer by just using VLAN's. This is achieved as we already have a mapping layer already in place. So assume there are 8000 tenants. We can divide them in groups of 4000 each and assign each tenant a particular id from say 2 to 4001. Now as long as we make sure we do not have tenants from 2 different groups on the same machine, there are just no issues. Only the mapping layer needs to be aware of the group a machine belongs to. There is thus no change in dataplane. Any comments? Thanks, Vishwas --0016369cffb484c3d004ae8e9b4a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi,
=A0
I was thinking of multi-tenancy and the way to achieve > 4094 witho= ut any changes in any layer by just using VLAN's. This is achieved as w= e already have a mapping layer already in place.
=A0
So assume there are 8000 tenants.=A0We can divide them in groups of 40= 00 each and assign each tenant a particular id from say 2 to 4001. Now as l= ong as we make sure we do not have tenants from 2 different groups on the s= ame machine, there are just no issues. Only the mapping layer needs to be a= ware of the group a machine belongs to. There is thus no change in dataplan= e.
=A0
Any comments?
=A0
Thanks,
Vishwas
--0016369cffb484c3d004ae8e9b4a-- From linda.dunbar@huawei.com Wed Oct 5 12:03:14 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3429E11E80C9 for ; Wed, 5 Oct 2011 12:03:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.416 X-Spam-Level: X-Spam-Status: No, score=-6.416 tagged_above=-999 required=5 tests=[AWL=0.182, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J1+HehBRSchy for ; Wed, 5 Oct 2011 12:03:13 -0700 (PDT) Received: from usaga04-in.huawei.com (usaga04-in.huawei.com [206.16.17.180]) by ietfa.amsl.com (Postfix) with ESMTP id 5D6AF11E80C2 for ; Wed, 5 Oct 2011 12:03:13 -0700 (PDT) Received: from huawei.com (usaga04-in [172.18.4.101]) by usaga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LSL003RMX2KW6@usaga04-in.huawei.com> for armd@ietf.org; Wed, 05 Oct 2011 14:06:21 -0500 (CDT) Received: from dfweml202-edg.china.huawei.com ([172.18.4.104]) by usaga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LSL00EMQX2K88@usaga04-in.huawei.com> for armd@ietf.org; Wed, 05 Oct 2011 14:06:20 -0500 (CDT) Received: from DFWEML404-HUB.china.huawei.com (10.193.5.203) by dfweml202-edg.china.huawei.com (172.18.9.108) with Microsoft SMTP Server (TLS) id 14.1.270.1; Wed, 05 Oct 2011 12:06:21 -0700 Received: from DFWEML506-MBX.china.huawei.com ([10.124.31.111]) by dfweml404-hub.china.huawei.com ([10.193.5.203]) with mapi id 14.01.0270.001; Wed, 05 Oct 2011 12:06:09 -0700 Date: Wed, 05 Oct 2011 19:06:09 +0000 From: Linda Dunbar In-reply-to: X-Originating-IP: [10.192.11.155] To: Vishwas Manral , "armd@ietf.org" Message-id: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_ecW72zECnyhttB/ZfZcrZQ)" Content-language: en-US Accept-Language: en-US, zh-CN Thread-topic: [armd] Multi-Tenancy without changes Thread-index: AQHMg3D0j3Ia6kdngkOCCwlMzdnQoJVuG8PA X-MS-Has-Attach: X-MS-TNEF-Correlator: References: Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 19:03:14 -0000 --Boundary_(ID_ecW72zECnyhttB/ZfZcrZQ) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Vishwas, Are you assuming that each group of tenants are connected to Gateway router via completely different physical ports? If yes, then you are absolutely correct. Linda From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas Manral Sent: Wednesday, October 05, 2011 10:10 AM To: armd@ietf.org Subject: [armd] Multi-Tenancy without changes Hi, I was thinking of multi-tenancy and the way to achieve > 4094 without any changes in any layer by just using VLAN's. This is achieved as we already have a mapping layer already in place. So assume there are 8000 tenants. We can divide them in groups of 4000 each and assign each tenant a particular id from say 2 to 4001. Now as long as we make sure we do not have tenants from 2 different groups on the same machine, there are just no issues. Only the mapping layer needs to be aware of the group a machine belongs to. There is thus no change in dataplane. Any comments? Thanks, Vishwas --Boundary_(ID_ecW72zECnyhttB/ZfZcrZQ) Content-type: text/html; charset=us-ascii Content-transfer-encoding: quoted-printable

Vishwas,

 <= /p>

Are you assuming that eac= h group of tenants are connected to Gateway router via completely different= physical ports? If yes, then you are absolutely correct.

 <= /p>

Linda

 <= /p>

From: armd-bou= nces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas Manral
Sent: Wednesday, October 05, 2011 10:10 AM
To: armd@ietf.org
Subject: [armd] Multi-Tenancy without changes

 

Hi,

 

I was thinking of multi-tenancy and the way to achie= ve > 4094 without any changes in any layer by just using VLAN's. This is= achieved as we already have a mapping layer already in place.

 

So assume there are 8000 tenants. We can divide= them in groups of 4000 each and assign each tenant a particular id from sa= y 2 to 4001. Now as long as we make sure we do not have tenants from 2 diff= erent groups on the same machine, there are just no issues. Only the mapping layer needs to be aware of the group = a machine belongs to. There is thus no change in dataplane.

 

Any comments?

 

Thanks,

Vishwas

--Boundary_(ID_ecW72zECnyhttB/ZfZcrZQ)-- From narten@us.ibm.com Wed Oct 5 12:59:50 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E3DEC11E80C9; Wed, 5 Oct 2011 12:59:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -105.961 X-Spam-Level: X-Spam-Status: No, score=-105.961 tagged_above=-999 required=5 tests=[AWL=0.638, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Llhbt6An7UpU; Wed, 5 Oct 2011 12:59:50 -0700 (PDT) Received: from e9.ny.us.ibm.com (e9.ny.us.ibm.com [32.97.182.139]) by ietfa.amsl.com (Postfix) with ESMTP id EB29111E80BB; Wed, 5 Oct 2011 12:59:49 -0700 (PDT) Received: from d01relay02.pok.ibm.com (d01relay02.pok.ibm.com [9.56.227.234]) by e9.ny.us.ibm.com (8.14.4/8.13.1) with ESMTP id p95JR77U016053; Wed, 5 Oct 2011 15:27:07 -0400 Received: from d01av01.pok.ibm.com (d01av01.pok.ibm.com [9.56.224.215]) by d01relay02.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id p95K13S8317468; Wed, 5 Oct 2011 16:01:05 -0400 Received: from d01av01.pok.ibm.com (loopback [127.0.0.1]) by d01av01.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id p95K11fX025510; Wed, 5 Oct 2011 16:01:03 -0400 Received: from cichlid.raleigh.ibm.com (sig-9-49-155-68.mts.ibm.com [9.49.155.68]) by d01av01.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id p95K0u5x025118 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 5 Oct 2011 16:00:57 -0400 Received: from cichlid.raleigh.ibm.com (cichlid.raleigh.ibm.com [127.0.0.1]) by cichlid.raleigh.ibm.com (8.14.4/8.12.5) with ESMTP id p95K0sCT011047; Wed, 5 Oct 2011 16:00:55 -0400 Message-Id: <201110052000.p95K0sCT011047@cichlid.raleigh.ibm.com> To: shares@ndzh.com In-reply-to: <589039012-1317823289-cardhu_decombobulator_blackberry.rim.net-164474641-@b18.c6.bise6.blackberry> References: <4A95BA014132FF49AE685FAB4B9F17F61209A7F3@dfweml506-mbx><201110041609.p94G9SVh019333@cichlid.raleigh.ibm.com> <589039012-1317823289-cardhu_decombobulator_blackberry.rim.net-164474641-@b18.c6.bise6.blackberry> Comments: In-reply-to shares@ndzh.com message dated "Wed, 05 Oct 2011 14:01:25 -0000." Date: Wed, 05 Oct 2011 16:00:54 -0400 From: Thomas Narten Cc: "armd@ietf.org" , armd-bounces@ietf.org Subject: Re: [armd] Call for Participation: Using IP Overlays to provide L2Virtualization X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 19:59:51 -0000 Sue, There is a third layer/level of addressing. The addresses a VM uses internal to the overlay are private (or can be, but that is not required). If the VM needs to communicate with the outside world, it will need a public address. This can be done via standard NAT (as is done today, if the VM is not using the public address directly). I would expect the NAT translation to be done at the edge of the overlay, i.e., as part of being decapsulated in order to exit the overlay. I.e., if a packet exits an overlay network, the addresses it uses outside of the overlay needs to have meaning outside of the overlay. If the VM address (as used within the overlay) is a globally unique public address, there is nothing to do. If it is a private address, you'd need to NAT it. The bigger point is that when a packet leaves on overlay, its a normal IP packet (no longer encapsulated). Whatever techniques are used currently with IP can then still be used outside of the overlay. Make sense? Thomas > Thomas: > Linda's comment about a applications which communicate outside the data center indicates that the address outside the data center are likely not to share the IP addresses within the data center. > This is a scenario we have heard from Igor at yahoo. It is a scenario I have seen in some goggle slides at nanog. > External addresses in these scenarios need to be resolved. > Does this make sense? > If so, we can go on to dc connected by by mpls le vpns. > Sue > Sent via BlackBerry by AT&T > -----Original Message----- > From: Thomas Narten > Sender: armd-bounces@ietf.org > Date: Tue, 04 Oct 2011 12:09:28 > To: Linda Dunbar > Cc: armd@ietf.org > Subject: Re: [armd] Call for Participation: Using IP Overlays to provide L2 > Virtualization > Hi Linda. > > I agree that "Overlay push some of the L2 scaling concerns" in your > > draft. But this statement should not be listed under the sub-section > > of ARMD. > I will tweak this in the next version. > > Overlay is to hide VMs' addresses from network interior > > nodes. > Agreed. > > Address Resolution is to map VMs' IP addresses to physical > > addresses. > Yes, but. Address resolution is also a sort of generic term and can > mean more than this. Anytime you have to map an "upper layer" address > to a "lower layer" address, you need address resolution or address > mapping. We usually think of ARP/ND doing address resolution between > an IP address and a physical (ethernet) address. > But, with overlays, you will have multiple layers doing address > mapping. In the overlay context, you will have the VM's IP address, > which needs to be mapped into the infrastructure IP address where that > VM current resides (and where packets can be tunneled to). That is > something that is handled at the overlay level. Separately, at some > point, a router/gateway will need to map that infrastructure IP > address into a physical address (just as is done today). I assume this > is what you mean by "address resolution". > The "mapping function" that the problem statement draft uses is a form > of Address Resolution. > > Applications (or VMs) within Data center need to communicate with > > peers outside data center. Regardless of overlay is deployed within > > the data center or not, the Gateway routers still need to resolve > > physical addresses for all VMs which have external communications. > I'm not sure I understand this fully (see above). There are two > IP addresses in use (private within the overlay and > infrastructure). Which IP addresses do you mean above? > > Therefore, Address resolution and Overlay are orthogonal to each > > other. > If you mean address resolution as in the ARP/ND sense vs. "mappings" > in the overlay sense, I agree. > > Overlay can only make address resolution more complex, > > because Gateway router not only needs to resolve physical address > > for each VM's IP, it also needs to resolve the overlay edge address > > for each VM's IP. > Yes. If the gateway is at the edge of the overlay, it will need to > encap/decap packets and perform address mappings. Separately, it will > need to do address resolution for infrastructure addresses. > Thomas > _______________________________________________ > armd mailing list > armd@ietf.org > https://www.ietf.org/mailman/listinfo/armd From vishwas.ietf@gmail.com Wed Oct 5 13:11:34 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6688C11E80C4 for ; Wed, 5 Oct 2011 13:11:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.288 X-Spam-Level: X-Spam-Status: No, score=-3.288 tagged_above=-999 required=5 tests=[AWL=0.310, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2JL3hinopDEE for ; Wed, 5 Oct 2011 13:11:33 -0700 (PDT) Received: from mail-qy0-f179.google.com (mail-qy0-f179.google.com [209.85.216.179]) by ietfa.amsl.com (Postfix) with ESMTP id 8579D11E80BB for ; Wed, 5 Oct 2011 13:11:33 -0700 (PDT) Received: by qyk33 with SMTP id 33so1797703qyk.10 for ; Wed, 05 Oct 2011 13:14:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=XjkuRbxynTZEv3xtRAzWoKr/AP1xypMX4r66HJhaSFU=; b=HWCUOxgFu0L0abQQLp7wqpwW8e+Gf78citoVJ/mOksxqjFLfgZOX0oaJfzu2OSKeMO KZBoL56fuTHCYeQxcqNYLemKt7Tnqp5XpXdFJcBOS4xvN0wy3nzH4JrQ7g5PHsPXDabl JDBxUd+PSIK8P1I+sFcGY+XTeZcC1FwZcGaXI= MIME-Version: 1.0 Received: by 10.229.65.86 with SMTP id h22mr2290538qci.225.1317845681720; Wed, 05 Oct 2011 13:14:41 -0700 (PDT) Received: by 10.229.91.131 with HTTP; Wed, 5 Oct 2011 13:14:41 -0700 (PDT) In-Reply-To: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx> References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx> Date: Wed, 5 Oct 2011 13:14:41 -0700 Message-ID: From: Vishwas Manral To: Linda Dunbar Content-Type: multipart/alternative; boundary=0016e64cbc02ba4f1204ae92db41 Cc: "armd@ietf.org" Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 20:11:34 -0000 --0016e64cbc02ba4f1204ae92db41 Content-Type: text/plain; charset=ISO-8859-1 Linda, I am unsure what you mean by Gateway router, but there is always a Hypervisor or the first hop router that does the encapsulation/ decapsulation. Thanks, Vishwas On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar wrote: > Vishwas, **** > > ** ** > > Are you assuming that each group of tenants are connected to Gateway router > via completely different physical ports? If yes, then you are absolutely > correct. **** > > ** ** > > Linda**** > > ** ** > > *From:* armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] *On Behalf Of > *Vishwas Manral > *Sent:* Wednesday, October 05, 2011 10:10 AM > *To:* armd@ietf.org > *Subject:* [armd] Multi-Tenancy without changes**** > > ** ** > > Hi,**** > > **** > > I was thinking of multi-tenancy and the way to achieve > 4094 without any > changes in any layer by just using VLAN's. This is achieved as we already > have a mapping layer already in place.**** > > **** > > So assume there are 8000 tenants. We can divide them in groups of 4000 each > and assign each tenant a particular id from say 2 to 4001. Now as long as we > make sure we do not have tenants from 2 different groups on the same > machine, there are just no issues. Only the mapping layer needs to be aware > of the group a machine belongs to. There is thus no change in dataplane.** > ** > > **** > > Any comments?**** > > **** > > Thanks,**** > > Vishwas**** > --0016e64cbc02ba4f1204ae92db41 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Linda,
=A0
I am unsure what you mean by Gateway router, but there is always a Hyp= ervisor or the first hop router that does the encapsulation/ decapsulation.=
=A0
Thanks,
Vishwas
On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar <linda.dunbar= @huawei.com> wrote:

Vish= was,

<= /u>=A0

Are = you assuming that each group of tenants are connected to Gateway router via= completely different physical ports? If yes, then you are absolutely corre= ct.

<= /u>=A0

Lind= a

<= /u>=A0

From:<= span style=3D"FONT-SIZE: 10pt"> armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf O= f Vishwas Manral
Sent: Wednesday, October 05, 2011 10:10 AM
To: armd@ietf.org
Subject: [armd] Multi-Tenancy without changes

=A0

Hi,

=A0

I was thinking of multi-tenancy and the way to achie= ve > 4094 without any changes in any layer by just using VLAN's. Thi= s is achieved as we already have a mapping layer already in place.

=A0

So assume there are 8000 tenants.=A0We can divide th= em in groups of 4000 each and assign each tenant a particular id from say 2= to 4001. Now as long as we make sure we do not have tenants from 2 differe= nt groups on the same machine, there are just no issues. Only the mapping l= ayer needs to be aware of the group a machine belongs to. There is thus no = change in dataplane.

=A0

Any comments?

=A0

Thanks,

Vishwas


--0016e64cbc02ba4f1204ae92db41-- From linda.dunbar@huawei.com Wed Oct 5 13:22:04 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 877171F0C62 for ; Wed, 5 Oct 2011 13:22:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.423 X-Spam-Level: X-Spam-Status: No, score=-6.423 tagged_above=-999 required=5 tests=[AWL=0.175, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2JeU2wLugylJ for ; Wed, 5 Oct 2011 13:22:02 -0700 (PDT) Received: from usaga02-in.huawei.com (usaga02-in.huawei.com [206.16.17.70]) by ietfa.amsl.com (Postfix) with ESMTP id 110D021F8B77 for ; Wed, 5 Oct 2011 13:22:02 -0700 (PDT) Received: from huawei.com (localhost [127.0.0.1]) by usaga02-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LSM00HJE0PXJ1@usaga02-in.huawei.com> for armd@ietf.org; Wed, 05 Oct 2011 15:25:10 -0500 (CDT) Received: from dfweml201-edg.china.huawei.com ([172.18.4.104]) by usaga02-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTPS id <0LSM00FYD0PXH0@usaga02-in.huawei.com> for armd@ietf.org; Wed, 05 Oct 2011 15:25:09 -0500 (CDT) Received: from DFWEML404-HUB.china.huawei.com (10.193.5.203) by dfweml201-edg.china.huawei.com (172.18.9.107) with Microsoft SMTP Server (TLS) id 14.1.270.1; Wed, 05 Oct 2011 13:25:04 -0700 Received: from DFWEML506-MBX.china.huawei.com ([10.124.31.111]) by dfweml404-hub.china.huawei.com ([10.193.5.203]) with mapi id 14.01.0270.001; Wed, 05 Oct 2011 13:25:01 -0700 Date: Wed, 05 Oct 2011 20:25:01 +0000 From: Linda Dunbar In-reply-to: X-Originating-IP: [10.192.11.155] To: Vishwas Manral Message-id: <4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_bIXPVNJXZoSMQCmjBO4tzg)" Content-language: en-US Accept-Language: en-US, zh-CN Thread-topic: [armd] Multi-Tenancy without changes Thread-index: AQHMg3D0j3Ia6kdngkOCCwlMzdnQoJVuG8PAgACJiID//4wQgA== X-MS-Has-Attach: X-MS-TNEF-Correlator: References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx> Cc: "armd@ietf.org" Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 20:22:04 -0000 --Boundary_(ID_bIXPVNJXZoSMQCmjBO4tzg) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT I mean "the first hop router". Each physical port on "the first hop router" can have its own 4095 VLANs. Each Hypervisor can also have its own 4095 VLANs if Hypervisor does the IP encapsulation with a key in the header to differentiate clients (e.g. GRE's KEY field, or VxLan's Client ID field). You can make VLAN locally significant at the overlay edge node if the Overlay Encapsulation header has a field to further differentiate the Clients. Linda From: Vishwas Manral [mailto:vishwas.ietf@gmail.com] Sent: Wednesday, October 05, 2011 3:15 PM To: Linda Dunbar Cc: armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes Linda, I am unsure what you mean by Gateway router, but there is always a Hypervisor or the first hop router that does the encapsulation/ decapsulation. Thanks, Vishwas On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar > wrote: Vishwas, Are you assuming that each group of tenants are connected to Gateway router via completely different physical ports? If yes, then you are absolutely correct. Linda From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas Manral Sent: Wednesday, October 05, 2011 10:10 AM To: armd@ietf.org Subject: [armd] Multi-Tenancy without changes Hi, I was thinking of multi-tenancy and the way to achieve > 4094 without any changes in any layer by just using VLAN's. This is achieved as we already have a mapping layer already in place. So assume there are 8000 tenants. We can divide them in groups of 4000 each and assign each tenant a particular id from say 2 to 4001. Now as long as we make sure we do not have tenants from 2 different groups on the same machine, there are just no issues. Only the mapping layer needs to be aware of the group a machine belongs to. There is thus no change in dataplane. Any comments? Thanks, Vishwas --Boundary_(ID_bIXPVNJXZoSMQCmjBO4tzg) Content-type: text/html; charset=us-ascii Content-transfer-encoding: 7BIT

I mean “the first hop router”. Each physical port on “the first hop router” can have its own 4095 VLANs. Each Hypervisor can also have its own 4095 VLANs if Hypervisor does the IP encapsulation with a key in the header to differentiate clients (e.g. GRE’s KEY field, or VxLan’s Client ID field).

 

You can make VLAN locally significant at the overlay edge node if the Overlay Encapsulation header has a field to further differentiate the Clients.

 

Linda

 

From: Vishwas Manral [mailto:vishwas.ietf@gmail.com]
Sent: Wednesday, October 05, 2011 3:15 PM
To: Linda Dunbar
Cc: armd@ietf.org
Subject: Re: [armd] Multi-Tenancy without changes

 

Linda,

 

I am unsure what you mean by Gateway router, but there is always a Hypervisor or the first hop router that does the encapsulation/ decapsulation.

 

Thanks,

Vishwas

On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar <linda.dunbar@huawei.com> wrote:

Vishwas,

 

Are you assuming that each group of tenants are connected to Gateway router via completely different physical ports? If yes, then you are absolutely correct.

 

Linda

 

From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas Manral
Sent: Wednesday, October 05, 2011 10:10 AM
To: armd@ietf.org
Subject: [armd] Multi-Tenancy without changes

 

Hi,

 

I was thinking of multi-tenancy and the way to achieve > 4094 without any changes in any layer by just using VLAN's. This is achieved as we already have a mapping layer already in place.

 

So assume there are 8000 tenants. We can divide them in groups of 4000 each and assign each tenant a particular id from say 2 to 4001. Now as long as we make sure we do not have tenants from 2 different groups on the same machine, there are just no issues. Only the mapping layer needs to be aware of the group a machine belongs to. There is thus no change in dataplane.

 

Any comments?

 

Thanks,

Vishwas

 

--Boundary_(ID_bIXPVNJXZoSMQCmjBO4tzg)-- From vishwas.ietf@gmail.com Wed Oct 5 13:32:21 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DD34F21F8C44 for ; Wed, 5 Oct 2011 13:32:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.295 X-Spam-Level: X-Spam-Status: No, score=-3.295 tagged_above=-999 required=5 tests=[AWL=0.303, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uTqvPmAjkRiE for ; Wed, 5 Oct 2011 13:32:21 -0700 (PDT) Received: from mail-qy0-f172.google.com (mail-qy0-f172.google.com [209.85.216.172]) by ietfa.amsl.com (Postfix) with ESMTP id 0089321F8C42 for ; Wed, 5 Oct 2011 13:32:20 -0700 (PDT) Received: by qyk32 with SMTP id 32so4236593qyk.10 for ; Wed, 05 Oct 2011 13:35:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=yJIyooiHAFaLLWfhTfqkFCLWjK+1u9z4YCy7xTsV19Q=; b=oAlOZmkVZ6nxJdwvIYg+793ttVov7lLdJW7J/9ftj5aLoZss6fcX74yhT7p/1EBtTC cAV5jx0kDU1n3/ZWH5aD4SPDOpZUY/JbeOhiF24sJVUsn81HB6aU2XFu/uXJl/3a7mSz dEvlnl8vwcYm9DAQD0siUKiLs4/Pjkh11FG6o= MIME-Version: 1.0 Received: by 10.229.37.84 with SMTP id w20mr2296115qcd.76.1317846929387; Wed, 05 Oct 2011 13:35:29 -0700 (PDT) Received: by 10.229.91.131 with HTTP; Wed, 5 Oct 2011 13:35:29 -0700 (PDT) In-Reply-To: <4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx> References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx> <4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx> Date: Wed, 5 Oct 2011 13:35:29 -0700 Message-ID: From: Vishwas Manral To: Linda Dunbar Content-Type: multipart/alternative; boundary=00163649a7271831ae04ae93262f Cc: "armd@ietf.org" Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 20:32:22 -0000 --00163649a7271831ae04ae93262f Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Linda, Thats is a very complicated way of thinking things, where you have multiple layers. A simpler solution is to just assume each hypervisor belongs to a particula= r group. So it will communicate with only members of the same group. The mapping layer/ IP has a unique IP address (not group specific). You have to understand in the cloud you have an orchesrtator so there is absolute flexibility in this approach, though there is some additional load on the orchestrator to know groups + VLAN's to identify a tenant and ofcourse managing the mappings to best use the physical infrastructure. Thanks, Vishwas On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar wrote= : > I mean =93the first hop router=94. Each physical port on =93the first ho= p > router=94 can have its own 4095 VLANs. Each Hypervisor can also have its = own > 4095 VLANs if Hypervisor does the IP encapsulation with a key in the head= er > to differentiate clients (e.g. GRE=92s KEY field, or VxLan=92s Client ID = field). > **** > > ** ** > > You can make VLAN locally significant at the overlay edge node if the > Overlay Encapsulation header has a field to further differentiate the > Clients. **** > > ** ** > > Linda**** > > ** ** > > *From:* Vishwas Manral [mailto:vishwas.ietf@gmail.com] > *Sent:* Wednesday, October 05, 2011 3:15 PM > *To:* Linda Dunbar > *Cc:* armd@ietf.org > *Subject:* Re: [armd] Multi-Tenancy without changes**** > > ** ** > > Linda,**** > > **** > > I am unsure what you mean by Gateway router, but there is always a > Hypervisor or the first hop router that does the encapsulation/ > decapsulation.**** > > **** > > Thanks,**** > > Vishwas**** > > On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar > wrote:**** > > Vishwas, **** > > **** > > Are you assuming that each group of tenants are connected to Gateway rout= er > via completely different physical ports? If yes, then you are absolutely > correct. **** > > **** > > Linda**** > > **** > > *From:* armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] *On Behalf O= f > *Vishwas Manral > *Sent:* Wednesday, October 05, 2011 10:10 AM > *To:* armd@ietf.org > *Subject:* [armd] Multi-Tenancy without changes**** > > **** > > Hi,**** > > **** > > I was thinking of multi-tenancy and the way to achieve > 4094 without any > changes in any layer by just using VLAN's. This is achieved as we already > have a mapping layer already in place.**** > > **** > > So assume there are 8000 tenants. We can divide them in groups of 4000 ea= ch > and assign each tenant a particular id from say 2 to 4001. Now as long as= we > make sure we do not have tenants from 2 different groups on the same > machine, there are just no issues. Only the mapping layer needs to be awa= re > of the group a machine belongs to. There is thus no change in dataplane.*= * > ** > > **** > > Any comments?**** > > **** > > Thanks,**** > > Vishwas**** > > ** ** > --00163649a7271831ae04ae93262f Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Linda,
=A0
Thats is a very complicated way of thinking things, where you have mul= tiple layers.
=A0
A simpler solution is to just assume each hypervisor belongs to a part= icular group. So it will communicate with only members of the same group.= =A0 The mapping layer/ IP has a unique IP address (not group specific).
=A0
You have to understand in the cloud you have an orchesrtator so there = is absolute flexibility in this approach, though there is some additional l= oad on the orchestrator to know groups +=A0VLAN's to identify a tenant = and ofcourse managing the mappings to best use the physical infrastructure.=
=A0
Thanks,
Vishwas
On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar <linda.dunbar@= huawei.com> wrote:

I me= an =93the first hop router=94. Each physical port on =93the first hop route= r=94 can have its own 4095 VLANs. Each Hypervisor can also have its own 409= 5 VLANs if Hypervisor does the IP encapsulation with a key in the header to= differentiate clients (e.g. GRE=92s KEY field, or VxLan=92s Client ID fiel= d).

<= /u>=A0

You = can make VLAN locally significant at the overlay edge node if the Overlay E= ncapsulation header has a field to further differentiate the Clients.

<= /u>=A0

Lind= a

<= /u>=A0

From:<= span style=3D"FONT-SIZE: 10pt"> Vishwas Manral [mailto:vishwas.ietf@gmail.com]
= Sent: Wednesday, October 05, 2011 3:15 PM
To: Linda Dunbar
Cc: armd@ietf.org
Subject: Re: [armd] Multi-Tenancy= without changes

=A0

Linda,

=A0

I am unsure what you mean by Gateway router, but the= re is always a Hypervisor or the first hop router that does the encapsulati= on/ decapsulation.

=A0

Thanks,

Vishwas

On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar <linda.dunbar@hua= wei.com> wrote:

Vish= was,

=A0<= /span>

Are = you assuming that each group of tenants are connected to Gateway router via= completely different physical ports? If yes, then you are absolutely corre= ct.

=A0<= /span>

Lind= a

=A0<= /span>

From:<= span style=3D"FONT-SIZE: 10pt"> armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf O= f Vishwas Manral
Sent: Wednesday, October 05, 2011 10:10 AM
To: armd@ietf.org
Subject: [armd] Multi-Tenancy without changes

=A0

Hi,

=A0

I was thinking of multi-tenancy and the way to achie= ve > 4094 without any changes in any layer by just using VLAN's. Thi= s is achieved as we already have a mapping layer already in place.

=A0

So assume there are 8000 tenants.=A0We can divide th= em in groups of 4000 each and assign each tenant a particular id from say 2= to 4001. Now as long as we make sure we do not have tenants from 2 differe= nt groups on the same machine, there are just no issues. Only the mapping l= ayer needs to be aware of the group a machine belongs to. There is thus no = change in dataplane.

=A0

Any comments?

=A0

Thanks,

Vishwas

=A0

<= /blockquote>

--00163649a7271831ae04ae93262f-- From bschlies@cisco.com Wed Oct 5 14:12:11 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 51D4421F8AE6 for ; Wed, 5 Oct 2011 14:12:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[AWL=-0.001, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k6IR1jzMHMqT for ; Wed, 5 Oct 2011 14:12:10 -0700 (PDT) Received: from mtv-iport-3.cisco.com (mtv-iport-3.cisco.com [173.36.130.14]) by ietfa.amsl.com (Postfix) with ESMTP id 5E11621F8ABD for ; Wed, 5 Oct 2011 14:12:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=bschlies@cisco.com; l=13951; q=dns/txt; s=iport; t=1317849319; x=1319058919; h=subject:mime-version:from:in-reply-to:date:cc:message-id: references:to; bh=VcHbOsMqwHEilAPfrw9Vx+lawoYRi6ctHv/q7rc+ltU=; b=jwKk2C0cUaG50Lntm9nh8MB5soO3J5qMLC/ea5qSHMeOvqBoGONqWvhV vFRuNdLdslBRoeJSVsvAlOtdmZndo27gIVt/pdOajb3C/4MZhLC8qhYwc sOJlxky771bEsmBAnbCcJ0/X0tBzuoGCfiXV4gMSSW1LN5VLmVMv58Tyv U=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUAAOHHjE6rRDoI/2dsb2JhbAA4CoJNljmPFIEFgVMBAQEBAgEBAQEPAVsLBQsLEQQBAQEnByEGHwkIBhMbB4dbBphgAZ4RA4N8gkxhBId8i3GFJ4R5h0I X-IronPort-AV: E=Sophos;i="4.68,493,1312156800"; d="scan'208,217";a="6170234" Received: from mtv-core-3.cisco.com ([171.68.58.8]) by mtv-iport-3.cisco.com with ESMTP; 05 Oct 2011 21:15:19 +0000 Received: from [192.168.0.133] (sjc-vpn2-937.cisco.com [10.21.115.169]) by mtv-core-3.cisco.com (8.14.3/8.14.3) with ESMTP id p95LFIcq022854; Wed, 5 Oct 2011 21:15:18 GMT Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: multipart/alternative; boundary=Apple-Mail-6-239701780 From: Benson Schliesser In-Reply-To: Date: Wed, 5 Oct 2011 16:15:18 -0500 Message-Id: References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx> <4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx> To: Vishwas Manral X-Mailer: Apple Mail (2.1084) Cc: "armd@ietf.org" Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 21:12:11 -0000 --Apple-Mail-6-239701780 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Hi, Vishwas - Yes, what you describe is possible. But the issue is that it only works = if we "do not have tenants from 2 different groups on the same machine". = Keep in mind that this applies to all "machines" in the network. In = addition to the hypervisor, all of the switch paths and router = interfaces must also be segregated. The only way this works is if we partition the hardware into = non-overlapping topologies, such that different tenants with a = conflicting VLAN ID aren't bridged together. And, as Linda pointed out, = each partition must have a different physical interface to the L3 = gateway. (E.g. by having different routers, or discrete interfaces on a = shared router) Effectively, we do this today when we build multiple datacenters = (including when we build multiple "logical" datacenters inside the same = physical building). We could pursue a more complicated approach where = an arbitrary number of topologies are overlaid on a common physical = network, constrained by the number of physical paths etc, but that would = be an operational nightmare. Cheers, -Benson On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote: > Linda, > =20 > Thats is a very complicated way of thinking things, where you have = multiple layers. > =20 > A simpler solution is to just assume each hypervisor belongs to a = particular group. So it will communicate with only members of the same = group. The mapping layer/ IP has a unique IP address (not group = specific). > =20 > You have to understand in the cloud you have an orchesrtator so there = is absolute flexibility in this approach, though there is some = additional load on the orchestrator to know groups + VLAN's to identify = a tenant and ofcourse managing the mappings to best use the physical = infrastructure. > =20 > Thanks, > Vishwas > On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar = wrote: > I mean =93the first hop router=94. Each physical port on =93the first = hop router=94 can have its own 4095 VLANs. Each Hypervisor can also have = its own 4095 VLANs if Hypervisor does the IP encapsulation with a key in = the header to differentiate clients (e.g. GRE=92s KEY field, or VxLan=92s = Client ID field). >=20 > =20 >=20 > You can make VLAN locally significant at the overlay edge node if the = Overlay Encapsulation header has a field to further differentiate the = Clients. >=20 > =20 >=20 > Linda >=20 > =20 >=20 > From: Vishwas Manral [mailto:vishwas.ietf@gmail.com]=20 > Sent: Wednesday, October 05, 2011 3:15 PM > To: Linda Dunbar > Cc: armd@ietf.org > Subject: Re: [armd] Multi-Tenancy without changes >=20 > =20 >=20 > Linda, >=20 > =20 >=20 > I am unsure what you mean by Gateway router, but there is always a = Hypervisor or the first hop router that does the encapsulation/ = decapsulation. >=20 > =20 >=20 > Thanks, >=20 > Vishwas >=20 > On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar = wrote: >=20 > Vishwas, >=20 > =20 >=20 > Are you assuming that each group of tenants are connected to Gateway = router via completely different physical ports? If yes, then you are = absolutely correct. >=20 > =20 >=20 > Linda >=20 > =20 >=20 > From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf = Of Vishwas Manral > Sent: Wednesday, October 05, 2011 10:10 AM > To: armd@ietf.org > Subject: [armd] Multi-Tenancy without changes >=20 > =20 >=20 > Hi, >=20 > =20 >=20 > I was thinking of multi-tenancy and the way to achieve > 4094 without = any changes in any layer by just using VLAN's. This is achieved as we = already have a mapping layer already in place. >=20 > =20 >=20 > So assume there are 8000 tenants. We can divide them in groups of 4000 = each and assign each tenant a particular id from say 2 to 4001. Now as = long as we make sure we do not have tenants from 2 different groups on = the same machine, there are just no issues. Only the mapping layer needs = to be aware of the group a machine belongs to. There is thus no change = in dataplane. >=20 > =20 >=20 > Any comments? >=20 > =20 >=20 > Thanks, >=20 > Vishwas >=20 > =20 >=20 >=20 > _______________________________________________ > armd mailing list > armd@ietf.org > https://www.ietf.org/mailman/listinfo/armd --Apple-Mail-6-239701780 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 Hi, = Vishwas -

Yes, what you describe is possible. =  But the issue is that it only works if we "do not have tenants = from 2 different groups on the same machine".  Keep in mind that = this applies to all "machines" in the network.  In addition to the = hypervisor, all of the switch paths and router interfaces must also be = segregated.

The only way this works is if we = partition the hardware into non-overlapping topologies, such that = different tenants with a conflicting VLAN ID aren't bridged together. =  And, as Linda pointed out, each partition must have a different = physical interface to the L3 gateway. (E.g. by having different routers, = or discrete interfaces on a shared = router)

Effectively, we do this today when we = build multiple datacenters (including when we build multiple "logical" = datacenters inside the same physical building).  We could pursue a = more complicated approach where an arbitrary number of topologies are = overlaid on a common physical network, constrained by the number of = physical paths etc, but that would be an operational = nightmare.

Cheers,
-Benson

On Oct 5, 2011, at 3:35 PM, Vishwas Manral = wrote:

Linda,
 
Thats is a very complicated way of thinking things, where you have = multiple layers.
 
A simpler solution is to just assume each hypervisor belongs to a = particular group. So it will communicate with only members of the same = group.  The mapping layer/ IP has a unique IP address (not group = specific).
 
You have to understand in the cloud you have an orchesrtator so = there is absolute flexibility in this approach, though there is some = additional load on the orchestrator to know groups + VLAN's to = identify a tenant and ofcourse managing the mappings to best use the = physical infrastructure.
 
Thanks,
Vishwas
On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar = <linda.dunbar@huawei.com> wrote:

I mean =93the first hop router=94. Each physical port on =93the = first hop router=94 can have its own 4095 VLANs. Each Hypervisor can = also have its own 4095 VLANs if Hypervisor does the IP encapsulation = with a key in the header to differentiate clients (e.g. GRE=92s KEY = field, or VxLan=92s Client ID field).

 

You can make VLAN locally = significant at the overlay edge node if the Overlay Encapsulation header = has a field to further differentiate the Clients. =

 

Linda

 

From: Vishwas Manral = [mailto:vishwas.ietf@gmail.com]
Sent: = Wednesday, October 05, 2011 3:15 PM
To: Linda Dunbar
Cc: armd@ietf.org
Subject: Re: [armd] = Multi-Tenancy without changes

 

Linda,

 

I am unsure what you mean by Gateway router, = but there is always a Hypervisor or the first hop router that does the = encapsulation/ decapsulation.

 

Thanks,

Vishwas

On Wed, Oct 5, 2011 at 12:06 PM, Linda = Dunbar <linda.dunbar@huawei.com> = wrote:

Vishwas,

 

Are you assuming that each = group of tenants are connected to Gateway router via completely = different physical ports? If yes, then you are absolutely correct. =

 

Linda

 

From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas = Manral
Sent: Wednesday, October 05, 2011 10:10 AM
To: armd@ietf.org
Subject: [armd] = Multi-Tenancy without changes

 

Hi,

 

I was thinking of multi-tenancy and the way = to achieve > 4094 without any changes in any layer by just using = VLAN's. This is achieved as we already have a mapping layer already in = place.

 

So assume there are 8000 tenants. We = can divide them in groups of 4000 each and assign each tenant a = particular id from say 2 to 4001. Now as long as we make sure we do not = have tenants from 2 different groups on the same machine, there are just = no issues. Only the mapping layer needs to be aware of the group a = machine belongs to. There is thus no change in = dataplane.

 

Any comments?

 

Thanks,

Vishwas

 

=

_______________________________________________
armd mailing = list
armd@ietf.org
https://www.ietf.org/ma= ilman/listinfo/armd

= --Apple-Mail-6-239701780-- From vishwas.ietf@gmail.com Wed Oct 5 14:17:59 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 789121F0C77 for ; Wed, 5 Oct 2011 14:17:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.302 X-Spam-Level: X-Spam-Status: No, score=-3.302 tagged_above=-999 required=5 tests=[AWL=0.296, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hdNjLXFiQNK7 for ; Wed, 5 Oct 2011 14:17:58 -0700 (PDT) Received: from mail-qy0-f172.google.com (mail-qy0-f172.google.com [209.85.216.172]) by ietfa.amsl.com (Postfix) with ESMTP id 3FCE21F0C5F for ; Wed, 5 Oct 2011 14:17:58 -0700 (PDT) Received: by qyk32 with SMTP id 32so4277695qyk.10 for ; Wed, 05 Oct 2011 14:21:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=vTAvWQbfdEYcaje3NB5EFWSFDXl2yTNk/fu7fhOfy6I=; b=heW1JnloaaJ9RwO0XF0Hj2BjzxiimVZF7ZXjeWliO9TyLaPJ3NDKhSnAmyiWse3AO7 Fjm9B7sVKjVGwgE2OyFLigiy9PXNdFQrygWneSPHgeT0YKFpDVEYgGulh5ZvImxobZi3 iGc1XRaWAIJ7skq0jt94rh9A3wwG+rLq4kLTs= MIME-Version: 1.0 Received: by 10.229.196.233 with SMTP id eh41mr2342925qcb.187.1317849666802; Wed, 05 Oct 2011 14:21:06 -0700 (PDT) Received: by 10.229.91.131 with HTTP; Wed, 5 Oct 2011 14:21:06 -0700 (PDT) In-Reply-To: References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx> <4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx> Date: Wed, 5 Oct 2011 14:21:06 -0700 Message-ID: From: Vishwas Manral To: Benson Schliesser Content-Type: multipart/alternative; boundary=0016369cffb441d3b704ae93c9e2 Cc: "armd@ietf.org" Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 21:17:59 -0000 --0016369cffb441d3b704ae93c9e2 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Benson, You got the idea perfectly right. Here is the catch however. > The only way this works is if we partition the hardware into non-overlapping > topologies, such that different tenants with a conflicting VLAN ID aren't bridged > together. There is no hard seperation required, because mapping can be done through a control plane. As the mapping is all logical, the orchestration layer can d= o intelligent things here and change group bindings on a logical basis. So if I know group 1 has VLAN 1, 2 and 3 and Group 2 has 4, 5, 6 we can still use the same group and same hardware, till there is an overlap. Thanks, Vishwas On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser wrote= : > Hi, Vishwas - > > Yes, what you describe is possible. But the issue is that it only works = if > we "do not have tenants from 2 different groups on the same machine". Ke= ep > in mind that this applies to all "machines" in the network. In addition = to > the hypervisor, all of the switch paths and router interfaces must also b= e > segregated. > > The only way this works is if we partition the hardware into > non-overlapping topologies, such that different tenants with a conflictin= g > VLAN ID aren't bridged together. And, as Linda pointed out, each partiti= on > must have a different physical interface to the L3 gateway. (E.g. by havi= ng > different routers, or discrete interfaces on a shared router) > > Effectively, we do this today when we build multiple datacenters (includi= ng > when we build multiple "logical" datacenters inside the same physical > building). We could pursue a more complicated approach where an arbitrar= y > number of topologies are overlaid on a common physical network, constrain= ed > by the number of physical paths etc, but that would be an operational > nightmare. > > Cheers, > -Benson > > > On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote: > > Linda, > > Thats is a very complicated way of thinking things, where you have multip= le > layers. > > A simpler solution is to just assume each hypervisor belongs to a > particular group. So it will communicate with only members of the same > group. The mapping layer/ IP has a unique IP address (not group specific= ). > > You have to understand in the cloud you have an orchesrtator so there is > absolute flexibility in this approach, though there is some additional lo= ad > on the orchestrator to know groups + VLAN's to identify a tenant and > ofcourse managing the mappings to best use the physical infrastructure. > > Thanks, > Vishwas > On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar wro= te: > >> I mean =93the first hop router=94. Each physical port on =93the first h= op >> router=94 can have its own 4095 VLANs. Each Hypervisor can also have its= own >> 4095 VLANs if Hypervisor does the IP encapsulation with a key in the hea= der >> to differentiate clients (e.g. GRE=92s KEY field, or VxLan=92s Client ID= field). >> **** >> >> ** ** >> >> You can make VLAN locally significant at the overlay edge node if the >> Overlay Encapsulation header has a field to further differentiate the >> Clients. **** >> >> ** ** >> >> Linda**** >> >> ** ** >> >> *From:* Vishwas Manral [mailto:vishwas.ietf@gmail.com] >> *Sent:* Wednesday, October 05, 2011 3:15 PM >> *To:* Linda Dunbar >> *Cc:* armd@ietf.org >> *Subject:* Re: [armd] Multi-Tenancy without changes**** >> >> ** ** >> >> Linda,**** >> >> **** >> >> I am unsure what you mean by Gateway router, but there is always a >> Hypervisor or the first hop router that does the encapsulation/ >> decapsulation.**** >> >> **** >> >> Thanks,**** >> >> Vishwas**** >> >> On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar >> wrote:**** >> >> Vishwas, **** >> >> **** >> >> Are you assuming that each group of tenants are connected to Gateway >> router via completely different physical ports? If yes, then you are >> absolutely correct. **** >> >> **** >> >> Linda**** >> >> **** >> >> *From:* armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] *On Behalf >> Of *Vishwas Manral >> *Sent:* Wednesday, October 05, 2011 10:10 AM >> *To:* armd@ietf.org >> *Subject:* [armd] Multi-Tenancy without changes**** >> >> **** >> >> Hi,**** >> >> **** >> >> I was thinking of multi-tenancy and the way to achieve > 4094 without an= y >> changes in any layer by just using VLAN's. This is achieved as we alread= y >> have a mapping layer already in place.**** >> >> **** >> >> So assume there are 8000 tenants. We can divide them in groups of 4000 >> each and assign each tenant a particular id from say 2 to 4001. Now as l= ong >> as we make sure we do not have tenants from 2 different groups on the sa= me >> machine, there are just no issues. Only the mapping layer needs to be aw= are >> of the group a machine belongs to. There is thus no change in dataplane.= * >> *** >> >> **** >> >> Any comments?**** >> >> **** >> >> Thanks,**** >> >> Vishwas**** >> >> ** ** >> > > _______________________________________________ > armd mailing list > armd@ietf.org > https://www.ietf.org/mailman/listinfo/armd > > > --0016369cffb441d3b704ae93c9e2 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Hi Benson,
=A0
You got the idea perfectly right. Here is the catch however.
> The only way this works is if we partition the hardware into non-= overlapping
>=A0topologies, such that different tenants with a conflicting VLAN= ID aren't bridged
> together.
There is no hard seperation required, because mapping can be done thro= ugh a control plane. As the mapping is all logical, the orchestration layer= can do intelligent things here and change group bindings on a logical basi= s. So if I know group 1 has VLAN 1, 2 and 3 and Group 2 has 4, 5, 6 we can = still use the same group and same hardware, till there is an overlap.
=A0
Thanks,
Vishwas
On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesse= r <bschlies@cisc= o.com> wrote:
Hi, Vishwas -=20

Yes, what you describe is possible. =A0But the issue is that it only w= orks if we "do not have tenants from 2 different groups on the same ma= chine". =A0Keep in mind that this applies to all "machines" = in the network. =A0In addition to the hypervisor, all of the switch paths a= nd router interfaces must also be segregated.

The only way this works is if we partition the hardware into non-overl= apping topologies, such that different tenants with a conflicting VLAN ID a= ren't bridged together. =A0And, as Linda pointed out, each partition mu= st have a different physical interface to the L3 gateway. (E.g. by having d= ifferent routers, or discrete interfaces on a shared router)

Effectively, we do this today when we build multiple datacenters (incl= uding when we build multiple "logical" datacenters inside the sam= e physical building). =A0We could pursue a more complicated approach where = an arbitrary number of topologies are overlaid on a common physical network= , constrained by the number of physical paths etc, but that would be an ope= rational nightmare.

Cheers,
-Benson


On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote:

Linda,
=A0
Thats is a very complicated way of thinking things, where you have mul= tiple layers.
=A0
A simpler solution is to just assume each hypervisor belongs to a part= icular group. So it will communicate with only members of the same group.= =A0 The mapping layer/ IP has a unique IP address (not group specific).
=A0
You have to understand in the cloud you have an orchesrtator so there = is absolute flexibility in this approach, though there is some additional l= oad on the orchestrator to know groups +=A0VLAN's to identify a tenant = and ofcourse managing the mappings to best use the physical infrastructure.=
=A0
Thanks,
Vishwas
On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar <linda.dunbar@huawei.com> wrote:

I me= an =93the first hop router=94. Each physical port on =93the first hop route= r=94 can have its own 4095 VLANs. Each Hypervisor can also have its own 409= 5 VLANs if Hypervisor does the IP encapsulation with a key in the header to= differentiate clients (e.g. GRE=92s KEY field, or VxLan=92s Client ID fiel= d).

<= /u>=A0

You = can make VLAN locally significant at the overlay edge node if the Overlay E= ncapsulation header has a field to further differentiate the Clients.

<= /u>=A0

Lind= a

<= /u>=A0

From:<= span style=3D"FONT-SIZE: 10pt"> Vishwas Manral [mailto:vishwas.ietf@gmail.com]
= Sent: Wednesday, October 05, 2011 3:15 PM
To: Linda Dunbar
Cc: armd@ietf.org
Subject: Re: [armd] Multi-Tenancy= without changes

=A0

Linda,

=A0

I am unsure what you mean by Gateway router, but the= re is always a Hypervisor or the first hop router that does the encapsulati= on/ decapsulation.

=A0

Thanks,

Vishwas

On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar <linda.dunbar@hua= wei.com> wrote:

Vish= was,

=A0<= /span>

Are = you assuming that each group of tenants are connected to Gateway router via= completely different physical ports? If yes, then you are absolutely corre= ct.

=A0<= /span>

Lind= a

=A0<= /span>

From:<= span style=3D"FONT-SIZE: 10pt"> armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf O= f Vishwas Manral
Sent: Wednesday, October 05, 2011 10:10 AM
To: armd@ietf.org
Subject: [armd] Multi-Tenancy without changes

=A0

Hi,

=A0

I was thinking of multi-tenancy and the way to achie= ve > 4094 without any changes in any layer by just using VLAN's. Thi= s is achieved as we already have a mapping layer already in place.

=A0

So assume there are 8000 tenants.=A0We can divide th= em in groups of 4000 each and assign each tenant a particular id from say 2= to 4001. Now as long as we make sure we do not have tenants from 2 differe= nt groups on the same machine, there are just no issues. Only the mapping l= ayer needs to be aware of the group a machine belongs to. There is thus no = change in dataplane.

=A0

Any comments?

=A0

Thanks,

Vishwas

=A0

<= /blockquote>

_________________________________________= ______
armd mailing list
armd@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/armd


--0016369cffb441d3b704ae93c9e2-- From bschlies@cisco.com Wed Oct 5 15:28:34 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DBFCF11E80B7 for ; Wed, 5 Oct 2011 15:28:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gdXz8UMsQ4zl for ; Wed, 5 Oct 2011 15:28:33 -0700 (PDT) Received: from mtv-iport-1.cisco.com (mtv-iport-1.cisco.com [173.36.130.12]) by ietfa.amsl.com (Postfix) with ESMTP id 9E14C11E8094 for ; Wed, 5 Oct 2011 15:28:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=bschlies@cisco.com; l=19529; q=dns/txt; s=iport; t=1317853902; x=1319063502; h=subject:mime-version:from:in-reply-to:date:cc:message-id: references:to; bh=2ib9DUVYfRCtNwBELlYJsIw9Qg7kGTaR6s6X6CPFOrY=; b=BFeGsSmfrv/IWPGV02darAL7DuO2/JaPIMd2yoxtgbxCxe5bCLb+GbJs 73PcLgkbjvXWPdvxuQFxR2RWcKpUf3tORHa6NHdmRByT2Yb6AYVK6sfd8 ijqTyYY0bsgugxCGOSXtFtCrCURGrZUiOAUb7itZbTwKpFX/Qm0pHmzgH c=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUAAKLZjE6rRDoG/2dsb2JhbAA4CoJNljmPFIEFgVMBAQEBAgEBAQEPAVsLBQsLEQEDAQEBJwchBh8DBggGExsHh1sGmEwBng8Dg3yCTGEEh3yLcYUnhHmHQg X-IronPort-AV: E=Sophos;i="4.68,494,1312156800"; d="scan'208,217";a="6164375" Received: from mtv-core-1.cisco.com ([171.68.58.6]) by mtv-iport-1.cisco.com with ESMTP; 05 Oct 2011 22:31:42 +0000 Received: from [192.168.0.133] (sjc-vpn2-937.cisco.com [10.21.115.169]) by mtv-core-1.cisco.com (8.14.3/8.14.3) with ESMTP id p95MVf30002682; Wed, 5 Oct 2011 22:31:41 GMT Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: multipart/alternative; boundary=Apple-Mail-8-244284728 From: Benson Schliesser In-Reply-To: Date: Wed, 5 Oct 2011 17:31:40 -0500 Message-Id: <7384885B-E229-4C91-9C45-C3982F620E74@cisco.com> References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx> <4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx> To: Vishwas Manral X-Mailer: Apple Mail (2.1084) Cc: "armd@ietf.org" Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 22:28:35 -0000 --Apple-Mail-8-244284728 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Hi, Vishwas. On Oct 5, 2011, at 4:21 PM, Vishwas Manral wrote: > You got the idea perfectly right. Here is the catch however. > > The only way this works is if we partition the hardware into = non-overlapping > > topologies, such that different tenants with a conflicting VLAN ID = aren't bridged > > together. > There is no hard seperation required, because mapping can be done = through a control plane. As the mapping is all logical, the = orchestration layer can do intelligent things here and change group = bindings on a logical basis. So if I know group 1 has VLAN 1, 2 and 3 = and Group 2 has 4, 5, 6 we can still use the same group and same = hardware, till there is an overlap. Your proposal was a design "just using VLAN's" so that there is "no = change in dataplane". Thus I'm not sure what a "group" means, in the = context of your response, unless it means that the control plane = enforces some kind of hard separation. Note that I referred to separation of "switch paths" in my previous = message. In today's average datacenter, this requires hard separation of = switches (or partitions within the switch). But there are alternative = switch designs; let me try to imagine what you're describing. I suppose = one could build a switch that completely localizes VLAN ID on each = interface, has a very large internal namespace for tenants, and rewrites = VLAN ID as frames are forwarded from ingress to egress interface. = (Sounds a bit like a Frame Relay switch...) But even with this = architecture, we're limited to 4k VLANs on a given path - that = limitation is built into the frame format. Bringing this back to the ARMD scope: As I wrote last week, it doesn't = really matter how we segregate the tenants. As long as the mechanism = provides a L2 service, the underlying method of segregation is = orthogonal to ARP/ND address resolution. However, the fact that we = segregate tenants might allow us to distribute the address resolution = function more intelligently. In your approach, do you imagine = distributing L3 gateways and/or ARP proxy functions throughout each = topology / partition? Cheers, -Benson > On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser = wrote: > Hi, Vishwas - >=20 > Yes, what you describe is possible. But the issue is that it only = works if we "do not have tenants from 2 different groups on the same = machine". Keep in mind that this applies to all "machines" in the = network. In addition to the hypervisor, all of the switch paths and = router interfaces must also be segregated. >=20 > The only way this works is if we partition the hardware into = non-overlapping topologies, such that different tenants with a = conflicting VLAN ID aren't bridged together. And, as Linda pointed out, = each partition must have a different physical interface to the L3 = gateway. (E.g. by having different routers, or discrete interfaces on a = shared router) >=20 > Effectively, we do this today when we build multiple datacenters = (including when we build multiple "logical" datacenters inside the same = physical building). We could pursue a more complicated approach where = an arbitrary number of topologies are overlaid on a common physical = network, constrained by the number of physical paths etc, but that would = be an operational nightmare. >=20 > Cheers, > -Benson >=20 >=20 > On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote: >=20 >> Linda, >> =20 >> Thats is a very complicated way of thinking things, where you have = multiple layers. >> =20 >> A simpler solution is to just assume each hypervisor belongs to a = particular group. So it will communicate with only members of the same = group. The mapping layer/ IP has a unique IP address (not group = specific). >> =20 >> You have to understand in the cloud you have an orchesrtator so there = is absolute flexibility in this approach, though there is some = additional load on the orchestrator to know groups + VLAN's to identify = a tenant and ofcourse managing the mappings to best use the physical = infrastructure. >> =20 >> Thanks, >> Vishwas >> On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar = wrote: >> I mean =93the first hop router=94. Each physical port on =93the first = hop router=94 can have its own 4095 VLANs. Each Hypervisor can also have = its own 4095 VLANs if Hypervisor does the IP encapsulation with a key in = the header to differentiate clients (e.g. GRE=92s KEY field, or VxLan=92s = Client ID field). >>=20 >> =20 >>=20 >> You can make VLAN locally significant at the overlay edge node if the = Overlay Encapsulation header has a field to further differentiate the = Clients. >>=20 >> =20 >>=20 >> Linda >>=20 >> =20 >>=20 >> From: Vishwas Manral [mailto:vishwas.ietf@gmail.com]=20 >> Sent: Wednesday, October 05, 2011 3:15 PM >> To: Linda Dunbar >> Cc: armd@ietf.org >> Subject: Re: [armd] Multi-Tenancy without changes >>=20 >> =20 >>=20 >> Linda, >>=20 >> =20 >>=20 >> I am unsure what you mean by Gateway router, but there is always a = Hypervisor or the first hop router that does the encapsulation/ = decapsulation. >>=20 >> =20 >>=20 >> Thanks, >>=20 >> Vishwas >>=20 >> On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar = wrote: >>=20 >> Vishwas, >>=20 >> =20 >>=20 >> Are you assuming that each group of tenants are connected to Gateway = router via completely different physical ports? If yes, then you are = absolutely correct. >>=20 >> =20 >>=20 >> Linda >>=20 >> =20 >>=20 >> From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf = Of Vishwas Manral >> Sent: Wednesday, October 05, 2011 10:10 AM >> To: armd@ietf.org >> Subject: [armd] Multi-Tenancy without changes >>=20 >> =20 >>=20 >> Hi, >>=20 >> =20 >>=20 >> I was thinking of multi-tenancy and the way to achieve > 4094 without = any changes in any layer by just using VLAN's. This is achieved as we = already have a mapping layer already in place. >>=20 >> =20 >>=20 >> So assume there are 8000 tenants. We can divide them in groups of = 4000 each and assign each tenant a particular id from say 2 to 4001. Now = as long as we make sure we do not have tenants from 2 different groups = on the same machine, there are just no issues. Only the mapping layer = needs to be aware of the group a machine belongs to. There is thus no = change in dataplane. >>=20 >> =20 >>=20 >> Any comments? >>=20 >> =20 >>=20 >> Thanks, >>=20 >> Vishwas >>=20 >> =20 >>=20 >>=20 >> _______________________________________________ >> armd mailing list >> armd@ietf.org >> https://www.ietf.org/mailman/listinfo/armd >=20 >=20 --Apple-Mail-8-244284728 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 Hi, = Vishwas.

On Oct 5, 2011, at 4:21 PM, Vishwas Manral = wrote:

You got the idea perfectly right. Here is the catch = however.
> The only way this works is if we partition the hardware into = non-overlapping
> topologies, such that different tenants with a = conflicting VLAN ID aren't bridged
> together.
There is no hard seperation required, because mapping can be done = through a control plane. As the mapping is all logical, the = orchestration layer can do intelligent things here and change group = bindings on a logical basis. So if I know group 1 has VLAN 1, 2 and 3 = and Group 2 has 4, 5, 6 we can still use the same group and same = hardware, till there is an overlap.

Your proposal was a design "just using = VLAN's" so that there is "no change in dataplane". Thus I'm not sure = what a "group" means, in the context of your response, unless it means = that the control plane enforces some kind of hard = separation.

Note that I referred to separation = of "switch paths" in my previous message. In today's average datacenter, = this requires hard separation of switches (or partitions within the = switch). But there are alternative switch designs; let me try to imagine = what you're describing.  I suppose one could build a switch that = completely localizes VLAN ID on each interface, has a very large = internal namespace for tenants, and rewrites VLAN ID as frames are = forwarded from ingress to egress interface. (Sounds a bit like a = Frame Relay switch...)  But even with this architecture, we're = limited to 4k VLANs on a given path - that limitation is built into the = frame format.

Bringing this back to the ARMD = scope:  As I wrote last week, it doesn't really matter how we = segregate the tenants.  As long as the mechanism provides a L2 = service, the underlying method of segregation is orthogonal to ARP/ND = address resolution.  However, the fact that we segregate tenants = might allow us to distribute the address resolution function more = intelligently.  In your approach, do you imagine distributing L3 = gateways and/or ARP proxy functions throughout each topology / = partition?

Cheers,
-Benson

On Wed, Oct 5, 2011 at 2:15 = PM, Benson Schliesser <bschlies@cisco.com> = wrote:
Hi, Vishwas -=20

Yes, what you describe is possible.  But the issue is that it = only works if we "do not have tenants from 2 different groups on the = same machine".  Keep in mind that this applies to all "machines" in = the network.  In addition to the hypervisor, all of the switch = paths and router interfaces must also be segregated.

The only way this works is if we partition the hardware into = non-overlapping topologies, such that different tenants with a = conflicting VLAN ID aren't bridged together.  And, as Linda pointed = out, each partition must have a different physical interface to the L3 = gateway. (E.g. by having different routers, or discrete interfaces on a = shared router)

Effectively, we do this today when we build multiple datacenters = (including when we build multiple "logical" datacenters inside the same = physical building).  We could pursue a more complicated approach = where an arbitrary number of topologies are overlaid on a common = physical network, constrained by the number of physical paths etc, but = that would be an operational nightmare.

Cheers,
-Benson


On Oct 5, 2011, at 3:35 PM, Vishwas Manral = wrote:

Linda,
 
Thats is a very complicated way of thinking things, where you have = multiple layers.
 
A simpler solution is to just assume each hypervisor belongs to a = particular group. So it will communicate with only members of the same = group.  The mapping layer/ IP has a unique IP address (not group = specific).
 
You have to understand in the cloud you have an orchesrtator so = there is absolute flexibility in this approach, though there is some = additional load on the orchestrator to know groups + VLAN's to = identify a tenant and ofcourse managing the mappings to best use the = physical infrastructure.
 
Thanks,
Vishwas
On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar = <linda.dunbar@huawei.com> wrote:

I mean =93the first hop router=94. Each physical port on =93the = first hop router=94 can have its own 4095 VLANs. Each Hypervisor can = also have its own 4095 VLANs if Hypervisor does the IP encapsulation = with a key in the header to differentiate clients (e.g. GRE=92s KEY = field, or VxLan=92s Client ID field).

 

You can make VLAN locally = significant at the overlay edge node if the Overlay Encapsulation header = has a field to further differentiate the Clients. =

 

Linda

 

From: Vishwas Manral = [mailto:vishwas.ietf@gmail.com]
Sent: = Wednesday, October 05, 2011 3:15 PM
To: Linda Dunbar
Cc: armd@ietf.org
Subject: Re: [armd] = Multi-Tenancy without changes

 

Linda,

 

I am unsure what you mean by Gateway router, = but there is always a Hypervisor or the first hop router that does the = encapsulation/ decapsulation.

 

Thanks,

Vishwas

On Wed, Oct 5, 2011 at 12:06 PM, Linda = Dunbar <linda.dunbar@huawei.com> = wrote:

Vishwas,

 

Are you assuming that each = group of tenants are connected to Gateway router via completely = different physical ports? If yes, then you are absolutely correct. =

 

Linda

 

From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas = Manral
Sent: Wednesday, October 05, 2011 10:10 AM
To: armd@ietf.org
Subject: [armd] = Multi-Tenancy without changes

 

Hi,

 

I was thinking of multi-tenancy and the way = to achieve > 4094 without any changes in any layer by just using = VLAN's. This is achieved as we already have a mapping layer already in = place.

 

So assume there are 8000 tenants. We = can divide them in groups of 4000 each and assign each tenant a = particular id from say 2 to 4001. Now as long as we make sure we do not = have tenants from 2 different groups on the same machine, there are just = no issues. Only the mapping layer needs to be aware of the group a = machine belongs to. There is thus no change in = dataplane.

 

Any comments?

 

Thanks,

Vishwas

 

=

_______________________________________= ________
armd mailing list
armd@ietf.org
https://www.ietf.org/mailman/listinfo/armd



= --Apple-Mail-8-244284728-- From vishwas.ietf@gmail.com Wed Oct 5 17:01:17 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5E5DE21F8B58 for ; Wed, 5 Oct 2011 17:01:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.315 X-Spam-Level: X-Spam-Status: No, score=-3.315 tagged_above=-999 required=5 tests=[AWL=0.283, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id if7QSRQk-zzX for ; Wed, 5 Oct 2011 17:01:16 -0700 (PDT) Received: from mail-qy0-f172.google.com (mail-qy0-f172.google.com [209.85.216.172]) by ietfa.amsl.com (Postfix) with ESMTP id DC57121F8B56 for ; Wed, 5 Oct 2011 17:01:15 -0700 (PDT) Received: by qyk32 with SMTP id 32so4353113qyk.10 for ; Wed, 05 Oct 2011 17:04:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MM29ejjhH7GN76J2SraqZK/SmfJsWMadev3vbJzO2cw=; b=NnPz3evHlLaeXgozlkcm+C0QnT5ZkzdN70FNasPxGVCcFD9pExCEW/rtEwCL7nItvR LXE84tF4lIJOosG7nXgcOpWXhvVgBwqyK9QxU8vO4xasotypg4UK4hVxLTt4Ssa5Es+B 3z6NR29CBg7TVeggKmkV0lRHfhnq6VC9WIv1g= MIME-Version: 1.0 Received: by 10.229.37.84 with SMTP id w20mr52874qcd.76.1317859464624; Wed, 05 Oct 2011 17:04:24 -0700 (PDT) Received: by 10.229.91.131 with HTTP; Wed, 5 Oct 2011 17:04:24 -0700 (PDT) In-Reply-To: <7384885B-E229-4C91-9C45-C3982F620E74@cisco.com> References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx> <4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx> <7384885B-E229-4C91-9C45-C3982F620E74@cisco.com> Date: Wed, 5 Oct 2011 17:04:24 -0700 Message-ID: From: Vishwas Manral To: Benson Schliesser Content-Type: multipart/alternative; boundary=00163649a72740ba0704ae96116f Cc: "armd@ietf.org" Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 00:01:17 -0000 --00163649a72740ba0704ae96116f Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Benson, > Your proposal was a design "just using VLAN's" so that there is > "no change in dataplane". Thus I'm not sure what a "group" means, > in the context of your response, unless it means that the control > plane enforces some kind of hard separation. Yes the concept of group is only in the control plane.A device at one point of time belongs to one group. Yes there is a limitation of 4k instances on = a particular server (however that sounds like a reasonable limitation) to me. No physical devices need to be dedicated for a aprticular group, as with virtualization different tenats can occupy a host machine and can be shifte= d around. > Note that I referred to separation of "switch paths" in my previous > message. In today's average datacenter, this requires hard separation > of switches (or partitions within the switch). But there are alternative > switch designs; let me try to imagine what you're describing. I suppose > one could build a switch that completely localizes VLAN ID on each > interface, has a very large internal namespace for tenants, and rewrites > VLAN ID as frames are forwarded from ingress to egress interface. > (Sounds a bit like a Frame Relay switch...) But even with this > architecture, we're limited to 4k VLANs on a given path - that limitation > is built into the frame format. > > Bringing this back to the ARMD scope: As I wrote last week, it doesn't > really matter how we segregate the tenants. As long as the > mechanism provides a L2 service, the underlying method of segregation > is orthogonal to ARP/ND address resolution. However, the fact that > we segregate tenants might allow us to distribute the address > resolution function more intelligently. In your approach, do you > imagine distributing L3 gateways and/or ARP proxy functions > throughout each topology / partition? Yes you are right with this design change in ARMD, in my view there is not much difference as such other than the fact that we can divide the ARP tables more intelligently and only to members of the group that belong to the group so a lot easier for sure. Thanks, Vishwas > On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser = wrote: > >> Hi, Vishwas - >> >> Yes, what you describe is possible. But the issue is that it only works >> if we "do not have tenants from 2 different groups on the same machine". >> Keep in mind that this applies to all "machines" in the network. In >> addition to the hypervisor, all of the switch paths and router interface= s >> must also be segregated. >> >> The only way this works is if we partition the hardware into >> non-overlapping topologies, such that different tenants with a conflicti= ng >> VLAN ID aren't bridged together. And, as Linda pointed out, each partit= ion >> must have a different physical interface to the L3 gateway. (E.g. by hav= ing >> different routers, or discrete interfaces on a shared router) >> >> Effectively, we do this today when we build multiple datacenters >> (including when we build multiple "logical" datacenters inside the same >> physical building). We could pursue a more complicated approach where a= n >> arbitrary number of topologies are overlaid on a common physical network= , >> constrained by the number of physical paths etc, but that would be an >> operational nightmare. >> >> Cheers, >> -Benson >> >> >> On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote: >> >> Linda, >> >> Thats is a very complicated way of thinking things, where you have >> multiple layers. >> >> A simpler solution is to just assume each hypervisor belongs to a >> particular group. So it will communicate with only members of the same >> group. The mapping layer/ IP has a unique IP address (not group specifi= c). >> >> You have to understand in the cloud you have an orchesrtator so there is >> absolute flexibility in this approach, though there is some additional l= oad >> on the orchestrator to know groups + VLAN's to identify a tenant and >> ofcourse managing the mappings to best use the physical infrastructure. >> >> Thanks, >> Vishwas >> On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar wr= ote: >> >>> I mean =93the first hop router=94. Each physical port on =93the first = hop >>> router=94 can have its own 4095 VLANs. Each Hypervisor can also have it= s own >>> 4095 VLANs if Hypervisor does the IP encapsulation with a key in the he= ader >>> to differentiate clients (e.g. GRE=92s KEY field, or VxLan=92s Client I= D field). >>> **** >>> >>> ** ** >>> >>> You can make VLAN locally significant at the overlay edge node if the >>> Overlay Encapsulation header has a field to further differentiate the >>> Clients. **** >>> >>> ** ** >>> >>> Linda**** >>> >>> ** ** >>> >>> *From:* Vishwas Manral [mailto:vishwas.ietf@gmail.com] >>> *Sent:* Wednesday, October 05, 2011 3:15 PM >>> *To:* Linda Dunbar >>> *Cc:* armd@ietf.org >>> *Subject:* Re: [armd] Multi-Tenancy without changes**** >>> >>> ** ** >>> >>> Linda,**** >>> >>> **** >>> >>> I am unsure what you mean by Gateway router, but there is always a >>> Hypervisor or the first hop router that does the encapsulation/ >>> decapsulation.**** >>> >>> **** >>> >>> Thanks,**** >>> >>> Vishwas**** >>> >>> On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar >>> wrote:**** >>> >>> Vishwas, **** >>> >>> **** >>> >>> Are you assuming that each group of tenants are connected to Gateway >>> router via completely different physical ports? If yes, then you are >>> absolutely correct. **** >>> >>> **** >>> >>> Linda**** >>> >>> **** >>> >>> *From:* armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] *On Behalf >>> Of *Vishwas Manral >>> *Sent:* Wednesday, October 05, 2011 10:10 AM >>> *To:* armd@ietf.org >>> *Subject:* [armd] Multi-Tenancy without changes**** >>> >>> **** >>> >>> Hi,**** >>> >>> **** >>> >>> I was thinking of multi-tenancy and the way to achieve > 4094 without a= ny >>> changes in any layer by just using VLAN's. This is achieved as we alrea= dy >>> have a mapping layer already in place.**** >>> >>> **** >>> >>> So assume there are 8000 tenants. We can divide them in groups of 4000 >>> each and assign each tenant a particular id from say 2 to 4001. Now as = long >>> as we make sure we do not have tenants from 2 different groups on the s= ame >>> machine, there are just no issues. Only the mapping layer needs to be a= ware >>> of the group a machine belongs to. There is thus no change in dataplane= . >>> **** >>> >>> **** >>> >>> Any comments?**** >>> >>> **** >>> >>> Thanks,**** >>> >>> Vishwas**** >>> >>> ** ** >>> >> >> _______________________________________________ >> armd mailing list >> armd@ietf.org >> https://www.ietf.org/mailman/listinfo/armd >> >> >> > > --00163649a72740ba0704ae96116f Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Hi Benson,
=A0
> Your proposal was a design "just using VLAN's" so t= hat there is
> "no change in dataplane". Thus I'm not sure what a = "group" means,
>=A0in the context of your response, unless it means that the contr= ol
> plane enforces some kind of hard separation.
=A0
Yes the concept of group is only in the control plane.A device at one = point of time belongs to one group. Yes there is a limitation of 4k instanc= es on a particular server (however that sounds like a reasonable limitation= ) to me. No physical devices need to be dedicated for a aprticular group, a= s with virtualization different tenats can occupy a host machine and can be= shifted around.
=A0
> Note that I referred to separation of "switch paths" in= my previous
> message. In today's average datacenter, this requires hard se= paration
>=A0of switches (or partitions within the switch). But there are al= ternative
> switch designs; let me try to imagine what you're describing.= =A0 I suppose
>=A0one could build a switch that completely localizes VLAN ID on e= ach
> interface, has a very large internal namespace for tenants, and r= ewrites
>=A0VLAN ID as frames are forwarded from ingress to egress interfac= e.=A0
> (Sounds a bit like a Frame Relay switch...) =A0But even with this=
> architecture, we're limited to 4k VLANs on a given path - tha= t limitation
>=A0is built into the frame format.
>
> Bringing this back to the ARMD scope: =A0As I wrote last week, it= doesn't
> really matter how we segregate the tenants. =A0As long as the
> mechanism provides a L2 service, the underlying method of segrega= tion
> =A0is orthogonal to ARP/ND address resolution. =A0However, the fa= ct that
> we segregate tenants might allow us to distribute the address
> resolution function more intelligently. =A0In your approach, do y= ou
> imagine distributing L3 gateways and/or ARP proxy functions
> throughout each topology / partition?
Yes you are right with this design change in ARMD, in my view there is= not much=A0difference=A0as such other than the fact that we can divide the= ARP tables more intelligently and only to members of the group that belong= to the group so a lot easier for sure.
=A0
Thanks,
Vishwas=A0

=A0
On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser &l= t;bschlies@cisco.co= m> wrote:
Hi, Vishwas -=20

Yes, what you describe is possible. =A0But the issue is that it only w= orks if we "do not have tenants from 2 different groups on the same ma= chine". =A0Keep in mind that this applies to all "machines" = in the network. =A0In addition to the hypervisor, all of the switch paths a= nd router interfaces must also be segregated.

The only way this works is if we partition the hardware into non-overl= apping topologies, such that different tenants with a conflicting VLAN ID a= ren't bridged together. =A0And, as Linda pointed out, each partition mu= st have a different physical interface to the L3 gateway. (E.g. by having d= ifferent routers, or discrete interfaces on a shared router)

Effectively, we do this today when we build multiple datacenters (incl= uding when we build multiple "logical" datacenters inside the sam= e physical building). =A0We could pursue a more complicated approach where = an arbitrary number of topologies are overlaid on a common physical network= , constrained by the number of physical paths etc, but that would be an ope= rational nightmare.

Cheers,
-Benson


On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote:

Linda,
=A0
Thats is a very complicated way of thinking things, where you have mul= tiple layers.
=A0
A simpler solution is to just assume each hypervisor belongs to a part= icular group. So it will communicate with only members of the same group.= =A0 The mapping layer/ IP has a unique IP address (not group specific).
=A0
You have to understand in the cloud you have an orchesrtator so there = is absolute flexibility in this approach, though there is some additional l= oad on the orchestrator to know groups +=A0VLAN's to identify a tenant = and ofcourse managing the mappings to best use the physical infrastructure.=
=A0
Thanks,
Vishwas
On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar <linda.dunbar@huawei.com> wrote:

I me= an =93the first hop router=94. Each physical port on =93the first hop route= r=94 can have its own 4095 VLANs. Each Hypervisor can also have its own 409= 5 VLANs if Hypervisor does the IP encapsulation with a key in the header to= differentiate clients (e.g. GRE=92s KEY field, or VxLan=92s Client ID fiel= d).

<= /u>=A0

You = can make VLAN locally significant at the overlay edge node if the Overlay E= ncapsulation header has a field to further differentiate the Clients.

<= /u>=A0

Lind= a

<= /u>=A0

From:<= span style=3D"FONT-SIZE: 10pt"> Vishwas Manral [mailto:vishwas.ietf@gmail.com]
= Sent: Wednesday, October 05, 2011 3:15 PM
To: Linda Dunbar
Cc: armd@ietf.org
Subject: Re: [armd] Multi-Tenancy= without changes

=A0

Linda,

=A0

I am unsure what you mean by Gateway router, but the= re is always a Hypervisor or the first hop router that does the encapsulati= on/ decapsulation.

=A0

Thanks,

Vishwas

On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar <linda.dunbar@hua= wei.com> wrote:

Vish= was,

=A0<= /span>

Are = you assuming that each group of tenants are connected to Gateway router via= completely different physical ports? If yes, then you are absolutely corre= ct.

=A0<= /span>

Lind= a

=A0<= /span>

From:<= span style=3D"FONT-SIZE: 10pt"> armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf O= f Vishwas Manral
Sent: Wednesday, October 05, 2011 10:10 AM
To: armd@ietf.org
Subject: [armd] Multi-Tenancy without changes

=A0

Hi,

=A0

I was thinking of multi-tenancy and the way to achie= ve > 4094 without any changes in any layer by just using VLAN's. Thi= s is achieved as we already have a mapping layer already in place.

=A0

So assume there are 8000 tenants.=A0We can divide th= em in groups of 4000 each and assign each tenant a particular id from say 2= to 4001. Now as long as we make sure we do not have tenants from 2 differe= nt groups on the same machine, there are just no issues. Only the mapping l= ayer needs to be aware of the group a machine belongs to. There is thus no = change in dataplane.

=A0

Any comments?

=A0

Thanks,

Vishwas

=A0

<= /blockquote>

_________________________________________= ______
armd mailing list
armd@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/armd




--00163649a72740ba0704ae96116f-- From adalela@cisco.com Wed Oct 5 20:56:24 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AD5A011E8081 for ; Wed, 5 Oct 2011 20:56:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.899 X-Spam-Level: X-Spam-Status: No, score=-5.899 tagged_above=-999 required=5 tests=[AWL=-3.300, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dv4O3pEhY16Y for ; Wed, 5 Oct 2011 20:56:20 -0700 (PDT) Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id DC6F421F8D3A for ; Wed, 5 Oct 2011 20:56:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=adalela@cisco.com; l=30513; q=dns/txt; s=iport; t=1317873569; x=1319083169; h=mime-version:subject:date:message-id:in-reply-to: references:from:to:cc; bh=Ap5VBmLTntW1QFcVxYgckLYmUtdSAMfjMt7uh40wTGw=; b=Btx4Di3t1vygltdGXd+oyPBd5LgTqUT4zoBOxa8I5aBP2UQSy2sBRgFX hiKvwyF/5Y/Sx1eGczLCdkaDl06B7dlYi9xwz46AwZzkOR5CqykveTQRF mm/b/72EZBlnu1PIutSaCoAAuwNqPFC5lWqPlOhrvI+rH2wHygo9m0M8J w=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AsIAAH8mjU5Io8US/2dsb2JhbAA6CoJNlkOPFYEFgVMBAQEBAgEBAQEPAQkLBgM+CwULAgEIEQEDAQELBhAHAQYBIAYfAwYIAQEEAQoICBMHh1sGl00BnXkDg3yCTGEEh3qRJIRwhyw X-IronPort-AV: E=Sophos;i="4.68,495,1312156800"; d="scan'208,217";a="418758" Received: from bgl-core-3.cisco.com ([72.163.197.18]) by ams-iport-4.cisco.com with ESMTP; 06 Oct 2011 03:59:22 +0000 Received: from xbh-bgl-411.cisco.com (xbh-bgl-411.cisco.com [72.163.129.201]) by bgl-core-3.cisco.com (8.14.3/8.14.3) with ESMTP id p963xMTB025726; Thu, 6 Oct 2011 03:59:22 GMT Received: from xmb-bgl-416.cisco.com ([72.163.129.212]) by xbh-bgl-411.cisco.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 6 Oct 2011 09:29:22 +0530 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01CC83DC.548BDF71" Date: Thu, 6 Oct 2011 09:29:16 +0530 Message-ID: <618BE8B40039924EB9AED233D4A09C51022D55FE@XMB-BGL-416.cisco.com> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [armd] Multi-Tenancy without changes thread-index: AcyDu4s+Vww6PLWZRIeLXCjjuVXLZgAHFuPw References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx><4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx><7384885B-E229-4C91-9C45-C3982F620E74@cisco.com> From: "Ashish Dalela (adalela)" To: "Vishwas Manral" , "Benson Schliesser (bschlies)" X-OriginalArrivalTime: 06 Oct 2011 03:59:22.0450 (UTC) FILETIME=[54BB3320:01CC83DC] Cc: armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 03:56:24 -0000 This is a multi-part message in MIME format. ------_=_NextPart_001_01CC83DC.548BDF71 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable =20 Assumptions about how we can use VLANs are broken down the moment you think of Hybrid Clouds - i.e. private VLAN stretching out into the public cloud. In effect, every customer has the full range of 4K VLANs inside the enterprise and when they span the VLAN into the public cloud, by implication the entire range is (potentially) spanned into the cloud as well. There are two ways to deal with this - (a) do VLAN translation at some boundary or (b) create segmentation orthogonal to VLAN. I prefer option (b). Option (a) will also work, but will not scale. =20 Another reason why VLAN as segmentation is not a scalable option because all inter-VLAN traffic gets pinned to some L3 gateway. Now, if you have L2 network, with some kind of ECMP, then you don't get ECMP for L3 traffic. That's a huge problem, IMO, especially if you have a VLAN spanning multiple subnets and the host thinks it is outside the subnet and the network thinks it is inside the VLAN. =20 You can design around this and pin the L3 GW at multiple points in the network - and now you have a different problem - all the L3->L2 bindings are stored at all the L3 GWs. If you had a million VMs each host talking to even one host outside the VLAN, then that traffic goes through the GW. Now, when you apply ECMP, the problem is quickly spread to all L3 GWs.=20 =20 In summary, using VLAN for segmentation is just inviting too many problems, that I would like to side step. =20 Ashish =20 From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas Manral Sent: Thursday, October 06, 2011 5:34 AM To: Benson Schliesser (bschlies) Cc: armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes =20 Hi Benson, =20 > Your proposal was a design "just using VLAN's" so that there is=20 > "no change in dataplane". Thus I'm not sure what a "group" means, > in the context of your response, unless it means that the control=20 > plane enforces some kind of hard separation. =20 Yes the concept of group is only in the control plane.A device at one point of time belongs to one group. Yes there is a limitation of 4k instances on a particular server (however that sounds like a reasonable limitation) to me. No physical devices need to be dedicated for a aprticular group, as with virtualization different tenats can occupy a host machine and can be shifted around. =20 > Note that I referred to separation of "switch paths" in my previous=20 > message. In today's average datacenter, this requires hard separation > of switches (or partitions within the switch). But there are alternative=20 > switch designs; let me try to imagine what you're describing. I suppose > one could build a switch that completely localizes VLAN ID on each=20 > interface, has a very large internal namespace for tenants, and rewrites > VLAN ID as frames are forwarded from ingress to egress interface.=20 > (Sounds a bit like a Frame Relay switch...) But even with this=20 > architecture, we're limited to 4k VLANs on a given path - that limitation > is built into the frame format. >=20 > Bringing this back to the ARMD scope: As I wrote last week, it doesn't=20 > really matter how we segregate the tenants. As long as the=20 > mechanism provides a L2 service, the underlying method of segregation > is orthogonal to ARP/ND address resolution. However, the fact that=20 > we segregate tenants might allow us to distribute the address=20 > resolution function more intelligently. In your approach, do you=20 > imagine distributing L3 gateways and/or ARP proxy functions=20 > throughout each topology / partition? Yes you are right with this design change in ARMD, in my view there is not much difference as such other than the fact that we can divide the ARP tables more intelligently and only to members of the group that belong to the group so a lot easier for sure. =20 Thanks, Vishwas=20 =20 On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser wrote: Hi, Vishwas -=20 =20 Yes, what you describe is possible. But the issue is that it only works if we "do not have tenants from 2 different groups on the same machine". Keep in mind that this applies to all "machines" in the network. In addition to the hypervisor, all of the switch paths and router interfaces must also be segregated. =20 The only way this works is if we partition the hardware into non-overlapping topologies, such that different tenants with a conflicting VLAN ID aren't bridged together. And, as Linda pointed out, each partition must have a different physical interface to the L3 gateway. (E.g. by having different routers, or discrete interfaces on a shared router) =20 Effectively, we do this today when we build multiple datacenters (including when we build multiple "logical" datacenters inside the same physical building). We could pursue a more complicated approach where an arbitrary number of topologies are overlaid on a common physical network, constrained by the number of physical paths etc, but that would be an operational nightmare. =20 Cheers, -Benson =20 =20 On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote: =20 Linda,=20 =20 Thats is a very complicated way of thinking things, where you have multiple layers.=20 =20 A simpler solution is to just assume each hypervisor belongs to a particular group. So it will communicate with only members of the same group. The mapping layer/ IP has a unique IP address (not group specific).=20 =20 You have to understand in the cloud you have an orchesrtator so there is absolute flexibility in this approach, though there is some additional load on the orchestrator to know groups + VLAN's to identify a tenant and ofcourse managing the mappings to best use the physical infrastructure. =20 Thanks, Vishwas On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar wrote: I mean "the first hop router". Each physical port on "the first hop router" can have its own 4095 VLANs. Each Hypervisor can also have its own 4095 VLANs if Hypervisor does the IP encapsulation with a key in the header to differentiate clients (e.g. GRE's KEY field, or VxLan's Client ID field).=20 =20 You can make VLAN locally significant at the overlay edge node if the Overlay Encapsulation header has a field to further differentiate the Clients.=20 =20 Linda =20 From: Vishwas Manral [mailto:vishwas.ietf@gmail.com]=20 Sent: Wednesday, October 05, 2011 3:15 PM To: Linda Dunbar Cc: armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes =20 Linda, =20 I am unsure what you mean by Gateway router, but there is always a Hypervisor or the first hop router that does the encapsulation/ decapsulation. =20 Thanks, Vishwas On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar wrote: Vishwas,=20 =20 Are you assuming that each group of tenants are connected to Gateway router via completely different physical ports? If yes, then you are absolutely correct.=20 =20 Linda =20 From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas Manral Sent: Wednesday, October 05, 2011 10:10 AM To: armd@ietf.org Subject: [armd] Multi-Tenancy without changes =20 Hi, =20 I was thinking of multi-tenancy and the way to achieve > 4094 without any changes in any layer by just using VLAN's. This is achieved as we already have a mapping layer already in place. =20 So assume there are 8000 tenants. We can divide them in groups of 4000 each and assign each tenant a particular id from say 2 to 4001. Now as long as we make sure we do not have tenants from 2 different groups on the same machine, there are just no issues. Only the mapping layer needs to be aware of the group a machine belongs to. There is thus no change in dataplane. =20 Any comments? =20 Thanks, Vishwas =20 =20 =09 _______________________________________________ armd mailing list armd@ietf.org =09 https://www.ietf.org/mailman/listinfo/armd =20 =20 =20 =20 ------_=_NextPart_001_01CC83DC.548BDF71 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

 

Assumptions about how we can use VLANs are broken down the moment you = think of Hybrid Clouds – i.e. private VLAN stretching out into the = public cloud. In effect, every customer has the full range of 4K VLANs = inside the enterprise and when they span the VLAN into the public cloud, = by implication the entire range is (potentially) spanned into the cloud = as well. There are two ways to deal with this – (a) do VLAN = translation at some boundary or (b) create segmentation orthogonal to = VLAN. I prefer option (b). Option (a) will also work, but will not = scale.

 

Another reason why VLAN as segmentation is not a scalable option = because all inter-VLAN traffic gets pinned to some L3 gateway. Now, if = you have L2 network, with some kind of ECMP, then you don’t get = ECMP for L3 traffic. That’s a huge problem, IMO, especially if you = have a VLAN spanning multiple subnets and the host thinks it is outside = the subnet and the network thinks it is inside the = VLAN.

 

You can design around this and pin the L3 GW at multiple points in = the network – and now you have a different problem – all the = L3->L2 bindings are stored at all the L3 GWs. If you had a million = VMs each host talking to even one host outside the VLAN, then that = traffic goes through the GW. Now, when you apply ECMP, the problem is = quickly spread to all L3 GWs.

 

In summary, using VLAN for segmentation is just inviting too many = problems, that I would like to side step.

 

Ashish

 

From:= = armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of = Vishwas Manral
Sent: Thursday, October 06, 2011 5:34 = AM
To: Benson Schliesser (bschlies)
Cc: = armd@ietf.org
Subject: Re: [armd] Multi-Tenancy without = changes

 

Hi = Benson,

 

> Your proposal was a design "just using = VLAN's" so that there is

> "no change in dataplane". Thus I'm not = sure what a "group" means,

> in the context of your response, unless it = means that the control

> plane enforces some kind of hard = separation.

 

Yes the concept of group is only in the control = plane.A device at one point of time belongs to one group. Yes there is a = limitation of 4k instances on a particular server (however that sounds = like a reasonable limitation) to me. No physical devices need to be = dedicated for a aprticular group, as with virtualization different = tenats can occupy a host machine and can be shifted = around.

 

> Note that I referred to separation of = "switch paths" in my previous

> message. In today's average datacenter, this = requires hard separation

> of switches (or partitions within the = switch). But there are alternative

> switch designs; let me try to imagine what you're = describing.  I suppose

> one could build a switch that completely = localizes VLAN ID on each

> interface, has a very large internal namespace = for tenants, and rewrites

> VLAN ID as frames are forwarded from ingress = to egress interface. 

> (Sounds a bit like a Frame Relay switch...) =  But even with this

> architecture, we're limited to 4k VLANs on a = given path - that limitation

> is built into the frame = format.

> 

> Bringing this back to the ARMD scope:  As I = wrote last week, it doesn't

> really matter how we segregate the tenants. =  As long as the

> = mechanism provides a L2 service, the underlying method of = segregation

>  is = orthogonal to ARP/ND address resolution.  However, the fact that =

> we segregate tenants = might allow us to distribute the address

> resolution function more intelligently.  In = your approach, do you

> imagine distributing L3 gateways and/or ARP proxy = functions

> throughout = each topology / partition?

Yes you are right with this design change in ARMD, in = my view there is not much difference as such other than the = fact that we can divide the ARP tables more intelligently and only to = members of the group that belong to the group so a lot easier for = sure.

 

Thanks,

Vishwas 


 

On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser = <bschlies@cisco.com> = wrote:

Hi, = Vishwas -

 

Yes, what you describe is possible.  But the = issue is that it only works if we "do not have tenants from 2 = different groups on the same machine".  Keep in mind that this = applies to all "machines" in the network.  In addition to = the hypervisor, all of the switch paths and router interfaces must also = be segregated.

 

The only way this works is if we partition the = hardware into non-overlapping topologies, such that different tenants = with a conflicting VLAN ID aren't bridged together.  And, as Linda = pointed out, each partition must have a different physical interface to = the L3 gateway. (E.g. by having different routers, or discrete = interfaces on a shared router)

 

Effectively, we do this today when we build multiple = datacenters (including when we build multiple "logical" = datacenters inside the same physical building).  We could pursue a = more complicated approach where an arbitrary number of topologies are = overlaid on a common physical network, constrained by the number of = physical paths etc, but that would be an operational = nightmare.

 

Cheers,

-Benson

 

 

On Oct 5, 2011, at 3:35 PM, Vishwas Manral = wrote:

 

Linda,

 

Thats is a very complicated way of thinking things, = where you have multiple layers.

 

A = simpler solution is to just assume each hypervisor belongs to a = particular group. So it will communicate with only members of the same = group.  The mapping layer/ IP has a unique IP address (not group = specific).

 

You have to understand in the cloud you have an = orchesrtator so there is absolute flexibility in this approach, though = there is some additional load on the orchestrator to know groups = + VLAN's to identify a tenant and ofcourse managing the mappings to = best use the physical infrastructure.

 

Thanks,

Vishwas

On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar <linda.dunbar@huawei.com> = wrote:

I mean “the first hop = router”. Each physical port on “the first hop router” = can have its own 4095 VLANs. Each Hypervisor can also have its own 4095 = VLANs if Hypervisor does the IP encapsulation with a key in the header = to differentiate clients (e.g. GRE’s KEY field, or VxLan’s = Client ID field).

 

You can make VLAN locally = significant at the overlay edge node if the Overlay Encapsulation header = has a field to further differentiate the Clients. =

 

Linda

 

From: Vishwas Manral [mailto:vishwas.ietf@gmail.com]
Sent: = Wednesday, October 05, 2011 3:15 PM
To: Linda = Dunbar
Cc: armd@ietf.org
Subject: Re: [armd] = Multi-Tenancy without = changes

 <= /o:p>

Linda,<= /o:p>

 <= /o:p>

I am unsure = what you mean by Gateway router, but there is always a Hypervisor or the = first hop router that does the encapsulation/ = decapsulation.

 <= /o:p>

Thanks,=

Vishwas=

On Wed, Oct = 5, 2011 at 12:06 PM, Linda Dunbar <linda.dunbar@huawei.com> = wrote:

Vishwas, =

 

Are you assuming that each = group of tenants are connected to Gateway router via completely = different physical ports? If yes, then you are absolutely correct. =

 

Linda

 

From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas = Manral
Sent: Wednesday, October 05, 2011 10:10 = AM
To: armd@ietf.org
Subject: [armd] = Multi-Tenancy without = changes

 <= /o:p>

Hi,

 <= /o:p>

I was = thinking of multi-tenancy and the way to achieve > 4094 without any = changes in any layer by just using VLAN's. This is achieved as we = already have a mapping layer already in = place.

 <= /o:p>

So assume = there are 8000 tenants. We can divide them in groups of 4000 each = and assign each tenant a particular id from say 2 to 4001. Now as long = as we make sure we do not have tenants from 2 different groups on the = same machine, there are just no issues. Only the mapping layer needs to = be aware of the group a machine belongs to. There is thus no change in = dataplane.

 <= /o:p>

Any = comments?

 <= /o:p>

Thanks,=

Vishwas=

 <= /o:p>

 

_______________________________________________
armd= mailing list
armd@ietf.org
https://www.ietf.org/mailman/listinfo/armd

 

 

 

 

------_=_NextPart_001_01CC83DC.548BDF71-- From skh@ndzh.com Thu Oct 6 09:25:49 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0F62221F8DAF for ; Thu, 6 Oct 2011 09:25:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.283 X-Spam-Level: X-Spam-Status: No, score=0.283 tagged_above=-999 required=5 tests=[AWL=0.265, BAYES_00=-2.599, FH_HOST_EQ_D_D_D_D=0.765, FH_HOST_EQ_D_D_D_DB=0.888, HELO_MISMATCH_COM=0.553, HOST_MISMATCH_NET=0.311, HTML_MESSAGE=0.001, RDNS_DYNAMIC=0.1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bS7hio7FbHUm for ; Thu, 6 Oct 2011 09:25:48 -0700 (PDT) Received: from hickoryhill-consulting.com (63-208-161-201.digitalrealm.net [63.208.161.201]) by ietfa.amsl.com (Postfix) with ESMTP id A611321F8D5F for ; Thu, 6 Oct 2011 09:25:47 -0700 (PDT) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=166.250.32.97; Received: from SKHPERSONALLT (unverified [166.250.32.97]) by hickoryhill-consulting.com (SurgeMail 5.2a) with ESMTP id 2727509-1945496 for multiple; Thu, 06 Oct 2011 12:28:40 -0400 From: "Susan Hares" To: "'Thomas Narten'" Date: Thu, 6 Oct 2011 12:28:41 -0400 Message-ID: <00bb01cc8445$02fd8240$08f886c0$@com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_00BC_01CC8423.7BEBE240" X-Mailer: Microsoft Office Outlook 12.0 Thread-index: AcyERQHROYsIMQkeSrmYfncGVe+R7Q== Content-Language: en-us X-Authenticated-User: skh@ndzh.com Cc: armd@ietf.org Subject: [armd] ARMD X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 16:25:49 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_00BC_01CC8423.7BEBE240 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Thomas: Comments below. Does this help you refine the problem statement on pain points? By the way, will we be seeing the first draft of the problems statement this week. Sue ------ Sue, There is a third layer/level of addressing. The addresses a VM uses internal to the overlay are private (or can be, but that is not required). If the VM needs to communicate with the outside world, it will need a public address. This can be done via standard NAT (as is done today, if the VM is not using the public address directly). I would expect the NAT translation to be done at the edge of the overlay, i.e., as part of being decapsulated in order to exit the overlay. ---[Sue] NAT works this way, but it is also important to provide an ---[Sue] non-NAT solution if you are using global addresses. I.e., if a packet exits an overlay network, the addresses it uses outside of the overlay needs to have meaning outside of the overlay. If the VM address (as used within the overlay) is a globally unique public address, there is nothing to do. If it is a private address, you'd need to NAT it. --- [Sue] agreed if private address. The bigger point is that when a packet leaves on overlay, its a normal IP packet (no longer encapsulated). Whatever techniques are used currently with IP can then still be used outside of the overlay. Make sense? --- yes it makes sense with the caveats on NAT/Non-NAT. Thomas > Thomas: > Linda's comment about a applications which communicate outside the data center indicates that the address outside the data center are likely not to share the IP addresses within the data center. > This is a scenario we have heard from Igor at yahoo. It is a scenario I have seen in some goggle slides at nanog. > External addresses in these scenarios need to be resolved. > Does this make sense? > If so, we can go on to dc connected by by mpls le vpns. > Sue > Sent via BlackBerry by AT&T > -----Original Message----- > From: Thomas Narten > Sender: armd-bounces at ietf.org > Date: Tue, 04 Oct 2011 12:09:28 > To: Linda Dunbar > Cc: armd at ietf.org > Subject: Re: [armd] Call for Participation: Using IP Overlays to provide L2 > Virtualization > Hi Linda. > > I agree that "Overlay push some of the L2 scaling concerns" in your > > draft. But this statement should not be listed under the sub-section > > of ARMD. > I will tweak this in the next version. > > Overlay is to hide VMs' addresses from network interior > > nodes. > Agreed. > > Address Resolution is to map VMs' IP addresses to physical > > addresses. > Yes, but. Address resolution is also a sort of generic term and can > mean more than this. Anytime you have to map an "upper layer" address > to a "lower layer" address, you need address resolution or address > mapping. We usually think of ARP/ND doing address resolution between > an IP address and a physical (ethernet) address. > But, with overlays, you will have multiple layers doing address > mapping. In the overlay context, you will have the VM's IP address, > which needs to be mapped into the infrastructure IP address where that > VM current resides (and where packets can be tunneled to). That is > something that is handled at the overlay level. Separately, at some > point, a router/gateway will need to map that infrastructure IP > address into a physical address (just as is done today). I assume this > is what you mean by "address resolution". > The "mapping function" that the problem statement draft uses is a form > of Address Resolution. > > Applications (or VMs) within Data center need to communicate with > > peers outside data center. Regardless of overlay is deployed within > > the data center or not, the Gateway routers still need to resolve > > physical addresses for all VMs which have external communications. > I'm not sure I understand this fully (see above). There are two > IP addresses in use (private within the overlay and > infrastructure). Which IP addresses do you mean above? > > Therefore, Address resolution and Overlay are orthogonal to each > > other. > If you mean address resolution as in the ARP/ND sense vs. "mappings" > in the overlay sense, I agree. > > Overlay can only make address resolution more complex, > > because Gateway router not only needs to resolve physical address > > for each VM's IP, it also needs to resolve the overlay edge address > > for each VM's IP. > Yes. If the gateway is at the edge of the overlay, it will need to > encap/decap packets and perform address mappings. Separately, it will > need to do address resolution for infrastructure addresses. > Thomas > _______________________________________________ > armd mailing list > armd at ietf.org > https://www.ietf.org/mailman/listinfo/armd ------=_NextPart_000_00BC_01CC8423.7BEBE240 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable 
Thomas:
 <=
/pre>
Comments below. Does this help you refine the problem =
statement on pain points? 
By the way, will we be =
seeing the first draft of the problems statement this week. =

 
Sue =

 
------
=

 
Sue,
 
There is a third layer/level of =
addressing.
 
The =
addresses a VM uses internal to the overlay are private (or =
can
be, but that is not =
required).
 
If the VM =
needs to communicate with the outside world, it will need =
a
public address. This can be done via standard NAT =
(as is done today,
if the VM is not using the =
public address directly). =

 
I would expect the NAT =
translation to be done at the edge of the
overlay, =
i.e., as part of being decapsulated in order to exit =
the
overlay.
---[Sue] NAT =
works this way, but it is also important to provide =
an
---[Sue] non-NAT solution if you are using =
global addresses. =
 
 
I.e., if a =
packet exits an overlay network, the addresses it =
uses
outside of the overlay needs to have meaning =
outside of the
overlay. If the VM address (as used =
within the overlay) is a globally
unique public =
address, there is nothing to do. If it is a =
private
address, you'd need to NAT =
it.
--- [Sue] agreed if private address. =

 
The bigger point is =
that when a packet leaves on overlay, its a =
normal
IP packet (no longer encapsulated). Whatever =
techniques are used
currently with IP can then =
still be used outside of the =
overlay.
 
Make =
sense?
--- yes it makes sense with the caveats on =
NAT/Non-NAT. =

  
 =

Thomas
 
> =
Thomas:
 
> Linda's =
comment about a applications which communicate outside the data center =
indicates that the address outside the data center are likely not to =
share the IP addresses within the data =
center.
 
> This is a =
scenario we have heard from Igor at yahoo.  It is a scenario I have =
seen in some goggle slides at =
nanog.
 
> External =
addresses in these scenarios need to be =
resolved.
 
> Does =
this make sense?
 
> =
If so, we can go on to dc connected by by mpls le =
vpns.
 
> =
Sue
> Sent via BlackBerry by =
AT&T
 
> =
-----Original Message-----
> From: Thomas Narten =
<narten at us.ibm.com>
> Sender: =
armd-bounces at ietf.org
> Date: Tue, 04 Oct =
2011 12:09:28 
> To: Linda =
Dunbar<linda.dunbar at huawei.com>
> Cc: =
armd at ietf.org<armd at ietf.org>
> =
Subject: Re: [armd] Call for Participation: Using IP Overlays to provide =
L2
>       =
Virtualization
 
> Hi =
Linda.
 
> > I =
agree that "Overlay push some of the L2 scaling concerns" in =
your
> > draft. But this statement should not =
be listed under the sub-section
> > of =
ARMD.
 
> I will tweak =
this in the next =
version.
 
> > =
Overlay is to hide VMs' addresses from network =
interior
> > =
nodes.
 
> =
Agreed.
 
> > =
Address Resolution is to map VMs' IP addresses to =
physical
> > =
addresses.
 
> Yes, =
but. Address resolution is also a sort of generic term and =
can
> mean more than this. Anytime you have to =
map an "upper layer" address
> to a =
"lower layer" address, you need address resolution or =
address
> mapping. We usually think of ARP/ND =
doing address resolution between
> an IP address =
and a physical (ethernet) =
address.
 
> But, with =
overlays, you will have multiple layers doing =
address
> mapping. In the overlay context, you =
will have the VM's IP address,
> which needs to =
be mapped into the infrastructure IP address where =
that
> VM current resides (and where packets can =
be tunneled to). That is
> something that is =
handled at the overlay level. Separately, at =
some
> point, a router/gateway will need to map =
that infrastructure IP
> address into a physical =
address (just as is done today). I assume this
> =
is what you mean by "address =
resolution".
 
> =
The "mapping function" that the problem statement draft uses =
is a form
> of Address =
Resolution.
 
> > =
Applications (or VMs) within Data center need to communicate =
with
> > peers outside data center. =
Regardless of overlay is deployed within
> > =
the data center or not, the Gateway routers still need to =
resolve
> > physical addresses for all VMs =
which have external =
communications.
 
> =
I'm not sure I understand this fully (see above). There are =
two
> IP addresses in use (private within the =
overlay and
> infrastructure). Which IP =
addresses do you mean =
above?
 
> > =
Therefore, Address resolution and Overlay are orthogonal to =
each
> > =
other.
 
> If you mean =
address resolution as in the ARP/ND sense vs. =
"mappings"
> in the overlay sense, I =
agree.
 
> > =
Overlay can only make address resolution more =
complex,
> > because Gateway router not only =
needs to resolve physical address
> > for =
each VM's IP, it also needs to resolve the overlay edge =
address
> > for each VM's =
IP.
 
> Yes. If the =
gateway is at the edge of the overlay, it will need =
to
> encap/decap packets and perform address =
mappings. Separately, it will
> need to do =
address resolution for infrastructure =
addresses.
 
> =
Thomas
> =
_______________________________________________
>=
 armd mailing list
> armd at =
ietf.org
> https://www.ietf.org/=
mailman/listinfo/armd
 
------=_NextPart_000_00BC_01CC8423.7BEBE240-- From skh@ndzh.com Thu Oct 6 09:39:00 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 584C821F8CC1 for ; Thu, 6 Oct 2011 09:39:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.151 X-Spam-Level: X-Spam-Status: No, score=0.151 tagged_above=-999 required=5 tests=[AWL=0.132, BAYES_00=-2.599, FH_HOST_EQ_D_D_D_D=0.765, FH_HOST_EQ_D_D_D_DB=0.888, HELO_MISMATCH_COM=0.553, HOST_MISMATCH_NET=0.311, HTML_MESSAGE=0.001, RDNS_DYNAMIC=0.1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sBlnVVNnswnm for ; Thu, 6 Oct 2011 09:38:55 -0700 (PDT) Received: from hickoryhill-consulting.com (63-208-161-201.digitalrealm.net [63.208.161.201]) by ietfa.amsl.com (Postfix) with ESMTP id 53FC421F8CD3 for ; Thu, 6 Oct 2011 09:38:55 -0700 (PDT) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=166.250.32.97; Received: from SKHPERSONALLT (unverified [166.250.32.97]) by hickoryhill-consulting.com (SurgeMail 5.2a) with ESMTP id 2727554-1945496 for multiple; Thu, 06 Oct 2011 12:42:00 -0400 From: "Susan Hares" To: "'Ashish Dalela \(adalela\)'" , "'Vishwas Manral'" , "'Benson Schliesser \(bschlies\)'" References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx><4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx><7384885B-E229-4C91-9C45-C3982F620E74@cisco.com> <618BE8B40039924EB9AED233D4A09C51022D55FE@XMB-BGL-416.cisco.com> In-Reply-To: <618BE8B40039924EB9AED233D4A09C51022D55FE@XMB-BGL-416.cisco.com> Date: Thu, 6 Oct 2011 12:42:01 -0400 Message-ID: <010301cc8446$e005dec0$a0119c40$@com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0104_01CC8425.58F43EC0" X-Mailer: Microsoft Office Outlook 12.0 Thread-index: AcyDu4s+Vww6PLWZRIeLXCjjuVXLZgAHFuPwABtaaAA= Content-Language: en-us X-Authenticated-User: skh@ndzh.com Cc: 'Thomas Narten' , armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 16:39:00 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0104_01CC8425.58F43EC0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Vishwas: As always, great comments. Thomas Narten is collecting pain points and issues. Can you give us details why option (a) VLAN translation at some boundary will not scale? It would help Thomas' pain points. Sue From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Ashish Dalela (adalela) Sent: Wednesday, October 05, 2011 11:59 PM To: Vishwas Manral; Benson Schliesser (bschlies) Cc: armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes Assumptions about how we can use VLANs are broken down the moment you think of Hybrid Clouds - i.e. private VLAN stretching out into the public cloud. In effect, every customer has the full range of 4K VLANs inside the enterprise and when they span the VLAN into the public cloud, by implication the entire range is (potentially) spanned into the cloud as well. There are two ways to deal with this - (a) do VLAN translation at some boundary or (b) create segmentation orthogonal to VLAN. I prefer option (b). Option (a) will also work, but will not scale. Another reason why VLAN as segmentation is not a scalable option because all inter-VLAN traffic gets pinned to some L3 gateway. Now, if you have L2 network, with some kind of ECMP, then you don't get ECMP for L3 traffic. That's a huge problem, IMO, especially if you have a VLAN spanning multiple subnets and the host thinks it is outside the subnet and the network thinks it is inside the VLAN. You can design around this and pin the L3 GW at multiple points in the network - and now you have a different problem - all the L3->L2 bindings are stored at all the L3 GWs. If you had a million VMs each host talking to even one host outside the VLAN, then that traffic goes through the GW. Now, when you apply ECMP, the problem is quickly spread to all L3 GWs. In summary, using VLAN for segmentation is just inviting too many problems, that I would like to side step. Ashish From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas Manral Sent: Thursday, October 06, 2011 5:34 AM To: Benson Schliesser (bschlies) Cc: armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes Hi Benson, > Your proposal was a design "just using VLAN's" so that there is > "no change in dataplane". Thus I'm not sure what a "group" means, > in the context of your response, unless it means that the control > plane enforces some kind of hard separation. Yes the concept of group is only in the control plane.A device at one point of time belongs to one group. Yes there is a limitation of 4k instances on a particular server (however that sounds like a reasonable limitation) to me. No physical devices need to be dedicated for a aprticular group, as with virtualization different tenats can occupy a host machine and can be shifted around. > Note that I referred to separation of "switch paths" in my previous > message. In today's average datacenter, this requires hard separation > of switches (or partitions within the switch). But there are alternative > switch designs; let me try to imagine what you're describing. I suppose > one could build a switch that completely localizes VLAN ID on each > interface, has a very large internal namespace for tenants, and rewrites > VLAN ID as frames are forwarded from ingress to egress interface. > (Sounds a bit like a Frame Relay switch...) But even with this > architecture, we're limited to 4k VLANs on a given path - that limitation > is built into the frame format. > > Bringing this back to the ARMD scope: As I wrote last week, it doesn't > really matter how we segregate the tenants. As long as the > mechanism provides a L2 service, the underlying method of segregation > is orthogonal to ARP/ND address resolution. However, the fact that > we segregate tenants might allow us to distribute the address > resolution function more intelligently. In your approach, do you > imagine distributing L3 gateways and/or ARP proxy functions > throughout each topology / partition? Yes you are right with this design change in ARMD, in my view there is not much difference as such other than the fact that we can divide the ARP tables more intelligently and only to members of the group that belong to the group so a lot easier for sure. Thanks, Vishwas On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser wrote: Hi, Vishwas - Yes, what you describe is possible. But the issue is that it only works if we "do not have tenants from 2 different groups on the same machine". Keep in mind that this applies to all "machines" in the network. In addition to the hypervisor, all of the switch paths and router interfaces must also be segregated. The only way this works is if we partition the hardware into non-overlapping topologies, such that different tenants with a conflicting VLAN ID aren't bridged together. And, as Linda pointed out, each partition must have a different physical interface to the L3 gateway. (E.g. by having different routers, or discrete interfaces on a shared router) Effectively, we do this today when we build multiple datacenters (including when we build multiple "logical" datacenters inside the same physical building). We could pursue a more complicated approach where an arbitrary number of topologies are overlaid on a common physical network, constrained by the number of physical paths etc, but that would be an operational nightmare. Cheers, -Benson On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote: Linda, Thats is a very complicated way of thinking things, where you have multiple layers. A simpler solution is to just assume each hypervisor belongs to a particular group. So it will communicate with only members of the same group. The mapping layer/ IP has a unique IP address (not group specific). You have to understand in the cloud you have an orchesrtator so there is absolute flexibility in this approach, though there is some additional load on the orchestrator to know groups + VLAN's to identify a tenant and ofcourse managing the mappings to best use the physical infrastructure. Thanks, Vishwas On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar wrote: I mean "the first hop router". Each physical port on "the first hop router" can have its own 4095 VLANs. Each Hypervisor can also have its own 4095 VLANs if Hypervisor does the IP encapsulation with a key in the header to differentiate clients (e.g. GRE's KEY field, or VxLan's Client ID field). You can make VLAN locally significant at the overlay edge node if the Overlay Encapsulation header has a field to further differentiate the Clients. Linda From: Vishwas Manral [mailto:vishwas.ietf@gmail.com] Sent: Wednesday, October 05, 2011 3:15 PM To: Linda Dunbar Cc: armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes Linda, I am unsure what you mean by Gateway router, but there is always a Hypervisor or the first hop router that does the encapsulation/ decapsulation. Thanks, Vishwas On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar wrote: Vishwas, Are you assuming that each group of tenants are connected to Gateway router via completely different physical ports? If yes, then you are absolutely correct. Linda From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas Manral Sent: Wednesday, October 05, 2011 10:10 AM To: armd@ietf.org Subject: [armd] Multi-Tenancy without changes Hi, I was thinking of multi-tenancy and the way to achieve > 4094 without any changes in any layer by just using VLAN's. This is achieved as we already have a mapping layer already in place. So assume there are 8000 tenants. We can divide them in groups of 4000 each and assign each tenant a particular id from say 2 to 4001. Now as long as we make sure we do not have tenants from 2 different groups on the same machine, there are just no issues. Only the mapping layer needs to be aware of the group a machine belongs to. There is thus no change in dataplane. Any comments? Thanks, Vishwas _______________________________________________ armd mailing list armd@ietf.org https://www.ietf.org/mailman/listinfo/armd ------=_NextPart_000_0104_01CC8425.58F43EC0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Vishwas:

 

As always, great comments.

 

Thomas Narten is collecting pain points and issues. =

Can you give us details why option (a) VLAN translation at some = boundary will not scale?

 

It would help Thomas’ pain points.

 

Sue

 

From:= = armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of = Ashish Dalela (adalela)
Sent: Wednesday, October 05, 2011 = 11:59 PM
To: Vishwas Manral; Benson Schliesser = (bschlies)
Cc: armd@ietf.org
Subject: Re: [armd] = Multi-Tenancy without changes

 

 

Assumptions about how we can use VLANs are broken down the moment you = think of Hybrid Clouds – i.e. private VLAN stretching out into the = public cloud. In effect, every customer has the full range of 4K VLANs = inside the enterprise and when they span the VLAN into the public cloud, = by implication the entire range is (potentially) spanned into the cloud = as well. There are two ways to deal with this – (a) do VLAN = translation at some boundary or (b) create segmentation orthogonal to = VLAN. I prefer option (b). Option (a) will also work, but will not = scale.

 

Another reason why VLAN as segmentation is not a scalable option = because all inter-VLAN traffic gets pinned to some L3 gateway. Now, if = you have L2 network, with some kind of ECMP, then you don’t get = ECMP for L3 traffic. That’s a huge problem, IMO, especially if you = have a VLAN spanning multiple subnets and the host thinks it is outside = the subnet and the network thinks it is inside the = VLAN.

 

You can design around this and pin the L3 GW at multiple points in = the network – and now you have a different problem – all the = L3->L2 bindings are stored at all the L3 GWs. If you had a million = VMs each host talking to even one host outside the VLAN, then that = traffic goes through the GW. Now, when you apply ECMP, the problem is = quickly spread to all L3 GWs.

 

In summary, using VLAN for segmentation is just inviting too many = problems, that I would like to side step.

 

Ashish

 

From:= = armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of = Vishwas Manral
Sent: Thursday, October 06, 2011 5:34 = AM
To: Benson Schliesser (bschlies)
Cc: = armd@ietf.org
Subject: Re: [armd] Multi-Tenancy without = changes

 

Hi = Benson,

 

> Your proposal was a design "just using = VLAN's" so that there is

> "no change in dataplane". Thus I'm not = sure what a "group" means,

> in the context of your response, unless it = means that the control

> plane enforces some kind of hard = separation.

 

Yes the concept of group is only in the control = plane.A device at one point of time belongs to one group. Yes there is a = limitation of 4k instances on a particular server (however that sounds = like a reasonable limitation) to me. No physical devices need to be = dedicated for a aprticular group, as with virtualization different = tenats can occupy a host machine and can be shifted = around.

 

> Note that I referred to separation of = "switch paths" in my previous

> message. In today's average datacenter, this = requires hard separation

> of switches (or partitions within the = switch). But there are alternative

> switch designs; let me try to imagine what you're = describing.  I suppose

> one could build a switch that completely = localizes VLAN ID on each

> interface, has a very large internal namespace = for tenants, and rewrites

> VLAN ID as frames are forwarded from ingress = to egress interface. 

> (Sounds a bit like a Frame Relay switch...) =  But even with this

> architecture, we're limited to 4k VLANs on a = given path - that limitation

> is built into the frame = format.

> 

> Bringing this back to the ARMD scope:  As I = wrote last week, it doesn't

> really matter how we segregate the tenants. =  As long as the

> = mechanism provides a L2 service, the underlying method of = segregation

>  is = orthogonal to ARP/ND address resolution.  However, the fact that =

> we segregate tenants = might allow us to distribute the address

> resolution function more intelligently.  In = your approach, do you

> imagine distributing L3 gateways and/or ARP proxy = functions

> throughout = each topology / partition?

Yes you are right with this design change in ARMD, in = my view there is not much difference as such other than the = fact that we can divide the ARP tables more intelligently and only to = members of the group that belong to the group so a lot easier for = sure.

 

Thanks,

Vishwas 


 

On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser = <bschlies@cisco.com> = wrote:

Hi, Vishwas -

 

Yes, what you describe is possible.  But the = issue is that it only works if we "do not have tenants from 2 = different groups on the same machine".  Keep in mind that this = applies to all "machines" in the network.  In addition to = the hypervisor, all of the switch paths and router interfaces must also = be segregated.

 

The only way this works is if we partition the = hardware into non-overlapping topologies, such that different tenants = with a conflicting VLAN ID aren't bridged together.  And, as Linda = pointed out, each partition must have a different physical interface to = the L3 gateway. (E.g. by having different routers, or discrete = interfaces on a shared router)

 

Effectively, we do this today when we build multiple = datacenters (including when we build multiple "logical" = datacenters inside the same physical building).  We could pursue a = more complicated approach where an arbitrary number of topologies are = overlaid on a common physical network, constrained by the number of = physical paths etc, but that would be an operational = nightmare.

 

Cheers,

-Benson

 

 

On Oct 5, 2011, at 3:35 PM, Vishwas Manral = wrote:

 

Linda,

 

Thats is a very complicated way of thinking things, = where you have multiple layers.

 

A = simpler solution is to just assume each hypervisor belongs to a = particular group. So it will communicate with only members of the same = group.  The mapping layer/ IP has a unique IP address (not group = specific).

 

You have to understand in the cloud you have an = orchesrtator so there is absolute flexibility in this approach, though = there is some additional load on the orchestrator to know groups = + VLAN's to identify a tenant and ofcourse managing the mappings to = best use the physical infrastructure.

 

Thanks,

Vishwas

On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar <linda.dunbar@huawei.com> = wrote:

I mean “the first hop = router”. Each physical port on “the first hop router” = can have its own 4095 VLANs. Each Hypervisor can also have its own 4095 = VLANs if Hypervisor does the IP encapsulation with a key in the header = to differentiate clients (e.g. GRE’s KEY field, or VxLan’s = Client ID field).

 

You can make VLAN locally = significant at the overlay edge node if the Overlay Encapsulation header = has a field to further differentiate the Clients. =

 

Linda

 

From: Vishwas Manral [mailto:vishwas.ietf@gmail.com]
Sent: = Wednesday, October 05, 2011 3:15 PM
To: Linda = Dunbar
Cc: armd@ietf.org
Subject: Re: [armd] = Multi-Tenancy without = changes

 <= /o:p>

Linda,<= /o:p>

 <= /o:p>

I am unsure = what you mean by Gateway router, but there is always a Hypervisor or the = first hop router that does the encapsulation/ = decapsulation.

 <= /o:p>

Thanks,=

Vishwas=

On Wed, Oct = 5, 2011 at 12:06 PM, Linda Dunbar <linda.dunbar@huawei.com> = wrote:

Vishwas, =

 

Are you assuming that each = group of tenants are connected to Gateway router via completely = different physical ports? If yes, then you are absolutely correct. =

 

Linda

 

From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas = Manral
Sent: Wednesday, October 05, 2011 10:10 = AM
To: armd@ietf.org
Subject: [armd] = Multi-Tenancy without = changes

 <= /o:p>

Hi,

 <= /o:p>

I was = thinking of multi-tenancy and the way to achieve > 4094 without any = changes in any layer by just using VLAN's. This is achieved as we = already have a mapping layer already in = place.

 <= /o:p>

So assume = there are 8000 tenants. We can divide them in groups of 4000 each = and assign each tenant a particular id from say 2 to 4001. Now as long = as we make sure we do not have tenants from 2 different groups on the = same machine, there are just no issues. Only the mapping layer needs to = be aware of the group a machine belongs to. There is thus no change in = dataplane.

 <= /o:p>

Any = comments?

 <= /o:p>

Thanks,=

Vishwas=

 <= /o:p>

 

_______________________________________________
armd= mailing list
armd@ietf.org
https://www.ietf.org/mailman/listinfo/armd

 

 

 

 

------=_NextPart_000_0104_01CC8425.58F43EC0-- From vishwas.ietf@gmail.com Thu Oct 6 21:02:25 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C361D21F85DB for ; Thu, 6 Oct 2011 21:02:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.321 X-Spam-Level: X-Spam-Status: No, score=-3.321 tagged_above=-999 required=5 tests=[AWL=0.277, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qrhx9PnuycFa for ; Thu, 6 Oct 2011 21:02:24 -0700 (PDT) Received: from mail-qy0-f172.google.com (mail-qy0-f172.google.com [209.85.216.172]) by ietfa.amsl.com (Postfix) with ESMTP id DA74421F8B0A for ; Thu, 6 Oct 2011 21:02:23 -0700 (PDT) Received: by qyk32 with SMTP id 32so207848qyk.10 for ; Thu, 06 Oct 2011 21:05:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=hy+BXFNtr9blpDfEfcI0Ffu23BKGHV+u1zDIOE+RgUM=; b=w90LW/3xxJaZXvBkgNoB2esJDPXQZzF2tEzdb0P9zvKqcW+0n35fXVbo15TgcFXUnK L+AIjIx77KYqXhhCSYI2j2N4bdZDxgAQVZGjBg9/WXKKIDcpICEvqkMEBQ+f9OBRrMHC sIHsTQiRydIZCML/Sa+KZFdoprDU+6ba7DgrA= MIME-Version: 1.0 Received: by 10.229.65.86 with SMTP id h22mr1159246qci.225.1317960335942; Thu, 06 Oct 2011 21:05:35 -0700 (PDT) Received: by 10.229.91.131 with HTTP; Thu, 6 Oct 2011 21:05:35 -0700 (PDT) In-Reply-To: <010301cc8446$e005dec0$a0119c40$@com> References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx> <4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx> <7384885B-E229-4C91-9C45-C3982F620E74@cisco.com> <618BE8B40039924EB9AED233D4A09C51022D55FE@XMB-BGL-416.cisco.com> <010301cc8446$e005dec0$a0119c40$@com> Date: Thu, 6 Oct 2011 21:05:35 -0700 Message-ID: From: Vishwas Manral To: Susan Hares Content-Type: multipart/alternative; boundary=0016e64cbc02a6e3a804aead8de8 Cc: Thomas Narten , armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 04:02:25 -0000 --0016e64cbc02a6e3a804aead8de8 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Sue, Sure I can let you know the details. Like I mentioned and as was details very well by Ashish, the main issues are: 1. Behind a particular encapsulator/ decapsulator there can be at most 4096 VLAN's and hence 4096 tenants. 2. Also with using the mechanism ther orchestrator has to have a lot of additional intelligence and will need to optimize the resources of a data center as the number of tenats grow. Thanks, Vishwas On Thu, Oct 6, 2011 at 9:42 AM, Susan Hares wrote: > Vishwas:**** > > ** ** > > As always, great comments. **** > > ** ** > > Thomas Narten is collecting pain points and issues. **** > > Can you give us details why option (a) VLAN translation at some boundary > will not scale? **** > > ** ** > > It would help Thomas=92 pain points. **** > > ** ** > > Sue**** > > ** ** > > *From:* armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] *On Behalf O= f > *Ashish Dalela (adalela) > *Sent:* Wednesday, October 05, 2011 11:59 PM > *To:* Vishwas Manral; Benson Schliesser (bschlies) > > *Cc:* armd@ietf.org > *Subject:* Re: [armd] Multi-Tenancy without changes**** > > ** ** > > ** ** > > Assumptions about how we can use VLANs are broken down the moment you thi= nk > of Hybrid Clouds =96 i.e. private VLAN stretching out into the public clo= ud. > In effect, every customer has the full range of 4K VLANs inside the > enterprise and when they span the VLAN into the public cloud, by implicat= ion > the entire range is (potentially) spanned into the cloud as well. There a= re > two ways to deal with this =96 (a) do VLAN translation at some boundary o= r (b) > create segmentation orthogonal to VLAN. I prefer option (b). Option (a) w= ill > also work, but will not scale.**** > > ** ** > > Another reason why VLAN as segmentation is not a scalable option because > all inter-VLAN traffic gets pinned to some L3 gateway. Now, if you have L= 2 > network, with some kind of ECMP, then you don=92t get ECMP for L3 traffic= . > That=92s a huge problem, IMO, especially if you have a VLAN spanning mult= iple > subnets and the host thinks it is outside the subnet and the network thin= ks > it is inside the VLAN.**** > > ** ** > > You can design around this and pin the L3 GW at multiple points in the > network =96 and now you have a different problem =96 all the L3->L2 bindi= ngs are > stored at all the L3 GWs. If you had a million VMs each host talking to e= ven > one host outside the VLAN, then that traffic goes through the GW. Now, wh= en > you apply ECMP, the problem is quickly spread to all L3 GWs. **** > > ** ** > > In summary, using VLAN for segmentation is just inviting too many problem= s, > that I would like to side step.**** > > ** ** > > Ashish**** > > ** ** > > *From:* armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] *On Behalf O= f > *Vishwas Manral > *Sent:* Thursday, October 06, 2011 5:34 AM > *To:* Benson Schliesser (bschlies) > *Cc:* armd@ietf.org > *Subject:* Re: [armd] Multi-Tenancy without changes**** > > ** ** > > Hi Benson,**** > > **** > > > Your proposal was a design "just using VLAN's" so that there is **** > > > "no change in dataplane". Thus I'm not sure what a "group" means,**** > > > in the context of your response, unless it means that the control **** > > > plane enforces some kind of hard separation.**** > > **** > > Yes the concept of group is only in the control plane.A device at one poi= nt > of time belongs to one group. Yes there is a limitation of 4k instances o= n a > particular server (however that sounds like a reasonable limitation) to m= e. > No physical devices need to be dedicated for a aprticular group, as with > virtualization different tenats can occupy a host machine and can be shif= ted > around.**** > > **** > > > Note that I referred to separation of "switch paths" in my previous ***= * > > > message. In today's average datacenter, this requires hard separation**= * > * > > > of switches (or partitions within the switch). But there are alternativ= e > **** > > > switch designs; let me try to imagine what you're describing. I suppos= e > **** > > > one could build a switch that completely localizes VLAN ID on each **** > > > interface, has a very large internal namespace for tenants, and rewrite= s > **** > > > VLAN ID as frames are forwarded from ingress to egress interface. **** > > > (Sounds a bit like a Frame Relay switch...) But even with this **** > > > architecture, we're limited to 4k VLANs on a given path - that limitati= on > **** > > > is built into the frame format.**** > > >** ** > > > Bringing this back to the ARMD scope: As I wrote last week, it doesn't > **** > > > really matter how we segregate the tenants. As long as the **** > > > mechanism provides a L2 service, the underlying method of segregation**= * > * > > > is orthogonal to ARP/ND address resolution. However, the fact that **= * > * > > > we segregate tenants might allow us to distribute the address **** > > > resolution function more intelligently. In your approach, do you **** > > > imagine distributing L3 gateways and/or ARP proxy functions **** > > > throughout each topology / partition?**** > > Yes you are right with this design change in ARMD, in my view there is no= t > much difference as such other than the fact that we can divide the ARP > tables more intelligently and only to members of the group that belong to > the group so a lot easier for sure.**** > > **** > > Thanks,**** > > Vishwas **** > > > **** > > On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser > wrote:**** > > Hi, Vishwas - **** > > ** ** > > Yes, what you describe is possible. But the issue is that it only works = if > we "do not have tenants from 2 different groups on the same machine". Ke= ep > in mind that this applies to all "machines" in the network. In addition = to > the hypervisor, all of the switch paths and router interfaces must also b= e > segregated.**** > > ** ** > > The only way this works is if we partition the hardware into > non-overlapping topologies, such that different tenants with a conflictin= g > VLAN ID aren't bridged together. And, as Linda pointed out, each partiti= on > must have a different physical interface to the L3 gateway. (E.g. by havi= ng > different routers, or discrete interfaces on a shared router)**** > > ** ** > > Effectively, we do this today when we build multiple datacenters (includi= ng > when we build multiple "logical" datacenters inside the same physical > building). We could pursue a more complicated approach where an arbitrar= y > number of topologies are overlaid on a common physical network, constrain= ed > by the number of physical paths etc, but that would be an operational > nightmare.**** > > ** ** > > Cheers,**** > > -Benson**** > > ** ** > > ** ** > > On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote:**** > > ** ** > > Linda, **** > > **** > > Thats is a very complicated way of thinking things, where you have multip= le > layers. **** > > **** > > A simpler solution is to just assume each hypervisor belongs to a > particular group. So it will communicate with only members of the same > group. The mapping layer/ IP has a unique IP address (not group specific= ). > **** > > **** > > You have to understand in the cloud you have an orchesrtator so there is > absolute flexibility in this approach, though there is some additional lo= ad > on the orchestrator to know groups + VLAN's to identify a tenant and > ofcourse managing the mappings to best use the physical infrastructure.**= * > * > > **** > > Thanks,**** > > Vishwas**** > > On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar > wrote:**** > > I mean =93the first hop router=94. Each physical port on =93the first hop= router=94 > can have its own 4095 VLANs. Each Hypervisor can also have its own 4095 > VLANs if Hypervisor does the IP encapsulation with a key in the header to > differentiate clients (e.g. GRE=92s KEY field, or VxLan=92s Client ID fie= ld). > **** > > **** > > You can make VLAN locally significant at the overlay edge node if the > Overlay Encapsulation header has a field to further differentiate the > Clients. **** > > **** > > Linda**** > > **** > > *From:* Vishwas Manral [mailto:vishwas.ietf@gmail.com] > *Sent:* Wednesday, October 05, 2011 3:15 PM > *To:* Linda Dunbar > *Cc:* armd@ietf.org > *Subject:* Re: [armd] Multi-Tenancy without changes**** > > **** > > Linda,**** > > **** > > I am unsure what you mean by Gateway router, but there is always a > Hypervisor or the first hop router that does the encapsulation/ > decapsulation.**** > > **** > > Thanks,**** > > Vishwas**** > > On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar > wrote:**** > > Vishwas, **** > > **** > > Are you assuming that each group of tenants are connected to Gateway rout= er > via completely different physical ports? If yes, then you are absolutely > correct. **** > > **** > > Linda**** > > **** > > *From:* armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] *On Behalf O= f > *Vishwas Manral > *Sent:* Wednesday, October 05, 2011 10:10 AM > *To:* armd@ietf.org > *Subject:* [armd] Multi-Tenancy without changes**** > > **** > > Hi,**** > > **** > > I was thinking of multi-tenancy and the way to achieve > 4094 without any > changes in any layer by just using VLAN's. This is achieved as we already > have a mapping layer already in place.**** > > **** > > So assume there are 8000 tenants. We can divide them in groups of 4000 ea= ch > and assign each tenant a particular id from say 2 to 4001. Now as long as= we > make sure we do not have tenants from 2 different groups on the same > machine, there are just no issues. Only the mapping layer needs to be awa= re > of the group a machine belongs to. There is thus no change in dataplane.*= * > ** > > **** > > Any comments?**** > > **** > > Thanks,**** > > Vishwas**** > > **** > > ** ** > > _______________________________________________ > armd mailing list > armd@ietf.org > https://www.ietf.org/mailman/listinfo/armd**** > > ** ** > > ** ** > > ** ** > > ** ** > --0016e64cbc02a6e3a804aead8de8 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Hi Sue,
=A0
Sure I can let you know the details. Like I mentioned and as was detai= ls=A0very well by Ashish, the main issues are:
=A0
1.=A0Behind a particular encapsulator/ decapsulator there can be at mo= st 4096 VLAN's and hence 4096 tenants.
2. Also with using the mechanism ther orchestrator has to have a lot o= f additional intelligence and will need to optimize the resources of a data= center as the number of tenats grow.
=A0
Thanks,
Vishwas
On Thu, Oct 6, 2011 at 9:42 AM, Susan Hares <skh@ndzh.com> wrote:

Vish= was:

<= /u>=A0

As a= lways, great comments.

<= /u>=A0

Thom= as Narten is collecting pain points and issues.

Can = you give us details why option (a) VLAN translation at some boundary will n= ot scale?

<= /u>=A0

It w= ould help Thomas=92 pain points.

<= /u>=A0

Sue<= u>

<= /u>=A0

From:<= span style=3D"FONT-SIZE: 10pt"> armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf O= f Ashish Dalela (adalela)
Sent: Wednesday, October 05, 2011 11:59 PM
To: Vishwas Man= ral; Benson Schliesser (bschlies)=20


Cc: armd@ietf.org
Subject: Re: [armd] Multi-Tenancy with= out changes

=A0

<= /u>=A0

Assu= mptions about how we can use VLANs are broken down the moment you think of = Hybrid Clouds =96 i.e. private VLAN stretching out into the public cloud. I= n effect, every customer has the full range of 4K VLANs inside the enterpri= se and when they span the VLAN into the public cloud, by implication the en= tire range is (potentially) spanned into the cloud as well. There are two w= ays to deal with this =96 (a) do VLAN translation at some boundary or (b) c= reate segmentation orthogonal to VLAN. I prefer option (b). Option (a) will= also work, but will not scale.

<= /u>=A0

Anot= her reason why VLAN as segmentation is not a scalable option because all in= ter-VLAN traffic gets pinned to some L3 gateway. Now, if you have L2 networ= k, with some kind of ECMP, then you don=92t get ECMP for L3 traffic. That= =92s a huge problem, IMO, especially if you have a VLAN spanning multiple s= ubnets and the host thinks it is outside the subnet and the network thinks = it is inside the VLAN.

<= /u>=A0

You = can design around this and pin the L3 GW at multiple points in the network = =96 and now you have a different problem =96 all the L3->L2 bindings are= stored at all the L3 GWs. If you had a million VMs each host talking to ev= en one host outside the VLAN, then that traffic goes through the GW. Now, w= hen you apply ECMP, the problem is quickly spread to all L3 GWs. =

<= /u>=A0

In s= ummary, using VLAN for segmentation is just inviting too many problems, tha= t I would like to side step.

<= /u>=A0

Ashi= sh

<= /u>=A0

From:<= span style=3D"FONT-SIZE: 10pt"> armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf O= f Vishwas Manral
Sent: Thursday, October 06, 2011 5:34 AM
To: Benson Schlie= sser (bschlies)
Cc: armd@ietf.org
Subject: Re: [armd] Multi-Tenancy without= changes

=A0

Hi Benson,

=A0

> Your proposal was a design "just using VLA= N's" so that there is

> "no change in dataplane". Thus I'= m not sure what a "group" means,

>=A0in the context of your response, unless it me= ans that the control

> plane enforces some kind of hard separation.=

=A0

Yes the concept of group is only in the control plan= e.A device at one point of time belongs to one group. Yes there is a limita= tion of 4k instances on a particular server (however that sounds like a rea= sonable limitation) to me. No physical devices need to be dedicated for a a= prticular group, as with virtualization different tenats can occupy a host = machine and can be shifted around.

=A0

> Note that I referred to separation of "swi= tch paths" in my previous

> message. In today's average datacenter, thi= s requires hard separation

>=A0of switches (or partitions within the switch)= . But there are alternative

> switch designs; let me try to imagine what you&= #39;re describing.=A0 I suppose

>=A0one could build a switch that completely loca= lizes VLAN ID on each

> interface, has a very large internal namespace = for tenants, and rewrites

>=A0VLAN ID as frames are forwarded from ingress = to egress interface.=A0

> (Sounds a bit like a Frame Relay switch...) =A0= But even with this

> architecture, we're limited to 4k VLANs on = a given path - that limitation

>=A0is built into the frame format.=

>=A0

> Bringing this back to the ARMD scope: =A0As I w= rote last week, it doesn't

> really matter how we segregate the tenants. =A0= As long as the

> mechanism provides a L2 service, the underlying= method of segregation

> =A0is orthogonal to ARP/ND address resolution. = =A0However, the fact that

> we segregate tenants might allow us to distribu= te the address

> resolution function more intelligently. =A0In y= our approach, do you

> imagine distributing L3 gateways and/or ARP pro= xy functions

> throughout each topology / partition?=

Yes you are right with this design change in ARMD, i= n my view there is not much=A0difference=A0as such other than the fact that= we can divide the ARP tables more intelligently and only to members of the= group that belong to the group so a lot easier for sure.

=A0

Thanks,

Vishwas=A0


=A0

On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser &l= t;bschlies@cisco.co= m> wrote:

Hi, Vishwas -

=A0

Yes, what you describe is possible. =A0But the issue= is that it only works if we "do not have tenants from 2 different gro= ups on the same machine". =A0Keep in mind that this applies to all &qu= ot;machines" in the network. =A0In addition to the hypervisor, all of = the switch paths and router interfaces must also be segregated.

=A0

The only way this works is if we partition the hardw= are into non-overlapping topologies, such that different tenants with a con= flicting VLAN ID aren't bridged together. =A0And, as Linda pointed out,= each partition must have a different physical interface to the L3 gateway.= (E.g. by having different routers, or discrete interfaces on a shared rout= er)

=A0

Effectively, we do this today when we build multiple= datacenters (including when we build multiple "logical" datacent= ers inside the same physical building). =A0We could pursue a more complicat= ed approach where an arbitrary number of topologies are overlaid on a commo= n physical network, constrained by the number of physical paths etc, but th= at would be an operational nightmare.

=A0

Cheers,

-Benson

=A0

=A0

On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote:=

=A0

Linda,

=A0

Thats is a very complicated way of thinking things, = where you have multiple layers.

=A0

A simpler solution is to just assume each hypervisor= belongs to a particular group. So it will communicate with only members of= the same group.=A0 The mapping layer/ IP has a unique IP address (not grou= p specific).

=A0

You have to understand in the cloud you have an orch= esrtator so there is absolute flexibility in this approach, though there is= some additional load on the orchestrator to know groups +=A0VLAN's to = identify a tenant and ofcourse managing the mappings to best use the physic= al infrastructure.

=A0

Thanks,

Vishwas

On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar <linda.dunbar@huaw= ei.com> wrote:

I me= an =93the first hop router=94. Each physical port on =93the first hop route= r=94 can have its own 4095 VLANs. Each Hypervisor can also have its own 409= 5 VLANs if Hypervisor does the IP encapsulation with a key in the header to= differentiate clients (e.g. GRE=92s KEY field, or VxLan=92s Client ID fiel= d).

=A0<= /span>

You = can make VLAN locally significant at the overlay edge node if the Overlay E= ncapsulation header has a field to further differentiate the Clients.

=A0<= /span>

Lind= a

=A0<= /span>

From:<= span style=3D"FONT-SIZE: 10pt"> Vishwas Manral [mailto:vishwas.ietf@gmail.com]
= Sent: Wednesday, October 05, 2011 3:15 PM
To: Linda Dunbar
Cc: armd@ietf.org
Subject: Re: [armd] Multi-Tenancy= without changes

=A0

Linda,

=A0

I am unsure what you mean by Gateway router, but the= re is always a Hypervisor or the first hop router that does the encapsulati= on/ decapsulation.

=A0

Thanks,

Vishwas

On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar <linda.dunbar@hua= wei.com> wrote:

Vish= was,

=A0<= /span>

Are = you assuming that each group of tenants are connected to Gateway router via= completely different physical ports? If yes, then you are absolutely corre= ct.

=A0<= /span>

Lind= a

=A0<= /span>

From:<= span style=3D"FONT-SIZE: 10pt"> armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf O= f Vishwas Manral
Sent: Wednesday, October 05, 2011 10:10 AM
To: armd@ietf.org
Subject: [armd] Multi-Tenancy without changes

=A0

Hi,

=A0

I was thinking of multi-tenancy and the way to achie= ve > 4094 without any changes in any layer by just using VLAN's. Thi= s is achieved as we already have a mapping layer already in place.

=A0

So assume there are 8000 tenants.=A0We can divide th= em in groups of 4000 each and assign each tenant a particular id from say 2= to 4001. Now as long as we make sure we do not have tenants from 2 differe= nt groups on the same machine, there are just no issues. Only the mapping l= ayer needs to be aware of the group a machine belongs to. There is thus no = change in dataplane.

=A0

Any comments?

=A0

Thanks,

Vishwas

=A0

<= /div>

=A0

_______________________________________________
a= rmd mailing list
armd= @ietf.org
https://www.ietf.org/mailman/listinfo/armd=

=A0

=A0

=A0

=A0


--0016e64cbc02a6e3a804aead8de8-- From adalela@cisco.com Thu Oct 6 22:03:33 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4FA9921F8ABB for ; Thu, 6 Oct 2011 22:03:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.248 X-Spam-Level: X-Spam-Status: No, score=-8.248 tagged_above=-999 required=5 tests=[AWL=2.350, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0pp516aqtkcD for ; Thu, 6 Oct 2011 22:03:27 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 1183821F8A95 for ; Thu, 6 Oct 2011 22:03:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=adalela@cisco.com; l=46851; q=dns/txt; s=iport; t=1317963999; x=1319173599; h=mime-version:subject:date:message-id:in-reply-to: references:from:to:cc; bh=D+nKf/fros6y5KR6w5j8V/2hpv2RTartkLAO855kbcU=; b=S4OOU80ZO18gMnSgKbBzux7xcYHOfa2SCY4Fnx86uSxWGjO6ZzvSQN7i WF5ov6mF5aKvBm873ioqhLF6E8pfOQmXLyliOxLW6pv2GuH4YUcQiFcsV 5bknC8521AmtSSQRoQsFek59xrddiI7OntjjzCcY33MXWQcqnjxzE6Nzu 4=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqkAAPKHjk5Io8UQ/2dsb2JhbAA6CoJNlk+PHIEFgVMBAQEBAgEBAQEPAQkLBgM+CwULAgEIEQEDAQELBhABBgEGASAGHwMGCAEBBAEKCAgTB4dcB5o4AZ4SA4QFgkxhBId7kSWEcYct X-IronPort-AV: E=Sophos;i="4.68,499,1312156800"; d="scan'208,217";a="57276483" Received: from bgl-core-1.cisco.com ([72.163.197.16]) by ams-iport-2.cisco.com with ESMTP; 07 Oct 2011 05:06:32 +0000 Received: from xbh-bgl-412.cisco.com (xbh-bgl-412.cisco.com [72.163.129.202]) by bgl-core-1.cisco.com (8.14.3/8.14.3) with ESMTP id p9756Dg0017019; Fri, 7 Oct 2011 05:06:20 GMT Received: from xmb-bgl-416.cisco.com ([72.163.129.212]) by xbh-bgl-412.cisco.com with Microsoft SMTPSVC(6.0.3790.4675); Fri, 7 Oct 2011 10:36:20 +0530 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01CC84AE.D9D72EB1" Date: Fri, 7 Oct 2011 10:36:15 +0530 Message-ID: <618BE8B40039924EB9AED233D4A09C51022D56ED@XMB-BGL-416.cisco.com> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [armd] Multi-Tenancy without changes thread-index: AcyEpmL6KZdMxcUHSCSHLZHgCf/08gAADoeA References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx><4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx><7384885B-E229-4C91-9C45-C3982F620E74@cisco.com><618BE8B40039924EB9AED233D4A09C51022D55FE@XMB-BGL-416.cisco.com><010301cc8446$e005dec0$a0119c40$@com> From: "Ashish Dalela (adalela)" To: "Vishwas Manral" , "Susan Hares" X-OriginalArrivalTime: 07 Oct 2011 05:06:20.0575 (UTC) FILETIME=[DA222AF0:01CC84AE] Cc: Thomas Narten , armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 05:03:33 -0000 This is a multi-part message in MIME format. ------_=_NextPart_001_01CC84AE.D9D72EB1 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Specifically on VLAN translation, if you have only 2 sites spanning VLANs with different id's then you need a table to statically map one VLAN to another. This doesn't seem too bad if you only use 4096 VLANs. But, if you have more than 2 sites, spanning the same VLAN as different VLAN id's then you need to have per MAC entry mapping it to a new VLAN. Total entries equal total MAC. Obviously, we will design for multiple sites. =20 If your DCI already required you to have per host entry at the edge, then you are not any worse off by adding VLAN translation to it. But, if not, VLAN translation forces you to have a per-host entry. Either way, it's not a scalable solution.=20 =20 Ashish =20 From: Vishwas Manral [mailto:vishwas.ietf@gmail.com]=20 Sent: Friday, October 07, 2011 9:36 AM To: Susan Hares Cc: Ashish Dalela (adalela); Benson Schliesser (bschlies); armd@ietf.org; Thomas Narten Subject: Re: [armd] Multi-Tenancy without changes =20 Hi Sue, =20 Sure I can let you know the details. Like I mentioned and as was details very well by Ashish, the main issues are: =20 1. Behind a particular encapsulator/ decapsulator there can be at most 4096 VLAN's and hence 4096 tenants. 2. Also with using the mechanism ther orchestrator has to have a lot of additional intelligence and will need to optimize the resources of a data center as the number of tenats grow. =20 Thanks, Vishwas On Thu, Oct 6, 2011 at 9:42 AM, Susan Hares wrote: Vishwas: =20 As always, great comments.=20 =20 Thomas Narten is collecting pain points and issues.=20 Can you give us details why option (a) VLAN translation at some boundary will not scale?=20 =20 It would help Thomas' pain points.=20 =20 Sue =20 From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Ashish Dalela (adalela) Sent: Wednesday, October 05, 2011 11:59 PM To: Vishwas Manral; Benson Schliesser (bschlies)=20 Cc: armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes =20 =20 Assumptions about how we can use VLANs are broken down the moment you think of Hybrid Clouds - i.e. private VLAN stretching out into the public cloud. In effect, every customer has the full range of 4K VLANs inside the enterprise and when they span the VLAN into the public cloud, by implication the entire range is (potentially) spanned into the cloud as well. There are two ways to deal with this - (a) do VLAN translation at some boundary or (b) create segmentation orthogonal to VLAN. I prefer option (b). Option (a) will also work, but will not scale. =20 Another reason why VLAN as segmentation is not a scalable option because all inter-VLAN traffic gets pinned to some L3 gateway. Now, if you have L2 network, with some kind of ECMP, then you don't get ECMP for L3 traffic. That's a huge problem, IMO, especially if you have a VLAN spanning multiple subnets and the host thinks it is outside the subnet and the network thinks it is inside the VLAN. =20 You can design around this and pin the L3 GW at multiple points in the network - and now you have a different problem - all the L3->L2 bindings are stored at all the L3 GWs. If you had a million VMs each host talking to even one host outside the VLAN, then that traffic goes through the GW. Now, when you apply ECMP, the problem is quickly spread to all L3 GWs.=20 =20 In summary, using VLAN for segmentation is just inviting too many problems, that I would like to side step. =20 Ashish =20 From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas Manral Sent: Thursday, October 06, 2011 5:34 AM To: Benson Schliesser (bschlies) Cc: armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes =20 Hi Benson, =20 > Your proposal was a design "just using VLAN's" so that there is=20 > "no change in dataplane". Thus I'm not sure what a "group" means, > in the context of your response, unless it means that the control=20 > plane enforces some kind of hard separation. =20 Yes the concept of group is only in the control plane.A device at one point of time belongs to one group. Yes there is a limitation of 4k instances on a particular server (however that sounds like a reasonable limitation) to me. No physical devices need to be dedicated for a aprticular group, as with virtualization different tenats can occupy a host machine and can be shifted around. =20 > Note that I referred to separation of "switch paths" in my previous=20 > message. In today's average datacenter, this requires hard separation > of switches (or partitions within the switch). But there are alternative=20 > switch designs; let me try to imagine what you're describing. I suppose > one could build a switch that completely localizes VLAN ID on each=20 > interface, has a very large internal namespace for tenants, and rewrites > VLAN ID as frames are forwarded from ingress to egress interface.=20 > (Sounds a bit like a Frame Relay switch...) But even with this=20 > architecture, we're limited to 4k VLANs on a given path - that limitation > is built into the frame format. >=20 > Bringing this back to the ARMD scope: As I wrote last week, it doesn't=20 > really matter how we segregate the tenants. As long as the=20 > mechanism provides a L2 service, the underlying method of segregation > is orthogonal to ARP/ND address resolution. However, the fact that=20 > we segregate tenants might allow us to distribute the address=20 > resolution function more intelligently. In your approach, do you=20 > imagine distributing L3 gateways and/or ARP proxy functions=20 > throughout each topology / partition? Yes you are right with this design change in ARMD, in my view there is not much difference as such other than the fact that we can divide the ARP tables more intelligently and only to members of the group that belong to the group so a lot easier for sure. =20 Thanks, Vishwas=20 =20 On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser wrote: Hi, Vishwas -=20 =20 Yes, what you describe is possible. But the issue is that it only works if we "do not have tenants from 2 different groups on the same machine". Keep in mind that this applies to all "machines" in the network. In addition to the hypervisor, all of the switch paths and router interfaces must also be segregated. =20 The only way this works is if we partition the hardware into non-overlapping topologies, such that different tenants with a conflicting VLAN ID aren't bridged together. And, as Linda pointed out, each partition must have a different physical interface to the L3 gateway. (E.g. by having different routers, or discrete interfaces on a shared router) =20 Effectively, we do this today when we build multiple datacenters (including when we build multiple "logical" datacenters inside the same physical building). We could pursue a more complicated approach where an arbitrary number of topologies are overlaid on a common physical network, constrained by the number of physical paths etc, but that would be an operational nightmare. =20 Cheers, -Benson =20 =20 On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote: =20 Linda,=20 =20 Thats is a very complicated way of thinking things, where you have multiple layers.=20 =20 A simpler solution is to just assume each hypervisor belongs to a particular group. So it will communicate with only members of the same group. The mapping layer/ IP has a unique IP address (not group specific).=20 =20 You have to understand in the cloud you have an orchesrtator so there is absolute flexibility in this approach, though there is some additional load on the orchestrator to know groups + VLAN's to identify a tenant and ofcourse managing the mappings to best use the physical infrastructure. =20 Thanks, Vishwas On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar wrote: I mean "the first hop router". Each physical port on "the first hop router" can have its own 4095 VLANs. Each Hypervisor can also have its own 4095 VLANs if Hypervisor does the IP encapsulation with a key in the header to differentiate clients (e.g. GRE's KEY field, or VxLan's Client ID field).=20 =20 You can make VLAN locally significant at the overlay edge node if the Overlay Encapsulation header has a field to further differentiate the Clients.=20 =20 Linda =20 From: Vishwas Manral [mailto:vishwas.ietf@gmail.com]=20 Sent: Wednesday, October 05, 2011 3:15 PM To: Linda Dunbar Cc: armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes =20 Linda, =20 I am unsure what you mean by Gateway router, but there is always a Hypervisor or the first hop router that does the encapsulation/ decapsulation. =20 Thanks, Vishwas On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar wrote: Vishwas,=20 =20 Are you assuming that each group of tenants are connected to Gateway router via completely different physical ports? If yes, then you are absolutely correct.=20 =20 Linda =20 From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas Manral Sent: Wednesday, October 05, 2011 10:10 AM To: armd@ietf.org Subject: [armd] Multi-Tenancy without changes =20 Hi, =20 I was thinking of multi-tenancy and the way to achieve > 4094 without any changes in any layer by just using VLAN's. This is achieved as we already have a mapping layer already in place. =20 So assume there are 8000 tenants. We can divide them in groups of 4000 each and assign each tenant a particular id from say 2 to 4001. Now as long as we make sure we do not have tenants from 2 different groups on the same machine, there are just no issues. Only the mapping layer needs to be aware of the group a machine belongs to. There is thus no change in dataplane. =20 Any comments? =20 Thanks, Vishwas =20 =20 =09 _______________________________________________ armd mailing list armd@ietf.org =09 https://www.ietf.org/mailman/listinfo/armd =20 =20 =20 =20 =20 ------_=_NextPart_001_01CC84AE.D9D72EB1 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Specifically on VLAN translation, if you have only 2 sites spanning = VLANs with different id’s then you need a table to statically map = one VLAN to another. This doesn’t seem too bad if you only use = 4096 VLANs. But, if you have more than 2 sites, spanning the same VLAN = as different VLAN id’s then you need to have per MAC entry mapping = it to a new VLAN. Total entries equal total MAC. Obviously, we will = design for multiple sites.

 

If your DCI already required you to have per host entry at the edge, = then you are not any worse off by adding VLAN translation to it. But, if = not, VLAN translation forces you to have a per-host entry. Either way, = it’s not a scalable solution.

 

Ashish

 

From:= = Vishwas Manral [mailto:vishwas.ietf@gmail.com]
Sent: Friday, = October 07, 2011 9:36 AM
To: Susan Hares
Cc: Ashish = Dalela (adalela); Benson Schliesser (bschlies); armd@ietf.org; Thomas = Narten
Subject: Re: [armd] Multi-Tenancy without = changes

 

Hi = Sue,

 

Sure I can let you know the details. Like I mentioned = and as was details very well by Ashish, the main issues = are:

 

1. Behind a particular encapsulator/ decapsulator = there can be at most 4096 VLAN's and hence 4096 = tenants.

2. Also with = using the mechanism ther orchestrator has to have a lot of additional = intelligence and will need to optimize the resources of a data center as = the number of tenats grow.

 

Thanks,

Vishwas

On Thu, Oct 6, 2011 at 9:42 AM, Susan Hares <skh@ndzh.com> = wrote:

Vishwas:

 

As always, great comments. =

 

Thomas Narten is collecting = pain points and issues.

Can you give us details why = option (a) VLAN translation at some boundary will not scale? =

 

It would help Thomas’ = pain points.

 

Sue

 

From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Ashish = Dalela (adalela)
Sent: Wednesday, October 05, 2011 11:59 = PM
To: Vishwas Manral; Benson Schliesser (bschlies) =


Cc: armd@ietf.org
Subject: Re: [armd] = Multi-Tenancy without = changes

 <= /o:p>

 

Assumptions about how we can = use VLANs are broken down the moment you think of Hybrid Clouds – = i.e. private VLAN stretching out into the public cloud. In effect, every = customer has the full range of 4K VLANs inside the enterprise and when = they span the VLAN into the public cloud, by implication the entire = range is (potentially) spanned into the cloud as well. There are two = ways to deal with this – (a) do VLAN translation at some boundary = or (b) create segmentation orthogonal to VLAN. I prefer option (b). = Option (a) will also work, but will not scale.

 

Another reason why VLAN as = segmentation is not a scalable option because all inter-VLAN traffic = gets pinned to some L3 gateway. Now, if you have L2 network, with some = kind of ECMP, then you don’t get ECMP for L3 traffic. That’s = a huge problem, IMO, especially if you have a VLAN spanning multiple = subnets and the host thinks it is outside the subnet and the network = thinks it is inside the VLAN.

 

You can design around this and = pin the L3 GW at multiple points in the network – and now you have = a different problem – all the L3->L2 bindings are stored at all = the L3 GWs. If you had a million VMs each host talking to even one host = outside the VLAN, then that traffic goes through the GW. Now, when you = apply ECMP, the problem is quickly spread to all L3 GWs. =

 

In summary, using VLAN for = segmentation is just inviting too many problems, that I would like to = side step.

 

Ashish

 

From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas = Manral
Sent: Thursday, October 06, 2011 5:34 AM
To: = Benson Schliesser (bschlies)
Cc: armd@ietf.org
Subject: Re: [armd] = Multi-Tenancy without changes

 <= /o:p>

Hi = Benson,

 <= /o:p>

> Your = proposal was a design "just using VLAN's" so that there is =

> = "no change in dataplane". Thus I'm not sure what a = "group" means,

> in= the context of your response, unless it means that the control =

> plane = enforces some kind of hard separation.

 <= /o:p>

Yes the = concept of group is only in the control plane.A device at one point of = time belongs to one group. Yes there is a limitation of 4k instances on = a particular server (however that sounds like a reasonable limitation) = to me. No physical devices need to be dedicated for a aprticular group, = as with virtualization different tenats can occupy a host machine and = can be shifted around.

 <= /o:p>

> Note = that I referred to separation of "switch paths" in my previous =

> = message. In today's average datacenter, this requires hard = separation

> of= switches (or partitions within the switch). But there are alternative =

> switch = designs; let me try to imagine what you're describing.  I = suppose

> on= e could build a switch that completely localizes VLAN ID on each =

> = interface, has a very large internal namespace for tenants, and = rewrites

> VL= AN ID as frames are forwarded from ingress to egress = interface. 

> = (Sounds a bit like a Frame Relay switch...)  But even with this =

> = architecture, we're limited to 4k VLANs on a given path - that = limitation

> is= built into the frame format.

> 

> = Bringing this back to the ARMD scope:  As I wrote last week, it = doesn't

> really = matter how we segregate the tenants.  As long as the =

> = mechanism provides a L2 service, the underlying method of = segregation

> =  is orthogonal to ARP/ND address resolution.  However, the = fact that

> we = segregate tenants might allow us to distribute the address =

> = resolution function more intelligently.  In your approach, do you =

> = imagine distributing L3 gateways and/or ARP proxy functions =

> = throughout each topology / partition?

Yes you are = right with this design change in ARMD, in my view there is not = much difference as such other than the fact that we can divide = the ARP tables more intelligently and only to members of the group that = belong to the group so a lot easier for = sure.

 <= /o:p>

Thanks,=

Vishwas = ;


 

On Wed, Oct = 5, 2011 at 2:15 PM, Benson Schliesser <bschlies@cisco.com> = wrote:

Hi, Vishwas = -

 <= /o:p>

Yes, what = you describe is possible.  But the issue is that it only works if = we "do not have tenants from 2 different groups on the same = machine".  Keep in mind that this applies to all = "machines" in the network.  In addition to the = hypervisor, all of the switch paths and router interfaces must also be = segregated.

 <= /o:p>

The only = way this works is if we partition the hardware into non-overlapping = topologies, such that different tenants with a conflicting VLAN ID = aren't bridged together.  And, as Linda pointed out, each partition = must have a different physical interface to the L3 gateway. (E.g. by = having different routers, or discrete interfaces on a shared = router)

 <= /o:p>

Effectively,= we do this today when we build multiple datacenters (including when we = build multiple "logical" datacenters inside the same physical = building).  We could pursue a more complicated approach where an = arbitrary number of topologies are overlaid on a common physical = network, constrained by the number of physical paths etc, but that would = be an operational nightmare.

 <= /o:p>

Cheers,=

-Benson=

 <= /o:p>

 <= /o:p>

On Oct 5, = 2011, at 3:35 PM, Vishwas Manral wrote:

 <= /o:p>

Linda, =

 <= /o:p>

Thats is a = very complicated way of thinking things, where you have multiple layers. =

 <= /o:p>

A simpler = solution is to just assume each hypervisor belongs to a particular = group. So it will communicate with only members of the same group.  = The mapping layer/ IP has a unique IP address (not group specific). =

 <= /o:p>

You have to = understand in the cloud you have an orchesrtator so there is absolute = flexibility in this approach, though there is some additional load on = the orchestrator to know groups + VLAN's to identify a tenant and = ofcourse managing the mappings to best use the physical = infrastructure.

 <= /o:p>

Thanks,=

Vishwas=

On Wed, Oct = 5, 2011 at 1:25 PM, Linda Dunbar <linda.dunbar@huawei.com> = wrote:

I mean “the first hop = router”. Each physical port on “the first hop router” = can have its own 4095 VLANs. Each Hypervisor can also have its own 4095 = VLANs if Hypervisor does the IP encapsulation with a key in the header = to differentiate clients (e.g. GRE’s KEY field, or VxLan’s = Client ID field).

 

You can make VLAN locally = significant at the overlay edge node if the Overlay Encapsulation header = has a field to further differentiate the Clients. =

 

Linda

 

From: Vishwas Manral [mailto:vishwas.ietf@gmail.com]
Sent: = Wednesday, October 05, 2011 3:15 PM
To: Linda = Dunbar
Cc: armd@ietf.org
Subject: Re: [armd] = Multi-Tenancy without = changes

 <= /o:p>

Linda,<= /o:p>

 <= /o:p>

I am unsure = what you mean by Gateway router, but there is always a Hypervisor or the = first hop router that does the encapsulation/ = decapsulation.

 <= /o:p>

Thanks,=

Vishwas=

On Wed, Oct = 5, 2011 at 12:06 PM, Linda Dunbar <linda.dunbar@huawei.com> = wrote:

Vishwas, =

 

Are you assuming that each = group of tenants are connected to Gateway router via completely = different physical ports? If yes, then you are absolutely correct. =

 

Linda

 

From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Vishwas = Manral
Sent: Wednesday, October 05, 2011 10:10 = AM
To: armd@ietf.org
Subject: [armd] = Multi-Tenancy without = changes

 <= /o:p>

Hi,

 <= /o:p>

I was = thinking of multi-tenancy and the way to achieve > 4094 without any = changes in any layer by just using VLAN's. This is achieved as we = already have a mapping layer already in = place.

 <= /o:p>

So assume = there are 8000 tenants. We can divide them in groups of 4000 each = and assign each tenant a particular id from say 2 to 4001. Now as long = as we make sure we do not have tenants from 2 different groups on the = same machine, there are just no issues. Only the mapping layer needs to = be aware of the group a machine belongs to. There is thus no change in = dataplane.

 <= /o:p>

Any = comments?

 <= /o:p>

Thanks,=

Vishwas=

 <= /o:p>

 <= /o:p>

____________= ___________________________________
armd mailing list
armd@ietf.org
https://www.ietf.org/mailman/listinfo/armd

 <= /o:p>

 <= /o:p>

 <= /o:p>

 <= /o:p>

 

------_=_NextPart_001_01CC84AE.D9D72EB1-- From bschlies@cisco.com Fri Oct 7 10:08:45 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 20A4121F8C58 for ; Fri, 7 Oct 2011 10:08:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nhEXlZBw0oWd for ; Fri, 7 Oct 2011 10:08:44 -0700 (PDT) Received: from mtv-iport-4.cisco.com (mtv-iport-4.cisco.com [173.36.130.15]) by ietfa.amsl.com (Postfix) with ESMTP id A218021F8C57 for ; Fri, 7 Oct 2011 10:08:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=bschlies@cisco.com; l=1638; q=dns/txt; s=iport; t=1318007519; x=1319217119; h=from:subject:date:message-id:to:mime-version; bh=Wa2dYf8Bc1GbhgpjLcbOWPW837agrvTqrKnJEnzWMvw=; b=C9OC25Mny23LB7xWFqsz0te5bLde13g6u7J7+0l/rv+VSq/iHyzrDepw fuh6QN6NrYbIvhLexE2eHCzxvVORyUq9taZbOEL2lGDHODlAAsFWafBmK vy7splQb+9WQ1XambX8W5AFb505CT40A/xoZ8SKFo2ksN0pXSS/Iij8hx k=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArANAJUyj06rRDoH/2dsb2JhbABEmWKHP4cdgQWBSgUdAYEEAYE7GYdUD5hfgSYBni2GUGEEh32LcpFm X-IronPort-AV: E=Sophos;i="4.68,503,1312156800"; d="scan'208,217";a="6547593" Received: from mtv-core-2.cisco.com ([171.68.58.7]) by mtv-iport-4.cisco.com with ESMTP; 07 Oct 2011 17:11:43 +0000 Received: from sjc-bschlies-8915.cisco.com (sjc-bschlies-8915.cisco.com [10.20.217.230]) by mtv-core-2.cisco.com (8.14.3/8.14.3) with ESMTP id p97HBfID003195 for ; Fri, 7 Oct 2011 17:11:42 GMT From: Benson Schliesser Content-Type: multipart/alternative; boundary=Apple-Mail-6-397885366 Date: Fri, 7 Oct 2011 12:11:41 -0500 Message-Id: To: armd@ietf.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Subject: [armd] Reminder: IETF82 draft cut-off dates X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 17:08:45 -0000 --Apple-Mail-6-397885366 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Just a brief reminder of the cut-off dates for IETF82. http://www.ietf.org/meeting/cutoff-dates-2011.html#IETF82 * 2011-10-24 (Monday): Internet Draft Cut-off for initial document (-00) = submission by 17:00 PT (00:00 UTC) * 2011-10-31 (Monday): Internet Draft final submission cut-off by 17:00 = PT (00:00 UTC) Please note: All presenters at the ARMD meeting in Taipei are expected = to have submitted a draft prior to the cut-off. Exceptions will only be = made in exceptional circumstances. Cheers, -Benson --Apple-Mail-6-397885366 Content-Transfer-Encoding: 7bit Content-Type: text/html; charset=us-ascii
Just a brief reminder of the cut-off dates for IETF82.

http://www.ietf.org/meeting/cutoff-dates-2011.html#IETF82

* 2011-10-24 (Monday): Internet Draft Cut-off for initial document (-00) submission by 17:00 PT (00:00 UTC)
* 2011-10-31 (Monday): Internet Draft final submission cut-off by 17:00 PT (00:00 UTC)

Please note:  All presenters at the ARMD meeting in Taipei are expected to have submitted a draft prior to the cut-off.  Exceptions will only be made in exceptional circumstances.

Cheers,
-Benson

--Apple-Mail-6-397885366-- From linda.dunbar@huawei.com Wed Oct 12 06:38:16 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1FEC521F8BCB for ; Wed, 12 Oct 2011 06:38:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.449 X-Spam-Level: X-Spam-Status: No, score=-6.449 tagged_above=-999 required=5 tests=[AWL=0.149, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jXyotsof+5Wt for ; Wed, 12 Oct 2011 06:38:15 -0700 (PDT) Received: from usaga04-in.huawei.com (usaga04-in.huawei.com [206.16.17.180]) by ietfa.amsl.com (Postfix) with ESMTP id 28FA221F8BB5 for ; Wed, 12 Oct 2011 06:38:15 -0700 (PDT) Received: from huawei.com (usaga04-in [172.18.4.101]) by usaga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LSY00FJ1GJQ5Z@usaga04-in.huawei.com> for armd@ietf.org; Wed, 12 Oct 2011 08:38:14 -0500 (CDT) Received: from dfweml201-edg.china.huawei.com ([172.18.4.104]) by usaga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LSY00G99GJPWI@usaga04-in.huawei.com> for armd@ietf.org; Wed, 12 Oct 2011 08:38:14 -0500 (CDT) Received: from DFWEML403-HUB.china.huawei.com (10.193.5.151) by dfweml201-edg.china.huawei.com (172.18.9.107) with Microsoft SMTP Server (TLS) id 14.1.270.1; Wed, 12 Oct 2011 06:38:08 -0700 Received: from DFWEML506-MBX.china.huawei.com ([10.124.31.111]) by dfweml403-hub.china.huawei.com ([10.193.5.151]) with mapi id 14.01.0270.001; Wed, 12 Oct 2011 06:38:06 -0700 Date: Wed, 12 Oct 2011 13:38:06 +0000 From: Linda Dunbar X-Originating-IP: [10.47.146.113] To: Thomas Narten , "armd@ietf.org" , Benson Schliesser Message-id: <4A95BA014132FF49AE685FAB4B9F17F61209FDAE@dfweml506-mbx> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_fPub6sNBD7CnP88t8RiL2w)" Content-language: en-US Accept-Language: en-US, zh-CN Thread-topic: Oct NANOG ARMD Lightning talk Thread-index: AcyI5CtF+TEWliZPSgmlOmfHZD/7fQ== X-MS-Has-Attach: X-MS-TNEF-Correlator: Cc: Ronald Bonica Subject: [armd] Oct NANOG ARMD Lightning talk X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2011 13:38:16 -0000 --Boundary_(ID_fPub6sNBD7CnP88t8RiL2w) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Benson and I gave an ARMD lightning talk at the Oct NANOG on Monday. Here are the slides: http://www.nanog.org/meetings/nanog53/presentations/Monday/Dunbar.pdf Many operators talked to us afterwards, confirmed the generic network designs and the associated pain points described in the slides. For operators who didn't attend the NANOG lightning talk or didn't get chance talking to us afterwards, we would appreciate your feedback either publicly or privately. Thanks, Linda & Benson --Boundary_(ID_fPub6sNBD7CnP88t8RiL2w) Content-type: text/html; charset=us-ascii Content-transfer-encoding: quoted-printable

Benson and I gave an ARMD lightning talk at the Oct = NANOG on Monday. Here are the slides: http://www.nanog.org/meetings/nanog53/presentations/Monday/Dunbar.pd= f

 

Many operators talked to us afterwards, confirmed th= e generic network designs and the associated pain points described in the s= lides. For operators who didn’t attend the NANOG lightning talk or di= dn’t get chance talking to us afterwards, we would appreciate your feedback either publicly or privately.

 

Thanks,

Linda & Benson

 

--Boundary_(ID_fPub6sNBD7CnP88t8RiL2w)-- From linda.dunbar@huawei.com Wed Oct 12 07:33:12 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 09C4F21F8B8B for ; Wed, 12 Oct 2011 07:33:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.523 X-Spam-Level: X-Spam-Status: No, score=-6.523 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1DPfGD4ybvXh for ; Wed, 12 Oct 2011 07:33:10 -0700 (PDT) Received: from usaga04-in.huawei.com (usaga04-in.huawei.com [206.16.17.180]) by ietfa.amsl.com (Postfix) with ESMTP id BE82521F899F for ; Wed, 12 Oct 2011 07:33:10 -0700 (PDT) Received: from huawei.com (usaga04-in [172.18.4.101]) by usaga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LSY00FEIJ395Z@usaga04-in.huawei.com> for armd@ietf.org; Wed, 12 Oct 2011 09:33:09 -0500 (CDT) Received: from dfweml201-edg.china.huawei.com ([172.18.4.104]) by usaga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LSY00LF0J383E@usaga04-in.huawei.com> for armd@ietf.org; Wed, 12 Oct 2011 09:33:09 -0500 (CDT) Received: from DFWEML404-HUB.china.huawei.com (10.193.5.203) by dfweml201-edg.china.huawei.com (172.18.9.107) with Microsoft SMTP Server (TLS) id 14.1.270.1; Wed, 12 Oct 2011 07:33:04 -0700 Received: from DFWEML506-MBX.china.huawei.com ([10.124.31.111]) by dfweml404-hub.china.huawei.com ([10.193.5.203]) with mapi id 14.01.0270.001; Wed, 12 Oct 2011 07:32:58 -0700 Date: Wed, 12 Oct 2011 14:32:57 +0000 From: Linda Dunbar In-reply-to: <4A95BA014132FF49AE685FAB4B9F17F61209FDAE@dfweml506-mbx> X-Originating-IP: [10.47.146.113] To: Linda Dunbar , Thomas Narten , "armd@ietf.org" , Benson Schliesser Message-id: <4A95BA014132FF49AE685FAB4B9F17F61209FEBA@dfweml506-mbx> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_0PIiEfTsqHAeN5fC9A+J0Q)" Content-language: en-US Accept-Language: en-US, zh-CN Thread-topic: Oct NANOG ARMD Lightning talk Thread-index: AcyI5CtF+TEWliZPSgmlOmfHZD/7fQAByoww X-MS-Has-Attach: X-MS-TNEF-Correlator: References: <4A95BA014132FF49AE685FAB4B9F17F61209FDAE@dfweml506-mbx> Cc: Ronald Bonica Subject: Re: [armd] Oct NANOG ARMD Lightning talk X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2011 14:33:12 -0000 --Boundary_(ID_0PIiEfTsqHAeN5fC9A+J0Q) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Just a little background, the goal of this talk is to solicit operators to challenge (publicly or privately) the generic DC network designs and their associated pain points identified so far. Several operators have indicated to us that they don't want to openly disclose their network issues for various reasons. Therefore, we put together some generic data center network design and associated pain points. - Linda & Benson From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Linda Dunbar Sent: Wednesday, October 12, 2011 8:38 AM To: Thomas Narten; armd@ietf.org; Benson Schliesser Cc: Ronald Bonica Subject: [armd] Oct NANOG ARMD Lightning talk Benson and I gave an ARMD lightning talk at the Oct NANOG on Monday. Here are the slides: http://www.nanog.org/meetings/nanog53/presentations/Monday/Dunbar.pdf Many operators talked to us afterwards, confirmed the generic network designs and the associated pain points described in the slides. For operators who didn't attend the NANOG lightning talk or didn't get chance talking to us afterwards, we would appreciate your feedback either publicly or privately. Thanks, Linda & Benson --Boundary_(ID_0PIiEfTsqHAeN5fC9A+J0Q) Content-type: text/html; charset=us-ascii Content-transfer-encoding: quoted-printable

Just a little background, the goal of this talk is to solicit operato= rs to challenge (publicly or privately) the generic DC network designs and = their associated pain points identified so far.  Several operators have indicated to us that they don’t= want to openly disclose their network issues for various reasons. Therefor= e, we put together some generic data center network design and associated p= ain points.

 

– Linda & Benson

 

 

From: armd-bou= nces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of Linda Dunbar
Sent: Wednesday, October 12, 2011 8:38 AM
To: Thomas Narten; armd@ietf.org; Benson Schliesser
Cc: Ronald Bonica
Subject: [armd] Oct NANOG ARMD Lightning talk

 

Benson and I gave an ARMD lightning talk at the Oct = NANOG on Monday. Here are the slides: http://www.nanog.org/meetings/nanog53/presentations/Monday/Dunbar.pd= f

 

Many operators talked to us afterwards, confirmed th= e generic network designs and the associated pain points described in the s= lides. For operators who didn’t attend the NANOG lightning talk or di= dn’t get chance talking to us afterwards, we would appreciate your feedback either publicly or privately.

 

Thanks,

Linda & Benson

 

--Boundary_(ID_0PIiEfTsqHAeN5fC9A+J0Q)-- From bschlies@cisco.com Thu Oct 13 14:22:06 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A40D721F8B17 for ; Thu, 13 Oct 2011 14:22:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3K1Yto0V4i+Y for ; Thu, 13 Oct 2011 14:22:03 -0700 (PDT) Received: from mtv-iport-2.cisco.com (mtv-iport-2.cisco.com [173.36.130.13]) by ietfa.amsl.com (Postfix) with ESMTP id 7239521F8A56 for ; Thu, 13 Oct 2011 14:22:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=bschlies@cisco.com; l=985; q=dns/txt; s=iport; t=1318540922; x=1319750522; h=from:content-transfer-encoding:subject:date:references: to:message-id:mime-version; bh=uHP644eCUfHZ8WBecpPrrwM/urup6Kcl2l4A6sMht+8=; b=gTNLEx8uLu284OTqWpy2ZA4JXKR5QDgcKgkzGHGhUOFqBpfHfrSGTjn1 gV+N1wiysWNnzeTOtJmTZ5jf+SqOKffEf04usPhOqkUe4zCitRSFECMd5 JkUlcsfAaHURGZv8PuNZxSZrLs9viSI7xfAUle0/djWBPdFqB7lln5LkO c=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Aj4IALpVl06rRDoJ/2dsb2JhbABDmXiOY4EFgVMBAQEDARIBJ0QLUU0KIhmHXAiZGwGeLIcMYQSTeJFx X-IronPort-AV: E=Sophos;i="4.69,342,1315180800"; d="scan'208";a="7740454" Received: from mtv-core-4.cisco.com ([171.68.58.9]) by mtv-iport-2.cisco.com with ESMTP; 13 Oct 2011 21:22:02 +0000 Received: from dhcp-222-99.meetings.nanog.org (bxb-vpn3-356.cisco.com [10.86.249.100]) by mtv-core-4.cisco.com (8.14.3/8.14.3) with ESMTP id p9DLM1cv007282 for ; Thu, 13 Oct 2011 21:22:01 GMT From: Benson Schliesser Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Thu, 13 Oct 2011 17:22:01 -0400 References: <20111013211312.B6C7421F8AFF@ietfa.amsl.com> To: armd@ietf.org Message-Id: <1FDF1406-9CAC-4ACB-B773-A9AB6F4FA148@cisco.com> Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Subject: [armd] Fwd: 82nd IETF DRAFT Agenda X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2011 21:22:06 -0000 The first-draft agenda is out for IETF-82. See the links below for = details. Currently, ARMD is scheduled to meet on Monday afternoon from 1-3pm. = Unfortunately this conflicts with L2VPN's first session - I will attempt = to fix this with the secretariat. As a result, one or both meetings may = end up getting rescheduled to a different timeslot. Cheers, -Benson Begin forwarded message: > The DRAFT agenda is ready for viewing. Please note the cutoff date = for > requests to reschedule Working Group and BOF meetings is October 17, = 2011 > 17:00 PT. The final agenda will be published on October 21, 2011. >=20 > https://datatracker.ietf.org/meeting/82/agenda.html > https://datatracker.ietf.org/meeting/82/agenda.txt >=20 > http://www.ietf.org/meeting/82/index.html >=20 >=20 > Thanks, > Wanda >=20 > Only 30 days until Taipei, 82nd IETF! > Online registration for the IETF meeting is at: > http://www.ietf.org/meeting/register.html >=20 From vishwas.ietf@gmail.com Fri Oct 14 18:38:26 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0BF3C21F8D22 for ; Fri, 14 Oct 2011 18:38:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e7a4NXT3tzHC for ; Fri, 14 Oct 2011 18:38:25 -0700 (PDT) Received: from mail-gy0-f172.google.com (mail-gy0-f172.google.com [209.85.160.172]) by ietfa.amsl.com (Postfix) with ESMTP id 65E4221F8D21 for ; Fri, 14 Oct 2011 18:38:25 -0700 (PDT) Received: by gyh20 with SMTP id 20so1959899gyh.31 for ; Fri, 14 Oct 2011 18:38:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=c1Xxu3v3B8bXOnMFwuNRBcBNJqZXDGwFN+ShncNXGJ8=; b=rD3idtIaSFdSiJHwPec/QT/quMbOeMdehkeFzX/gOdSyS6zdv/q3MJ96cSBalUJS47 h1L6vLlHz3NfcO2KMH4RtEXMry3lP/Ok3La3/QHWycUaMinFq60d0rVOb+6h9W1GdS9e bbmHGiO3b8lB/0zN9ulqM8Do3FrQhaQV55qTM= MIME-Version: 1.0 Received: by 10.182.227.7 with SMTP id rw7mr5866414obc.70.1318642704897; Fri, 14 Oct 2011 18:38:24 -0700 (PDT) Received: by 10.182.11.7 with HTTP; Fri, 14 Oct 2011 18:38:24 -0700 (PDT) In-Reply-To: <1058595951-1318017071-cardhu_decombobulator_blackberry.rim.net-1171647282-@b18.c6.bise6.blackberry> References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx> <4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx> <7384885B-E229-4C91-9C45-C3982F620E74@cisco.com> <618BE8B40039924EB9AED233D4A09C51022D55FE@XMB-BGL-416.cisco.com> <010301cc8446$e005dec0$a0119c40$@com> <1058595951-1318017071-cardhu_decombobulator_blackberry.rim.net-1171647282-@b18.c6.bise6.blackberry> Date: Fri, 14 Oct 2011 18:38:24 -0700 Message-ID: From: Vishwas Manral To: shares@ndzh.com Content-Type: multipart/alternative; boundary=f46d044472bb02d81d04af4c6ea9 Cc: Thomas Narten , Susan Hares , armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Oct 2011 01:38:26 -0000 --f46d044472bb02d81d04af4c6ea9 Content-Type: text/plain; charset=ISO-8859-1 Hi Sue, Sorry I ave been away on some work. If we use MAC-in-MAC the I-SID looks very similar to the TNI or other multi-tenancy service indicators. I know multi-tenancy support for TRILL plans to extend to use Q-in-Q and other such mechanisms. BTW SPB-V uses Q-in-Q too though I do not know if it is deployed at all. Thanks, Vishwas On Fri, Oct 7, 2011 at 10:59 AM, wrote: > Vishwas: > > Thanks. That clears up a lot. > > I'm not a Q-inQ expert so this question maybe off base.... > > How does Q-in-Q support impact the vlan approac:? Is there any DC use. Of > Q-in-Q Vlans? > > Sue > > > Sent via BlackBerry by AT&T > > -----Original Message----- > From: Vishwas Manral > Sender: armd-bounces@ietf.org > Date: Thu, 6 Oct 2011 21:05:35 > To: Susan Hares > Cc: Thomas Narten; > Subject: Re: [armd] Multi-Tenancy without changes > --f46d044472bb02d81d04af4c6ea9 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi Sue,
=A0
Sorry I ave been away on some work. If we use MAC-in-MAC the I-SID loo= ks very similar to the TNI or other multi-tenancy service indicators.
=A0
I know multi-tenancy support for TRILL plans to extend to use Q-in-Q a= nd other such mechanisms. BTW SPB-V uses Q-in-Q too though I do not know if= it is deployed at all.
=A0
Thanks,
Vishwas
On Fri, Oct 7, 2011 at 10:59 AM, <shares@ndzh.com> wr= ote:
Vishwas:

Thanks. =A0That = clears up a lot.

I'm not a Q-inQ expert so this question maybe o= ff base....

How does Q-in-Q support impact the vlan approac:? =A0Is there any DC us= e. Of Q-in-Q Vlans?

Sue


Sent via BlackBerry by AT&T

-----Original Message-----
From: Vishwas Manral &l= t;vishwas.ietf@gmail.com><= br>Sender: armd-bounces@ietf.org
Date: Thu, 6 Oct 2011 21:05:35
To: Susan Hares<skh@ndzh.com>
Cc: Thomas Narten<narten@us.ibm.com>; <armd@ietf.org>
Subject: Re: [armd] Multi-Tenancy without changes
--f46d044472bb02d81d04af4c6ea9-- From skh@ndzh.com Sat Oct 15 05:21:35 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E55A121F8A7D for ; Sat, 15 Oct 2011 05:21:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aAJlcAGzhTq8 for ; Sat, 15 Oct 2011 05:21:35 -0700 (PDT) Received: from smtp.hsia.fairmont.com (smtp.hsia.fairmont.com [142.131.15.59]) by ietfa.amsl.com (Postfix) with ESMTP id 4B96B21F8A7B for ; Sat, 15 Oct 2011 05:21:33 -0700 (PDT) Received: from p2258.superclick.com (unverified [203.116.152.2]) by smtp.hsia.fairmont.com (Vircom SMTPRS 4.7.840.0) with ESMTP id ; Sat, 15 Oct 2011 08:21:01 -0400 X-Modus-BlackList: 203.116.152.2=OK;skh@ndzh.com=OK X-Modus-Trusted: 203.116.152.2=NO X-Modus-Audit: FALSE;0;0;0 Received: from SKHPERSONALLT ([172.16.94.167]) (authenticated bits=0) by p2258.superclick.com (8.13.1/8.13.1) with ESMTP id p9FCLHB3007114; Sat, 15 Oct 2011 20:21:18 +0800 From: "Susan Hares" To: "'Vishwas Manral'" , References: <4A95BA014132FF49AE685FAB4B9F17F61209D00D@dfweml506-mbx> <4A95BA014132FF49AE685FAB4B9F17F61209D0DF@dfweml506-mbx> <7384885B-E229-4C91-9C45-C3982F620E74@cisco.com> <618BE8B40039924EB9AED233D4A09C51022D55FE@XMB-BGL-416.cisco.com> <010301cc8446$e005dec0$a0119c40$@com> <1058595951-1318017071-cardhu_decombobulator_blackberry.rim.net-1171647282-@b18.c6.bise6.blackberry> In-Reply-To: Date: Sat, 15 Oct 2011 08:21:22 -0400 Message-ID: <002101cc8b34$f51becb0$df53c610$@com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0022_01CC8B13.6E0A4CB0" X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcyK2yFqP8bekZM0QOm5tVrOhXKheAAWYKLQ Content-Language: en-us Cc: 'Thomas Narten' , armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Oct 2011 12:21:36 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0022_01CC8B13.6E0A4CB0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Vishwas: Your understanding matches mine on the mac-in-mac and I-SID, Trills Q-in-Q planned support, and SPB-V Q-in-Q support. I think a common ARP solution would be useful for all of these. Sue From: Vishwas Manral [mailto:vishwas.ietf@gmail.com] Sent: Friday, October 14, 2011 9:38 PM To: shares@ndzh.com Cc: Susan Hares; Thomas Narten; armd@ietf.org Subject: Re: [armd] Multi-Tenancy without changes Hi Sue, Sorry I ave been away on some work. If we use MAC-in-MAC the I-SID looks very similar to the TNI or other multi-tenancy service indicators. I know multi-tenancy support for TRILL plans to extend to use Q-in-Q and other such mechanisms. BTW SPB-V uses Q-in-Q too though I do not know if it is deployed at all. Thanks, Vishwas On Fri, Oct 7, 2011 at 10:59 AM, wrote: Vishwas: Thanks. That clears up a lot. I'm not a Q-inQ expert so this question maybe off base.... How does Q-in-Q support impact the vlan approac:? Is there any DC use. Of Q-in-Q Vlans? Sue Sent via BlackBerry by AT&T -----Original Message----- From: Vishwas Manral Sender: armd-bounces@ietf.org Date: Thu, 6 Oct 2011 21:05:35 To: Susan Hares Cc: Thomas Narten; Subject: Re: [armd] Multi-Tenancy without changes ------=_NextPart_000_0022_01CC8B13.6E0A4CB0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Vishwas:

 

Your understanding matches mine on the mac-in-mac  and I-SID, = Trills Q-in-Q planned support, and SPB-V Q-in-Q = support.

 

I think a common ARP solution would be useful for all of these.  =

 

Sue

 

From:= = Vishwas Manral [mailto:vishwas.ietf@gmail.com]
Sent: Friday, = October 14, 2011 9:38 PM
To: shares@ndzh.com
Cc: = Susan Hares; Thomas Narten; armd@ietf.org
Subject: Re: [armd] = Multi-Tenancy without changes

 

Hi = Sue,

 

Sorry I ave been away on some work. If we use = MAC-in-MAC the I-SID looks very similar to the TNI or other = multi-tenancy service indicators.

 

I = know multi-tenancy support for TRILL plans to extend to use Q-in-Q and = other such mechanisms. BTW SPB-V uses Q-in-Q too though I do not know if = it is deployed at all.

 

Thanks,

Vishwas

On Fri, Oct 7, 2011 at 10:59 AM, <shares@ndzh.com> = wrote:

Vishwas:

Thanks. =  That clears up a lot.

I'm not a Q-inQ expert so this = question maybe off base....

How does Q-in-Q support impact the = vlan approac:?  Is there any DC use. Of Q-in-Q = Vlans?

Sue


Sent via BlackBerry by = AT&T


-----Original = Message-----
From: Vishwas Manral <vishwas.ietf@gmail.com>
= Sender: armd-bounces@ietf.org
Date: = Thu, 6 Oct 2011 21:05:35
To: Susan Hares<skh@ndzh.com>
Cc: Thomas = Narten<narten@us.ibm.com>; <armd@ietf.org>
Subject: Re: = [armd] Multi-Tenancy without = changes

------=_NextPart_000_0022_01CC8B13.6E0A4CB0-- From internet-drafts@ietf.org Mon Oct 17 15:34:19 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3266B11E80A4; Mon, 17 Oct 2011 15:34:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.55 X-Spam-Level: X-Spam-Status: No, score=-102.55 tagged_above=-999 required=5 tests=[AWL=0.049, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UI6czrUascjw; Mon, 17 Oct 2011 15:34:18 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C7C5F11E8095; Mon, 17 Oct 2011 15:34:18 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 3.60 Message-ID: <20111017223418.17182.59408.idtracker@ietfa.amsl.com> Date: Mon, 17 Oct 2011 15:34:18 -0700 Cc: armd@ietf.org Subject: [armd] I-D Action: draft-ietf-armd-problem-statement-00.txt X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2011 22:34:19 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the Address Resolution for Massive number= s of hosts in the Data center Working Group of the IETF. Title : Problem Statement for ARMD Author(s) : Thomas Narten Filename : draft-ietf-armd-problem-statement-00.txt Pages : 11 Date : 2011-10-17 This document examines issues related to the massive scaling of data centers. Our initial scope is relatively narrow. Specifically, we focus on address resolution (ARP and ND) within the data center. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-armd-problem-statement-00.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-armd-problem-statement-00.txt From bschlies@cisco.com Wed Oct 19 14:46:32 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AFB7A21F8A56 for ; Wed, 19 Oct 2011 14:46:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.027 X-Spam-Level: X-Spam-Status: No, score=-6.027 tagged_above=-999 required=5 tests=[AWL=0.571, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 48RZ8Yh4UEVV for ; Wed, 19 Oct 2011 14:46:32 -0700 (PDT) Received: from mtv-iport-4.cisco.com (mtv-iport-4.cisco.com [173.36.130.15]) by ietfa.amsl.com (Postfix) with ESMTP id 0F9F121F8A4E for ; Wed, 19 Oct 2011 14:46:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=bschlies@cisco.com; l=1053; q=dns/txt; s=iport; t=1319060792; x=1320270392; h=from:subject:date:message-id:to:mime-version; bh=aqjYvZ4QTfOIkyZEf7Mp+8a2010ICrKNBlGfuD084Mo=; b=irmM55UXBIrEntmLzownSmhY2mBbmlWeD6IzX+iiys7t86FCvOJjw756 jmi9zLH+zLLLKjS+VFJ12/0vf+FET6rgMocPG9Jbff+1k8Ti6K36+rQSC SKQGi0qdX9t2mA//LpBLwPgW1EB+k/0aGVCae/LhtUxRa27RFZdpCc+Qq 0=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AroGADpEn06rRDoJ/2dsb2JhbABEmhSOboEFgWUFHQGBBAECgVKFJoIwEJY2gSYBnkcEhzphBIgCi3yFKoxN X-IronPort-AV: E=Sophos;i="4.69,374,1315180800"; d="scan'208,217";a="8952251" Received: from mtv-core-4.cisco.com ([171.68.58.9]) by mtv-iport-4.cisco.com with ESMTP; 19 Oct 2011 21:46:31 +0000 Received: from [10.22.238.45] ([10.21.72.253]) by mtv-core-4.cisco.com (8.14.3/8.14.3) with ESMTP id p9JLkUf8013758 for ; Wed, 19 Oct 2011 21:46:30 GMT From: Benson Schliesser Content-Type: multipart/alternative; boundary=Apple-Mail-23--696309272 Date: Wed, 19 Oct 2011 14:46:30 -0700 Message-Id: <0E8B288A-11C5-4C52-8FA0-6110A85FDFBC@cisco.com> To: armd@ietf.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Subject: [armd] updated ARMD schedule X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2011 21:46:32 -0000 --Apple-Mail-23--696309272 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii At our request, the secretariat has rescheduled the ARMD meeting to the = Tue Afternoon III timeslot (5:10-6:10pm). This appears to avoid the = most important conflicts. Please see https://datatracker.ietf.org/meeting/82/agenda.html for all = the details. Cheers, -Benson --Apple-Mail-23--696309272 Content-Transfer-Encoding: 7bit Content-Type: text/html; charset=us-ascii At our request, the secretariat has rescheduled the ARMD meeting to the Tue Afternoon III timeslot (5:10-6:10pm).  This appears to avoid the most important conflicts.

Please see https://datatracker.ietf.org/meeting/82/agenda.html for all the details.

Cheers,
-Benson
--Apple-Mail-23--696309272-- From bschlies@cisco.com Wed Oct 19 14:53:23 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A8CC321F8B0C for ; Wed, 19 Oct 2011 14:53:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.099 X-Spam-Level: X-Spam-Status: No, score=-6.099 tagged_above=-999 required=5 tests=[AWL=0.500, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WLtqYQgfsUpi for ; Wed, 19 Oct 2011 14:53:23 -0700 (PDT) Received: from mtv-iport-3.cisco.com (mtv-iport-3.cisco.com [173.36.130.14]) by ietfa.amsl.com (Postfix) with ESMTP id 23EFC21F84FB for ; Wed, 19 Oct 2011 14:53:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=bschlies@cisco.com; l=501; q=dns/txt; s=iport; t=1319061203; x=1320270803; h=from:content-transfer-encoding:subject:date:message-id: to:mime-version; bh=ulfB3W+mvDALV4is8D8+PVzflJUV8c8zpE9YHGPdBk0=; b=EUeR7VjMD6F/3C1BiATUEagRInff95Fo/T+jzS1PPvSlCVeG+kwO9fOS IxIckdtBmSxekgVu0hRx3l6NjaMPYfojxuhJY4ynmKM9CToK+zv49+tMT ry3RtZU4Qtx3NVqydrzpRFH+nYgBt1xNvTTB3z7B9Ym7yHzBQ1FNKrpdt 8=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AroGAJRGn06rRDoG/2dsb2JhbABEmhSOboEFggcBCh00gVwJGYdmlieBJgGeS4c6YQSIAot8hSqMTQ X-IronPort-AV: E=Sophos;i="4.69,374,1315180800"; d="scan'208";a="8986488" Received: from mtv-core-1.cisco.com ([171.68.58.6]) by mtv-iport-3.cisco.com with ESMTP; 19 Oct 2011 21:53:04 +0000 Received: from [10.22.238.45] ([10.21.72.253]) by mtv-core-1.cisco.com (8.14.3/8.14.3) with ESMTP id p9JLr3QK012165 for ; Wed, 19 Oct 2011 21:53:03 GMT From: Benson Schliesser Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Wed, 19 Oct 2011 14:53:03 -0700 Message-Id: To: armd@ietf.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Subject: [armd] Call for Agenda Items X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2011 21:53:23 -0000 ARMD Contributors - If you wish to give a presentation at the upcoming ARMD meeting at = IETF-82, please send a message to the chairs = (armd-chairs@tools.ietf.org) and/or this mailing list (armd@ietf.org). All requests must be received by 01-NOV-2011. Also, all speakers must = have submitted a draft by the cut-off. (See = http://www.ietf.org/meeting/cutoff-dates-2011.html#IETF82 for details.) = Exceptions will only be made in exceptional circumstances. Cheers, -Benson & Linda From mkarir@merit.edu Fri Oct 21 07:56:57 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1A3AB21F8AFE for ; Fri, 21 Oct 2011 07:56:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id idkWGfjfFa8L for ; Fri, 21 Oct 2011 07:56:56 -0700 (PDT) Received: from merit-proxy02.merit.edu (merit-proxy02.merit.edu [207.75.116.194]) by ietfa.amsl.com (Postfix) with ESMTP id A038621F8AB8 for ; Fri, 21 Oct 2011 07:56:56 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by merit-proxy02.merit.edu (Postfix) with ESMTP id 15E48203984B; Fri, 21 Oct 2011 10:56:54 -0400 (EDT) X-Virus-Scanned: amavisd-new at merit-proxy02.merit.edu Received: from merit-proxy02.merit.edu ([127.0.0.1]) by localhost (merit-proxy02.merit.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qwIfCVfZI1UY; Fri, 21 Oct 2011 10:56:52 -0400 (EDT) Received: from [10.255.255.13] (sfpop-vpn01.merit.edu [192.203.195.134]) by merit-proxy02.merit.edu (Postfix) with ESMTPSA id D7881203984A; Fri, 21 Oct 2011 10:56:52 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Manish Karir In-Reply-To: Date: Fri, 21 Oct 2011 10:56:52 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <9860EBA1-7BA8-43D2-A79F-755CC1D89371@merit.edu> References: To: Benson Schliesser X-Mailer: Apple Mail (2.1084) Cc: armd@ietf.org Subject: Re: [armd] Call for Agenda Items X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2011 14:56:57 -0000 Hi Benson, Linda, I would like to request an agenda slot for discussion of a generalized = data center reference document for the ARMD group. The goal is go have folks agree on something that is generic enough so that the subsequent = discussion of pain points or issues can have something to reference in = terms of what and where those pain points are. Thanks. -manish On Oct 19, 2011, at 5:53 PM, Benson Schliesser wrote: > ARMD Contributors - >=20 > If you wish to give a presentation at the upcoming ARMD meeting at = IETF-82, please send a message to the chairs = (armd-chairs@tools.ietf.org) and/or this mailing list (armd@ietf.org). >=20 > All requests must be received by 01-NOV-2011. Also, all speakers must = have submitted a draft by the cut-off. (See = http://www.ietf.org/meeting/cutoff-dates-2011.html#IETF82 for details.) = Exceptions will only be made in exceptional circumstances. >=20 > Cheers, > -Benson & Linda >=20 > _______________________________________________ > armd mailing list > armd@ietf.org > https://www.ietf.org/mailman/listinfo/armd From linda.dunbar@huawei.com Thu Oct 27 14:23:47 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6D26111E8086 for ; Thu, 27 Oct 2011 14:23:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.577 X-Spam-Level: X-Spam-Status: No, score=-6.577 tagged_above=-999 required=5 tests=[AWL=0.021, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DPgEYsqGVlT7 for ; Thu, 27 Oct 2011 14:23:46 -0700 (PDT) Received: from usaga04-in.huawei.com (usaga04-in.huawei.com [206.16.17.180]) by ietfa.amsl.com (Postfix) with ESMTP id 738FC11E807F for ; Thu, 27 Oct 2011 14:23:46 -0700 (PDT) Received: from huawei.com (usaga04-in [172.18.4.101]) by usaga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LTQ007UBU3KPJ@usaga04-in.huawei.com> for armd@ietf.org; Thu, 27 Oct 2011 16:23:45 -0500 (CDT) Received: from dfweml201-edg.china.huawei.com ([172.18.4.104]) by usaga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LTQ00NGDU3K43@usaga04-in.huawei.com> for armd@ietf.org; Thu, 27 Oct 2011 16:23:44 -0500 (CDT) Received: from DFWEML404-HUB.china.huawei.com (10.193.5.203) by dfweml201-edg.china.huawei.com (172.18.9.107) with Microsoft SMTP Server (TLS) id 14.1.270.1; Thu, 27 Oct 2011 14:23:38 -0700 Received: from DFWEML505-MBX.china.huawei.com ([10.124.31.100]) by dfweml404-hub.china.huawei.com ([10.193.5.203]) with mapi id 14.01.0270.001; Thu, 27 Oct 2011 14:23:35 -0700 Date: Thu, 27 Oct 2011 21:23:34 +0000 From: Linda Dunbar X-Originating-IP: [10.192.11.155] To: Thomas Narten Message-id: <4A95BA014132FF49AE685FAB4B9F17F6120AEBDD@dfweml505-mbx> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_HjAOVFQSWFb95U3IOx2/9g)" Content-language: en-US Accept-Language: en-US, zh-CN Thread-topic: What is the proper definition for Broadcast Domain? which is used in draft-ietf-armd-problem-statement-00 Thread-index: AcyU7q65xFFeVyD+TEeBHWM0Egqwdg== X-MS-Has-Attach: X-MS-TNEF-Correlator: Cc: "armd@ietf.org" Subject: [armd] What is the proper definition for Broadcast Domain? which is used in draft-ietf-armd-problem-statement-00 X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Oct 2011 21:23:47 -0000 --Boundary_(ID_HjAOVFQSWFb95U3IOx2/9g) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT draft-ietf-armd-problem-statement-00 has the following definition for Broadcast Domain: Broadcast Domain: The set of all links and switches that are traversed in order to reach all nodes that are members of a given L2 domain. For example, when sending a broadcast packet on a VLAN, the domain would include all the links and switches that the packet traverses when broadcast traffic is sent. The phrase of "are members of a given L2 domain" is a bit confusing because L2 domain could have up to 4095 VLANS, with each one being its own broadcast domain. Therefore, I think it is more accurate to say "The set of all links and switches that are traversed in order to reach all nodes that are members of a given VLAN of a L2 domain." Any other suggestions? Linda --Boundary_(ID_HjAOVFQSWFb95U3IOx2/9g) Content-type: text/html; charset=us-ascii Content-transfer-encoding: 7BIT

draft-ietf-armd-problem-statement-00 has the following definition for Broadcast Domain:

 

Broadcast Domain: The set of all links and switches that are

traversed in order to reach all nodes that are members of a given

L2 domain. For example, when sending a broadcast packet on a

VLAN, the domain would include all the links and switches that the

packet traverses when broadcast traffic is sent.

 

The phrase of “are members of a given L2 domain” is a bit confusing because L2 domain could have up to 4095 VLANS, with each one being its own broadcast domain.

 

Therefore, I think it is more accurate to say “The set of all links and switches that are traversed in order to reach all nodes that are members of a given VLAN of a L2 domain.”

 

Any other suggestions?

 

Linda

--Boundary_(ID_HjAOVFQSWFb95U3IOx2/9g)-- From mkarir@merit.edu Fri Oct 28 09:02:03 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D4E0021F8AC9 for ; Fri, 28 Oct 2011 09:02:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g06DHsAVDsmY for ; Fri, 28 Oct 2011 09:02:03 -0700 (PDT) Received: from merit-proxy01.merit.edu (merit-proxy01.merit.edu [207.75.116.193]) by ietfa.amsl.com (Postfix) with ESMTP id 3717121F8AD2 for ; Fri, 28 Oct 2011 09:02:03 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by merit-proxy01.merit.edu (Postfix) with ESMTP id 940CD203984C for ; Fri, 28 Oct 2011 12:01:53 -0400 (EDT) X-Virus-Scanned: amavisd-new at merit-proxy01.merit.edu Received: from merit-proxy01.merit.edu ([127.0.0.1]) by localhost (merit-proxy01.merit.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3laDceNh8P63 for ; Fri, 28 Oct 2011 12:01:53 -0400 (EDT) Received: from [10.255.255.5] (sfpop-vpn01.merit.edu [192.203.195.134]) by merit-proxy01.merit.edu (Postfix) with ESMTPSA id 0C235203984A for ; Fri, 28 Oct 2011 12:01:53 -0400 (EDT) From: Manish Karir Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Fri, 28 Oct 2011 12:01:52 -0400 Message-Id: <905C201F-E6DD-4FA2-A65A-38472BA39571@merit.edu> To: armd@ietf.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Subject: [armd] datacenter reference architecture draft X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Oct 2011 16:02:04 -0000 The following draft was submitted to hopefully help focus the ARMD = discussion around a common architecture. Comments and feedback are welcome. The goal of the writeup really is to = abstract away specific datacenter designs each of which focuses on solving a particular application/traffic = pattern by trying to talk about what is common between=20 the various designs. Hopefully this will help some of the very varied = discussion that has taken place in this WG so far. Thanks. -manish http://www.ietf.org/id/draft-armd-datacenter-reference-arch-01.txt ----------------------------------------- Filename: draft-karir-armd-datacenter-reference-arch Revision: 00 Title: Data Center Reference Architectures Creation date: 2011-10-24 WG ID: Individual Submission Number of pages: 11 Abstract: The continued growth of large-scale data centers has resulted in a wide range of architectures and designs. Each design is tuned to address the challenges and requirements of the specific applications and workload that the data is being built for. Each design evolves as engineering solutions are developed to workaround limitations of existing protocols, hardware, as well as software implementations. The goal of this document is to characterize this problem space in detail in order to better understand if there is any gap in making address resolution scale in various network designs for data centers. In particular it is our goal to peel back the various optimization and engineering solutions to develop generalized reference architectures for a data center. We also discuss the various factors that influence design choices in developing various data center designs. = --------------------------------------------------------------------------= From joelja@bogus.com Mon Oct 31 11:28:55 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B426B1F0CB0 for ; Mon, 31 Oct 2011 11:28:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7A5ucVoPHbb4 for ; Mon, 31 Oct 2011 11:28:55 -0700 (PDT) Received: from nagasaki.bogus.com (nagasaki.bogus.com [IPv6:2001:418:1::81]) by ietfa.amsl.com (Postfix) with ESMTP id 141981F0CAF for ; Mon, 31 Oct 2011 11:28:55 -0700 (PDT) Received: from Zorch.local (adsl-99-173-15-226.dsl.pltn13.sbcglobal.net [99.173.15.226]) (authenticated bits=0) by nagasaki.bogus.com (8.14.4/8.14.4) with ESMTP id p9VISrGm002015 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NOT); Mon, 31 Oct 2011 18:28:54 GMT (envelope-from joelja@bogus.com) Message-ID: <4EAEE8E4.90302@bogus.com> Date: Mon, 31 Oct 2011 11:28:52 -0700 From: Joel jaeggli User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: Manish Karir References: <905C201F-E6DD-4FA2-A65A-38472BA39571@merit.edu> In-Reply-To: <905C201F-E6DD-4FA2-A65A-38472BA39571@merit.edu> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (nagasaki.bogus.com [147.28.0.81]); Mon, 31 Oct 2011 18:28:54 +0000 (UTC) Cc: armd@ietf.org Subject: Re: [armd] datacenter reference architecture draft X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Oct 2011 18:28:55 -0000 so, I looked at it for a while... I'm a bit mystified by 3.4.1-3.4.4 we've already arrived I guess at what we conclude is the ideal topology, and a particular model of mobility. When faced with this choice and a desire to constrain both the complexity and the diameter failure domain one response is to not make the network the arbiter of mobility, e.g. move that to provisioning or the application layer. the result is a lot closer to 3.4.1 than it is the others. One of he problems I have with managing a large l2 domain, particularly one constructed as an overlay is that there's effectively no upper bound appart from physics and good taste for how far it can spread, first they want across the rack, the module, then across the whole datacenter, then to the adjacent datacenter, across the country, to other cloud providers, etc. if you constrain it sufficiently small that availability is not a design consideration (1 or 2 switches is big l2 domain) then it doesn't become a dependency. Insisting that ip addresses move around with virtualized machines is one way to view the world but it not the only way. nor is constraining tenets to common l2 buckets the only way to segment applications from each other, the hosts for the virtual machines can just as well be (are) policy enforcement points. On 10/28/11 09:01 , Manish Karir wrote: > > The following draft was submitted to hopefully help focus the ARMD discussion around a common architecture. > > Comments and feedback are welcome. The goal of the writeup really is to abstract away specific datacenter designs > each of which focuses on solving a particular application/traffic pattern by trying to talk about what is common between > the various designs. Hopefully this will help some of the very varied discussion that has taken place in this WG so far. > > Thanks. > -manish > > http://www.ietf.org/id/draft-armd-datacenter-reference-arch-01.txt > ----------------------------------------- > Filename: draft-karir-armd-datacenter-reference-arch > Revision: 00 > Title: Data Center Reference Architectures > Creation date: 2011-10-24 > WG ID: Individual Submission > Number of pages: 11 > > Abstract: > The continued growth of large-scale data centers has resulted in a > wide range of architectures and designs. Each design is tuned to > address the challenges and requirements of the specific applications > and workload that the data is being built for. Each design evolves > as engineering solutions are developed to workaround limitations of > existing protocols, hardware, as well as software implementations. > > The goal of this document is to characterize this problem space in > detail in order to better understand if there is any gap in making > address resolution scale in various network designs for data > centers. In particular it is our goal to peel back the various > optimization and engineering solutions to develop generalized > reference architectures for a data center. We also discuss the > various factors that influence design choices in developing various > data center designs. > -------------------------------------------------------------------------- > _______________________________________________ > armd mailing list > armd@ietf.org > https://www.ietf.org/mailman/listinfo/armd > From linda.dunbar@huawei.com Mon Oct 31 12:33:10 2011 Return-Path: X-Original-To: armd@ietfa.amsl.com Delivered-To: armd@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D55981F0C8E for ; Mon, 31 Oct 2011 12:33:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.569 X-Spam-Level: X-Spam-Status: No, score=-6.569 tagged_above=-999 required=5 tests=[AWL=0.030, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dvD5OK04n8iN for ; Mon, 31 Oct 2011 12:33:10 -0700 (PDT) Received: from usaga02-in.huawei.com (usaga02-in.huawei.com [206.16.17.70]) by ietfa.amsl.com (Postfix) with ESMTP id 3542B1F0C3D for ; Mon, 31 Oct 2011 12:33:10 -0700 (PDT) Received: from huawei.com (localhost [127.0.0.1]) by usaga02-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LTY0081U3N3MZ@usaga02-in.huawei.com> for armd@ietf.org; Mon, 31 Oct 2011 14:33:04 -0500 (CDT) Received: from dfweml202-edg.china.huawei.com ([172.18.4.104]) by usaga02-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTPS id <0LTY00GYR3MFDU@usaga02-in.huawei.com> for armd@ietf.org; Mon, 31 Oct 2011 14:33:03 -0500 (CDT) Received: from DFWEML401-HUB.china.huawei.com (10.193.5.101) by dfweml202-edg.china.huawei.com (172.18.9.108) with Microsoft SMTP Server (TLS) id 14.1.270.1; Mon, 31 Oct 2011 12:33:02 -0700 Received: from DFWEML505-MBX.china.huawei.com ([10.124.31.100]) by DFWEML401-HUB.china.huawei.com ([fe80::f07f:889f:78ef:8df3%13]) with mapi id 14.01.0270.001; Mon, 31 Oct 2011 12:32:55 -0700 Date: Mon, 31 Oct 2011 19:32:56 +0000 From: Linda Dunbar In-reply-to: <4EAEE8E4.90302@bogus.com> X-Originating-IP: [10.192.11.155] To: Joel jaeggli , Manish Karir Message-id: <4A95BA014132FF49AE685FAB4B9F17F6120AFBCF@dfweml505-mbx> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-language: en-US Content-transfer-encoding: 7BIT Accept-Language: en-US Thread-topic: [armd] datacenter reference architecture draft Thread-index: AQHMlYs3s+iG/L4rDECFpB4Hx6gBHpWXQCMA//+YgdA= X-MS-Has-Attach: X-MS-TNEF-Correlator: References: <905C201F-E6DD-4FA2-A65A-38472BA39571@merit.edu> <4EAEE8E4.90302@bogus.com> Cc: "armd@ietf.org" Subject: Re: [armd] datacenter reference architecture draft X-BeenThere: armd@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Oct 2011 19:33:10 -0000 Joel, See comments inserted below: > -----Original Message----- > From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of > Joel jaeggli > Sent: Monday, October 31, 2011 1:29 PM > To: Manish Karir > Cc: armd@ietf.org > Subject: Re: [armd] datacenter reference architecture draft > > so, I looked at it for a while... > > I'm a bit mystified by 3.4.1-3.4.4 > > we've already arrived I guess at what we conclude is the ideal topology, > and a particular model of mobility. > > When faced with this choice and a desire to constrain both the > complexity and the diameter failure domain one response is to not make > the network the arbiter of mobility, e.g. move that to provisioning or > the application layer. the result is a lot closer to 3.4.1 than it is > the others. [Linda] Are you saying that the data center which you manage prefers 3.4.1 or majority of data centers should be 3.4.1? > > One of he problems I have with managing a large l2 domain, particularly > one constructed as an overlay is that there's effectively no upper > bound > appart from physics and good taste for how far it can spread, first > they > want across the rack, the module, then across the whole datacenter, > then > to the adjacent datacenter, across the country, to other cloud > providers, etc. if you constrain it sufficiently small that > availability is not a design consideration (1 or 2 switches is big l2 > domain) then it doesn't become a dependency. [Linda] Do you mean that simply not let L2 go beyond ToR switches? > > Insisting that ip addresses move around with virtualized machines is > one > way to view the world but it not the only way. nor is constraining > tenets to common l2 buckets the only way to segment applications from > each other, the hosts for the virtual machines can just as well be (are) > policy enforcement points. [Linda] Can you elaborate this a bit more on "the hosts for VM can just as well be policy enforcement points"? > > On 10/28/11 09:01 , Manish Karir wrote: > > > > The following draft was submitted to hopefully help focus the ARMD > discussion around a common architecture. > > > > Comments and feedback are welcome. The goal of the writeup really is > to abstract away specific datacenter designs > > each of which focuses on solving a particular application/traffic > pattern by trying to talk about what is common between > > the various designs. Hopefully this will help some of the very > varied discussion that has taken place in this WG so far. > > > > Thanks. > > -manish > > > > http://www.ietf.org/id/draft-armd-datacenter-reference-arch-01.txt > > ----------------------------------------- > > Filename: draft-karir-armd-datacenter-reference-arch > > Revision: 00 > > Title: Data Center Reference Architectures > > Creation date: 2011-10-24 > > WG ID: Individual Submission > > Number of pages: 11 > > > > Abstract: > > The continued growth of large-scale data centers has resulted in a > > wide range of architectures and designs. Each design is tuned to > > address the challenges and requirements of the specific > applications > > and workload that the data is being built for. Each design evolves > > as engineering solutions are developed to workaround limitations of > > existing protocols, hardware, as well as software implementations. > > > > The goal of this document is to characterize this problem space in > > detail in order to better understand if there is any gap in making > > address resolution scale in various network designs for data > > centers. In particular it is our goal to peel back the various > > optimization and engineering solutions to develop generalized > > reference architectures for a data center. We also discuss the > > various factors that influence design choices in developing various > > data center designs. > > --------------------------------------------------------------------- > ----- > > _______________________________________________ > > armd mailing list > > armd@ietf.org > > https://www.ietf.org/mailman/listinfo/armd > > > > _______________________________________________ > armd mailing list > armd@ietf.org > https://www.ietf.org/mailman/listinfo/armd