Subject: [dc] Comment of draft-dalela-dc-requirements

From: Lizhong Jin <[email-masked]>
Date: Mon, 9 Jan 2012 23:00:10 +0800

Hi Ashish,
I have several comments to this requirement. Thanks.

Section 5.3, Multi-tenancy problem. From my side, the multi-tenancy problem
is not only the scalability of segmentation-ID problem, it is also required
to isolate performance and security among multi-tenant. For example, one
tenant should not suffer denial-of-service attacks by other tenant within
same datacenter. A detailed example is, the huge broadcast traffic from one
tenant should not influence the performace and availability of other
tenants.

Section 5.3, "The use of L3 VRFs also poses similar challenges of scaling".
The challenge for VRF is quite different with VLAN. The challenge for VRF
is the scalability of forwarding table. And you also said:"With VRFs, these
entries will be present even if there is no traffic from a host to other
hosts in the VRF". I think this could be optimized that the FIB would store
only active route entries, while the RIB would store all route entries.

Section 5.5, last paragraph about mobility. It seems that this paragraph is
not much related with "network convergence", but is about "host mobility
impact to the network resource". While I find that, section 5.1 is about
the "host mobility impact to L2/3 forwarding", 5.10 is about the "host
mobility impact to forwarding tables". Suggest to re-organize the three
parts which are all related with impact of host mobility.

Regards
Lizhong

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Tue, 10 Jan 2012 15:56:04 +0530

Hi Lizhong,

Please see inline.

Thanks, Ashish

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Lizhong Jin
Sent: Monday, January 09, 2012 8:30 PM
To: Ashish Dalela (adalela)
Cc: Lizhong Jin; [email-masked]
Subject: [dc] Comment of draft-dalela-dc-requirements

Hi Ashish,

Section 5.3, Multi-tenancy problem. From my side, the multi-tenancy
problem is not only the scalability of segmentation-ID problem, it is
also required to isolate performance and security among multi-tenant.
For example, one tenant should not suffer denial-of-service attacks by
other tenant within same datacenter. A detailed example is, the huge
broadcast traffic from one tenant should not influence the performace
and availability of other tenants.

[AD] Ok, will add this to the multi-tenancy section.

Section 5.3, "The use of L3 VRFs also poses similar challenges of
scaling". The challenge for VRF is quite different with VLAN. The
challenge for VRF is the scalability of forwarding table. And you also
said:"With VRFs, these entries will be present even if there is no
traffic from a host to other hosts in the VRF". I think this could be
optimized that the FIB would store only active route entries, while the
RIB would store all route entries.

[AD] Others have expressed reservations about these "dynamically learnt"
entries. E.g. if a IP scanning attack is launched to unknown IP
addresses, it will load the control plane, although there is nothing in
the RIB. But, in any case, we are just trying to define the problem and
optimizations are a solution are possible. I will modify the VRF
statement to reflect your concern.

[AD] The same scaling concern also exists in the L2/VLAN case where a
MAC address has to be learnt. They are host routes, either L2 or L3 or
done via some sort of encapsulation.

Section 5.5, last paragraph about mobility. It seems that this paragraph
is not much related with "network convergence", but is about "host
mobility impact to the network resource". 

[AD] This paragraph is describing what happens when a whole server is
replicated (not just a single VM). This will happen in case of disaster
recovery. Note that in this case there is change to host routes, but
also change to network routes (the routes to reach the virtual switch
have changed). Let me know if this needs to be reworded or clarified
more.

While I find that, section 5.1 is about the "host mobility impact to
L2/3 forwarding", 5.10 is about the "host mobility impact to forwarding
tables". Suggest to re-organize the three parts which are all related
with impact of host mobility.

[AD] Agree with your comment. Section 5.5 describes control plane scale
while 5.10 describes forwarding plane scale issues. Section 5.1 is
overlapping. Would you suggest removing 5.1? 

Regards

Lizhong

---

From: Lizhong Jin<[email-masked]>
Date: Tue, 10 Jan 2012 20:35:12 +0800

Hi Ashish,
Please see inline below. Thanks.

Lizhong

"Ashish Dalela (adalela)" <[email-masked]> 写于 2012/01/10 18:26:04:

> problem is not only the scalability of segmentation-ID problem, it 
> is also required to isolate performance and security among multi-
> tenant. For example, one tenant should not suffer denial-of-service 
> attacks by other tenant within same datacenter. A detailed example 
> is, the huge broadcast traffic from one tenant should not influence 
> the performace and availability of other tenants.
> 
> challenge for VRF is the scalability of forwarding table. And you 
> also said:"With VRFs, these entries will be present even if there is
> no traffic from a host to other hosts in the VRF". I think this 
> could be optimized that the FIB would store only active route 
> entries, while the RIB would store all route entries.
> 
> [AD] Others have expressed reservations about these “dynamically 
> learnt” entries. E.g. if a IP scanning attack is launched to unknown
> IP addresses, it will load the control plane, although there is 
> nothing in the RIB. But, in any case, we are just trying to define 
> the problem and optimizations are a solution are possible. I will 
> modify the VRF statement to reflect your concern.
> 
> [AD] The same scaling concern also exists in the L2/VLAN case where 
> a MAC address has to be learnt. They are host routes, either L2 or 
> L3 or done via some sort of encapsulation.
[Lizhong] yes, agree
> 
> Section 5.5, last paragraph about mobility. It seems that this 
> paragraph is not much related with "network convergence", but is 
> about "host mobility impact to the network resource". 
> 
> [AD] This paragraph is describing what happens when a whole server 
> is replicated (not just a single VM). This will happen in case of 
> disaster recovery. Note that in this case there is change to host 
> routes, but also change to network routes (the routes to reach the 
> virtual switch have changed). Let me know if this needs to be 
> reworded or clarified more.
[Lizhong] if you refer the virtual switch and firewall relocation as 
network convergence, then the word is OK.
> L2/3 forwarding", 5.10 is about the "host mobility impact to 
> forwarding tables". Suggest to re-organize the three parts which are
> all related with impact of host mobility.
> 
> [AD] Agree with your comment. Section 5.5 describes control plane 
> scale while 5.10 describes forwarding plane scale issues. Section 5.
> 1 is overlapping. Would you suggest removing 5.1? 
[Lizhong] merging 5.1 to 5.10 is OK.
> Lizhong

--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail is solely property of the sender's organization. This mail communication is confidential. Recipients named above are obligated to maintain secrecy and are not permitted to disclose the contents of this communication to others.
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the originator of the message. Any views expressed in this message are those of the individual sender.
This message has been scanned for viruses and Spam by ZTE Anti-Spam system.

---

From: Bhumip Khasnabish <[email-masked]>
Date: Fri, 13 Jan 2012 19:07:33 -0500

Hello Ashish,
Hello Mr. Lizhong,

Please note that, we are also working on a draft related to virtual machine
(and virtual network element) mobility and interconnection
(http://www.ietf.org/id/draft-khasnabish-vmmi-problems-00.txt).
Would appreciate comments and  suggestions. Thanks.
Best.
Bhumip

On Mon, Jan 9, 2012 at 10:00 AM, Lizhong Jin <[email-masked]> wrote:

> problem is not only the scalability of segmentation-ID problem, it is also
> required to isolate performance and security among multi-tenant. For
> example, one tenant should not suffer denial-of-service attacks by other
> tenant within same datacenter. A detailed example is, the huge broadcast
> traffic from one tenant should not influence the performace and
> availability of other tenants.
>
> scaling". The challenge for VRF is quite different with VLAN. The challenge
> for VRF is the scalability of forwarding table. And you also said:"With
> VRFs, these entries will be present even if there is no traffic from a host
> to other hosts in the VRF". I think this could be optimized that the FIB
> would store only active route entries, while the RIB would store all route
> entries.
>
> is not much related with "network convergence", but is about "host mobility
> _______________________________________________
> dc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dc
>
>

---

From: Thomas Narten <[email-masked]>
Date: Tue, 17 Jan 2012 10:40:23 -0500

Bhumip,

I skimmed this document and am having trouble figuring out what it is
intended to do.

The draft name itself has "problem" in it, but there is no single (or
small set of) succinct problems listed. It's all very high level and
hand wavy. I need help making the connection to an IETF action that
could come out of this document.

For example, it talks about VM migration.

Is VM Migration a "problem" today? There are properietary approaches
that the market seems to like OK. 

What is wrong with the current approaches? What is "broken" that needs
fixing? Why should the IETF get involved in this space? What value
would the IETF bring?

Do you want to be able to do VM migration from one vendor's hypervisor
to another vendor's?  If so, please just say so. Then we can see
whether others here think that is an area the IETF (or some other SDO)
should get involved in.

Thomas

---

From: Bhumip Khasnabish <[email-masked]>
Date: Wed, 18 Jan 2012 01:33:50 -0500

Tom,

Thanks.

Yes, seamless migration of VM and VNE can be problematic in both intra- and
inter-data-center environments, especially in multi-hypervisor case.

It may be very helpful to bring one or more of these
proprietary VM migration approaches to IETF for consideration
for standardization, if that is appropriate.
Sure, we'll update the draft to articulate these requirements.

Best.

Bhumip

---

From: Christopher LILJENSTOLPE <[email-masked]>
Date: Wed, 18 Jan 2012 17:00:44 -0800

Greetings Bhumip,

> inter-data-center environments, especially in multi-hypervisor case.

Agreed, but it's more than just a network problem, is it not?  ESPECIALLY in a multi-hypervisor environment.  So, the big question is, are you interested in just addressing the network portion, or the larger problem (including containers, storage concurrency, etc)?  If the later, do we have the experience and remit to work on that space?  

	Chris

--  
李柯睿
Check my PGP key here: https://www.asgaard.org/~cdl/cdl.asc
Current vCard here: https://www.asgaard.org/~cdl/cdl.vcf

---

From: Thomas Narten <[email-masked]>
Date: Thu, 19 Jan 2012 09:19:28 -0500

> for standardization, if that is appropriate.

Well, let's start with this as a premise. Sure, in the utopia of open
standards everwhere, having all this stuff standardized would be
great.

But let's be realistic. Are any of the vendors/implementors of these
systems coming to the IETF saying they want or need a standard? (If
so, I must have missed this.)

If the market heavyweights are not indicating that they will
participate and/or implement such a standard, would anything the IETF
does in this space be relevant? (Based on experience, I fear the
answer is a very clear "no".)

Finally, do we see operators who use the existing systems coming to
the IETF saying they want open standards here? So far, the silence has
been deafening so far...

> Sure, we'll update the draft to articulate these requirements.

Frankly, until and unless the questions above can be answered in a
more positive manner, IMO there is little point in spending any cycles
in this area, other than getting positive answers to the questions.

Or am I missing something?

Thomas

---

From: Steven Blake <[email-masked]>
Date: Thu, 19 Jan 2012 11:07:56 -0500

> Or am I missing something?

Several system vendors (myself included) stood up in Taipei and said
"one encapsulation, please".  If IETF can facilitate industry
convergence on a small set of NVO3 encapsulations (preferably one), that
would be a big win for Ethernet switch vendors.

Regards,

/////////////////////////////////////////////
Steven Blake       [email-masked]
Extreme Networks              +1 919-884-3211

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Thu, 19 Jan 2012 21:53:16 +0530

>> "one encapsulation, please".  

How do you propose we reconcile the present scenario:

- Hypervisor based encapsulations
- Network based encapsulations
- L2 in L2 encapsulation
- L2 in L3 encapsulation
- L3 in L3 encapsulation

Thanks, Ashish

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Steven Blake
Sent: Thursday, January 19, 2012 9:38 PM
To: Thomas Narten
Cc: [email-masked]
Subject: Re: [dc] draft-khasnabish-vmmi-problems-00.txt

Regards,

---

From: Steven Blake <[email-masked]>
Date: Thu, 19 Jan 2012 11:54:53 -0500

> - Network based encapsulations

I don't see any reason why these need to differ.

> - L2 in L2 encapsulation

IEEE owns this space.

> - L2 in L3 encapsulation

This is NVO3.  I would prefer to have to support the minimum number of
NVO3 encapsulations in HW.

> - L3 in L3 encapsulation

There are already plenty to choose from.

Regards,

---

From: Thomas Narten <[email-masked]>
Date: Thu, 19 Jan 2012 12:47:27 -0500

Steven,

> would be a big win for Ethernet switch vendors.

I agree completely.

But my questions were asking about the apparent lack of  interest from
operators/implementers/market players regarding Bhumip's draft and the
apparent desire to have some sort of standards work related to the
general VM migration problem.

Is there such interest?

Thomas

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 20 Jan 2012 06:37:33 +0530

> I don't see any reason why these need to differ.

The fact is that they do differ. A hypervisor based solution will not run a routing protocol (control plane), or we haven't seen that yet. That means many types of information will be pulled through configuration / mgmt control from a hypervisor controller rather than the network control plane. 

> IEEE owns this space.

Then there is TRILL in IETF.

> NVO3 encapsulations in HW.

Minimum is not quantified :-) I thought you said you wanted only one.

> There are already plenty to choose from.

We agree.

Thanks, Ashish

-----Original Message-----
From: Steven Blake [mailto:[email-masked]] 
Sent: Thursday, January 19, 2012 10:25 PM
To: Ashish Dalela (adalela)
Cc: Thomas Narten; [email-masked]
Subject: RE: [dc] draft-khasnabish-vmmi-problems-00.txt

IEEE owns this space.

Regards,

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 20 Jan 2012 06:50:22 +0530

I think it is fair to say that there is a difference between mobility
and portability. Mobility is live migration, but portability is
specifying a VM's properties, delete in one location and create in
another. The new location can be another hypervisor. In many cases, you
don't need mobility, just portability. E.g. if you have a disaster
recovery situation, then you aren't going to get mobility anyway.

DMTF has specified a standard called OVF (Open Virtualization Format)
that addresses the "description" of the VM. This format is supported by
various hypervisor vendors. So, some level of VM migration
standardization has already happened (albeit portability and not
mobility). 

The questions are:

- Do we need a "control plane" to transfer VM state from point A to B -
the mobility problem?
- Do we need a "control plane" to transfer OVF specification from point
A to B - the portability problem?

The problem is relevant in the inter-datacenter, public-private, or
inter-cloud spaces, where there will be more than one hypervisor
controller by definition. Are we hitting the live migration issue today?
Maybe not. Is it conceivable that we will hit this issue? I think so.

However, the question has to be asked to the provider/operators and not
to the vendors.

Thanks, Ashish

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Thomas Narten
Sent: Thursday, January 19, 2012 11:17 PM
To: Steven Blake
Cc: [email-masked]
Subject: Re: [dc] draft-khasnabish-vmmi-problems-00.txt

Steven,

I agree completely.

Is there such interest?

Thomas

---

From: Christopher LILJENSTOLPE <[email-masked]>
Date: Thu, 19 Jan 2012 18:11:20 -0800

Greetings,

	I agree with some subset of L2-in-L2, L2-in-L3, and L3-in-L3 encaps.  I'm not sure what we mean by network vs hypervisor based encaps.  Do we really want to have a different encap depending on where it's originated or terminated?  Anyone see problems with this if one end of the connection is an appliance (and connected to, say a switch) and one end that's a hypervisor guest?  When folks were asking for a small subset (where small is 1<=n<=3 maybe) I didn't think that they were saying that this was a set of two, one for hypervisors, and one for "network elements" what ever that might mean (I would tend to view a hypervisor switch as an element in the network)

	Chris

---

From: Melinda Shore <[email-masked]>
Date: Thu, 19 Jan 2012 17:16:01 -0900

On 01/19/2012 05:11 PM, Christopher LILJENSTOLPE wrote:
> 	I agree with some subset of L2-in-L2, L2-in-L3, and L3-in-L3 encaps.
 > I'm not sure what we mean by network vs hypervisor based encaps.  Do
 > we really want to have a different encap depending on where it's
 > originated or terminated?  Anyone see problems with this if one end
 > of the connection is an appliance (and connected to, say a switch)
 > and one end that's a hypervisor guest?  When folks were asking for a
 > small subset (where small is 1<=n<=3 maybe) I didn't think that they
 > were saying that this was a set of two, one for hypervisors, and one
 > for "network elements" what ever that might mean (I would tend to
 > view a hypervisor switch as an element in the network)

I'm similarly unclear on what these distinctions actually are.  It
seems to me that in either case (network vs. "hypervisor") we're really
talking about tunnel endpoints.  Can someone state clearly what
the difference is in practice?

Thanks,

Melinda

---

From: <[email-masked]>
Date: Thu, 19 Jan 2012 21:48:28 -0500

> A to B - the portability problem?

Was that supposed to be a serious question?

If it was, I suggest FTP or NFS, both of which are already used to move VM
images in practice, and are already specified in RFCs ;-).  OVF is fundamentally
a VM image format.

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------
________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Ashish Dalela (adalela) [[email-masked]]
Sent: Thursday, January 19, 2012 8:20 PM
To: Thomas Narten; Steven Blake
Cc: [email-masked]
Subject: Re: [dc] draft-khasnabish-vmmi-problems-00.txt

The questions are:

Thanks, Ashish

-----Original Message-----

Steven,

I agree completely.

Is there such interest?

Thomas

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 20 Jan 2012 09:40:08 +0530

>> Was that supposed to be a serious question?

Yes, it is a serious question, because VM mobility goes beyond the VM.

>> images in practice, and are already specified in RFCs ;-).  OVF is
fundamentally

That's one approach. Another approach is to use a SOAP/REST APIs. Yet
another one is to define a cloud control plane, that does more than just
move VMs. E.g. when you move a VM, you have to move the firewall rules,
the VLAN association, the bandwidth, VRF configuration, GRE tunnel
configuration, etc.

Thanks, Ashish

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Friday, January 20, 2012 8:18 AM
To: Ashish Dalela (adalela)
Cc: [email-masked]
Subject: RE: [dc] draft-khasnabish-vmmi-problems-00.txt

If it was, I suggest FTP or NFS, both of which are already used to move
VM
images in practice, and are already specified in RFCs ;-).  OVF is
fundamentally
a VM image format.

Thanks,
--David
From: [email-masked] [[email-masked]] On Behalf Of Ashish
Dalela (adalela) [[email-masked]]

The questions are:

Thanks, Ashish

-----Original Message-----

Steven,

I agree completely.

Is there such interest?

Thomas

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 20 Jan 2012 09:56:15 +0530

>> the difference is in practice?

Tunnel endpoint creation needs a control plane (you don't to be
configuring tunnel end-points manually).

What kind of control plane would you run in the hypervisor? Network type
of control plane that has link-state or distance vector association? Or,
something that interacts with a management controller? People will
prefer the management type of interaction, as you can't have every
hypervisor interact with the network control plane, because it will
inject too many network devices, and break the control plane scale
problem.

As an example, a L2 over L3 needs to map a VLAN to a multicast group.
How do you know which multicast group is available? If the multicast
group creation is happening across many hypervisors, how do these
hypervisors coordinate the multicast group publishing between them? They
have to run PIM?

And, what do you do when a VM talks to a physical server (that doesn't
have a hypervisor). Terminate the encapsulation in the network access?
If yes, how does the network access know which encapsulation to
terminate, and which ones not to terminate? That needs an additional
control plane between a hypervisor to the network that says - "I'm a
hypervisor and I have decided to do XYZ type of encapsulation within
myself, but host targets PQR are unaware of this scheme, and I need you
to do the needful on my behalf". This additional control plane has to
now propagate to the other endpoint.

Then there are issues about QoS, Bandwidth, Security etc. E.g. VM-One
and VM-Two belong to different tenants, but sit on the same hypervisor.
VM-One and VM-Two have different QoS, Bandwidth needs, but they have the
same tunnel. How do I distinguish between them based on the tunnel? In
fact, if the tenant isolation is in the hypervisor, then the underlying
network has no clue which tenant needs what policy. 

So, in short, yes they are both tunnel endpoints. But, when you start to
worry about the control plane needed to automate these tunnels, there
are many differences between network and hypervisor approaches.

Thanks, Ashish

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Melinda Shore
Sent: Friday, January 20, 2012 7:46 AM
To: [email-masked]
Subject: Re: [dc] draft-khasnabish-vmmi-problems-00.txt

Thanks,

Melinda

---

From: Melinda Shore <[email-masked]>
Date: Thu, 19 Jan 2012 19:36:26 -0900

On 1/19/12 7:26 PM, Ashish Dalela (adalela) wrote:
>Bandwidth needs, but they have the

Well, that's not true.  In the case of IPSec we've got SPIs, and
there are similar demultiplexing mechanisms in other technologies.

But frankly I think that if you're going to distinguish between
tunnel endpoints in the hypervisor and tunnel endpoints in other
sorts of network devices I think you're going to be somewhat
hard-pressed to make the case for working on the former in
the IETF.

Melinda

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 20 Jan 2012 10:37:17 +0530

I would not arrive at the conclusion that hypervisor work should or
should not be done in IETF. That's a separate question. VXLAN and NVGRE
are hypervisor based approaches. But, they don't have control planes
(yet). My point is that finding a common map-encap scheme isn't that
hard. The harder part is how to make the hypervisor and network based
map-encap *control planes* work the same way. 

If they don't work the same way, then L2-in-L2, L2-in-L3, L3-in-L3 has a
network flavor and a hypervisor flavor.

Thanks, Ashish

-----Original Message-----
From: Melinda Shore [mailto:[email-masked]] 
Sent: Friday, January 20, 2012 10:06 AM
To: Ashish Dalela (adalela)
Cc: [email-masked]
Subject: Re: [dc] draft-khasnabish-vmmi-problems-00.txt

Melinda

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 20 Jan 2012 00:28:18 -0500

While one can construct strawman hypothesis in whcih there are reasons 
to have different tunnel control protocols depepnding upon end-point 
location, equally one can construct reasonably hypothesis in which the 
same protocol mechanisms work whether the end-point is at the VM, the 
Hypervisor, the ToR switch, or an aggregation switch.

Yours,
Joel

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 20 Jan 2012 11:22:55 +0530

Joel,

Hypervisor control is in the hypervisor manager. Switch control is in
the network control plane. These are parallel silos, that don't
interact. 

Either the hypervisor manager defers control to the network control
plane, or the switches defer control to the manager. Or, some new third
entity emerges to control both, and both hypervisor and switch defer
control to that entity. 

We can't have separate control models and expect this to work in the
same way. 

Which of these (or other) models you think presents a reasonable
approach to reconcile hypervisor control and network control?

Thanks, Ashish

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]] 
Sent: Friday, January 20, 2012 10:58 AM
To: Ashish Dalela (adalela)
Cc: [email-masked]
Subject: Re: [dc] draft-khasnabish-vmmi-problems-00.txt

Yours,
Joel

---

From: Melinda Shore <[email-masked]>
Date: Thu, 19 Jan 2012 21:01:50 -0900

> the network control plane. These are parallel silos, that don't

It seems to me that you're implying that we can't put protocols on
the hypervisor - is that correct?

I'm still having a hard time understanding your argument.  Can you
talk about specific information elements that you'd need for a
hypervisor but not a VPN concentrator (or vice versa), or protocol
elements ditto?  I'd find it helpful if you could illustrate what
the issue is.

Melinda

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 20 Jan 2012 01:04:59 -0500

Several of the proposal use mechanisms that are equally applicable.  For 
a general description of one class of such approaches, look at the dcop 
draft Warren Kumari and I wrote.  I am sure that there are other classes 
as well.

Yours,
Joel

---

From: <[email-masked]>
Date: Thu, 19 Jan 2012 22:20:13 -0800

Hi Ashish,

If you look at technologies like EVB you will see the interaction between the hypervisor and the network control plane.

_Vishwas

-- Sent from my HP TouchPad
On Jan 19, 2012 10:05 PM, Joel M. Halpern <[email-masked]> wrote:
Several of the proposal use mechanisms that are equally applicable. For

as well.

Yours,

Joel

>> should not be done in IETF. That's a separate question. VXLAN and

> NVGRE

>> If they don't work the same way, then L2-in-L2, L2-in-L3, L3-in-L3 has

> a

>>> same tunnel. How do I distinguish between them based on the tunnel?

> In

>>> fact, if the tenant isolation is in the hypervisor, then the

>> underlying

dc mailing list

[email-masked]

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 20 Jan 2012 12:15:12 +0530

>> Several of the proposal use mechanisms that are equally applicable.  

The question is - are hypervisor and network based encap-decap control
planes identical in all respects? The question is not if some mechanisms
can be common. The question is - can all the mechanisms be common?

As a point of note, the draft you mention doesn't address multicast and
broadcast. It defines a database to lookup unicast addresses through a
new protocol. That mechanism won't work for multicast and broadcast. 

Thanks, Ashish

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]] 
Sent: Friday, January 20, 2012 11:35 AM
To: Ashish Dalela (adalela)
Cc: [email-masked]
Subject: Re: [dc] draft-khasnabish-vmmi-problems-00.txt

as well.

Yours,
Joel

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 20 Jan 2012 22:02:19 +0530

Let's assume we are building protocols for hypervisors. Can you suggest
what type of protocols will allow a hypervisor based solution interact
with a non-hypervisor based server? I.e. encap-decap done in the
hypervisor is not understood by the non-virtualized server.

Thanks, Ashish

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Melinda Shore
Sent: Friday, January 20, 2012 11:32 AM
To: [email-masked]
Subject: Re: [dc] draft-khasnabish-vmmi-problems-00.txt

Melinda

---

From: Melinda Shore <[email-masked]>
Date: Fri, 20 Jan 2012 10:28:33 -0900

On 01/20/2012 07:32 AM, Ashish Dalela (adalela) wrote:
> Let's assume we are building protocols for hypervisors.

This may be the source of some of the confusion.  This is
a very vertical framing of the question, when more typically
we're dealing with horizontal communication.   Network
layering is not as clean as we'd necessarily like but what
we're usually dealing with is protocol peers talking
to each other.  That is to say, we're not building protocols
for *hypervisors*, we're building protocols for some functional
component of the network.  From what I've seen a bunch of us
are unclear on why you think that a hypervisor cannot be
where that component lives.

Melinda

---

From: Thomas Narten <[email-masked]>
Date: Fri, 20 Jan 2012 16:36:26 -0500

Ashish,

> hypervisor is not understood by the non-virtualized server.

In which case, in something like NVO3/VXLAN/NVGRE, the switch that the
physical server connects to can do the encap/decap.

But the kind of question you are asking above about protocols
completely misses the point.

We are not building "protocols for hypervisors". Hopefully, at some
point we talk about building "solutions" to "real concrete
problems". One of the requirements for such a solution might be that
hypervisors can particpate in the protocol. And that non-virtualized
server are not excluded (and indeed that is what the NVO3 problem
statement talks about). If so, those sorts of requirement get factored
in. But the key thing is not "hypervisors" but how well a proposed
solution addresses the problem and all of its assocated requirements.

<broken-record-mode>

So, first, we articulate a problem or pain point that we need a fix
for. Then we start talking about requirements a workable solution must
address, then mechanisms that address the problem.

Talking generically about "hypervisor based solution" needing to talk
to "non-hypervisor based servers" seems like a pointless exercise. At
least not without being able to answer "solution to what? part of the
question"

</broken-record-mode>

Thomas

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 20 Jan 2012 15:18:15 -0800

> the hypervisor - is that correct?
Hi Melinda,

An interesting thing to note is that the more the functionality you
put in the hypervisor, the more you stress the single point of failure
in the virtualized system.

Thanks,
Vishwas

---

From: Melinda Shore <[email-masked]>
Date: Fri, 20 Jan 2012 14:31:05 -0900

> in the virtualized system.

Well, there are a few ways to look at it.  For example,
the fewer components you've got the larger the mean time
between failures.  But aside from that it's been a
general rule of thumb that you want to minimize the
impact of failed components on non-failed components
(the fate sharing principle).

At any rate the hypervisor (at least the ones with which
I'm familiar) basically *are* network devices - they
function as a switch, or even a NAT.  If you're going to
suggest that they can't be used to terminate control plane
sessions I hope there's a more compelling reason for it
than what has been offered so far.

Melinda

---

From: Christopher LILJENSTOLPE <[email-masked]>
Date: Fri, 20 Jan 2012 15:41:40 -0800

Greetings,

	Another way of looking at it is a hypervisor is really an operating system distribution.  Many of the hypervisors out there have the network "switch" as a separate process (actually I believe all of them do).  So, if we are saying that networking intel doesn't belong in an OS distribution, that is a departure from current thinking :)

	Chris

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 20 Jan 2012 15:47:10 -0800

Hi Chistopher,

I totally agree.

The point I was making was that the hypervisor is the single point of
failure which can cause all the guest OS to fail. The more complex you
make the functionality the higer the chances of failure. So we should
work with that thought in mind.

Thanks,
Vishwas

On 1/20/12, Christopher LILJENSTOLPE <[email-masked]> wrote:
> Greetings,
>
> 	Another way of looking at it is a hypervisor is really an operating system
> distribution.  Many of the hypervisors out there have the network "switch"
> as a separate process (actually I believe all of them do).  So, if we are
> saying that networking intel doesn't belong in an OS distribution, that is a
> departure from current thinking :)
>
> 	Chris
>

---

From: Christopher LILJENSTOLPE <[email-masked]>
Date: Fri, 20 Jan 2012 16:06:19 -0800

Greetings Vishwas,

	And I guess I am saying that I'm not sure I agree.  If a single process running can kill all the other processes on a given system (especially if it is in "user space" - which I would assume for control plane functions), I would say that there are more substantial issues with the architecture of that particular VM distribution.  Similar answer if someone actually put a control-plane engine (think routing protocols) in kernel space.  We, as a standards organization can't save the world from less-than-inteligent developers....

	Chris

---

From: <[email-masked]>
Date: Fri, 20 Jan 2012 20:22:38 -0500

Ashish,

Unfortunately, this is digging in the "wrong place" because it recreates the
problem that OVF was designed to solve.  OVF is intended to be a self-contained
packaging and distribution format that contains everything needed to instantiate
one or more VMs.  As such, OVF can be moved by all of the protocols noted below,
plus a variety of other means, such as sneaker-net.

If OVF is insufficient for the portability use case, then I suggest going to DMTF
to work on adding what's missing instead of inventing a "control plane" that is
at odds with OVF's design intent.

Thanks,
--David

> From: Ashish Dalela (adalela) [mailto:[email-masked]]
> Sent: Thursday, January 19, 2012 11:10 PM
> To: Black, David
> Cc: [email-masked]

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 20 Jan 2012 17:23:25 -0800

Hi Chris,

It is not about crashing alone, it could be a wrong entry added into
the table that could cause the packets to be routed wrongly. The idea
is about increasing the complexity in a single point of failure.

I guess this is not the most critical discussion and we can probably
move forward.

Thanks,
Vishwas

On 1/20/12, Christopher LILJENSTOLPE <[email-masked]> wrote:
> Greetings Vishwas,
>
> 	And I guess I am saying that I'm not sure I agree.  If a single process
> running can kill all the other processes on a given system (especially if it
> is in "user space" - which I would assume for control plane functions), I
> would say that there are more substantial issues with the architecture of
> that particular VM distribution.  Similar answer if someone actually put a
> control-plane engine (think routing protocols) in kernel space.  We, as a
> standards organization can't save the world from less-than-inteligent
> developers....
>
>
> 	Chris
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 21 Jan 2012 09:00:42 +0100

> 	And I guess I am saying that I'm not sure I agree.  If a single process running can kill all the other processes on a given system (especially if it is in "user space" - which I would assume for c…

The Linux-based hypervisors I have used happen to have the bridge(s)
in the kernel plus user space utilities to configure/control them, as
it should be. But then I am not using products but just open source
implementations... ;-)

A general observation: I think what this discussion really is lacking
is implementors familiar with real-world implementations. If there
were more implementors involved here with a decent understanding of
code bases, I assume we would be quickly able to figure out what and
if anything needs to be dealt with in the IETF.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Bhumip Khasnabish <[email-masked]>
Date: Mon, 30 Jan 2012 21:39:04 -0500

Hello Chris,

Thanks.

>So, the big question is, are you interested in just addressing the network
>portion, or the larger problem (including containers, storage concurrency,
etc)?

We can start with a focus on the "network portion" with only as much as
needed of others (concurrency, containers, de-duplication, etc.). Hope this
helps.

Best.

Bhumip

On Wed, Jan 18, 2012 at 8:00 PM, Christopher LILJENSTOLPE
<[email-masked]>wrote:

> Agreed, but it's more than just a network problem, is it not?  ESPECIALLY
> in a multi-hypervisor environment.  So, the big question is, are you
> interested in just addressing the network portion, or the larger problem
> (including containers, storage concurrency, etc)?  If the later, do we have
> the experience and remit to work on that space?
>
>        Chris
>
> >

---

From: Bhumip Khasnabish <[email-masked]>
Date: Tue, 31 Jan 2012 12:09:31 -0500

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title           : Network Virtualization Overlay Control Protocol Requirements
	Author(s)       : Lawrence Kreeger
                          Dinesh Dutt
                          Thomas Narten
                          David Black
                          Murari Sridharan
	Filename        : draft-kreeger-nvo3-overlay-cp-00.txt
	Pages           : 13
	Date            : 2012-01-30

   The document draft-narten-nvo3-overlay-problem-statement-01 discusses
   the needs for network virtualization using overlay networks in highly
   virtualized data centers.  The problem statement outlines a need for
   control protocols to facilitate running these overlay networks.  This
   document outlines the high level requirements to be fulfilled by the
   control protocols.

A URL for this Internet-Draft
is:http://www.ietf.org/internet-drafts/draft-kreeger-nvo3-overlay-cp-00.txt

Internet-Drafts are also available by anonymous FTP
at:ftp://ftp.ietf.org/internet-drafts/

This Internet-Draft can be retrieved
at:ftp://ftp.ietf.org/internet-drafts/draft-kreeger-nvo3-overlay-cp-00.txt

I-D Action: draft-kreeger-nvo3-overlay-cp-00.txt
------------------------------

   - *To*: i-d-announce at ietf.org <[email-masked]>
   - *Subject*: I-D Action: draft-kreeger-nvo3-overlay-cp-00.txt
   - *From*: internet-drafts at ietf.org <[email-masked]>
   - *Date*: Mon, 30 Jan 2012 11:39:16 -0800
   - *Delivered-to*: i-d-announce at ietfa.amsl.com<[email-masked]>
   - *List-archive*: <http://www.ietf.org/mail-archive/web/i-d-announce>
   - *List-help*:
<mailto:[email-masked]?subject=help<[email-masked]?subject=help>>

   - *List-id*: Internet Draft Announcements only <i-d-announce.ietf.org>
   - *List-post*: <mailto:[email-masked] <[email-masked]>>
   - *List-subscribe*: <https://www.ietf.org/mailman/listinfo/i-d-announce>,
   <mailto:[email-masked]?subject=subscribe<[email-masked]?subject=subscribe>>

   - *List-unsubscribe*: <https://www.ietf.org/mailman/options/i-d-announce>,
   <mailto:[email-masked]?subject=unsubscribe<[email-masked]?subject=unsubscribe>>

   - *Reply-to*: internet-drafts at ietf.org <[email-masked]>

------------------------------

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title           : Network Virtualization Overlay Control Protocol Requirements
	Author(s)       : Lawrence Kreeger
                          Dinesh Dutt
                          Thomas Narten

---

From: <[email-masked]>
Date: Tue, 31 Jan 2012 12:24:40 -0500

OVF is not mentioned anywhere in that draft because:

"OVF is intended to be a self-contained packaging and distribution format"

OVF is not generally used as a runtime execution format for VMs.

Thanks,
--David (draft co-author)

From: [email-masked] [mailto:[email-masked]] On Behalf Of Bhumip Khasnabish
Sent: Tuesday, January 31, 2012 12:10 PM
To: Black, David
Cc: [email-masked]; [email-masked]
Subject: Re: [dc] OVF "control plane" - Not a good idea

                          Dinesh Dutt

                          Thomas Narten

                          David Black

                          Murari Sridharan

       Pages           : 13

       Date            : 2012-01-30

   control protocols.

A URL for this Internet-Draft is:

http://www.ietf.org/internet-drafts/draft-kreeger-nvo3-overlay-cp-00.txt

Internet-Drafts are also available by anonymous FTP at:

ftp://ftp.ietf.org/internet-drafts/

This Internet-Draft can be retrieved at:

ftp://ftp.ietf.org/internet-drafts/draft-kreeger-nvo3-overlay-cp-00.txt

I-D Action: draft-kreeger-nvo3-overlay-cp-00.txt
________________________________

 *   To: i-d-announce at ietf.org<mailto:[email-masked]>
 *   Subject: I-D Action: draft-kreeger-nvo3-overlay-cp-00.txt
 *   From: internet-drafts at ietf.org<mailto:[email-masked]>
 *   Date: Mon, 30 Jan 2012 11:39:16 -0800
 *   Delivered-to: i-d-announce at ietfa.amsl.com<mailto:[email-masked]>
 *   List-archive: <http://www.ietf.org/mail-archive/web/i-d-announce>
 *   List-help: <mailto:[email-masked]?subject=help>
 *   List-id: Internet Draft Announcements only <i-d-announce.ietf.org<http://i-d-announce.ietf.org>>
 *   List-post: <mailto:[email-masked]>
 *   List-subscribe: <https://www.ietf.org/mailman/listinfo/i-d-announce>, <mailto:[email-masked]?subject=subscribe>
 *   List-unsubscribe: <https://www.ietf.org/mailman/options/i-d-announce>, <mailto:[email-masked]?subject=unsubscribe>
 *   Reply-to: internet-drafts at ietf.org<mailto:[email-masked]>

________________________________

                          Dinesh Dutt

                          Thomas Narten

On Fri, Jan 20, 2012 at 8:22 PM, <[email-masked]<mailto:[email-masked]>> wrote:
Ashish,

Thanks,
--David

> From: Ashish Dalela (adalela) [mailto:[email-masked]<mailto:[email-masked]>]
> Cc: [email-masked]<mailto:[email-masked]>
> From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>]
> Cc: [email-masked]<mailto:[email-masked]>
> +1 (508) 293-7953<tel:%2B1%20%28508%29%20293-7953>             FAX: +1 (508) 293-7786<tel:%2B1%20%28508%29%20293-7786>
> [email-masked]<mailto:[email-masked]>        Mobile: +1 (978) 394-7754<tel:%2B1%20%28978%29%20394-7754>
> From: [email-masked]<mailto:[email-masked]> [[email-masked]<mailto:[email-masked]>] On Behalf Of Ashish
> Dalela (adalela) [[email-masked]<mailto:[email-masked]>]
> Cc: [email-masked]<mailto:[email-masked]>
> From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of
> Thomas Narten
> Cc: [email-masked]<mailto:[email-masked]>
> [email-masked]<mailto:[email-masked]>
> [email-masked]<mailto:[email-masked]>

_______________________________________________
dc mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dc

---

From: Bhumip Khasnabish <[email-masked]>
Date: Tue, 31 Jan 2012 12:28:00 -0500

Bhumip Khasnabish [email-masked]
To Christopher LILJENSTOLPE [email-masked]
ccThomas Narten <[email-masked]>, Ronald Bonica <[email-masked]>,
[email-masked],"So, Ning" <[email-masked]>
date Fri, Dec 30, 2011 at 12:56 PM
subjectRe: [dc] Elevator Pitch (was: Scoping the Interim meeting)

mailed-bygmail.com

hide details 12/30/11

Hello Chris,

Thank you very much for your comments and suggestions.

If you do not mind, please provide write ups, as appropriate, on Amazon
API, ONF, OpenFlow, Open Stack, etc. that you mention below for inclusion
in the next version of the SDO survey draft.

For the work item survey draft, the intention is to provide a list. We have
seen many follow up drafts during the last two IETF mtgs articulating
problems and possible solutions  (VDCS, VPN4DC, VEPC, VPN-O-CS, VPN-O-DCS,
VRM, security framework for VDCS, etc.) from both service providers and
solution providers.

We'll be updating this draft soon, and can work on providing priority, if
that is helpful.

*If we want to work on one near-term high-priority work item, may be
virtual machine (and virtual network element) mobility and interconnection
can be the focus..(may be we need to use distributed control, distributed
hash, etc.) *

Any others?

Thanks and best wishes.

Bhumip
===========================================================

On Tue, Jan 31, 2012 at 12:24 PM, <[email-masked]> wrote:

>  OVF is not mentioned anywhere in that draft because:****
>
> ** **
>
> “OVF is intended to be a self-contained packaging and distribution format”
> ****
>
> ** **
>
> OVF is not generally used as a runtime execution format for VMs.****
>
> ** **
>
> Thanks,
> ****
> ** **
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of *Bhumip
> Khasnabish
> *Sent:* Tuesday, January 31, 2012 12:10 PM
> *To:* Black, David
> *Cc:* [email-masked]; [email-masked]
> *Subject:* Re: [dc] OVF "control plane" - Not a good idea****
>
> ** **
>
> *A New Internet-Draft is available from the on-line Internet-Drafts directories.*
>
> * *
>
> *       Title           : Network Virtualization Overlay Control Protocol Requirements*
>
> *       Author(s)       : Lawrence Kreeger*
>
> *                          Dinesh Dutt*
>
> *                          Thomas Narten*
>
> *                          David Black*
>
> *                          Murari Sridharan*
>
> *       Filename        : draft-kreeger-nvo3-overlay-cp-00.txt*
>
> *       Pages           : 13*
>
> *       Date            : 2012-01-30*
>
> * *
>
> *   The document draft-narten-nvo3-overlay-problem-statement-01 discusses*
>
> *   the needs for network virtualization using overlay networks in highly*
>
> *   virtualized data centers.  The problem statement outlines a need for*
>
> *   control protocols to facilitate running these overlay networks.  This*
>
> *   document outlines the high level requirements to be fulfilled by the*
>
> *   control protocols.*
>
> * *
>
> * *
>
> *A URL for this Internet-Draft is:*
>
> *http://www.ietf.org/internet-drafts/draft-kreeger-nvo3-overlay-cp-00.txt*
>
> * *
>
> *Internet-Drafts are also available by anonymous FTP at:*
>
> *ftp://ftp.ietf.org/internet-drafts/*
>
> * *
>
> *This Internet-Draft can be retrieved at:*
>
> *ftp://ftp.ietf.org/internet-drafts/draft-kreeger-nvo3-overlay-cp-00.txt*
>
> * *
>
> I-D Action: draft-kreeger-nvo3-overlay-cp-00.txt****
>    - *To*: i-d-announce at ietf.org <[email-masked]> ****
>    - *Subject*: I-D Action: draft-kreeger-nvo3-overlay-cp-00.txt ****
>    - *From*: internet-drafts at ietf.org <[email-masked]> *
>    ***
>    - *Date*: Mon, 30 Jan 2012 11:39:16 -0800 ****
>    ****
>    - *List-archive*: <http://www.ietf.org/mail-archive/web/i-d-announce> *
>    ***
>    - *List-help*: <mailto:[email-masked]?subject=help<[email-masked]?subject=help>>
>    ****
>    ****
>    - *List-post*: <mailto:[email-masked] <[email-masked]>> *
>    ***
>    ****
>    - *List-unsubscribe*: <
>    https://www.ietf.org/mailman/options/i-d-announce>, <
>    mailto:[email-masked]?subject=unsubscribe<[email-masked]?subject=unsubscribe>>
>    ****
>    - *Reply-to*: internet-drafts at ietf.org<[email-masked]>
>    ****
>
> A New Internet-Draft is available from the on-line Internet-Drafts directories.****
>
> ** **
>
>        Title           : Network Virtualization Overlay Control Protocol Requirements****
>
>        Author(s)       : Lawrence Kreeger****
>
>                           Dinesh Dutt****
>
>                           Thomas Narten****
>
> ** **
>
>  On Fri, Jan 20, 2012 at 8:22 PM, <[email-masked]> wrote:****
>
> Ashish,
>
> https://www.ietf.org/mailman/listinfo/dc****
>
>
>
>
>   ****
>

---

From: Donald Eastlake <[email-masked]>
Date: Tue, 31 Jan 2012 14:13:20 -0500

See below

> Then there is TRILL in IETF.

TRILL is not L2 in L2. TRILL as specified defines a new layer that is
above all Layer 2 bridging but below Layer 3 routing. TRILL switches
appear to be end stations to bridges but are as transparent to L3
routers as bridges are.

Thanks,
Donald
=============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 155 Beaver Street, Milford, MA 01757 USA
 [email-masked]