From wesley.george@twcable.com Sun Apr 1 04:29:33 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2E05521F8668 for ; Sun, 1 Apr 2012 04:29:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.863 X-Spam-Level: X-Spam-Status: No, score=-0.863 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, J_CHICKENPOX_13=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BTqSdMUbHevt for ; Sun, 1 Apr 2012 04:29:32 -0700 (PDT) Received: from cdpipgw01.twcable.com (cdpipgw01.twcable.com [165.237.59.22]) by ietfa.amsl.com (Postfix) with ESMTP id 28C8521F85B8 for ; Sun, 1 Apr 2012 04:29:29 -0700 (PDT) X-SENDER-IP: 10.136.163.12 X-SENDER-REPUTATION: None X-IronPort-AV: E=Sophos;i="4.75,352,1330923600"; d="scan'208";a="361946209" Received: from unknown (HELO PRVPEXHUB03.corp.twcable.com) ([10.136.163.12]) by cdpipgw01.twcable.com with ESMTP/TLS/RC4-MD5; 01 Apr 2012 07:29:04 -0400 Received: from PRVPEXVS03.corp.twcable.com ([10.136.163.27]) by PRVPEXHUB03.corp.twcable.com ([10.136.163.12]) with mapi; Sun, 1 Apr 2012 07:29:17 -0400 From: "George, Wes" To: Christopher LILJENSTOLPE , "opsawg@ietf.org" Date: Sun, 1 Apr 2012 07:29:15 -0400 Thread-Topic: [OPSAWG] adoption of Draft-kuarsingh-lsn-deployment as WG item Thread-Index: Ac0MwD3Qs1L2auPPSwGB4Y9DVRB0bACn6OzQ Message-ID: References: <4F9931EB-E20A-4A7F-A670-47B587397C69@cdl.asgaard.org> In-Reply-To: <4F9931EB-E20A-4A7F-A670-47B587397C69@cdl.asgaard.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Cc: opsawg-chairs Subject: Re: [OPSAWG] adoption of Draft-kuarsingh-lsn-deployment as WG item X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Apr 2012 11:29:33 -0000 VGhpcyBzZWVtcyBsaWtlIGFub3RoZXIgb2YgdGhlIGRyYWZ0cyBvcnBoYW5lZCBieSBCRUhBVkUu IEl0J3Mgc3BlY2lmaWMgdG8gQ0dOLCBhbmQgdGhlcmVmb3JlIG5vdCBleGFjdGx5IHN1aXRlZCB0 byBPcHNBV0cgYnV0IEknbSBub3Qgb3Bwb3NlZCB0byB1cyBhZG9wdGluZyBpdCBhYnNlbnQgYSBi ZXR0ZXIgc3VnZ2VzdGlvbi4NCg0KDQpTb21lIGNvbW1lbnRzIG9uIHRoZSBkcmFmdC0NCkkgdGhp bmsgaXQgd291bGQgbWFrZSBzZW5zZSBmb3Igc2VjdGlvbiAxLzIgYW5kIHRoZSBkcmFmdCBvdmVy YWxsIHRvIGZvY3VzIGEgbG90IGxlc3Mgb24gdGhlIHJhdGlvbmFsZSBmb3IgZGVwbG95aW5nIENH Ti4gWWVzLCB0aGVyZSBhcmUgYSBsb3Qgb2YgZm9sa3MgdGhhdCB2aWV3IGl0IGFzIGluaGVyZW50 bHkgZXZpbCBhbmQgYWJob3JyZW50LCBidXQgd2UgbmVlZCB0byBzb3J0IG9mIGdldCBvdmVyIHRo YXQgaW4gb3JkZXIgdG8gbWFrZSBhbnkgZm9yd2FyZCBwcm9ncmVzcy4gQ0dOIGV4aXN0cywgYW5k IGl0J3MgZ29pbmcgdG8gZ2V0IHVzZWQgbm8gbWF0dGVyIGhvdyBtdWNoIHNvbWUgcHJvdGVzdC4g U28gcmF0aGVyIHRoYW4gdGFsa2luZyBhYm91dCB3aHkgY2FycmllcnMgbWlnaHQgZGVwbG95IGl0 IGFuZCBleHBlbmRpbmcgY29uc2lkZXJhYmxlIHRleHQgcGVyZm9ybWluZyBhcG9sb2dldGljcyBh Ym91dCBpdCwgSSdkIHJhdGhlciB0aGUgZHJhZnQgZ2V0IHN0cmFpZ2h0IHRvIHRoZSBwb2ludCBh Ym91dCB0aGUgcHJvYmxlbXMgdGhhdCB0aGlzIGRyYWZ0IHNvbHZlcy4NClRoYXQgaXMsIGFzc3Vt aW5nIHRoYXQgdGhlIGNhcnJpZXIgaGFzIGFscmVhZHkgZGVjaWRlZCB0aGV5IHdpbGwgdXNlIENH TiwgdGhleSBoYXZlIHRoZSBmb2xsb3dpbmcgcHJvYmxlbXMuIFRoaXMgZHJhZnQgc29sdmVzIHRo ZW0gYnkgLi4uDQoiIFRoaXMgZG9jdW1lbnQgc2hvd3MgaG93IE1QTFMvVlBOcyBhcyBkZXNjcmli ZWQgaW4gW1JGQzQzNjRdIGNhbiBiZQ0KICAgdXNlZCB0byBpbnRlZ3JhdGUgdGhlIENHTiBpbmZy YXN0cnVjdHVyZSBzb2x2aW5nIGtleSBwcm9ibGVtcyBmYWNlZA0KICAgYnkgdGhlIG9wZXJhdG9y LiINCkl0J2QgYmUgaGVscGZ1bCBpZiB0aG9zZSAia2V5IHByb2JsZW1zIiBjb3VsZCBiZSBzdW1t YXJpemVkIGluIHRoZSBpbnRybyBvciBzZWNvbmQgc2VjdGlvbiwgYW5kIHRoZW4gZXhwb3VuZGVk IHVwb24gaW4gbGF0ZXIgc2VjdGlvbnMgYWxvbmcgd2l0aCB0aGUgTVBMUyBWUE4tYmFzZWQgc29s dXRpb24uDQoNCkknbSBhbHNvIG5vdCBjbGVhciBvbiB3aGF0IHBvcnRpb25zIG9mIHNlY3Rpb24g MyBoYXZlIHRvIGRvIHdpdGggdGhlIHByb3Bvc2VkIHByb2JsZW0vc29sdXRpb24gc3BhY2UuIFBv cnRpb25zIG9mIGl0IHNlZW1zIHRvIGJlIHRvIGJlIGdlbmVyYWxpemVkIENHTiBkZXBsb3ltZW50 IGNvbnNpZGVyYXRpb25zLiBUaGF0IG1heSBiZSBmaW5lLCBidXQgdGhhdCdzIG5vdCB0ZWNobmlj YWxseSB3aGF0IHRoaXMgZG9jdW1lbnQgcHVycG9ydHMgdG8gZGlzY3VzcywgYW5kIG1heWJlIHNo b3VsZCBiZSBpbiBhbm90aGVyIGRvY3VtZW50LCBwZXJoYXBzIGV2ZW4gdGhlIGxzbi1yZXF1aXJl bWVudHMgZG9jdW1lbnQuIEkgY2FuIHNvcnQgb2Ygc2VlIHdoZXJlIEwzVlBOIG1pZ2h0IGhlbHAg d2l0aCBzb21lIG9mIHRoZXNlIHJlcXVpcmVtZW50cywgYnV0IHRoaXMgc2VjdGlvbiBzaG91bGQg Zm9jdXMgbW9yZSBjcmlzcGx5IG9uIHRoZSByZXF1aXJlbWVudHMgdGhhdCBhcmUgZGlmZmljdWx0 IHRvIHNvbHZlIHdpdGhvdXQgdGhlIHVzZSBvZiB0aGUgc29sdXRpb25zIHByb3Bvc2VkIGluIHRo aXMgZG9jdW1lbnQuDQoNCllvdSBoYXZlIGFuIG9ycGhhbi9lbXB0eSBzZWN0aW9uIDUNCg0KSSdk IGFsc28gZHJvcCB0aGUgInByb2JsZW1zIHRoaXMgZG9lc24ndCBzb2x2ZSIgaW4gc2VjdGlvbiA3 IHVubGVzcyB5b3UnZCBsaWtlIHRvIHR1cm4gaXQgaW50byBhIGdhcCBhbmFseXNpcyB3aXRoIGEg bWluZCB0b3dhcmRzIGFza2luZyBJRVRGIHRvIHNvbHZlIHRob3NlIHByb2JsZW1zIHdpdGggcHJv dG9jb2wgY2hhbmdlcy4gSWYgaXQncyBwdXJlbHkgaW5mb3JtYXRpb25hbCwgeW91J2xsIGxpa2Vs eSBuZXZlciBiZSBkb25lIGRvY3VtZW50aW5nIHRoZSBwcm9ibGVtcyB3aXRoIENHTiB0aGF0IGFy ZW4ndCBzb2x2ZWQuIFRoaXMgaXMgdGhlIGVzc2VuY2Ugb2YgYSAid2UgZG9uJ3Qga25vdyB3aGF0 IHdlIGRvbid0IGtub3ciIHN5c3RlbSwgc28gaXQncyBtaXNsZWFkaW5nIHRvIGRpc2N1c3MgaXQg YXMgaWYgaXQgd2FzIGEgY29tcGxldGUgbGlzdC4NCg0KDQpUaGFua3MsDQoNCldlcyBHZW9yZ2UN Cg0KDQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogb3BzYXdnLWJvdW5j ZXNAaWV0Zi5vcmcgW21haWx0bzpvcHNhd2ctYm91bmNlc0BpZXRmLm9yZ10gT24gQmVoYWxmIE9m DQo+IENocmlzdG9waGVyIExJTEpFTlNUT0xQRQ0KPiBTZW50OiBXZWRuZXNkYXksIE1hcmNoIDI4 LCAyMDEyIDQ6NTMgQU0NCj4gVG86IG9wc2F3Z0BpZXRmLm9yZw0KPiBDYzogb3BzYXdnLWNoYWly cw0KPiBTdWJqZWN0OiBbT1BTQVdHXSBhZG9wdGlvbiBvZiBEcmFmdC1rdWFyc2luZ2gtbHNuLWRl cGxveW1lbnQgYXMgV0cgaXRlbQ0KPg0KPiBHcmVldGluZ3MsDQo+DQo+ICAgICAgIFRoaXMgaXMg dG8gc3RhcnQgYSBwb2xsIGluIHRoZSB3b3JraW5nIGdyb3VwIHRvIHNlZSBpZiB3ZSB3YW50IHRv IGFjY2VwdA0KPiBEcmFmdC1rdWFyc2luZ2gtbHNuLWRlcGxveW1lbnQgYXMgYSB3b3JraW5nIGdy b3VwIGl0ZW0uICBXZSB3aWxsIGxlYXZlIHRoaXMNCj4gcG9sbCBvcGVuIHVudGlsIDIzNTkgVVRD IG9uIDQgQXByaWwuICBQbGVhc2Ugc3BlYWsgZWFybHkgYW5kIG9mdGVuLg0KPg0KPiAgICAgICBD aHJpcw0KPg0KPiAtLQ0KPiDmnY7mn6/nnb8NCj4gQ2hlY2sgbXkgUEdQIGtleSBoZXJlOiBodHRw czovL3d3dy5hc2dhYXJkLm9yZy9+Y2RsL2NkbC5hc2MNCj4gQ3VycmVudCB2Q2FyZCBoZXJlOiBo dHRwczovL3d3dy5hc2dhYXJkLm9yZy9+Y2RsL2NkbC52Y2YNCj4gQ2hlY2sgbXkgY2FsZW5kYXIg YXZhaWxhYmlsaXR5OiBodHRwczovL3R1bmdsZS5tZS9jZGwNCj4NCj4gX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gT1BTQVdHIG1haWxpbmcgbGlzdA0K PiBPUFNBV0dAaWV0Zi5vcmcNCj4gaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5m by9vcHNhd2cNCg0KVGhpcyBFLW1haWwgYW5kIGFueSBvZiBpdHMgYXR0YWNobWVudHMgbWF5IGNv bnRhaW4gVGltZSBXYXJuZXIgQ2FibGUgcHJvcHJpZXRhcnkgaW5mb3JtYXRpb24sIHdoaWNoIGlz IHByaXZpbGVnZWQsIGNvbmZpZGVudGlhbCwgb3Igc3ViamVjdCB0byBjb3B5cmlnaHQgYmVsb25n aW5nIHRvIFRpbWUgV2FybmVyIENhYmxlLiBUaGlzIEUtbWFpbCBpcyBpbnRlbmRlZCBzb2xlbHkg Zm9yIHRoZSB1c2Ugb2YgdGhlIGluZGl2aWR1YWwgb3IgZW50aXR5IHRvIHdoaWNoIGl0IGlzIGFk ZHJlc3NlZC4gSWYgeW91IGFyZSBub3QgdGhlIGludGVuZGVkIHJlY2lwaWVudCBvZiB0aGlzIEUt bWFpbCwgeW91IGFyZSBoZXJlYnkgbm90aWZpZWQgdGhhdCBhbnkgZGlzc2VtaW5hdGlvbiwgZGlz dHJpYnV0aW9uLCBjb3B5aW5nLCBvciBhY3Rpb24gdGFrZW4gaW4gcmVsYXRpb24gdG8gdGhlIGNv bnRlbnRzIG9mIGFuZCBhdHRhY2htZW50cyB0byB0aGlzIEUtbWFpbCBpcyBzdHJpY3RseSBwcm9o aWJpdGVkIGFuZCBtYXkgYmUgdW5sYXdmdWwuIElmIHlvdSBoYXZlIHJlY2VpdmVkIHRoaXMgRS1t YWlsIGluIGVycm9yLCBwbGVhc2Ugbm90aWZ5IHRoZSBzZW5kZXIgaW1tZWRpYXRlbHkgYW5kIHBl cm1hbmVudGx5IGRlbGV0ZSB0aGUgb3JpZ2luYWwgYW5kIGFueSBjb3B5IG9mIHRoaXMgRS1tYWls IGFuZCBhbnkgcHJpbnRvdXQuDQo= From victor.kuarsingh@gmail.com Sun Apr 1 12:13:29 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9AF0121F878B for ; Sun, 1 Apr 2012 12:13:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.399 X-Spam-Level: X-Spam-Status: No, score=-0.399 tagged_above=-999 required=5 tests=[BAYES_50=0.001, J_CHICKENPOX_13=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bfu958Xq1Cb7 for ; Sun, 1 Apr 2012 12:13:28 -0700 (PDT) Received: from mail-wi0-f172.google.com (mail-wi0-f172.google.com [209.85.212.172]) by ietfa.amsl.com (Postfix) with ESMTP id 4BE6121F873B for ; Sun, 1 Apr 2012 12:13:28 -0700 (PDT) Received: by wibhj6 with SMTP id hj6so1651004wib.13 for ; Sun, 01 Apr 2012 12:13:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :in-reply-to:mime-version:content-type:content-transfer-encoding; bh=dnsLUt0c5jVLicSiBRoJeseBEUZ2FG32XOqvzDej9ks=; b=VKZS9ttfVfP/sn7w2suo6LEkbgaMrPR3u7IGpssSIxwuS8SU4XeJcQUnMyMsdWMwuU zQ+IErNRvv6mB2CEsqMTLlQRc5NHGFXXapyhoVetBd5s34puT2Ha4peEvsadxsQBjh4U neNmY1F20PT/BtGOtU7xPpowTbd9ffn3681ojrlr8zLbYReqSOYy+wfRffEA9rt3+f4e F46Esr9Nv5yLEFOZWPVKks2gZcFmeeowhaeohQQid0byz8wnkKPmMQQfaB9AwRsbrWqq kLsFd1KAKSyqCbkk1jgVASRdzG3yq7GG3zxitYA1TORg1qmd+vnAEveWCabRN+lh2C1k LxkQ== Received: by 10.180.103.35 with SMTP id ft3mr17386016wib.0.1333307607465; Sun, 01 Apr 2012 12:13:27 -0700 (PDT) Received: from [10.59.90.251] (access-49.80.rev.fr.colt.net. [213.41.80.49]) by mx.google.com with ESMTPS id gg2sm44443580wib.7.2012.04.01.12.13.24 (version=SSLv3 cipher=OTHER); Sun, 01 Apr 2012 12:13:26 -0700 (PDT) User-Agent: Microsoft-MacOutlook/14.0.0.100825 Date: Sun, 01 Apr 2012 21:13:23 +0200 From: Victor Kuarsingh To: "George, Wes" , Christopher LILJENSTOLPE , "opsawg@ietf.org" Message-ID: Thread-Topic: [OPSAWG] adoption of Draft-kuarsingh-lsn-deployment as WG item In-Reply-To: Mime-version: 1.0 Content-type: text/plain; charset="ISO-2022-JP" Content-transfer-encoding: 7bit Cc: opsawg-chairs Subject: Re: [OPSAWG] adoption of Draft-kuarsingh-lsn-deployment as WG item X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Apr 2012 19:13:29 -0000 Wes, On 12-04-01 1:29 PM, "George, Wes" wrote: > > >Some comments on the draft- >I think it would make sense for section 1/2 and the draft overall to >focus a lot less on the rationale for deploying CGN. Understood. The original -00 version was drafted early on in the CGN discussion (July 2010) when such work required a boiler plate in front to just get people to keep reading. It's likely that enough other material is available as reference that we can just point there and remove some of this text. >Yes, there are a lot of folks that view it as inherently evil and >abhorrent, but we need to sort of get over that in order to make any >forward progress. CGN exists, and it's going to get used no matter how >much some protest. So rather than talking about why carriers might deploy >it and expending considerable text performing apologetics about it, I'd >rather the draft get straight to the point about the problems that this >draft solves. Understood. This document is not directed at such dissenters. It's targeted to those operators who have by due diligence decided that CGN plays a part of there network service for a period of time. >That is, assuming that the carrier has already decided they will use CGN, >they have the following problems. This draft solves them by ... >" This document shows how MPLS/VPNs as described in [RFC4364] can be > used to integrate the CGN infrastructure solving key problems faced > by the operator." >It'd be helpful if those "key problems" could be summarized in the intro >or second section, and then expounded upon in later sections along with >the MPLS VPN-based solution. I can look into doing that. > >I'm also not clear on what portions of section 3 have to do with the >proposed problem/solution space. Portions of it seems to be to be >generalized CGN deployment considerations. That may be fine, but that's >not technically what this document purports to discuss, and maybe should >be in another document, perhaps even the lsn-requirements document. I can >sort of see where L3VPN might help with some of these requirements, but >this section should focus more crisply on the requirements that are >difficult to solve without the use of the solutions proposed in this >document. Wes, I can revisit the assessed requirements vs. what is in the lsn-requiremetns document (which was not a WG draft when I first started). That said, some of the requirements were not about how an LSN should work, but more about how to make an LNS work in a real network that has running IPv4 legacy and may have IPv6 soon. I can rework the requirements to make sure I don't overlap with the lsn-requirement draft (draft-ietf-behave-lsn-requirements-05). I will keep mine closer to "network" related issues. > >You have an orphan/empty section 5 Sorry, fallout from previous version. I had included experiences in this draft until a better draft was published. I have since pulled this out. Reference - draft-donley-nat444-impacts-03. I can just point to that now (given our experiences where ported there). > >I'd also drop the "problems this doesn't solve" in section 7 unless you'd >like to turn it into a gap analysis with a mind towards asking IETF to >solve those problems with protocol changes. If it's purely informational, >you'll likely never be done documenting the problems with CGN that aren't >solved. This is the essence of a "we don't know what we don't know" >system, so it's misleading to discuss it as if it was a complete list. Seems fair. I can address that. > > >Thanks, > >Wes George I appreciate your review. I spin a new version once I get some additional feedback. Victor K > > > >> -----Original Message----- >> From: opsawg-bounces@ietf.org [mailto:opsawg-bounces@ietf.org] On >>Behalf Of >> Christopher LILJENSTOLPE >> Sent: Wednesday, March 28, 2012 4:53 AM >> To: opsawg@ietf.org >> Cc: opsawg-chairs >> Subject: [OPSAWG] adoption of Draft-kuarsingh-lsn-deployment as WG item >> >> Greetings, >> >> This is to start a poll in the working group to see if we want to >>accept >> Draft-kuarsingh-lsn-deployment as a working group item. We will leave >>this >> poll open until 2359 UTC on 4 April. Please speak early and often. >> >> Chris >> >> -- >> $BM{[IbO(B >> Check my PGP key here: https://www.asgaard.org/~cdl/cdl.asc >> Current vCard here: https://www.asgaard.org/~cdl/cdl.vcf >> Check my calendar availability: https://tungle.me/cdl >> >> _______________________________________________ >> OPSAWG mailing list >> OPSAWG@ietf.org >> https://www.ietf.org/mailman/listinfo/opsawg > >This E-mail and any of its attachments may contain Time Warner Cable >proprietary information, which is privileged, confidential, or subject to >copyright belonging to Time Warner Cable. This E-mail is intended solely >for the use of the individual or entity to which it is addressed. If you >are not the intended recipient of this E-mail, you are hereby notified >that any dissemination, distribution, copying, or action taken in >relation to the contents of and attachments to this E-mail is strictly >prohibited and may be unlawful. If you have received this E-mail in >error, please notify the sender immediately and permanently delete the >original and any copy of this E-mail and any printout. >_______________________________________________ >OPSAWG mailing list >OPSAWG@ietf.org >https://www.ietf.org/mailman/listinfo/opsawg From vumip1@gmail.com Tue Apr 3 19:38:11 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DEADC21F856C; Tue, 3 Apr 2012 19:38:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.825 X-Spam-Level: X-Spam-Status: No, score=-2.825 tagged_above=-999 required=5 tests=[AWL=0.773, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dBQWpDA-oaDb; Tue, 3 Apr 2012 19:38:11 -0700 (PDT) Received: from mail-ob0-f172.google.com (mail-ob0-f172.google.com [209.85.214.172]) by ietfa.amsl.com (Postfix) with ESMTP id 37C8F21F8568; Tue, 3 Apr 2012 19:38:11 -0700 (PDT) Received: by obbtb4 with SMTP id tb4so587840obb.31 for ; Tue, 03 Apr 2012 19:38:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=6uEBZ9heJkgKZQdg8XGDIyAo9fY484LtFO+LXwB22SQ=; b=Cu4yBE1GGi/5dcs1VLoLR22aNE1Qv2js3jXoSf9I07NQruwi+Z5VDOnV7xMNmtzIRj p3aSWAOnoCMGNcxLsfUiAZKLNvXuxhUPsnfD+IXO8LumZTY0Y1e5OXVSKvH5ENEWMWo8 fJt7R9LWMqjON1LB7iiqM3amHfO7ATkUH/pMXd4kFPlS/sqi5Wh+G8OSgyJVvFYtaMGf INnv4b+RNORrsXVPdXXnlMskUpl+uAxnynFbC/dRLiQj9N5+zpx+6wHtEdp1eOrccVtJ 87z4OAskACWmZXcjQ1XwzQKtnTrdfhJVPVlBuFjUT6KR5LX9UUE8xz1OW3m5lWlTWnM4 3WpA== MIME-Version: 1.0 Received: by 10.182.188.38 with SMTP id fx6mr22118610obc.77.1333507090880; Tue, 03 Apr 2012 19:38:10 -0700 (PDT) Received: by 10.182.171.104 with HTTP; Tue, 3 Apr 2012 19:38:10 -0700 (PDT) Date: Tue, 3 Apr 2012 22:38:10 -0400 Message-ID: From: Bhumip Khasnabish To: opsawg@ietf.org, apps-discuss@ietf.org, dc@ietf.org, cdni@ietf.org, scim@ietf.org, dcrg-interest@irtf.org Content-Type: multipart/alternative; boundary=f46d04478bdd7518e904bcd150de Subject: [OPSAWG] Fwd: slides from IETF83 Cloud Storage Talk by Giorgio X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Apr 2012 02:38:12 -0000 --f46d04478bdd7518e904bcd150de Content-Type: text/plain; charset=ISO-8859-1 The slides from IETF83 Cloud Storage Talk by Giorgio are now available at the following Website: http://trac.tools.ietf.org/area/app/trac/wiki/Clouds File names are as follows: IETF83-Cloud-Storage-Talk-Giorgio-29Mar2012-part1.pdf and IETF83-Cloud-Storage-Talk-Giorgio-29Mar2012-part2.pdf --f46d04478bdd7518e904bcd150de Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

The slides from IETF83 Cloud Storage Talk by Giorgio are now available a= t the following Website:
http://trac.tools.ietf.org/area/app/tra= c/wiki/Clouds

File names are as follows:
=A0
IETF83-Cloud-Storage-Talk-Giorgio-2= 9Mar2012-part1.pdf
and
IETF83-Cloud-Storage-Talk-Giorgio-29Mar2012-= part2.pdf
=A0
=A0
=A0

--f46d04478bdd7518e904bcd150de-- From ietf@cdl.asgaard.org Thu Apr 5 09:56:42 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3013421F8680 for ; Thu, 5 Apr 2012 09:56:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.11 X-Spam-Level: X-Spam-Status: No, score=-5.11 tagged_above=-999 required=5 tests=[BAYES_05=-1.11, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BXZcwIkqL0pM for ; Thu, 5 Apr 2012 09:56:41 -0700 (PDT) Received: from asgaard.org (odin.asgaard.org [204.29.151.68]) by ietfa.amsl.com (Postfix) with ESMTP id 9DD3C21F864E for ; Thu, 5 Apr 2012 09:56:41 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by asgaard.org (Postfix) with ESMTP id 078BEF73067 for ; Thu, 5 Apr 2012 16:56:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at asgaard.org Received: from asgaard.org ([127.0.0.1]) by localhost (odin.asgaard.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rV4L77pStZ8b for ; Thu, 5 Apr 2012 16:56:39 +0000 (UTC) Received: from [192.168.254.47] (74-93-4-130-sfba.hfc.comcastbusiness.net [74.93.4.130]) by asgaard.org (Postfix) with ESMTPSA id 9EA81F7305B for ; Thu, 5 Apr 2012 16:56:39 +0000 (UTC) From: Christopher LILJENSTOLPE Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Date: Thu, 5 Apr 2012 08:42:34 -0700 Message-Id: <43BD6172-7F7B-4412-A0D9-A85033CAC81E@cdl.asgaard.org> To: opsawg@ietf.org Mime-Version: 1.0 (Apple Message framework v1257) X-Mailer: Apple Mail (2.1257) Subject: [OPSAWG] automatic-network-config, and lsn-deployment WG Last call X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Apr 2012 16:56:42 -0000 Greetings, We've had at least one comment on lsn-deployment, but that seems = about it. The WG calls for them ended yesterday. At this point, I see = thundering silence. Can more of the working group please speak up, one = way or the other? I can't say one response is wg consensus. Remember, we are asking if we should adopt lsn-deployment as a wg item, = and if automatic-network-config is ready to be published? Thx Chris -- =20 =E6=9D=8E=E6=9F=AF=E7=9D=BF Check my PGP key here: https://www.asgaard.org/~cdl/cdl.asc Current vCard here: https://www.asgaard.org/~cdl/cdl.vcf Check my calendar availability: https://tungle.me/cdl From acmorton@att.com Mon Apr 9 07:11:22 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6D37921F8736; Mon, 9 Apr 2012 07:11:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -101.667 X-Spam-Level: X-Spam-Status: No, score=-101.667 tagged_above=-999 required=5 tests=[AWL=-1.629, BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_HTML_ONLY=1.457, MSGID_FROM_MTA_HEADER=0.803, SARE_WEOFFER=0.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KwB1GZIcmtVL; Mon, 9 Apr 2012 07:11:17 -0700 (PDT) Received: from nbfkord-smmo05.seg.att.com (nbfkord-smmo05.seg.att.com [209.65.160.92]) by ietfa.amsl.com (Postfix) with ESMTP id 405D821F871E; Mon, 9 Apr 2012 07:11:17 -0700 (PDT) Received: from unknown [144.160.20.145] (EHLO mlpd192.enaf.sfdc.sbc.com) by nbfkord-smmo05.seg.att.com(mxl_mta-6.11.0-8) over TLS secured channel with ESMTP id 40ee28f4.0.230930.00-497.614085.nbfkord-smmo05.seg.att.com (envelope-from ); Mon, 09 Apr 2012 14:11:17 +0000 (UTC) X-MXL-Hash: 4f82ee0566290b58-2cad6eb82c02b886279e967eac455db82a466348 Received: from enaf.sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlpd192.enaf.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id q39EBGuE023088; Mon, 9 Apr 2012 10:11:16 -0400 Received: from sflint02.pst.cso.att.com (sflint02.pst.cso.att.com [144.154.234.229]) by mlpd192.enaf.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id q39EB9Y8023068 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 9 Apr 2012 10:11:11 -0400 Received: from alpd052.aldc.att.com (alpd052.aldc.att.com [130.8.42.31]) by sflint02.pst.cso.att.com (RSA Interceptor); Mon, 9 Apr 2012 10:10:30 -0400 Received: from aldc.att.com (localhost.localdomain [127.0.0.1]) by alpd052.aldc.att.com (8.14.4/8.14.4) with ESMTP id q39EATtP020090; Mon, 9 Apr 2012 10:10:30 -0400 Received: from dns.maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by alpd052.aldc.att.com (8.14.4/8.14.4) with ESMTP id q39EARvq020008; Mon, 9 Apr 2012 10:10:27 -0400 Message-Id: <201204091410.q39EARvq020008@alpd052.aldc.att.com> Received: from acmt.att.com (martym.mt.att.com[135.16.251.71](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20120409140729gw1004or3ee>; Mon, 9 Apr 2012 14:07:29 +0000 X-Originating-IP: [135.16.251.71] X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 09 Apr 2012 10:11:31 -0400 To: opsawg-chairs@tools.ietf.org, opsawg@ietf.org, draft-tempia-opsawg-p3m@tools.ietf.org From: Al Morton Mime-Version: 1.0 Content-Type: text/html; charset="us-ascii" X-RSA-Inspected: yes X-RSA-Classifications: public X-RSA-Action: allow X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010122901)] X-MAIL-FROM: X-SOURCE-IP: [144.160.20.145] X-AnalysisOut: [v=1.0 c=1 a=dit6Jc5sqrAA:10 a=PGkD3MPGI9wA:10 a=ofMgfj31e3] X-AnalysisOut: [cA:10 a=BLceEmwcHowA:10 a=kj9zAlcOel0A:10 a=ZRNLZ4dFUbCvG8] X-AnalysisOut: [UMqPvVAA==:17 a=48vgC7mUAAAA:8 a=ULWbJayO_MKlLpVnT7EA:9 a=] X-AnalysisOut: [5sxecbfyaHv68ll3pjwA:7 a=CjuIK1q_8ugA:10 a=_W_S_7VecoQA:10] X-AnalysisOut: [ a=HlLFuPE95S0A:10] Cc: "pmol@ietf.org" , Vinayak Hegde Subject: [OPSAWG] Performance Metrics Directorate Review of draft-tempia-opsawg-p3m X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Apr 2012 14:11:22 -0000 OPSAWG,

Vinayak Hegde, a member of the Performance Metrics Directorate,
has reviewed the subject draft.

We offer this early review as part of our directorate mission.
http://www.ietf.org/iesg/directorate/performance-metrics.html

regards,
Al
directorate admin

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Comments on draft-tempia-opsawg-p3m
( http://datatracker.ietf.org/doc/draft-tempia-opsawg-p3m/)

These comments are for the -01 version on this draft.

1. Doesn't reference prior work in IPPM by RFC number (in the
References section). This is important as the approaches for passive
measurement seem to borrow heavily from the concepts of the IPPM
drafts (active measurements)

IPPM Delay http://tools.ietf.org/html/rfc2679
IPPM Loss http://tools.ietf.org/html/rfc2680

2. The draft does not take into out-of-order packet arrivals which is
possible in a real world scenario. Would the counters be updated or
how will they be switched off. Are there any conditions for timeouts.
If so, they are not explicitly mentioned.

3. Adding more implementation details of how colouring is done will be
done will help as it one of the main proposals of the draft. It has
not be covered in sufficient detail - only a few sentences mention
that certain DSCP bits be set but elsewhere the draft mentions that
several "colours" can be used in traffic flows.

4. In sections 6.3 monitoring nodes when there is periodic reading of
counters, it seems that the implicit upper bound on timeout of 5
minutes as counters are read every 5 minutes. Also there are time
synchnoisation issues when multiple intermediate elements are involved
in measurement.

5. The authors should elaborate on the data collection methods from
various routers since such measurements may be near real-time to
enable reactions from people monitoring the network. Scalability
concerns apply here when multiple monitoring nodes are needed.

From j.schoenwaelder@jacobs-university.de Tue Apr 10 03:30:25 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3B92C11E809D for ; Tue, 10 Apr 2012 03:30:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.096 X-Spam-Level: X-Spam-Status: No, score=-103.096 tagged_above=-999 required=5 tests=[AWL=0.153, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q44KtWKnfXJA for ; Tue, 10 Apr 2012 03:30:24 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 652B111E8087 for ; Tue, 10 Apr 2012 03:30:24 -0700 (PDT) Received: from localhost (demetrius2.jacobs-university.de [212.201.44.47]) by hermes.jacobs-university.de (Postfix) with ESMTP id B153520C11; Tue, 10 Apr 2012 12:30:23 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius2.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id 1puF68umFwe7; Tue, 10 Apr 2012 12:30:23 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 2C99720BF7; Tue, 10 Apr 2012 12:30:23 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 111811E5BC83; Tue, 10 Apr 2012 12:30:24 +0200 (CEST) Date: Tue, 10 Apr 2012 12:30:23 +0200 From: Juergen Schoenwaelder To: Christopher LILJENSTOLPE Message-ID: <20120410103023.GC24219@elstar.local> Mail-Followup-To: Christopher LILJENSTOLPE , opsawg@ietf.org References: <43BD6172-7F7B-4412-A0D9-A85033CAC81E@cdl.asgaard.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <43BD6172-7F7B-4412-A0D9-A85033CAC81E@cdl.asgaard.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] automatic-network-config, and lsn-deployment WG Last call X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2012 10:30:25 -0000 On Thu, Apr 05, 2012 at 08:42:34AM -0700, Christopher LILJENSTOLPE wrote: > > Remember, we are asking if we should adopt lsn-deployment as a wg item, and if automatic-network-config is ready to be published? > For the automatic-network-config document, this is the third WG last call and it seems Wes is fine with the latest edits. Do you want to have those who previously stated their support to restate it? If so, you may want to make this clear. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From mrm@vmware.com Tue Apr 10 22:40:35 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 40E1C21F86B7 for ; Tue, 10 Apr 2012 22:40:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.599 X-Spam-Level: X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rVERePjgsdwM for ; Tue, 10 Apr 2012 22:40:34 -0700 (PDT) Received: from smtp-outbound-1.vmware.com (smtp-outbound-1.vmware.com [208.91.2.12]) by ietfa.amsl.com (Postfix) with ESMTP id 505E621F86B2 for ; Tue, 10 Apr 2012 22:40:34 -0700 (PDT) Received: from sc9-mailhost1.vmware.com (sc9-mailhost1.vmware.com [10.113.161.71]) by smtp-outbound-1.vmware.com (Postfix) with ESMTP id 149D7282DB; Tue, 10 Apr 2012 22:40:34 -0700 (PDT) Received: from zimbra-prod-mta-3.vmware.com (zimbra-prod-mta-3.vmware.com [10.113.160.227]) by sc9-mailhost1.vmware.com (Postfix) with ESMTP id 124C0183AA; Tue, 10 Apr 2012 22:40:34 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by zimbra-prod-mta-3.vmware.com (Postfix) with ESMTP id 0ADEE12F9A6; Tue, 10 Apr 2012 22:40:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra-prod-mta-3.vmware.com Received: from zimbra-prod-mta-3.vmware.com ([127.0.0.1]) by localhost (zimbra-prod-mta-3.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Hjhv6j16FAqn; Tue, 10 Apr 2012 22:40:33 -0700 (PDT) Received: from zimbra-prod-mbox-2.vmware.com (zimbra-prod-mbox-2.vmware.com [10.113.160.202]) by zimbra-prod-mta-3.vmware.com (Postfix) with ESMTP id E8E6B12F9A3; Tue, 10 Apr 2012 22:40:33 -0700 (PDT) Date: Tue, 10 Apr 2012 22:40:33 -0700 (PDT) From: Michael MacFaden To: Juergen Schoenwaelder Message-ID: <1271859589.1843596.1334122833846.JavaMail.root@zimbra-prod-mbox-2.vmware.com> In-Reply-To: <146241634.1843589.1334122789855.JavaMail.root@zimbra-prod-mbox-2.vmware.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.113.60.13] X-Mailer: Zimbra 7.1.3_GA_3374 (ZimbraWebClient - GC18 (Mac)/7.1.3_GA_3346) Cc: opsawg@ietf.org Subject: [OPSAWG] Hypervisor MIB document X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Apr 2012 05:40:35 -0000 Hey Juergen, Am copying the opsawg list per your suggestion as we continue our conversation on the draft hypervisor mib module. >j.schoenwaelder@jacobs-university.de wrote: >>mrm@vmware.com wrote: >> 1) Which operators are we trying to service with this mib module? >> - enterprise >> - service providers (old school telcos or new school 'cloud') >> - all of the above >> - other >I guess it is all of the above. Do you think there is a major >difference from the MIB module perspective between enterprise clouds >and clouds provided as a service? Yes. I'll explain below. >And if there are differences, would >there not likely also be differences be clouds run by native cloud >providers and clouds operated by telcos as part of their portfolio? Don't personally know if ops models being used differ for native cloud providers vs telcos. Maybe someone on the list will chime in here. Today I work mostly with large corporate IT accounts and integrators that embed VMware ESX/vCenter directly into their products (ex: Cisco Integrated Service Router, Motorola Astro 25) and (in the past) with telcos and the nanog community. >I assume the difference (if any) is the integration into existing >management and monitoring infrastructures where SNMP likely plays a >more important role for telcos and enterprises than for specialized >cloud operators (but I am guessing here). What is your experience with >VMware MIBs? The VMWARE-VMNINFO-MIB[1] has been in deployed in product since 2001. In that time it has not changed all that much. Other than converting to SMIv2, here are three changes I can recall off hand: 1) A common problem among mgmt apps was unique vm identification. Apps would use the VMWARE-VMINFO-MIB::vmwVmTable table index vmwVmIdx . At point I added vmwVmUUID to allow tracking of virtual machine movement cross ESX systems as this could confuse mgmt apps like CA SPECTRUM. Notifications being sent still generate power off and power on and its up to mgmt apps to sync on UUID to know a VM moved. 2) Then when virtual SMP was added new managed objects were needed to track that, vmwVmCpus. A comment on the opsawg list pointed out that virtualized CPUs (VCPU) can be 32 or 64 but its even more than that. In order to move a live virtual machine to move from one host to another there is a whole raft of VCPU properties that must also be identical. 3) At some point counters for network i/o were made obsolete & not provided in ESXi (VMWARE-OBSOLETE-MIB:: vmwNetPktsTx, etc), yet these were highly popular. Modeling these as part of the VM were a mistake to begin with imo as there are standard objects that should have been populated in the virtual switch/bridge mib port that a given VM is attached to. So the right balance of what to show as "config" and what to show for "performance" will take some time to get right but then again that's what SNMP community has shown time again it is good at coming up with -- a data model done right (regardless of SPPI/YANG/SMIv2 syntax). Also would point out DMTF has existing standards for virtual machines described here: http://www.dmtf.org/sites/default/files/standards/documents/DSP2013_1.0.0.pdf Anyway I believe a main difference between an Enterprise and a Service provider is how they do tenancy and scale. Some background on tenancy: http://www.netapp.com/us/technology/secure-multi-tenancy.html http://www.cisco.com/en/US/solutions/ns340/ns414/ns742/ns743/ns1050/landing_dcVDDC.html Also for scale, here is what Igor Gashinsky/YAHOO posted on ARMD list Feb/2012: >To me, (and perhaps i'm in a very small minority here?), massive implies cloud scale datacenters, so, 10-20k physical hosts *per >cluster*, and somewhere around 400-500k VM's is the absolute *minimum* for what I would concider to be massive (and, really, I >think 100k physical, 2M+ logical is what i believe a realistic aiming point these days should be). Have never seen such large numbers in enterprise accounts. So might consider using timeFilter indexed tables and other techniques for scale. Mike MacFaden Staff Engineer, R&D Apps, VMware Palo Alto [1] http://communities.vmware.com/community/developer/forums/managementapi From ietf@cdl.asgaard.org Wed Apr 11 18:04:57 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1258A11E8112 for ; Wed, 11 Apr 2012 18:04:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.854 X-Spam-Level: X-Spam-Status: No, score=-5.854 tagged_above=-999 required=5 tests=[AWL=0.744, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E5bIE6UCLQsV for ; Wed, 11 Apr 2012 18:04:56 -0700 (PDT) Received: from asgaard.org (odin.asgaard.org [204.29.151.68]) by ietfa.amsl.com (Postfix) with ESMTP id 55C6C11E80FD for ; Wed, 11 Apr 2012 18:04:53 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by asgaard.org (Postfix) with ESMTP id 00287104055B; Thu, 12 Apr 2012 01:04:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at asgaard.org Received: from asgaard.org ([127.0.0.1]) by localhost (odin.asgaard.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Aoq3LBjBB+qu; Thu, 12 Apr 2012 01:04:51 +0000 (UTC) Received: from fenrir.bigswitch.com (74-93-4-129-sfba.hfc.comcastbusiness.net [74.93.4.129]) by asgaard.org (Postfix) with ESMTPSA id 82183104054E; Thu, 12 Apr 2012 01:04:51 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: text/plain; charset=utf-8 From: Christopher LILJENSTOLPE In-Reply-To: <20120410103023.GC24219@elstar.local> Date: Wed, 11 Apr 2012 18:04:50 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <43BD6172-7F7B-4412-A0D9-A85033CAC81E@cdl.asgaard.org> <20120410103023.GC24219@elstar.local> To: Juergen Schoenwaelder X-Mailer: Apple Mail (2.1257) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] automatic-network-config, and lsn-deployment WG Last call X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Apr 2012 01:04:57 -0000 Greetings, There was some noise after the IPR came out about the draft. = Now that the IPR issues have been clarified, I was hoping for some sense = of "I'm ok with it now" or "no, I'm not". As it stands, you are correct = wrt it being third call, and wes seems to be ok with the changes. Last = chance for folks to speak up or hold their peace. Chris On 10Apr2012, at 03.30, Juergen Schoenwaelder wrote: > On Thu, Apr 05, 2012 at 08:42:34AM -0700, Christopher LILJENSTOLPE = wrote: >>=20 >> Remember, we are asking if we should adopt lsn-deployment as a wg = item, and if automatic-network-config is ready to be published? >>=20 >=20 > For the automatic-network-config document, this is the third WG last > call and it seems Wes is fine with the latest edits. Do you want to > have those who previously stated their support to restate it? If so, > you may want to make this clear. >=20 > /js >=20 > --=20 > Juergen Schoenwaelder Jacobs University Bremen gGmbH > Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany > Fax: +49 421 200 3103 -- =20 =E6=9D=8E=E6=9F=AF=E7=9D=BF Check my PGP key here: https://www.asgaard.org/~cdl/cdl.asc Current vCard here: https://www.asgaard.org/~cdl/cdl.vcf Check my calendar availability: https://tungle.me/cdl From ietf-ipr@ietf.org Fri Apr 13 07:52:35 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 602D221F87D4; Fri, 13 Apr 2012 07:52:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.394 X-Spam-Level: X-Spam-Status: No, score=-102.394 tagged_above=-999 required=5 tests=[AWL=0.205, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ui6lJNWtkqVL; Fri, 13 Apr 2012 07:52:34 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D942021F85CF; Fri, 13 Apr 2012 07:52:34 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: IETF Secretariat To: tena@huawei.com, j.schoenwaelder@jacobs-university.de, shiyang1@huawei.com, tom111.taylor@bell.net, iamyanggl@gmail.com X-Test-IDTracker: no X-IETF-IDTracker: 4.00 Message-ID: <20120413145234.20723.93298.idtracker@ietfa.amsl.com> Date: Fri, 13 Apr 2012 07:52:34 -0700 Cc: opsawg@ietf.org, ipr-announce@ietf.org Subject: [OPSAWG] IPR Disclosure: Huawei Technologies Co., Ltd's Statement about IPR related to draft-ietf-opsawg-automated-network-configuration-03 X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Apr 2012 14:52:35 -0000 Dear Tina Tsou (Ting ZOU), Juergen Schoenwaelder, Yang Shi, Tom Taylor, Guo= Liang Yang: An IPR disclosure that pertains to your Internet-Draft entitled "Problem Statement for the Automated Configuration of Large IP Networks" (draft-ietf- opsawg-automated-network-configuration) was submitted to the IETF Secretari= at on 2012-03-27 and has been posted on the "IETF Page of Intellectual Property R= ights Disclosures" (https://datatracker.ietf.org/ipr/1735/). The title of the IPR disclosure is "Huawei Technologies Co.,Ltd's Statement about IPR related to draft-ietf-opsawg-automated-network-configuration-03.""); The IETF Secretariat From cdl@asgaard.org Wed Apr 18 15:57:24 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D4E4721F8470 for ; Wed, 18 Apr 2012 15:57:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pRIg2C2YvmYG for ; Wed, 18 Apr 2012 15:57:20 -0700 (PDT) Received: from asgaard.org (odin.asgaard.org [204.29.151.68]) by ietfa.amsl.com (Postfix) with ESMTP id CA0A021F846F for ; Wed, 18 Apr 2012 15:57:20 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by asgaard.org (Postfix) with ESMTP id 0BC9E11084F0; Wed, 18 Apr 2012 22:57:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at asgaard.org Received: from asgaard.org ([127.0.0.1]) by localhost (odin.asgaard.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P9XP4TiA3ipq; Wed, 18 Apr 2012 22:57:15 +0000 (UTC) Received: from [10.10.238.223] (63-145-238-4.dia.static.qwest.net [63.145.238.4]) by asgaard.org (Postfix) with ESMTPSA id 6906511084E4; Wed, 18 Apr 2012 22:57:13 +0000 (UTC) From: Christopher LILJENSTOLPE Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Date: Wed, 18 Apr 2012 15:57:12 -0700 Message-Id: <7363771F-B336-4A8C-9800-E0EDAF3FA77B@asgaard.org> To: opsawg@ietf.org Mime-Version: 1.0 (Apple Message framework v1257) X-Mailer: Apple Mail (2.1257) Cc: draft-ietf-opsawg-automated-network-configuration@tools.ietf.org, draft-kuarsingh-lsn-deployment@tools.ietf.org, draft-baker-opsawg-firewalls@tools.ietf.org Subject: [OPSAWG] wg calls X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Apr 2012 22:57:24 -0000 Greetings, As per the results of the calls, we are sending = draft-ietf-automated-network-configuration to the IESG requesting = publication. We believe that we have consensus on adopting both = draft-kuarsingh-lsn-deployment and draft-baker-opsawg-firewalls as = working group items. We will do so tomorrow unless we hear dissent. = Please speak up now. Chris -- =20 =E6=9D=8E=E6=9F=AF=E7=9D=BF Check my PGP key here: https://www.asgaard.org/~cdl/cdl.asc Current vCard here: https://www.asgaard.org/~cdl/cdl.vcf Check my calendar availability: https://tungle.me/cdl From phil@juniper.net Wed Apr 18 22:11:35 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CC11C21F84E2 for ; Wed, 18 Apr 2012 22:11:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qM4QrgwmJXhM for ; Wed, 18 Apr 2012 22:11:34 -0700 (PDT) Received: from exprod7og107.obsmtp.com (exprod7og107.obsmtp.com [64.18.2.167]) by ietfa.amsl.com (Postfix) with ESMTP id 3316D21F8499 for ; Wed, 18 Apr 2012 22:11:34 -0700 (PDT) Received: from P-EMHUB01-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob107.postini.com ([64.18.6.12]) with SMTP ID DSNKT4+efsw0fg9KNm368K0oopyFKNevNEti@postini.com; Wed, 18 Apr 2012 22:11:34 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB01-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Wed, 18 Apr 2012 22:10:31 -0700 Received: from idle.juniper.net (idleski.juniper.net [172.25.4.26]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q3J5AU125682; Wed, 18 Apr 2012 22:10:30 -0700 (PDT) (envelope-from phil@juniper.net) Received: from idle.juniper.net (localhost [127.0.0.1]) by idle.juniper.net (8.14.4/8.14.3) with ESMTP id q3J5BYUL028931; Thu, 19 Apr 2012 01:11:34 -0400 (EDT) (envelope-from phil@idle.juniper.net) Message-ID: <201204190511.q3J5BYUL028931@idle.juniper.net> To: Christopher LILJENSTOLPE In-Reply-To: <7363771F-B336-4A8C-9800-E0EDAF3FA77B@asgaard.org> Date: Thu, 19 Apr 2012 01:11:34 -0400 From: Phil Shafer MIME-Version: 1.0 Content-Type: text/plain Cc: draft-baker-opsawg-firewalls@tools.ietf.org, opsawg@ietf.org, draft-ietf-opsawg-automated-network-configuration@tools.ietf.org, draft-kuarsingh-lsn-deployment@tools.ietf.org Subject: Re: [OPSAWG] wg calls X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Apr 2012 05:11:35 -0000 Christopher LILJENSTOLPE writes: > As per the results of the calls, we are sending draft-ietf-automated-network-con >figuration to the IESG requesting publication. Were the IPR issues ever sorted out and/or the details announced? Thanks, Phil From j.schoenwaelder@jacobs-university.de Thu Apr 19 00:00:42 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9B08021F84D0 for ; Thu, 19 Apr 2012 00:00:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.158 X-Spam-Level: X-Spam-Status: No, score=-103.158 tagged_above=-999 required=5 tests=[AWL=0.091, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p917M-w+rBGN for ; Thu, 19 Apr 2012 00:00:38 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 7D12021F84C3 for ; Thu, 19 Apr 2012 00:00:38 -0700 (PDT) Received: from localhost (demetrius1.jacobs-university.de [212.201.44.46]) by hermes.jacobs-university.de (Postfix) with ESMTP id 9BA6320C55; Thu, 19 Apr 2012 09:00:37 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius1.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id PpVGX9wqttoa; Thu, 19 Apr 2012 09:00:37 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 3895220C4E; Thu, 19 Apr 2012 09:00:37 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 3E0851E6B9FF; Thu, 19 Apr 2012 09:00:38 +0200 (CEST) Date: Thu, 19 Apr 2012 09:00:37 +0200 From: Juergen Schoenwaelder To: Phil Shafer Message-ID: <20120419070037.GB61326@elstar.local> Mail-Followup-To: Phil Shafer , Christopher LILJENSTOLPE , opsawg@ietf.org References: <7363771F-B336-4A8C-9800-E0EDAF3FA77B@asgaard.org> <201204190511.q3J5BYUL028931@idle.juniper.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201204190511.q3J5BYUL028931@idle.juniper.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: opsawg@ietf.org, Christopher LILJENSTOLPE Subject: Re: [OPSAWG] wg calls X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Apr 2012 07:00:42 -0000 On Thu, Apr 19, 2012 at 01:11:34AM -0400, Phil Shafer wrote: > Christopher LILJENSTOLPE writes: > > As per the results of the calls, we are sending draft-ietf-automated-network-con > >figuration to the IESG requesting publication. > > Were the IPR issues ever sorted out and/or the details announced? Phil, please check these messages that went over the WG mailing list: http://www.ietf.org/mail-archive/web/opsawg/current/msg02184.html http://www.ietf.org/mail-archive/web/opsawg/current/msg02175.html /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From tom.taylor.stds@gmail.com Thu Apr 19 06:35:52 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3C5FF21F863D for ; Thu, 19 Apr 2012 06:35:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.383 X-Spam-Level: X-Spam-Status: No, score=-3.383 tagged_above=-999 required=5 tests=[AWL=0.216, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ty6anTvf8b0d for ; Thu, 19 Apr 2012 06:35:48 -0700 (PDT) Received: from mail-yw0-f44.google.com (mail-yw0-f44.google.com [209.85.213.44]) by ietfa.amsl.com (Postfix) with ESMTP id 2737C21F8636 for ; Thu, 19 Apr 2012 06:35:48 -0700 (PDT) Received: by yhkk25 with SMTP id k25so5023071yhk.31 for ; Thu, 19 Apr 2012 06:35:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding :x-antivirus:x-antivirus-status; bh=LVQhfT3LmhvOIax9W/UsuMwfDeE4BoDXnNoV9poA4sg=; b=PuvitzWJ4F92mfUrhEYyl1ab/g3QROffHNif7D78HavEVItRSo8QKV0VQYCOkDRKHI q6OaD7TpNPbkJulaEcSnHg6SU7PVqYx5NVIhxIlJWw1x8YK3of5yuV+BRjgvwIinjAgl aWGJCdy2qEns0JLx+XuvHJy4dS7eRQP6N4EuwhF9iYzGC+0FREMYQsAEun0vtAwJ7j1i ev2i+ToWvMR/xPbOgTO5ih7CpDte2DgVNfd7yidNmNTs5bqXcYYJR5pzIbqDQ0g7usPZ FCMAlx4E/OI0mNtGg/Wh6k+UN2BJ8ukDlDrsR3h++gWzlSJJtegdA5+aoG/6qIyK2lrM uWhQ== Received: by 10.60.18.137 with SMTP id w9mr3055810oed.7.1334842547710; Thu, 19 Apr 2012 06:35:47 -0700 (PDT) Received: from [127.0.0.1] (dsl-207-112-91-137.tor.primus.ca. [207.112.91.137]) by mx.google.com with ESMTPS id w4sm2116181oeg.12.2012.04.19.06.35.45 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 19 Apr 2012 06:35:46 -0700 (PDT) Message-ID: <4F9014B0.8020006@gmail.com> Date: Thu, 19 Apr 2012 09:35:44 -0400 From: Tom Taylor User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0) Gecko/20120327 Thunderbird/11.0.1 MIME-Version: 1.0 To: Phil Shafer References: <201204190511.q3J5BYUL028931@idle.juniper.net> In-Reply-To: <201204190511.q3J5BYUL028931@idle.juniper.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Antivirus: avast! (VPS 120419-0, 19/04/2012), Outbound message X-Antivirus-Status: Clean Cc: opsawg@ietf.org, draft-ietf-opsawg-automated-network-configuration@tools.ietf.org, draft-kuarsingh-lsn-deployment@tools.ietf.org Subject: Re: [OPSAWG] wg calls X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Apr 2012 13:35:52 -0000 See Scott Bradner's message to the list, 31/03/2012. On 19/04/2012 1:11 AM, Phil Shafer wrote: > Christopher LILJENSTOLPE writes: >> As per the results of the calls, we are sending draft-ietf-automated-network-con >> figuration to the IESG requesting publication. > > Were the IPR issues ever sorted out and/or the details announced? > > Thanks, > Phil > _______________________________________________ > OPSAWG mailing list > OPSAWG@ietf.org > https://www.ietf.org/mailman/listinfo/opsawg > . > From phil@juniper.net Thu Apr 19 10:43:00 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B3F2321F86B9 for ; Thu, 19 Apr 2012 10:43:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q2-eSCEnjVGU for ; Thu, 19 Apr 2012 10:43:00 -0700 (PDT) Received: from exprod7og124.obsmtp.com (exprod7og124.obsmtp.com [64.18.2.26]) by ietfa.amsl.com (Postfix) with ESMTP id C64C021F86B6 for ; Thu, 19 Apr 2012 10:42:53 -0700 (PDT) Received: from P-EMHUB01-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob124.postini.com ([64.18.6.12]) with SMTP ID DSNKT5BOneE2MwFksI12S5jhluXMz4RAkUbp@postini.com; Thu, 19 Apr 2012 10:42:59 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB01-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Thu, 19 Apr 2012 10:42:48 -0700 Received: from idle.juniper.net (idleski.juniper.net [172.25.4.26]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q3JHgl151717; Thu, 19 Apr 2012 10:42:48 -0700 (PDT) (envelope-from phil@juniper.net) Received: from idle.juniper.net (localhost [127.0.0.1]) by idle.juniper.net (8.14.4/8.14.3) with ESMTP id q3JHhq2g036079; Thu, 19 Apr 2012 13:43:53 -0400 (EDT) (envelope-from phil@idle.juniper.net) Message-ID: <201204191743.q3JHhq2g036079@idle.juniper.net> To: Juergen Schoenwaelder In-Reply-To: <20120419070037.GB61326@elstar.local> Date: Thu, 19 Apr 2012 13:43:52 -0400 From: Phil Shafer MIME-Version: 1.0 Content-Type: text/plain Cc: opsawg@ietf.org, Christopher LILJENSTOLPE Subject: Re: [OPSAWG] wg calls X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Apr 2012 17:43:00 -0000 Juergen Schoenwaelder writes: >http://www.ietf.org/mail-archive/web/opsawg/current/msg02184.html >http://www.ietf.org/mail-archive/web/opsawg/current/msg02175.html Was the root of the IPR disclosed? Thanks, Phil From ietf@cdl.asgaard.org Fri Apr 20 13:43:25 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AEB8821F85A4 for ; Fri, 20 Apr 2012 13:43:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i7FezA5x3H+q for ; Fri, 20 Apr 2012 13:43:24 -0700 (PDT) Received: from asgaard.org (odin.asgaard.org [204.29.151.68]) by ietfa.amsl.com (Postfix) with ESMTP id B679F21F85A0 for ; Fri, 20 Apr 2012 13:43:24 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by asgaard.org (Postfix) with ESMTP id 04966111AF6F for ; Fri, 20 Apr 2012 20:43:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at asgaard.org Received: from asgaard.org ([127.0.0.1]) by localhost (odin.asgaard.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xrBFRbm56SBh for ; Fri, 20 Apr 2012 20:43:22 +0000 (UTC) Received: from fenrir.asgaard.org (50-76-34-185-ip-static.hfc.comcastbusiness.net [50.76.34.185]) by asgaard.org (Postfix) with ESMTPSA id B6144111AF64 for ; Fri, 20 Apr 2012 20:43:22 +0000 (UTC) From: Christopher LILJENSTOLPE Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Date: Fri, 20 Apr 2012 13:43:21 -0700 References: <13205C286662DE4387D9AF3AC30EF456D76A76104A@EMBX01-WF.jnpr.net> To: opsawg@ietf.org Message-Id: <32E93727-1B28-4853-9232-2547F00CABBF@cdl.asgaard.org> Mime-Version: 1.0 (Apple Message framework v1257) X-Mailer: Apple Mail (2.1257) Subject: [OPSAWG] Fwd: OPS Area Office Hours X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Apr 2012 20:43:25 -0000 Begin forwarded message: > From: Ronald Bonica > Subject: OPS Area Office Hours > Date: 20 April 2012 09.37.42 -0700 > To: "ops-area@ietf.org" , "ops-chairs@ietf.org" = >=20 > Folks, >=20 > Benoit and I would like to make ourselves available for office hours = every other week. A schedule of office hours is posted at the following = URL: >=20 > - https://svn.tools.ietf.org/area/ops/trac/wiki/OfficeHours >=20 > Feel free to share this information with your WGs if you wish. >=20 > -------------------------- > Ron Bonica > vcard: www.bonica.org/ron/ronbonica.vcf >=20 -- =20 =E6=9D=8E=E6=9F=AF=E7=9D=BF Check my PGP key here: https://www.asgaard.org/~cdl/cdl.asc Current vCard here: https://www.asgaard.org/~cdl/cdl.vcf Check my calendar availability: https://tungle.me/cdl From j.schoenwaelder@jacobs-university.de Mon Apr 23 04:41:21 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D174321F869D for ; Mon, 23 Apr 2012 04:41:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -101.875 X-Spam-Level: X-Spam-Status: No, score=-101.875 tagged_above=-999 required=5 tests=[AWL=-1.226, BAYES_50=0.001, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id a64fDowafICb for ; Mon, 23 Apr 2012 04:41:20 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 37C2F21F8692 for ; Mon, 23 Apr 2012 04:41:16 -0700 (PDT) Received: from localhost (demetrius1.jacobs-university.de [212.201.44.46]) by hermes.jacobs-university.de (Postfix) with ESMTP id C95B720DD8; Mon, 23 Apr 2012 13:41:15 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius1.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id Z07kH2DZvw_K; Mon, 23 Apr 2012 13:41:15 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 30C4920E08; Mon, 23 Apr 2012 13:41:15 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id DE9CC1E9B153; Mon, 23 Apr 2012 13:41:16 +0200 (CEST) Date: Mon, 23 Apr 2012 13:41:16 +0200 From: Juergen Schoenwaelder To: Michael MacFaden Message-ID: <20120423114116.GA75467@elstar.local> Mail-Followup-To: Michael MacFaden , opsawg@ietf.org References: <146241634.1843589.1334122789855.JavaMail.root@zimbra-prod-mbox-2.vmware.com> <1271859589.1843596.1334122833846.JavaMail.root@zimbra-prod-mbox-2.vmware.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1271859589.1843596.1334122833846.JavaMail.root@zimbra-prod-mbox-2.vmware.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Hypervisor MIB document X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2012 11:41:22 -0000 On Tue, Apr 10, 2012 at 10:40:33PM -0700, Michael MacFaden wrote: > The VMWARE-VMNINFO-MIB[1] has been in deployed in product since 2001. > In that time it has not changed all that much. Other than converting > to SMIv2, here are three changes I can recall off hand: > > 1) A common problem among mgmt apps was unique vm identification. > > Apps would use the VMWARE-VMINFO-MIB::vmwVmTable table index vmwVmIdx . > At point I added vmwVmUUID to allow tracking of virtual machine movement cross > ESX systems as this could confuse mgmt apps like CA SPECTRUM. > Notifications being sent still generate power off and power on > and its up to mgmt apps to sync on UUID to know a VM moved. Good. The VM-MIB has a UUID for each guest so we are save. > 2) Then when virtual SMP was added new managed objects were needed to > track that, vmwVmCpus. A comment on the opsawg list pointed out that > virtualized CPUs (VCPU) can be 32 or 64 but its even more than that. > In order to move a live virtual machine to move from one host to another there > is a whole raft of VCPU properties that must also be identical. There was already a comment that we need to identify the CPU type especially since some platforms allow to emulate CPUs. So the question is how we model this properly. I should probably start a separate thread on this. Can you provide more information what the other "whole raft of VCPU properties" entails? > Also for scale, here is what Igor Gashinsky/YAHOO posted on ARMD > list Feb/2012: >To me, (and perhaps i'm in a very small minority > here?), massive implies cloud scale datacenters, so, 10-20k physical > hosts *per >cluster*, and somewhere around 400-500k VM's is the > absolute *minimum* for what I would concider to be massive (and, > really, I >think 100k physical, 2M+ logical is what i believe a > realistic aiming point these days should be). > > Have never seen such large numbers in enterprise accounts. So might > consider using timeFilter indexed tables and other techniques for > scale. Since the VM-MIB is implemented on physical hosts, what matters is really the ration of guests (VMs) per physical host. With the numbers above, 2M+ logical and 100k physical, we are still at an average of 20 guests per physical host. Assuming a long tailed distribution, we might consider 100 guests per physical machine, if its a cluster perhaps even more but nothing really unmanageable. The big number mentioned above is the huge number of physical machines. Whether timeFilter indexed tables are needed I do not know. Perhaps just having a few lastChanged objects is sufficient to allow for simple polls to check whether new guests popped up or guests changed their status? /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From rstory@tislabs.com Mon Apr 23 06:02:36 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5542F21F85DB for ; Mon, 23 Apr 2012 06:02:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qZI7QUl-Dnlb for ; Mon, 23 Apr 2012 06:02:33 -0700 (PDT) Received: from walnut.tislabs.com (walnut.tislabs.com [192.94.214.200]) by ietfa.amsl.com (Postfix) with ESMTP id BAD6D21F85CD for ; Mon, 23 Apr 2012 06:02:25 -0700 (PDT) Received: from nova.tislabs.com (unknown [10.66.1.77]) by walnut.tislabs.com (Postfix) with ESMTP id 4CE9B28B003A; Mon, 23 Apr 2012 09:02:25 -0400 (EDT) Received: from tp.vb.futz.org (localhost.localdomain [127.0.0.1]) by nova.tislabs.com (Postfix) with ESMTP id D7D081F8032; Mon, 23 Apr 2012 09:02:24 -0400 (EDT) Date: Mon, 23 Apr 2012 09:02:19 -0400 From: Robert Story To: Juergen Schoenwaelder Message-ID: <20120423090219.37857aca@tp.vb.futz.org> In-Reply-To: <20120423114116.GA75467@elstar.local> References: <146241634.1843589.1334122789855.JavaMail.root@zimbra-prod-mbox-2.vmware.com> <1271859589.1843596.1334122833846.JavaMail.root@zimbra-prod-mbox-2.vmware.com> <20120423114116.GA75467@elstar.local> Organization: SPARTA X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.8; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/l+.8V3Btz6NYR/6JA0tRL7s"; protocol="application/pgp-signature" Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Hypervisor MIB document X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2012 13:02:36 -0000 --Sig_/l+.8V3Btz6NYR/6JA0tRL7s Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Mon, 23 Apr 2012 13:41:16 +0200 Juergen wrote: JS> There was already a comment that we need to identify the CPU type JS> especially since some platforms allow to emulate CPUs. So the question JS> is how we model this properly. I should probably start a separate JS> thread on this. Can you provide more information what the other "whole JS> raft of VCPU properties" entails? KVM virtual machines keep track of features (e.g. sse2, apic) supported by the virtual machine's CPU. When you migrate a VM to another host, that host must support the same feature set. Here's are two pages about this: http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualiz= ation_Getting_Started_Guide/para-CPU_Models.html http://berrange.com/posts/2010/02/15/guest-cpu-model-configuration-in-libvi= rt-with-qemukvm/ Robert -- Senior Software Engineer SPARTA, Inc., a Parsons Company --Sig_/l+.8V3Btz6NYR/6JA0tRL7s Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) iEYEARECAAYFAk+VUuAACgkQ7/fVLLY1mnjU+ACbBzOXH1r4F32M/f8wwVTcJ8zU Wp4An0YUNRA5bGKcbAv085E0Qb5+vfzX =VHUC -----END PGP SIGNATURE----- --Sig_/l+.8V3Btz6NYR/6JA0tRL7s-- From j.schoenwaelder@jacobs-university.de Tue Apr 24 03:29:28 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 79B3E21F85D4 for ; Tue, 24 Apr 2012 03:29:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.139 X-Spam-Level: X-Spam-Status: No, score=-103.139 tagged_above=-999 required=5 tests=[AWL=0.110, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZlOvrgvcqvTS for ; Tue, 24 Apr 2012 03:29:27 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 41A5E21F853C for ; Tue, 24 Apr 2012 03:29:27 -0700 (PDT) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id D1BD020DD5; Tue, 24 Apr 2012 12:29:25 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id pzXtJcrkqjWW; Tue, 24 Apr 2012 12:29:25 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 2239620C6A; Tue, 24 Apr 2012 12:29:25 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 740471EA2A3B; Tue, 24 Apr 2012 12:29:25 +0200 (CEST) Date: Tue, 24 Apr 2012 12:29:25 +0200 From: Juergen Schoenwaelder To: Robert Story Message-ID: <20120424102924.GA81808@elstar.local> Mail-Followup-To: Robert Story , Melinda Shore , "opsawg@ietf.org" References: <4F707305.7040701@gmail.com> <6665BC1FEA04AB47B1F75FA641C43BC0A0B8C76C@FHDP1LUMXC7V41.us.one.verizon.com> <4F7074A8.2050105@gmail.com> <6665BC1FEA04AB47B1F75FA641C43BC0A0B8C782@FHDP1LUMXC7V41.us.one.verizon.com> <2FAFA474-1F24-40A6-9D54-9CC0A1EC5252@hongo.wide.ad.jp> <4F707C81.8030501@gmail.com> <20120326125113.69de8a87@tp.vb.futz.org> <20120328080723.GB38721@elstar.local> <20120328085040.1534d299@tp.vb.futz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120328085040.1534d299@tp.vb.futz.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "opsawg@ietf.org" Subject: Re: [OPSAWG] Hypervisor MIB document X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 10:29:28 -0000 Robert, I am not sure what to do here. More inline... On Wed, Mar 28, 2012 at 08:50:40AM -0400, Robert Story wrote: > JS> > Most hypervisors have multiple network configurations available. An > JS> > interface may be bridged to a hosy interface, use NAT on a host > JS> > interface, or be a private network (which may or may not be shared > JS> > with other VMs). I think there should be a vmNetTable. The vmIfTable > JS> > should have a column indication which network, if any, it is attached > JS> > to. > JS> > JS> Can you elaborate? What would be in the vmNetTable? > > vmNetIndex - index of network > vmNetName - name > vmNetDesc (optional) > vmNetPhysIfIndex - ifIndex of physical interface (or 0 if none) > vmNetifType - vmNetNAT(0), vmNetPublicBridge(1), vmNetPrivateBridge(2) I am not sure what the difference between public bridge and private bridge is. My understanding is that we have at least the following options: a) The hypervisor emulates a bridge and attaches VMs to the bridge. This is a pure layer two solution. b) The hypervisor emulates a router and attaches VMs by routing traffic to them. This is a layer three solution. There are two sub-options: b1) The router has an assigned dedicates IP address space for this purpose. b2) The router uses a NAT function to provide (NAT limited) connectivity. Both, b1 and b2 might be combined with a bridge in the hypervisor that provides direct connectivity between the VMs. This is what I get out of the documentation I have seen (although not fully clear since terminology tends to differ). Anyway, is this a suitable view of the world? Is there stuff missing? > Usually these virtual network offer DHCP addresses to guests, so it might > be useful to have that configuration information available too, either in > additional columns or in another table. > > The vmIfTable would then look something like > > vmIfIndex - index of virtual interface; no relation to ifIndex > vmGuestIndex - index of guest to which interface is assigned > vmNetIndex - index of network to which interface is assigned > vmIfState - vmIfConnected(0), vmIfDisconnected(0) > vmIfPhysAddr > > I don't think that guest or network index should be part of the index for > this table, as either one can be changed at any time. I am confused about a virtual interface that has no relation to an ifIndex identifiable interface. How is that going to work? How does the hypervisor forward traffic (independent of whether forwarding is done via layer two or layer three) to the VM if there is no interface? I am also unsure what the definition of vmIfConnected and vmIfDisconnected would be? What makes an interface connected in the above scenarios a), b1), and b2)? > It may be useful to have a few convenience tables: > > * vmGuestIfTable { vmGuestIndex, vmGuestIfIndex } vmNetIndex > * vmNetIfTable { vmNetIndex, vmIfIndex} vmGuestIndex I guess I am still not totally clear what the vmNetTable really represents. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From j.schoenwaelder@jacobs-university.de Tue Apr 24 03:35:06 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7E4CE21F8634 for ; Tue, 24 Apr 2012 03:35:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.142 X-Spam-Level: X-Spam-Status: No, score=-103.142 tagged_above=-999 required=5 tests=[AWL=0.107, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zL+JlGgHih0A for ; Tue, 24 Apr 2012 03:35:05 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id BA3E121F863B for ; Tue, 24 Apr 2012 03:35:05 -0700 (PDT) Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id 183CF20CCC; Tue, 24 Apr 2012 12:35:05 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id uwLrSTY__Q2D; Tue, 24 Apr 2012 12:35:04 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id A20D620DC4; Tue, 24 Apr 2012 12:35:04 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 7F60B1EA2AAE; Tue, 24 Apr 2012 12:35:06 +0200 (CEST) Date: Tue, 24 Apr 2012 12:35:06 +0200 From: Juergen Schoenwaelder To: opsawg@ietf.org Message-ID: <20120424103506.GA81983@elstar.local> Mail-Followup-To: opsawg@ietf.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Subject: [OPSAWG] VM-MIB issue vm-mib-01: storage sizes X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 10:35:06 -0000 Hi, * vm-mib-01: storage sizes The MIB does not provide storage sizes, assuming this is provided by the hrStorageTable of the HOST-RESOURCES-MIB. However, some well known implementations of the HOST-RESOURCES-MIB only report about file systems used by the host system and not file systems residing in files used by virtual machines. Furthermore, the hrStorageTable reports sizes "usable by the requesting entity", "excluding loss due to formatting of file system reference information". For storage provided to virtual machines, this information is often not readily available since all you have is the raw block size. ** Solution #01-01 Provide the storage block sizes as part of the VM-MIB. Provide a pointer to the hrStorageTable on systems that can provide this linkage but allow the pointer to be NULL. ** Resolution TBD /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From j.schoenwaelder@jacobs-university.de Tue Apr 24 03:35:34 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7B56A21F863B for ; Tue, 24 Apr 2012 03:35:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.145 X-Spam-Level: X-Spam-Status: No, score=-103.145 tagged_above=-999 required=5 tests=[AWL=0.104, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id o4Y-qnO+s9aX for ; Tue, 24 Apr 2012 03:35:33 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 80F1521F8634 for ; Tue, 24 Apr 2012 03:35:33 -0700 (PDT) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id C6E6C20E04; Tue, 24 Apr 2012 12:35:32 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id sdPP9rdSTtzJ; Tue, 24 Apr 2012 12:35:32 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 5AE1F20DF9; Tue, 24 Apr 2012 12:35:32 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 3E5F01EA2ACE; Tue, 24 Apr 2012 12:35:34 +0200 (CEST) Date: Tue, 24 Apr 2012 12:35:34 +0200 From: Juergen Schoenwaelder To: opsawg@ietf.org Message-ID: <20120424103534.GB81983@elstar.local> Mail-Followup-To: opsawg@ietf.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Subject: [OPSAWG] VM-MIB issue vm-mib-02: scaling and caching support X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 10:35:34 -0000 Hi, * vm-mib-02: scaling and caching support It was mentioned that large data centers are characterized by 100.000 physical hosts running 2.000.000 virtual machines. The NASA is reported with 1.000.000 physical hosts and 60.000.000 virtual machines. Bottom line is that we need to make the MIB module scalable. We can assume up hundreds of VMs running on a single virtual machine. ** Solution #02-01 Add ...LastChange objects to tables so that management applications can easily validate cached information without having to read through potentially larger tables. For the vmGuestTable, we might also provide a ...LastStateChange object so that state changes can be polled with reading a simple scalar. ** Solution #02-02 Make some tables time filtered. Unclear which tables would have to be time filtered. ** Resolution TBD /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From j.schoenwaelder@jacobs-university.de Tue Apr 24 03:36:15 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 951A721F8653 for ; Tue, 24 Apr 2012 03:36:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.148 X-Spam-Level: X-Spam-Status: No, score=-103.148 tagged_above=-999 required=5 tests=[AWL=0.101, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 46KUvBvnA6xp for ; Tue, 24 Apr 2012 03:36:15 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 006AF21F8643 for ; Tue, 24 Apr 2012 03:36:14 -0700 (PDT) Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id 68DD020E1D; Tue, 24 Apr 2012 12:36:05 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id XhyDJS5z2kqc; Tue, 24 Apr 2012 12:36:05 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 06BC820CC4; Tue, 24 Apr 2012 12:36:05 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id D95881EA2B03; Tue, 24 Apr 2012 12:36:06 +0200 (CEST) Date: Tue, 24 Apr 2012 12:36:06 +0200 From: Juergen Schoenwaelder To: opsawg@ietf.org Message-ID: <20120424103606.GC81983@elstar.local> Mail-Followup-To: opsawg@ietf.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Subject: [OPSAWG] VM-MIB issue vm-mib-03: cpu type identification X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 10:36:15 -0000 Hi, * vm-mib-03: cpu type identification It is necessary to identify the CPU architecture or type since some virtual machine systems can emulate different CPU types. ** Solution #03-01 Provide an IANA controlled enumeration that provides a CPU classification. The problem will be to provide rules about what constitutes a new CPU type and what not. ** Solution #03-02 Use OBJECT IDENTITIES to identify CPU types. Such a distributed enumeration will not achieve a great deal of interoperability and is likely close to #03-03. ** Solution #03-03 Use a string data type and rely on systems to put meaningful information there, perhaps provide guidelines how to structure the CPU type names, e.g. --*(-) that is amd-x86_64-opteron or intel-i686-pentium3-vmx-acpi (perhaps using a different separator character since a dash might easily clash). Applications may have to do some normalization across VM-MIB implementations (e.g., regular expression matching) but on the other hand this allows to provide details where necessary. ** Solution #03-04 Following #03-03, we provide ...GuestCpuVendor, ...GuestCpuArch and ...GuestCpuModel objects plus an additional table that provides details about the features of the CPUs used by a certain virtual machine. This essentially breaks the string into a set of separate MIB objects. ** Solution #03-05 Following #03-04, we provide ...GuestCpuVendor, ...GuestCpuArch and ...GuestCpuModel objects plus a string object containing a list of features. This way, things are more compact but still the most important components (vendor, arch, model) are broken out as separate objects. ** Resolution TBD /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From rstory@tislabs.com Tue Apr 24 07:34:09 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 43F5921F885C for ; Tue, 24 Apr 2012 07:34:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZrOvCukVZqBH for ; Tue, 24 Apr 2012 07:34:08 -0700 (PDT) Received: from walnut.tislabs.com (walnut.tislabs.com [192.94.214.200]) by ietfa.amsl.com (Postfix) with ESMTP id BA42F21F8859 for ; Tue, 24 Apr 2012 07:34:05 -0700 (PDT) Received: from nova.tislabs.com (unknown [10.66.1.77]) by walnut.tislabs.com (Postfix) with ESMTP id 2CB1A28B003C; Tue, 24 Apr 2012 10:34:05 -0400 (EDT) Received: from tp.vb.futz.org (localhost.localdomain [127.0.0.1]) by nova.tislabs.com (Postfix) with ESMTP id EC6D71F8032; Tue, 24 Apr 2012 10:34:04 -0400 (EDT) Date: Tue, 24 Apr 2012 10:34:00 -0400 From: Robert Story To: Juergen Schoenwaelder Message-ID: <20120424103400.73a4b631@tp.vb.futz.org> In-Reply-To: <20120424103534.GB81983@elstar.local> References: <20120424103534.GB81983@elstar.local> Organization: SPARTA X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.8; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/ppVRl+D6G2kNl7a1Tg6LJJ6"; protocol="application/pgp-signature" Cc: opsawg@ietf.org Subject: Re: [OPSAWG] VM-MIB issue vm-mib-02: scaling and caching support X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 14:34:09 -0000 --Sig_/ppVRl+D6G2kNl7a1Tg6LJJ6 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 24 Apr 2012 12:35:34 +0200 Juergen wrote: JS> * vm-mib-02: scaling and caching support JS>=20 JS> It was mentioned that large data centers are characterized by JS> 100.000 physical hosts running 2.000.000 virtual machines. The NASA JS> is reported with 1.000.000 physical hosts and 60.000.000 virtual JS> machines. Bottom line is that we need to make the MIB module JS> scalable. We can assume up hundreds of VMs running on a single JS> virtual machine. JS>=20 JS> ** Solution #02-01 JS>=20 JS> Add ...LastChange objects to tables so that management applications JS> can easily validate cached information without having to read JS> through potentially larger tables. For the vmGuestTable, we might JS> also provide a ...LastStateChange object so that state changes can JS> be polled with reading a simple scalar. JS>=20 JS> ** Solution #02-02 JS>=20 JS> Make some tables time filtered. Unclear which tables would have to JS> be time filtered. I vote for #02-01... Robert -- Senior Software Engineer SPARTA, Inc., a Parsons Company --Sig_/ppVRl+D6G2kNl7a1Tg6LJJ6 Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) iEYEARECAAYFAk+WudwACgkQ7/fVLLY1mnhk+ACfXycV5QgWeuydn4ofyQMEdnWd McYAoIX0vdWKCpFyy3BYPOuIc15BQRew =4MLQ -----END PGP SIGNATURE----- --Sig_/ppVRl+D6G2kNl7a1Tg6LJJ6-- From rstory@tislabs.com Tue Apr 24 08:12:24 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 713C421F86B6 for ; Tue, 24 Apr 2012 08:12:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MfZGlIl2j4Uc for ; Tue, 24 Apr 2012 08:12:24 -0700 (PDT) Received: from walnut.tislabs.com (walnut.tislabs.com [192.94.214.200]) by ietfa.amsl.com (Postfix) with ESMTP id E84DB21F86AD for ; Tue, 24 Apr 2012 08:12:23 -0700 (PDT) Received: from nova.tislabs.com (unknown [10.66.1.77]) by walnut.tislabs.com (Postfix) with ESMTP id A179528B003C; Tue, 24 Apr 2012 11:12:23 -0400 (EDT) Received: from tp.vb.futz.org (localhost.localdomain [127.0.0.1]) by nova.tislabs.com (Postfix) with ESMTP id 6A0451F8032; Tue, 24 Apr 2012 11:12:23 -0400 (EDT) Date: Tue, 24 Apr 2012 11:12:19 -0400 From: Robert Story To: Juergen Schoenwaelder Message-ID: <20120424111219.7db59451@tp.vb.futz.org> In-Reply-To: <20120424103606.GC81983@elstar.local> References: <20120424103606.GC81983@elstar.local> Organization: SPARTA X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.8; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/VeqG0e8Xyns/6Pcl_T.OcVf"; protocol="application/pgp-signature" Cc: opsawg@ietf.org Subject: Re: [OPSAWG] VM-MIB issue vm-mib-03: cpu type identification X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 15:12:24 -0000 --Sig_/VeqG0e8Xyns/6Pcl_T.OcVf Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 24 Apr 2012 12:36:06 +0200 Juergen wrote: JS> Hi, JS>=20 JS> * vm-mib-03: cpu type identification JS>=20 JS> It is necessary to identify the CPU architecture or type since some JS> virtual machine systems can emulate different CPU types. JS> ** Solution #03-03 JS>=20 JS> Use a string data type and rely on systems to put meaningful JS> information there, perhaps provide guidelines how to structure the JS> CPU type names, e.g. --*(-) that is JS> amd-x86_64-opteron or intel-i686-pentium3-vmx-acpi (perhaps using a JS> different separator character since a dash might easily clash). JS> Applications may have to do some normalization across VM-MIB JS> implementations (e.g., regular expression matching) but on the JS> other hand this allows to provide details where necessary. JS>=20 I like this option, but with two string columns. A name, and the configuration. There is likely going to be a limited number of supported configurations on given machine, and the (vendor provided) name will be what managers are looking for when trying to find another host when migrating machines. e.g. "486", "x86 fpu vme pse" "pentium", "x86 fpu vme pse de tsc msr mce cx8 mmx" ... I don't think we need to worry about interoperability, because I'm not aware of any VM solution that allows you to migrate to another vendor's solution. So a VMWare host cpu table will look different from a KVM host cpu table. Robert -- Senior Software Engineer SPARTA, Inc., a Parsons Company --Sig_/VeqG0e8Xyns/6Pcl_T.OcVf Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) iEYEARECAAYFAk+WwtcACgkQ7/fVLLY1mnjPewCgnu+8wW6LVDbHOw5/hpbkhsLb dvEAn2mJDHDGaXZ2iHyiO7Ov2S2nEF0+ =uYOA -----END PGP SIGNATURE----- --Sig_/VeqG0e8Xyns/6Pcl_T.OcVf-- From mrm@vmware.com Tue Apr 24 09:58:15 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0473F21E8086 for ; Tue, 24 Apr 2012 09:58:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.599 X-Spam-Level: X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hQiQmetR0GX9 for ; Tue, 24 Apr 2012 09:58:14 -0700 (PDT) Received: from smtp-outbound-2.vmware.com (smtp-outbound-2.vmware.com [208.91.2.13]) by ietfa.amsl.com (Postfix) with ESMTP id 54CCF21E8015 for ; Tue, 24 Apr 2012 09:58:14 -0700 (PDT) Received: from sc9-mailhost1.vmware.com (sc9-mailhost1.vmware.com [10.113.161.71]) by smtp-outbound-2.vmware.com (Postfix) with ESMTP id 27562281B2; Tue, 24 Apr 2012 09:58:14 -0700 (PDT) Received: from zimbra-prod-mta-3.vmware.com (zimbra-prod-mta-3.vmware.com [10.113.160.227]) by sc9-mailhost1.vmware.com (Postfix) with ESMTP id 2439D183B3; Tue, 24 Apr 2012 09:58:14 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by zimbra-prod-mta-3.vmware.com (Postfix) with ESMTP id 1F78AFA29; Tue, 24 Apr 2012 09:58:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra-prod-mta-3.vmware.com Received: from zimbra-prod-mta-3.vmware.com ([127.0.0.1]) by localhost (zimbra-prod-mta-3.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H5qg4vRX-DvO; Tue, 24 Apr 2012 09:58:14 -0700 (PDT) Received: from zimbra-prod-mbox-2.vmware.com (lbv-sc9-t2prod2-int.vmware.com [10.113.160.246]) by zimbra-prod-mta-3.vmware.com (Postfix) with ESMTP id 04234F9F8; Tue, 24 Apr 2012 09:58:14 -0700 (PDT) Date: Tue, 24 Apr 2012 09:58:13 -0700 (PDT) From: Michael MacFaden To: Juergen Schoenwaelder Message-ID: <2124595851.2575473.1335286693783.JavaMail.root@zimbra-prod-mbox-2.vmware.com> In-Reply-To: <20120423114116.GA75467@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.113.60.13] X-Mailer: Zimbra 7.1.3_GA_3374 (ZimbraWebClient - GC18 (Mac)/7.1.3_GA_3346) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Hypervisor MIB document X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 16:58:15 -0000 > There was already a comment that we need to identify the CPU type > especially since some platforms allow to emulate CPUs. So the > question > is how we model this properly. I should probably start a separate > thread on this. Can you provide more information what the other > "whole raft of VCPU properties" entails? CPU Features which make the VMware products capable of migrating VMs (what VMware calls vMotion) between physical processors are described here: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1993 Hence the following must be exposed for a given hardware system Cpu mfg, process family, feature sets. IMO this data would not be added to this draft mib module, we just augment RFC 2790 HOST-RESOURCES-MIB or if need be have a separate mib module for CPU related reporting in the draft. Notice that if you know your workload you can mask out features to increase compatibility which would be something for this draft. > Since the VM-MIB is implemented on physical hosts, what matters is > really the ration of guests (VMs) per physical host. With the numbers > above, 2M+ logical and 100k physical, we are still at an average of > 20 guests per physical host. Assuming a long tailed distribution, we > might consider 100 guests per physical machine, if its a cluster > perhaps even more but nothing really unmanageable. The big number > mentioned above is the huge number of physical machines. Indeed. There are two general types of servers, those that run backend vms (databases, etc) where the ratio of vms to host system is low and desktop vms, where the ratio is hundreds of vms per server. > Whether timeFilter indexed tables are needed I do not know. Perhaps > just having a few lastChanged objects is sufficient to allow for > simple polls to check whether new guests popped up or guests changed > their status? I'll see if I can collect some evidence of what frequently vs what does not. I prefer to not have to sync full tables to find one out difference when a last changed goes off would be my preference when the tables are large. Thanks, Mike MacFaden From mrm@vmware.com Tue Apr 24 10:43:35 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2621A21E8024 for ; Tue, 24 Apr 2012 10:43:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.599 X-Spam-Level: X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qQQ+nry74QvO for ; Tue, 24 Apr 2012 10:43:34 -0700 (PDT) Received: from smtp-outbound-2.vmware.com (smtp-outbound-2.vmware.com [208.91.2.13]) by ietfa.amsl.com (Postfix) with ESMTP id 109CA21E80A7 for ; Tue, 24 Apr 2012 10:43:34 -0700 (PDT) Received: from sc9-mailhost1.vmware.com (sc9-mailhost1.vmware.com [10.113.161.71]) by smtp-outbound-2.vmware.com (Postfix) with ESMTP id F0537281E0; Tue, 24 Apr 2012 10:43:33 -0700 (PDT) Received: from zimbra-prod-mta-1.vmware.com (zimbra-prod-mta-1.vmware.com [10.113.160.173]) by sc9-mailhost1.vmware.com (Postfix) with ESMTP id E24B7182C8; Tue, 24 Apr 2012 10:43:33 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by zimbra-prod-mta-1.vmware.com (Postfix) with ESMTP id D9FB8D1241; Tue, 24 Apr 2012 10:43:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra-prod-mta-1.vmware.com Received: from zimbra-prod-mta-1.vmware.com ([127.0.0.1]) by localhost (zimbra-prod-mta-1.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7wxZRzWLE2Lm; Tue, 24 Apr 2012 10:43:33 -0700 (PDT) Received: from zimbra-prod-mbox-2.vmware.com (zimbra-prod-mbox-2.vmware.com [10.113.160.202]) by zimbra-prod-mta-1.vmware.com (Postfix) with ESMTP id C5616D1026; Tue, 24 Apr 2012 10:43:33 -0700 (PDT) Date: Tue, 24 Apr 2012 10:43:33 -0700 (PDT) From: Michael MacFaden To: Juergen Schoenwaelder Message-ID: <611439813.2583111.1335289413661.JavaMail.root@zimbra-prod-mbox-2.vmware.com> In-Reply-To: <20120424103606.GC81983@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.113.60.13] X-Mailer: Zimbra 7.1.3_GA_3374 (ZimbraWebClient - GC18 (Mac)/7.1.3_GA_3346) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] VM-MIB issue vm-mib-03: cpu type identification X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 17:43:35 -0000 Juergen writes: > * vm-mib-03: cpu type identification > > It is necessary to identify the CPU architecture or type since some > virtual machine systems can emulate different CPU types. Yes. VMs can emulate a single-core cpu or multi-core cpu with or without hyper-threading. > ** Solution #03-01 > > Provide an IANA controlled enumeration that provides a CPU > classification. The problem will be to provide rules about what > constitutes a new CPU type and what not. This model has generally worked even with the noise and confusion of various persons registering whatever they like. Take the IF-MIB ifTypes where we'd end up with the official types and non-official types as anyone might register anything. RFCs then point out what official numbers *should* be used. Example: ifType 6 vs other gig ethernet types in http://www.iana.org/assignments/ianaiftype-mib > ** Solution #03-02 > > Use OBJECT IDENTITIES to identify CPU types. Such a distributed > enumeration will not achieve a great deal of interoperability > and is likely close to #03-03. >From experience, this style was not adopted by industry for a number of mib modules including RFC 2790. > ** Solution #03-03 > > Use a string data type and rely on systems to put meaningful > information there, perhaps provide guidelines how to structure the > CPU type names, e.g. --*(-) that is > amd-x86_64-opteron or intel-i686-pentium3-vmx-acpi (perhaps using > a different separator character since a dash might easily clash). > Applications may have to do some normalization across VM-MIB > implementations (e.g., regular expression matching) but on the > other hand this allows to provide details where necessary. That's something I might do in a proprietary mib, don't prefer it in a standard mib module. On practice I believe formatted octet strings makes interoperability harder to achieve. > ** Solution #03-04 > > Following #03-03, we provide ...GuestCpuVendor, ...GuestCpuArch > and ...GuestCpuModel objects plus an additional table that > provides details about the features of the CPUs used by a certain virtual > machine. This essentially breaks the string into a set of separate > MIB objects. Yes. That's what I'd expect from an IETF standard. > ** Solution #03-05 > > Following #03-04, we provide ...GuestCpuVendor, ...GuestCpuArch > and ...GuestCpuModel objects plus a string object containing a list of > features. This way, things are more compact but still the most > important components (vendor, arch, model) are broken out as > separate objects. Personally like this the best, but presents the same issue as 03-03 Namely these octet string values then need to be codified precisely for any chance of having interoperability. Variation on a theme here, consider using enumerations with BITS to simplify parsing, make it compact ala RFC 2674 PortList TC. The thing is that new CPU feature sets will continue to be added over time. If we don't want to rev this MIB module every time then this solution works great. How to do this such that new feature sets show up in a consistent way in fielded products is the crux. The purpose of this CPU data is to match a given VM and its cpu state to a real CPU to determine if vm migration will succeed. Would like to see, as I previously mentioned, that we model the real cpu's properly in the HOST-RESOURCES-MIB and in this draft focus on what a given VM's cpu(s) expects of a x86 server's cpu(s). Mike MacFaden From panda@hongo.wide.ad.jp Tue Apr 24 11:44:04 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3948D21E8086 for ; Tue, 24 Apr 2012 11:44:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.904 X-Spam-Level: X-Spam-Status: No, score=0.904 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_JP=1.244, HOST_EQ_JP=1.265, RELAY_IS_203=0.994] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E-GQRPkBkxJD for ; Tue, 24 Apr 2012 11:44:03 -0700 (PDT) Received: from mail.hongo.wide.ad.jp (mail.hongo.wide.ad.jp [203.178.135.13]) by ietfa.amsl.com (Postfix) with ESMTP id 6BB5121E8015 for ; Tue, 24 Apr 2012 11:44:03 -0700 (PDT) Received: from [172.16.78.2] (tc.b.scyphus.co.jp [219.117.248.118]) by mail.hongo.wide.ad.jp (Postfix) with ESMTPSA id 6C6EB108DF47; Wed, 25 Apr 2012 03:44:02 +0900 (JST) From: Hirochika Asai Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Wed, 25 Apr 2012 03:44:02 +0900 To: opsawg@ietf.org Message-Id: <17694FEE-3C81-4A9E-9526-A64E39B2B47D@hongo.wide.ad.jp> Mime-Version: 1.0 (Apple Message framework v1257) X-Mailer: Apple Mail (2.1257) Subject: [OPSAWG] Yet another hypervisor MIB X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 18:44:04 -0000 Hi all, As I previously posted, we have implemented and been testing a prototype = code of yet another hypervisor MIB. Now you can browse the current version = of code in python and MIB here: . Since we implemented this before was submitted, our MIB and implementation don't follow it. At the moment, we don't care on write operations and trap notifications. Moreover, we haven't implemented the storage part although we define some objects for virtual disks in our MIB file. This is because storage is more complex than other resources, i.e., CPU, memory, and network interfaces. For example, current resource allocation (e.g., actual size of sparse virtual disks) and I/O statistics rather than the static = definition of the storage size are more important from the viewpoint of operations and management of hypervisor storage. Thus, deep discussions are required for storage MIB, so we haven't implemented the storage part yet. After the implementation of the prototype code, I'd like to discuss the "direction" of interface statistics, which should be defined, here. This is a common issue to . Looking from hypervisors, TX and RX mean packets to a VM and those from = a VM, respectively. On the other hand, looking from VMs, TX and RX mean = packets from a VM (to HV) and those to a VM (from HV). Libvirt API follows the former manner, but our implementation follows the later. The former is the same way of switches, but the vmIfTable/vmIfEntry is defined under vmEntry (perspective from VMs), not directly under the hypervisor MIB root. Consequently, we have chosen the later. I agree that some of you think the former is more natural. So let me = bring it up for discussion in this community. And, one minor question to WG chairs: Is it better to submit a document summarizing MIB with our target environment and requirements in detail = as an I-D? Or better to avoid the conflict with draft-schoenw-opsawg-vm-mib (and = keep discussion in ML)? Thanks, Hirochika --=20 Hirochika Asai , The University of Tokyo From j.schoenwaelder@jacobs-university.de Tue Apr 24 11:53:54 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7998F21E8042 for ; Tue, 24 Apr 2012 11:53:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.153 X-Spam-Level: X-Spam-Status: No, score=-103.153 tagged_above=-999 required=5 tests=[AWL=0.096, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7tGnSqDCizsZ for ; Tue, 24 Apr 2012 11:53:53 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 972C221E8015 for ; Tue, 24 Apr 2012 11:53:53 -0700 (PDT) Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id DDB8B20D5F; Tue, 24 Apr 2012 20:53:52 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id YG9nTI8Escda; Tue, 24 Apr 2012 20:53:52 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 7853720D25; Tue, 24 Apr 2012 20:53:52 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 51C841EA3E80; Tue, 24 Apr 2012 20:53:54 +0200 (CEST) Date: Tue, 24 Apr 2012 20:53:54 +0200 From: Juergen Schoenwaelder To: Hirochika Asai Message-ID: <20120424185354.GA84350@elstar.local> Mail-Followup-To: Hirochika Asai , opsawg@ietf.org References: <17694FEE-3C81-4A9E-9526-A64E39B2B47D@hongo.wide.ad.jp> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <17694FEE-3C81-4A9E-9526-A64E39B2B47D@hongo.wide.ad.jp> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Yet another hypervisor MIB X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 18:53:54 -0000 On Wed, Apr 25, 2012 at 03:44:02AM +0900, Hirochika Asai wrote: > As I previously posted, we have implemented and been testing a > prototype code of yet another hypervisor MIB. Now you can browse > the current version of code in python and MIB here: > . I will look at this in more detail tomorrow but it is clear already that this is not a valid MIB module. SMIv2 does not allow SEQUENCES inside of SEQUENCES. Anyway, I will try to figure out what the modeling differences are. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From mrm@vmware.com Tue Apr 24 14:18:39 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D130211E8083 for ; Tue, 24 Apr 2012 14:18:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.599 X-Spam-Level: X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 51IBd2NEJ-XG for ; Tue, 24 Apr 2012 14:18:38 -0700 (PDT) Received: from smtp-outbound-2.vmware.com (smtp-outbound-2.vmware.com [208.91.2.13]) by ietfa.amsl.com (Postfix) with ESMTP id 6C14D11E8081 for ; Tue, 24 Apr 2012 14:18:38 -0700 (PDT) Received: from sc9-mailhost1.vmware.com (sc9-mailhost1.vmware.com [10.113.161.71]) by smtp-outbound-2.vmware.com (Postfix) with ESMTP id 093FF2824E; Tue, 24 Apr 2012 14:18:36 -0700 (PDT) Received: from zimbra-prod-mta-1.vmware.com (zimbra-prod-mta-1.vmware.com [10.113.160.173]) by sc9-mailhost1.vmware.com (Postfix) with ESMTP id 04453183F7; Tue, 24 Apr 2012 14:18:36 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by zimbra-prod-mta-1.vmware.com (Postfix) with ESMTP id F30D7D3C6D; Tue, 24 Apr 2012 14:18:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra-prod-mta-1.vmware.com Received: from zimbra-prod-mta-1.vmware.com ([127.0.0.1]) by localhost (zimbra-prod-mta-1.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hcSk0q3mJLn1; Tue, 24 Apr 2012 14:18:35 -0700 (PDT) Received: from zimbra-prod-mbox-2.vmware.com (zimbra-prod-mbox-2.vmware.com [10.113.160.202]) by zimbra-prod-mta-1.vmware.com (Postfix) with ESMTP id E1962D3C14; Tue, 24 Apr 2012 14:18:35 -0700 (PDT) Date: Tue, 24 Apr 2012 14:18:35 -0700 (PDT) From: Michael MacFaden To: Juergen Schoenwaelder Message-ID: <1848448462.2607134.1335302315758.JavaMail.root@zimbra-prod-mbox-2.vmware.com> In-Reply-To: <20120424185354.GA84350@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.113.60.13] X-Mailer: Zimbra 7.1.3_GA_3374 (ZimbraWebClient - GC18 (Mac)/7.1.3_GA_3346) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Yet another hypervisor MIB X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 21:18:40 -0000 js wrote: > I will look at this in more detail tomorrow but it is clear already > that this is not a valid MIB module. SMIv2 does not allow SEQUENCES > inside of SEQUENCES. Anyway, I will try to figure out what the > modeling differences are. Re: http://hg.scyphus.co.jp/virtsnmp The vm nic state is represented by copying selected objects from ifTable/ifXTable. Designing how to model the plumbing of a vm's nic needs to support quite a few configurations. a) hypervisors that plumb a virtual machines set of nics directly to one or more virtual layer 2 switches : (IEEE8021-BRIDGE-MIB / index ieee8021BridgeBasePortComponenId and IEEE8021BridgePortNumber) for the up/down state, source vm's nic mac address, and per vm/per port/per q-vlan in/out/error byte/packet stats. b) virtualization software that plumbs to pseudo interfaces in host OS visible in ifTable/ifStackTable. (seen with ifconfig -a) c) virtualization software that shunts nics directly to the host os nic (no pseudo device). Others? Mike MacFaden From warren@kumari.net Tue Apr 24 15:27:07 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EBE1721F8633 for ; Tue, 24 Apr 2012 15:27:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.504 X-Spam-Level: X-Spam-Status: No, score=-106.504 tagged_above=-999 required=5 tests=[AWL=0.095, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vWAaVEPm-Hv1 for ; Tue, 24 Apr 2012 15:27:07 -0700 (PDT) Received: from vimes.kumari.net (vimes.kumari.net [198.186.192.250]) by ietfa.amsl.com (Postfix) with ESMTP id 3C59321F86D1 for ; Tue, 24 Apr 2012 15:27:07 -0700 (PDT) Received: from [192.168.0.12] (unknown [64.13.52.115]) by vimes.kumari.net (Postfix) with ESMTPSA id 98FB21B40621; Tue, 24 Apr 2012 18:27:06 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: text/plain; charset=windows-1252 From: Warren Kumari In-Reply-To: <20120424103534.GB81983@elstar.local> Date: Tue, 24 Apr 2012 18:26:54 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <7E23F859-237D-42D7-930E-6AB5B953C7EA@kumari.net> References: <20120424103534.GB81983@elstar.local> To: Juergen Schoenwaelder X-Mailer: Apple Mail (2.1257) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] VM-MIB issue vm-mib-02: scaling and caching support X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 22:27:08 -0000 On Apr 24, 2012, at 6:35 AM, Juergen Schoenwaelder wrote: > Hi, >=20 > * vm-mib-02: scaling and caching support >=20 > It was mentioned that large data centers are characterized by > 100.000 physical hosts running 2.000.000 virtual machines. The NASA > is reported with 1.000.000 physical hosts and 60.000.000 virtual > machines. Do you have a references for the NASA numbers? "Design goals" and "wanting to build a=85" do not equal what is actually = built. People say all sorts of numbers, and bigger numbers sound more = impressive=85 I'm sorry to sound grumpy, but many of the discussions in (for example) = ARMD WG and the DC lists wildly over (or under) estimate the scale of = the issue=85. > Bottom line is that we need to make the MIB module > scalable. Yes, fully agree... > We can assume up hundreds of VMs running on a single > virtual machine. Yup. W >=20 > ** Solution #02-01 >=20 > Add ...LastChange objects to tables so that management applications > can easily validate cached information without having to read > through potentially larger tables. For the vmGuestTable, we might > also provide a ...LastStateChange object so that state changes can > be polled with reading a simple scalar. >=20 > ** Solution #02-02 >=20 > Make some tables time filtered. Unclear which tables would have to > be time filtered. >=20 > ** Resolution >=20 > TBD >=20 > /js >=20 > --=20 > Juergen Schoenwaelder Jacobs University Bremen gGmbH > Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany > Fax: +49 421 200 3103 > _______________________________________________ > OPSAWG mailing list > OPSAWG@ietf.org > https://www.ietf.org/mailman/listinfo/opsawg >=20 From ietf@cdl.asgaard.org Tue Apr 24 16:10:46 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5E1EA11E8091 for ; Tue, 24 Apr 2012 16:10:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.227 X-Spam-Level: X-Spam-Status: No, score=-6.227 tagged_above=-999 required=5 tests=[AWL=0.372, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v8SxccF3EO2K for ; Tue, 24 Apr 2012 16:10:41 -0700 (PDT) Received: from asgaard.org (odin.asgaard.org [204.29.151.68]) by ietfa.amsl.com (Postfix) with ESMTP id BA16B11E8079 for ; Tue, 24 Apr 2012 16:10:39 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by asgaard.org (Postfix) with ESMTP id 411191143892; Tue, 24 Apr 2012 23:10:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at asgaard.org Received: from asgaard.org ([127.0.0.1]) by localhost (odin.asgaard.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oYNw7aKbkY5c; Tue, 24 Apr 2012 23:10:35 +0000 (UTC) Received: from fenrir.bigswitch.com (74-93-4-129-sfba.hfc.comcastbusiness.net [74.93.4.129]) by asgaard.org (Postfix) with ESMTPSA id D26BD1143887; Tue, 24 Apr 2012 23:10:34 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: text/plain; charset=utf-8 From: Christopher LILJENSTOLPE In-Reply-To: <201204191743.q3JHhq2g036079@idle.juniper.net> Date: Tue, 24 Apr 2012 13:52:54 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201204191743.q3JHhq2g036079@idle.juniper.net> To: Phil Shafer X-Mailer: Apple Mail (2.1257) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] wg calls X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 23:10:46 -0000 Greetings, The messages below are summaries. The IPR does state that the = IPR claim is more around a potential solution to the problem space = described in the ID, rather than the problem space itself. There are no = further details. However, the patent is on-line if you want to look it = up. Please let the list know if you see this as a blocking issue = before publishing, as of yet we have had no "disagree" statements on the = list, and we were about to publish. Chris On 19Apr2012, at 10.43, Phil Shafer wrote: > Juergen Schoenwaelder writes: >> http://www.ietf.org/mail-archive/web/opsawg/current/msg02184.html >> http://www.ietf.org/mail-archive/web/opsawg/current/msg02175.html >=20 > Was the root of the IPR disclosed? >=20 > Thanks, > Phil > _______________________________________________ > OPSAWG mailing list > OPSAWG@ietf.org > https://www.ietf.org/mailman/listinfo/opsawg -- =20 =E6=9D=8E=E6=9F=AF=E7=9D=BF Check my PGP key here: https://www.asgaard.org/~cdl/cdl.asc Current vCard here: https://www.asgaard.org/~cdl/cdl.vcf Check my calendar availability: https://tungle.me/cdl From ietf@cdl.asgaard.org Tue Apr 24 16:10:46 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7707D11E8093 for ; Tue, 24 Apr 2012 16:10:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.351 X-Spam-Level: X-Spam-Status: No, score=-6.351 tagged_above=-999 required=5 tests=[AWL=0.248, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aBnkgGupa4u3 for ; Tue, 24 Apr 2012 16:10:44 -0700 (PDT) Received: from asgaard.org (odin.asgaard.org [204.29.151.68]) by ietfa.amsl.com (Postfix) with ESMTP id CAEAE11E8085 for ; Tue, 24 Apr 2012 16:10:44 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by asgaard.org (Postfix) with ESMTP id 3ED5A11438A6; Tue, 24 Apr 2012 23:10:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at asgaard.org Received: from asgaard.org ([127.0.0.1]) by localhost (odin.asgaard.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0xYxh5Yk2RwW; Tue, 24 Apr 2012 23:10:42 +0000 (UTC) Received: from fenrir.bigswitch.com (74-93-4-129-sfba.hfc.comcastbusiness.net [74.93.4.129]) by asgaard.org (Postfix) with ESMTPSA id 9903F1143893; Tue, 24 Apr 2012 23:10:42 +0000 (UTC) From: Christopher LILJENSTOLPE Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Date: Tue, 24 Apr 2012 13:56:51 -0700 Message-Id: <074A9139-4BD1-417C-BC44-E13A13824E53@cdl.asgaard.org> To: draft-baker-opsawg-firewalls@tools.ietf.org, draft-kuarsingh-lsn-deployment@tools.ietf.org, opsawg@ietf.org Mime-Version: 1.0 (Apple Message framework v1257) X-Mailer: Apple Mail (2.1257) Cc: opsawg-chairs@tools.ietf.org Subject: [OPSAWG] Acceptance as WG items X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 23:10:46 -0000 Greetings, The other chairs and I believe we have (quiet) consensus on = draft-baker-opsawg-firewalls and draft-kuarsingh-lsn-deployment being = accepted as working group documents. It may be that the lsn-deployment = document may get moved to a v4-transition style working group in the = future, but for right now we will continue the work in this WG. =20 Will the authors please update these documents as wg documents = and submit initial IDs with the new names? Thank you, Christopher -- =20 =E6=9D=8E=E6=9F=AF=E7=9D=BF Check my PGP key here: https://www.asgaard.org/~cdl/cdl.asc Current vCard here: https://www.asgaard.org/~cdl/cdl.vcf Check my calendar availability: https://tungle.me/cdl From mrm@vmware.com Tue Apr 24 20:27:03 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CE9AF21E8012 for ; Tue, 24 Apr 2012 20:27:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.599 X-Spam-Level: X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F-XnBlZx1Jvj for ; Tue, 24 Apr 2012 20:27:03 -0700 (PDT) Received: from smtp-outbound-2.vmware.com (smtp-outbound-2.vmware.com [208.91.2.13]) by ietfa.amsl.com (Postfix) with ESMTP id 4FC1711E8074 for ; Tue, 24 Apr 2012 20:27:03 -0700 (PDT) Received: from sc9-mailhost2.vmware.com (sc9-mailhost2.vmware.com [10.113.161.72]) by smtp-outbound-2.vmware.com (Postfix) with ESMTP id AFF0E282B5; Tue, 24 Apr 2012 20:27:02 -0700 (PDT) Received: from zimbra-prod-mta-2.vmware.com (zimbra-prod-mta-2.vmware.com [10.113.160.174]) by sc9-mailhost2.vmware.com (Postfix) with ESMTP id A6416B049F; Tue, 24 Apr 2012 20:27:02 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by zimbra-prod-mta-2.vmware.com (Postfix) with ESMTP id A0E5353F1A; Tue, 24 Apr 2012 20:27:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra-prod-mta-2.vmware.com Received: from zimbra-prod-mta-2.vmware.com ([127.0.0.1]) by localhost (zimbra-prod-mta-2.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZwZ-DgrJ5o3h; Tue, 24 Apr 2012 20:27:02 -0700 (PDT) Received: from zimbra-prod-mbox-2.vmware.com (lbv-sc9-t2prod2-int.vmware.com [10.113.160.246]) by zimbra-prod-mta-2.vmware.com (Postfix) with ESMTP id 8024C53EF1; Tue, 24 Apr 2012 20:27:02 -0700 (PDT) Date: Tue, 24 Apr 2012 20:27:02 -0700 (PDT) From: Michael MacFaden To: Juergen Schoenwaelder Message-ID: <190091609.2639094.1335324422428.JavaMail.root@zimbra-prod-mbox-2.vmware.com> In-Reply-To: <20120424103506.GA81983@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.113.60.13] X-Mailer: Zimbra 7.1.3_GA_3374 (ZimbraWebClient - GC18 (Mac)/7.1.3_GA_3346) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] VM-MIB issue vm-mib-01: storage sizes X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 03:27:03 -0000 > * vm-mib-01: storage sizes > > The MIB does not provide storage sizes, assuming this is provided > by the hrStorageTable of the HOST-RESOURCES-MIB. However, some well > known implementations of the HOST-RESOURCES-MIB only report about > file systems used by the host system and not file systems residing > in files used by virtual machines. Furthermore, the hrStorageTable > reports sizes "usable by the requesting entity", "excluding loss > due to formatting of file system reference information". For storage > provided to virtual machines, this information is often not readily > available since all you have is the raw block size. Would think if one wanted to see the file systems residing in files used by virtual machines, they'd install another SNMP agent in the guest OS. Have seen in large enterprises two sets of admins, some admins maintain block storage devices and are not allowed access to host/filesystems, then there is another group of admins to who manage the host/filesystems. > ** Solution #01-01 > > Provide the storage block sizes as part of the VM-MIB. Provide a > pointer to the hrStorageTable on systems that can provide this > linkage but allow the pointer to be NULL. Virtual disks reside in host OS filesystems which can be NFS, or local or possibly in a native partition. Would prefer we model this better than just sticking stuff in hrStorageTable. My experience, HOST-RESOURCES-MIB, in particular the hrFSTable and hrStorageTable provide useful data about the storage holding one or more virtual disks or a VM. Would start with a reference to the containing filesystem then just use the hr mib as it is defined to drill down into the partitions and to the underlying local or remote disk. Local disks are then reported in hrDeviceTable which reports generic hardware health. The hrStorageTable, while it is a nice catch all and quite useful in practice think it hard to standardize what goes in it and then try to expand indexing into other tables to understand how it fits into the overall storage stack and related indexing (ISCSI-MIB, Fiber Channel, ...). Thanks, Mike MacFaden From mrm@vmware.com Tue Apr 24 20:31:15 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6C35511E8087 for ; Tue, 24 Apr 2012 20:31:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.599 X-Spam-Level: X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9Qlhw2uR1n0A for ; Tue, 24 Apr 2012 20:31:15 -0700 (PDT) Received: from smtp-outbound-1.vmware.com (smtp-outbound-1.vmware.com [208.91.2.12]) by ietfa.amsl.com (Postfix) with ESMTP id F18CC11E8074 for ; Tue, 24 Apr 2012 20:31:14 -0700 (PDT) Received: from sc9-mailhost1.vmware.com (sc9-mailhost1.vmware.com [10.113.161.71]) by smtp-outbound-1.vmware.com (Postfix) with ESMTP id 536D1282B9; Tue, 24 Apr 2012 20:31:13 -0700 (PDT) Received: from zimbra-prod-mta-1.vmware.com (zimbra-prod-mta-1.vmware.com [10.113.160.173]) by sc9-mailhost1.vmware.com (Postfix) with ESMTP id 4D0F3183E1; Tue, 24 Apr 2012 20:31:13 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by zimbra-prod-mta-1.vmware.com (Postfix) with ESMTP id 4167FD6A5B; Tue, 24 Apr 2012 20:31:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra-prod-mta-1.vmware.com Received: from zimbra-prod-mta-1.vmware.com ([127.0.0.1]) by localhost (zimbra-prod-mta-1.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bjapYhBWInEb; Tue, 24 Apr 2012 20:31:13 -0700 (PDT) Received: from zimbra-prod-mbox-2.vmware.com (zimbra-prod-mbox-2.vmware.com [10.113.160.202]) by zimbra-prod-mta-1.vmware.com (Postfix) with ESMTP id 2F33FD6A31; Tue, 24 Apr 2012 20:31:13 -0700 (PDT) Date: Tue, 24 Apr 2012 20:31:13 -0700 (PDT) From: Michael MacFaden To: Juergen Schoenwaelder Message-ID: <1640073524.2639266.1335324673078.JavaMail.root@zimbra-prod-mbox-2.vmware.com> In-Reply-To: <190091609.2639094.1335324422428.JavaMail.root@zimbra-prod-mbox-2.vmware.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.113.60.13] X-Mailer: Zimbra 7.1.3_GA_3374 (ZimbraWebClient - GC18 (Mac)/7.1.3_GA_3346) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] VM-MIB issue vm-mib-01: storage sizes X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 03:31:15 -0000 > My experience, HOST-RESOURCES-MIB, in particular the hrFSTable > and hrStorageTable provide useful data about > the storage holding one or more virtual disks or a VM. > Would start with a reference to the containing filesystem then just > use the hr mib as it is defined to drill down into the partitions > and to the underlying local or remote disk. Local disks are then > reported in hrDeviceTable which reports generic hardware health. Ah just remembered that some virtualization software supports using native/raw disk partitions not mounted in the host OS, so then a pointer into the hrPartitionTable would be better than he hrFSTable to cover more configurations. Mike MacFaden From j.schoenwaelder@jacobs-university.de Wed Apr 25 00:11:00 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2FD6921F866A for ; Wed, 25 Apr 2012 00:11:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.155 X-Spam-Level: X-Spam-Status: No, score=-103.155 tagged_above=-999 required=5 tests=[AWL=0.094, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id THqoOl0QNU0f for ; Wed, 25 Apr 2012 00:10:59 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 00E2221F85D1 for ; Wed, 25 Apr 2012 00:10:58 -0700 (PDT) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id 24FBA20DA7; Wed, 25 Apr 2012 09:10:58 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id ATPpHUEdRzA0; Wed, 25 Apr 2012 09:10:58 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 288AF20DDE; Wed, 25 Apr 2012 09:10:57 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id BA3801EA4628; Wed, 25 Apr 2012 09:10:57 +0200 (CEST) Date: Wed, 25 Apr 2012 09:10:57 +0200 From: Juergen Schoenwaelder To: Michael MacFaden Message-ID: <20120425071057.GA85712@elstar.local> Mail-Followup-To: Michael MacFaden , opsawg@ietf.org References: <20120424185354.GA84350@elstar.local> <1848448462.2607134.1335302315758.JavaMail.root@zimbra-prod-mbox-2.vmware.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1848448462.2607134.1335302315758.JavaMail.root@zimbra-prod-mbox-2.vmware.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Yet another hypervisor MIB X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 07:11:00 -0000 On Tue, Apr 24, 2012 at 02:18:35PM -0700, Michael MacFaden wrote: > js wrote: > > I will look at this in more detail tomorrow but it is clear already > > that this is not a valid MIB module. SMIv2 does not allow SEQUENCES > > inside of SEQUENCES. Anyway, I will try to figure out what the > > modeling differences are. > > Re: http://hg.scyphus.co.jp/virtsnmp > > The vm nic state is represented by copying selected objects > from ifTable/ifXTable. Copying MIB objects is generally not the way we like to do things. Interface statistics are in the ifTable - so all that is needed is to identify the interfaces associated with virtual machine. > Designing how to model the plumbing of a vm's nic > needs to support quite a few configurations. > > a) hypervisors that plumb a virtual machines set of nics directly to one or > more virtual layer 2 switches : (IEEE8021-BRIDGE-MIB / index > ieee8021BridgeBasePortComponenId and IEEE8021BridgePortNumber) > for the up/down state, source vm's nic mac address, and per vm/per port/per q-vlan > in/out/error byte/packet stats. > > b) virtualization software that plumbs to pseudo interfaces in host OS > visible in ifTable/ifStackTable. (seen with ifconfig -a) > > c) virtualization software that shunts nics directly to the host os nic > (no pseudo device). I think c) is pretty much out of scope for a standards solution. The architectural model used in the IETF is that IP packets flow through interfaces. For bridges, we have ports but there is a common mapping of bridge ports to interfaces. Implementations that do pass packets magically to a virtual machine might exist but I doubt it is something worth spending extra efforts on. (I think such implementations should better be encouraged to expose their mechanism as an interface.) /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From j.schoenwaelder@jacobs-university.de Wed Apr 25 02:32:50 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8643221F86EC for ; Wed, 25 Apr 2012 02:32:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.158 X-Spam-Level: X-Spam-Status: No, score=-103.158 tagged_above=-999 required=5 tests=[AWL=0.091, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Zo9k6GH3++XL for ; Wed, 25 Apr 2012 02:32:49 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 54FFC21F86E4 for ; Wed, 25 Apr 2012 02:32:49 -0700 (PDT) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id 9EEC220E27; Wed, 25 Apr 2012 11:32:48 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id 1TzCAyJAXbvE; Wed, 25 Apr 2012 11:32:48 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 3CAE820E26; Wed, 25 Apr 2012 11:32:47 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 6CF071EA4C71; Wed, 25 Apr 2012 11:32:48 +0200 (CEST) Date: Wed, 25 Apr 2012 11:32:47 +0200 From: Juergen Schoenwaelder To: Hirochika Asai , opsawg@ietf.org Message-ID: <20120425093245.GA86754@elstar.local> Mail-Followup-To: Hirochika Asai , opsawg@ietf.org References: <17694FEE-3C81-4A9E-9526-A64E39B2B47D@hongo.wide.ad.jp> <20120424185354.GA84350@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120424185354.GA84350@elstar.local> User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: [OPSAWG] Yet another hypervisor MIB X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 09:32:50 -0000 On Tue, Apr 24, 2012 at 08:53:54PM +0200, Juergen Schoenwaelder wrote: > On Wed, Apr 25, 2012 at 03:44:02AM +0900, Hirochika Asai wrote: > > > As I previously posted, we have implemented and been testing a > > prototype code of yet another hypervisor MIB. Now you can browse > > the current version of code in python and MIB here: > > . > > I will look at this in more detail tomorrow but it is clear already > that this is not a valid MIB module. SMIv2 does not allow SEQUENCES > inside of SEQUENCES. Anyway, I will try to figure out what the > modeling differences are. Ignoring a number of technical issues, I believe the major differences are that a) your MIB duplicate objects already found in other MIBs (SNMPv2-MIB, IF-MIB) which I tried to avoid b) your MIB exports information about each virtual CPU assigned to a VM - I have no clear opinion whether that is valuable or not, in particular since CPU states may change pretty quickly (it would probably then make more sense how much type a CPU has spent in the various states) c) your MIB explicitely models a notion of disks as storage units which is probably a good thing Do you agree? Is this is fair summary? /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From panda@hongo.wide.ad.jp Wed Apr 25 04:04:07 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0CE0A21F8737 for ; Wed, 25 Apr 2012 04:04:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.904 X-Spam-Level: X-Spam-Status: No, score=0.904 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_JP=1.244, HOST_EQ_JP=1.265, RELAY_IS_203=0.994] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id O8Z61Om2YtCU for ; Wed, 25 Apr 2012 04:04:06 -0700 (PDT) Received: from mail.hongo.wide.ad.jp (mail.hongo.wide.ad.jp [203.178.135.13]) by ietfa.amsl.com (Postfix) with ESMTP id BBC9521F8762 for ; Wed, 25 Apr 2012 04:04:05 -0700 (PDT) Received: from [10.100.2.215] (lupin.hongo.wide.ad.jp [203.178.135.47]) by mail.hongo.wide.ad.jp (Postfix) with ESMTPSA id 6C809108DF47; Wed, 25 Apr 2012 20:04:02 +0900 (JST) Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: text/plain; charset=us-ascii From: Hirochika Asai In-Reply-To: <20120425071057.GA85712@elstar.local> Date: Wed, 25 Apr 2012 20:04:02 +0900 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20120424185354.GA84350@elstar.local> <1848448462.2607134.1335302315758.JavaMail.root@zimbra-prod-mbox-2.vmware.com> <20120425071057.GA85712@elstar.local> To: Juergen Schoenwaelder X-Mailer: Apple Mail (2.1257) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Yet another hypervisor MIB X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 11:04:07 -0000 On Apr 25, 2012, at 4:10 PM, Juergen Schoenwaelder wrote: > On Tue, Apr 24, 2012 at 02:18:35PM -0700, Michael MacFaden wrote: >> js wrote: >>> I will look at this in more detail tomorrow but it is clear already >>> that this is not a valid MIB module. SMIv2 does not allow SEQUENCES >>> inside of SEQUENCES. Anyway, I will try to figure out what the >>> modeling differences are. >>=20 >> Re: http://hg.scyphus.co.jp/virtsnmp >>=20 >> The vm nic state is represented by copying selected objects >> from ifTable/ifXTable.=20 >=20 > Copying MIB objects is generally not the way we like to do things. > Interface statistics are in the ifTable - so all that is needed is to > identify the interfaces associated with virtual machine. Thanks, I understood. Just copying isn't a good nor smart idea. As you suggested, vmEntry can have pointers to ifTable or ifEntry objects instead of holding vmIfTable in vmEntry. >> Designing how to model the plumbing of a vm's nic >> needs to support quite a few configurations. >>=20 >> a) hypervisors that plumb a virtual machines set of nics directly to = one or >> more virtual layer 2 switches : (IEEE8021-BRIDGE-MIB / index >> ieee8021BridgeBasePortComponenId and IEEE8021BridgePortNumber) >> for the up/down state, source vm's nic mac address, and per vm/per = port/per q-vlan=20 >> in/out/error byte/packet stats. >>=20 >> b) virtualization software that plumbs to pseudo interfaces in host = OS >> visible in ifTable/ifStackTable. (seen with ifconfig -a) >>=20 >> c) virtualization software that shunts nics directly to the host os = nic=20 >> (no pseudo device). >=20 > I think c) is pretty much out of scope for a standards solution. The > architectural model used in the IETF is that IP packets flow through > interfaces. For bridges, we have ports but there is a common mapping > of bridge ports to interfaces. Implementations that do pass packets > magically to a virtual machine might exist but I doubt it is something > worth spending extra efforts on. (I think such implementations should > better be encouraged to expose their mechanism as an interface.) Our current implementation assumes b). Both Xen and KVM create pseudo NICs (e.g., vif/tap/vnet) on a hypervisor and each VM attaches one or more of them when using bridge mode. I agree to js's opinion. Moreover, as far as I know, a) is the case of VMware; which plumbs vSwitches and each VM's NIC attaches a vSwitch, although I don't know the internal implementation of VMware's virtual NICs. c) is cases such as using MacVTap and directly attaching a physical NIC. However, in both cases, hypervisors must internally have virtual (pseudo) NICs or equivalent mechanisms to forward packets from/to VMs. Thus, I also think we don't have to care the internal mechanisms, and SNMP agents can generalize and provide virtual NICs' information in the same scheme. Only one exception is the case that a hypervisor bypasses "PCI" (i.e., physical NIC) to a VM using IOMMU. In this case, hypervisors cannot recognize any packets passing through the NIC. regards, Hirochika --=20 Hirochika Asai , The University of Tokyo From j.schoenwaelder@jacobs-university.de Wed Apr 25 06:02:48 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EF60021F86AD for ; Wed, 25 Apr 2012 06:02:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.162 X-Spam-Level: X-Spam-Status: No, score=-103.162 tagged_above=-999 required=5 tests=[AWL=0.087, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VgwndPCLf5ZQ for ; Wed, 25 Apr 2012 06:02:47 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 46BA721F86A8 for ; Wed, 25 Apr 2012 06:02:47 -0700 (PDT) Received: from localhost (demetrius1.jacobs-university.de [212.201.44.46]) by hermes.jacobs-university.de (Postfix) with ESMTP id 87C7820D92; Wed, 25 Apr 2012 15:02:46 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius1.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id 0w4wvffvxw8K; Wed, 25 Apr 2012 15:02:46 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id ED05220D02; Wed, 25 Apr 2012 15:02:45 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id CE35D1EA5144; Wed, 25 Apr 2012 15:02:46 +0200 (CEST) Date: Wed, 25 Apr 2012 15:02:46 +0200 From: Juergen Schoenwaelder To: Hirochika Asai Message-ID: <20120425130244.GA86934@elstar.local> Mail-Followup-To: Hirochika Asai , Michael MacFaden , opsawg@ietf.org References: <20120424185354.GA84350@elstar.local> <1848448462.2607134.1335302315758.JavaMail.root@zimbra-prod-mbox-2.vmware.com> <20120425071057.GA85712@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Yet another hypervisor MIB X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 13:02:48 -0000 On Wed, Apr 25, 2012 at 08:04:02PM +0900, Hirochika Asai wrote: > > Only one exception is the case that a hypervisor bypasses "PCI" (i.e., > physical NIC) to a VM using IOMMU. In this case, hypervisors cannot > recognize any packets passing through the NIC. > Yes, but in this case the hypervisor can't be expected to report about traffic it never sees. Hence in this case, a management application has to read the statistics directly from the virtual machine that is directly controlling the physical NIC. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From j.schoenwaelder@jacobs-university.de Wed Apr 25 06:48:36 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4F92921F86B2 for ; Wed, 25 Apr 2012 06:48:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.166 X-Spam-Level: X-Spam-Status: No, score=-103.166 tagged_above=-999 required=5 tests=[AWL=0.083, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2xMdeMCgYZs8 for ; Wed, 25 Apr 2012 06:48:35 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 849EC21F86AD for ; Wed, 25 Apr 2012 06:48:35 -0700 (PDT) Received: from localhost (demetrius1.jacobs-university.de [212.201.44.46]) by hermes.jacobs-university.de (Postfix) with ESMTP id C3CBD20D9B; Wed, 25 Apr 2012 15:48:34 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius1.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id QR3RM1dgTsBu; Wed, 25 Apr 2012 15:48:34 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 3DCD620CD4; Wed, 25 Apr 2012 15:48:34 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 17DE51EA54A5; Wed, 25 Apr 2012 15:48:34 +0200 (CEST) Date: Wed, 25 Apr 2012 15:48:34 +0200 From: Juergen Schoenwaelder To: Michael MacFaden Message-ID: <20120425134834.GA87475@elstar.local> Mail-Followup-To: Michael MacFaden , opsawg@ietf.org References: <20120423114116.GA75467@elstar.local> <2124595851.2575473.1335286693783.JavaMail.root@zimbra-prod-mbox-2.vmware.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2124595851.2575473.1335286693783.JavaMail.root@zimbra-prod-mbox-2.vmware.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Hypervisor MIB document X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 13:48:36 -0000 On Tue, Apr 24, 2012 at 09:58:13AM -0700, Michael MacFaden wrote: > > There was already a comment that we need to identify the CPU type > > especially since some platforms allow to emulate CPUs. So the > > question > > is how we model this properly. I should probably start a separate > > thread on this. Can you provide more information what the other > > "whole raft of VCPU properties" entails? > > CPU Features which make the VMware products capable of migrating > VMs (what VMware calls vMotion) between physical processors > are described here: > > http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1993 > > Hence the following must be exposed for a given hardware system > Cpu mfg, process family, feature sets. > > IMO this data would not be added to this draft mib module, > we just augment RFC 2790 HOST-RESOURCES-MIB or if need be > have a separate mib module for CPU related reporting in the draft. > > Notice that if you know your workload you can mask out features > to increase compatibility which would be something for this draft. Just to be clear: My understanding is that you are talking about the physical CPUs of a system, not the CPUs emulated and given to virtual machines (as qemu for instance does). Am I correct? /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From j.schoenwaelder@jacobs-university.de Wed Apr 25 06:54:27 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 31C6221F86FE for ; Wed, 25 Apr 2012 06:54:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.167 X-Spam-Level: X-Spam-Status: No, score=-103.167 tagged_above=-999 required=5 tests=[AWL=0.082, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WC1TW2b3KOkX for ; Wed, 25 Apr 2012 06:54:26 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id F06EF21F86C5 for ; Wed, 25 Apr 2012 06:54:25 -0700 (PDT) Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id 3FC7920E24; Wed, 25 Apr 2012 15:54:25 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id gx_WaJow07Sm; Wed, 25 Apr 2012 15:54:25 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id B510C20D97; Wed, 25 Apr 2012 15:54:24 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 5A4DA1EA5558; Wed, 25 Apr 2012 15:54:26 +0200 (CEST) Date: Wed, 25 Apr 2012 15:54:26 +0200 From: Juergen Schoenwaelder To: Warren Kumari Message-ID: <20120425135426.GA87576@elstar.local> Mail-Followup-To: Warren Kumari , opsawg@ietf.org References: <20120424103534.GB81983@elstar.local> <7E23F859-237D-42D7-930E-6AB5B953C7EA@kumari.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <7E23F859-237D-42D7-930E-6AB5B953C7EA@kumari.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] VM-MIB issue vm-mib-02: scaling and caching support X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 13:54:27 -0000 On Tue, Apr 24, 2012 at 06:26:54PM -0400, Warren Kumari wrote: > > On Apr 24, 2012, at 6:35 AM, Juergen Schoenwaelder wrote: > > > Hi, > > > > * vm-mib-02: scaling and caching support > > > > It was mentioned that large data centers are characterized by > > 100.000 physical hosts running 2.000.000 virtual machines. The NASA > > is reported with 1.000.000 physical hosts and 60.000.000 virtual > > machines. > > > Do you have a references for the NASA numbers? > "Design goals" and "wanting to build a…" do not equal what is actually built. > > People say all sorts of numbers, and bigger numbers sound more impressive… > > I'm sorry to sound grumpy, but many of the discussions in (for example) ARMD WG and the DC lists wildly over (or under) estimate the scale of the issue…. I have no clue whether this was really built or is really being built. The average still is in the order of 60 VMs per physical machine. This is what matters here. > > Bottom line is that we need to make the MIB module > > scalable. > > Yes, fully agree... > > > We can assume up hundreds of VMs running on a single > > virtual machine. So lets plan with a couple of hundreds of VMs when we are talking about what this MIB might have to "scale" to. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From j.schoenwaelder@jacobs-university.de Wed Apr 25 07:04:59 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8FDF021F8737 for ; Wed, 25 Apr 2012 07:04:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.169 X-Spam-Level: X-Spam-Status: No, score=-103.169 tagged_above=-999 required=5 tests=[AWL=0.080, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JwFEos9DMvKc for ; Wed, 25 Apr 2012 07:04:59 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id C648C21F86DA for ; Wed, 25 Apr 2012 07:04:58 -0700 (PDT) Received: from localhost (demetrius1.jacobs-university.de [212.201.44.46]) by hermes.jacobs-university.de (Postfix) with ESMTP id 255AB20E24; Wed, 25 Apr 2012 16:04:58 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius1.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id 9G5FQa0YNal4; Wed, 25 Apr 2012 16:04:57 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 4C3B320D6D; Wed, 25 Apr 2012 16:04:57 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 08DF61EA561F; Wed, 25 Apr 2012 16:04:57 +0200 (CEST) Date: Wed, 25 Apr 2012 16:04:57 +0200 From: Juergen Schoenwaelder To: Michael MacFaden Message-ID: <20120425140457.GB87576@elstar.local> Mail-Followup-To: Michael MacFaden , opsawg@ietf.org References: <190091609.2639094.1335324422428.JavaMail.root@zimbra-prod-mbox-2.vmware.com> <1640073524.2639266.1335324673078.JavaMail.root@zimbra-prod-mbox-2.vmware.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1640073524.2639266.1335324673078.JavaMail.root@zimbra-prod-mbox-2.vmware.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] VM-MIB issue vm-mib-01: storage sizes X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 14:04:59 -0000 On Tue, Apr 24, 2012 at 08:31:13PM -0700, Michael MacFaden wrote: > > My experience, HOST-RESOURCES-MIB, in particular the hrFSTable > > and hrStorageTable provide useful data about > > the storage holding one or more virtual disks or a VM. > > Would start with a reference to the containing filesystem then just > > use the hr mib as it is defined to drill down into the partitions > > and to the underlying local or remote disk. Local disks are then > > reported in hrDeviceTable which reports generic hardware health. > > Ah just remembered that some virtualization software supports > using native/raw disk partitions not mounted in the host OS, > so then a pointer into the hrPartitionTable would be better > than he hrFSTable to cover more configurations. But then, a file in my file system that I provide a virtual machine as a disk drive is not a partition either. And networked partition are not covered by the hrPartitionTable (but extremely useful if you want to migrate VMs). I get the feeling that the host resources MIB does not really help us here. There are apparently many different ways to provide storage to a virtual machine and neither the hrStorageTable nor the hrPartitionTable nor the hrFSTable nor the hrDiskStorageTable is doing what we need. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From rstory@tislabs.com Wed Apr 25 08:29:31 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C8F3F21F87C9 for ; Wed, 25 Apr 2012 08:29:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.299 X-Spam-Level: X-Spam-Status: No, score=-2.299 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, J_CHICKENPOX_25=0.6] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uR+8SMVDvYzX for ; Wed, 25 Apr 2012 08:29:31 -0700 (PDT) Received: from walnut.tislabs.com (walnut.tislabs.com [192.94.214.200]) by ietfa.amsl.com (Postfix) with ESMTP id 4BEC621F8715 for ; Wed, 25 Apr 2012 08:29:31 -0700 (PDT) Received: from nova.tislabs.com (unknown [10.66.1.77]) by walnut.tislabs.com (Postfix) with ESMTP id E5FFA28B003A; Wed, 25 Apr 2012 11:29:30 -0400 (EDT) Received: from tp.vb.futz.org (localhost.localdomain [127.0.0.1]) by nova.tislabs.com (Postfix) with ESMTP id A96411F8032; Wed, 25 Apr 2012 11:29:30 -0400 (EDT) Date: Wed, 25 Apr 2012 11:29:25 -0400 From: Robert Story To: Juergen Schoenwaelder Message-ID: <20120425112925.78d2a24a@tp.vb.futz.org> In-Reply-To: <20120425140457.GB87576@elstar.local> References: <190091609.2639094.1335324422428.JavaMail.root@zimbra-prod-mbox-2.vmware.com> <1640073524.2639266.1335324673078.JavaMail.root@zimbra-prod-mbox-2.vmware.com> <20120425140457.GB87576@elstar.local> Organization: SPARTA X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.8; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/ppRiP_G.AuLjok2o9GqTrU_"; protocol="application/pgp-signature" Cc: opsawg@ietf.org Subject: Re: [OPSAWG] VM-MIB issue vm-mib-01: storage sizes X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 15:29:31 -0000 --Sig_/ppRiP_G.AuLjok2o9GqTrU_ Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Wed, 25 Apr 2012 16:04:57 +0200 Juergen wrote: JS> I get the feeling that the host resources MIB does not really help us JS> here. There are apparently many different ways to provide storage to a JS> virtual machine and neither the hrStorageTable nor the JS> hrPartitionTable nor the hrFSTable nor the hrDiskStorageTable is doing JS> what we need. +1. But I also don't object to providing an OID to link to a hr*Table entry if there is a 1-1 mapping... Robert -- Senior Software Engineer SPARTA, Inc., a Parsons Company --Sig_/ppRiP_G.AuLjok2o9GqTrU_ Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) iEYEARECAAYFAk+YGFoACgkQ7/fVLLY1mng19gCeOejU0qtkvJN9tuDkazJCzDqH NDAAn3cMDC4DZZLKClL4E9ZkRBDCMvJK =FK2B -----END PGP SIGNATURE----- --Sig_/ppRiP_G.AuLjok2o9GqTrU_-- From mrm@vmware.com Wed Apr 25 10:16:22 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E65DB21F87E6 for ; Wed, 25 Apr 2012 10:16:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.599 X-Spam-Level: X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MwX4uWJ9GCFS for ; Wed, 25 Apr 2012 10:16:22 -0700 (PDT) Received: from smtp-outbound-1.vmware.com (smtp-outbound-1.vmware.com [208.91.2.12]) by ietfa.amsl.com (Postfix) with ESMTP id 7018521F87E1 for ; Wed, 25 Apr 2012 10:16:22 -0700 (PDT) Received: from sc9-mailhost2.vmware.com (sc9-mailhost2.vmware.com [10.113.161.72]) by smtp-outbound-1.vmware.com (Postfix) with ESMTP id DAE8C2826B; Wed, 25 Apr 2012 10:16:21 -0700 (PDT) Received: from zimbra-prod-mta-2.vmware.com (zimbra-prod-mta-2.vmware.com [10.113.160.174]) by sc9-mailhost2.vmware.com (Postfix) with ESMTP id D75BEB04DD; Wed, 25 Apr 2012 10:16:21 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by zimbra-prod-mta-2.vmware.com (Postfix) with ESMTP id CEB3438A24; Wed, 25 Apr 2012 10:16:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra-prod-mta-2.vmware.com Received: from zimbra-prod-mta-2.vmware.com ([127.0.0.1]) by localhost (zimbra-prod-mta-2.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I6QAMauocZjV; Wed, 25 Apr 2012 10:16:21 -0700 (PDT) Received: from zimbra-prod-mbox-2.vmware.com (lbv-sc9-t2prod2-int.vmware.com [10.113.160.246]) by zimbra-prod-mta-2.vmware.com (Postfix) with ESMTP id B3B5638A03; Wed, 25 Apr 2012 10:16:21 -0700 (PDT) Date: Wed, 25 Apr 2012 10:16:21 -0700 (PDT) From: Michael MacFaden To: Juergen Schoenwaelder Message-ID: <1407744686.2681126.1335374181637.JavaMail.root@zimbra-prod-mbox-2.vmware.com> In-Reply-To: <20120425134834.GA87475@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.113.60.13] X-Mailer: Zimbra 7.1.3_GA_3374 (ZimbraWebClient - GC18 (Mac)/7.1.3_GA_3346) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Hypervisor MIB document X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 17:16:23 -0000 > Just to be clear: My understanding is that you are talking about the > physical CPUs of a system, not the CPUs emulated and given to virtual > machines (as qemu for instance does). Am I correct? In short, HOST-RESOURCES-MIB should report what it is capable of. The draft should report what it expects. One fielded implementation allows for masking out what features will be compared in order to determine if a virtual machine can migrate. Mike From mrm@vmware.com Wed Apr 25 10:58:12 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BA98921F8859 for ; Wed, 25 Apr 2012 10:58:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.599 X-Spam-Level: X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TRWbDBIOfgoY for ; Wed, 25 Apr 2012 10:58:12 -0700 (PDT) Received: from smtp-outbound-2.vmware.com (smtp-outbound-2.vmware.com [208.91.2.13]) by ietfa.amsl.com (Postfix) with ESMTP id 4BA2B21F8858 for ; Wed, 25 Apr 2012 10:58:12 -0700 (PDT) Received: from sc9-mailhost2.vmware.com (sc9-mailhost2.vmware.com [10.113.161.72]) by smtp-outbound-2.vmware.com (Postfix) with ESMTP id 3986928377; Wed, 25 Apr 2012 10:58:12 -0700 (PDT) Received: from zimbra-prod-mta-3.vmware.com (zimbra-prod-mta-3.vmware.com [10.113.160.227]) by sc9-mailhost2.vmware.com (Postfix) with ESMTP id 35361B056C; Wed, 25 Apr 2012 10:58:12 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by zimbra-prod-mta-3.vmware.com (Postfix) with ESMTP id 2DB71F1A77; Wed, 25 Apr 2012 10:58:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra-prod-mta-3.vmware.com Received: from zimbra-prod-mta-3.vmware.com ([127.0.0.1]) by localhost (zimbra-prod-mta-3.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JyRjWYqLNcfK; Wed, 25 Apr 2012 10:58:12 -0700 (PDT) Received: from zimbra-prod-mbox-2.vmware.com (lbv-sc9-t2prod2-int.vmware.com [10.113.160.246]) by zimbra-prod-mta-3.vmware.com (Postfix) with ESMTP id 12388F1A33; Wed, 25 Apr 2012 10:58:12 -0700 (PDT) Date: Wed, 25 Apr 2012 10:58:11 -0700 (PDT) From: Michael MacFaden To: Juergen Schoenwaelder Message-ID: <499367579.2686841.1335376691865.JavaMail.root@zimbra-prod-mbox-2.vmware.com> In-Reply-To: <20120425140457.GB87576@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.113.60.13] X-Mailer: Zimbra 7.1.3_GA_3374 (ZimbraWebClient - GC18 (Mac)/7.1.3_GA_3346) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] VM-MIB issue vm-mib-01: storage sizes X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 17:58:12 -0000 js writes: > But then, a file in my file system that I provide a virtual machine > as a disk drive is not a partition either. If your filesystem is remote mounted filesystem then true there is no partition. That's but one type of configuration. To recap, places where a virtual disk may be situated: 1) a file (virtual disk) in a mounted filesystem of the host. a) local filesystem which can be traced to a partition and a disk/lun and a storage adapter. Note the LUN might attached via SAN:FC/ISCSI b) remote hrFSTable provides the remote mount information This is common with desktop as well as hypervisor software systems. 2) A raw partition (/dev/sd...) is configured for use by the VM. 3) other? This is common with the desktop virtualization software systems. > There are apparently many different ways to provide storage to > a virtual machine and neither the hrStorageTable nor the > hrPartitionTable nor the hrFSTable nor the hrDiskStorageTable is > doing what we need. What is it we need to accomplish? a) Report vm disk configuration? accurately portray how a given virtual disk maps to physical storage b) Report vm disk capacity? storage allocations at the block level, filesystem level of the host c) Report vm disk performance? Be able to index into the host bus adapter, and disk, network stacks. d) Report VM/Guest OS view a-c? One last bit that might impact this draft, virtual disk are not only simple files with a 1:1 relationship to a vm. Fielded products allow multiple VMs to share one or more virtual disk files but each having their copy-on-write logs. irtual disks can be configured to take up their full measure of storage blocks when created, others grow up to their limit over time (aka thick or thin provisioned). 'Snapshots' of the virtual disk at some point in time (possibly along with the vm's memory) will be common as well. Mike From j.schoenwaelder@jacobs-university.de Wed Apr 25 11:11:26 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8EAB721F8805 for ; Wed, 25 Apr 2012 11:11:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.171 X-Spam-Level: X-Spam-Status: No, score=-103.171 tagged_above=-999 required=5 tests=[AWL=0.078, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OeoZWXV2krci for ; Wed, 25 Apr 2012 11:11:25 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id C797121F8713 for ; Wed, 25 Apr 2012 11:11:04 -0700 (PDT) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id 1317F20E23; Wed, 25 Apr 2012 20:11:04 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id 2qnSRVEa6XDm; Wed, 25 Apr 2012 20:11:03 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 7F84120E22; Wed, 25 Apr 2012 20:11:03 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id DC5891EA61F6; Wed, 25 Apr 2012 20:11:03 +0200 (CEST) Date: Wed, 25 Apr 2012 20:11:03 +0200 From: Juergen Schoenwaelder To: Michael MacFaden Message-ID: <20120425181103.GA88418@elstar.local> Mail-Followup-To: Michael MacFaden , opsawg@ietf.org References: <20120425134834.GA87475@elstar.local> <1407744686.2681126.1335374181637.JavaMail.root@zimbra-prod-mbox-2.vmware.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1407744686.2681126.1335374181637.JavaMail.root@zimbra-prod-mbox-2.vmware.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Hypervisor MIB document X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 18:11:26 -0000 On Wed, Apr 25, 2012 at 10:16:21AM -0700, Michael MacFaden wrote: > > > Just to be clear: My understanding is that you are talking about the > > physical CPUs of a system, not the CPUs emulated and given to virtual > > machines (as qemu for instance does). Am I correct? > > In short, HOST-RESOURCES-MIB should report what it is capable of. > The draft should report what it expects. > > One fielded implementation allows for masking out what features will > be compared in order to determine if a virtual machine can migrate. Well, the HOST-RESOURCES-MIB is likely not revised anytime soon; all we can reasonably do is to extend it. That said, there is also the ENTITY-MIB that is reporting about the physical components and it has a CPU class. It is not clear to me whether the HOST-RESOURCES-MIB or the ENTITY-MIB should be able to report these hardware characteristics (as long as we talk about physical CPUs). For the matching of physical CPUs and their features, I am not even sure a MIB is the right tool. There usually is a software tool that controls migration of VMs and I am not sure such a tool will rely on information provided by MIBs. In fact, I see the MIBs more as a way to do scalable monitoring and hence I am not sure we really have to solve the CPU matching problem. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From rstory@tislabs.com Wed Apr 25 12:45:51 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A80E721F86CA for ; Wed, 25 Apr 2012 12:45:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.524 X-Spam-Level: X-Spam-Status: No, score=-2.524 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ClYmyu-lMwFO for ; Wed, 25 Apr 2012 12:45:50 -0700 (PDT) Received: from walnut.tislabs.com (walnut.tislabs.com [192.94.214.200]) by ietfa.amsl.com (Postfix) with ESMTP id BB44F21F842E for ; Wed, 25 Apr 2012 12:45:49 -0700 (PDT) Received: from nova.tislabs.com (unknown [10.66.1.77]) by walnut.tislabs.com (Postfix) with ESMTP id 75D3B28B003A; Wed, 25 Apr 2012 15:45:41 -0400 (EDT) Received: from tp.vb.futz.org (localhost.localdomain [127.0.0.1]) by nova.tislabs.com (Postfix) with ESMTP id 345A51F8032; Wed, 25 Apr 2012 15:45:41 -0400 (EDT) Date: Wed, 25 Apr 2012 15:45:32 -0400 From: Robert Story To: Juergen Schoenwaelder Message-ID: <20120425154532.7c45c9f6@tp.vb.futz.org> In-Reply-To: <20120425181103.GA88418@elstar.local> References: <20120425134834.GA87475@elstar.local> <1407744686.2681126.1335374181637.JavaMail.root@zimbra-prod-mbox-2.vmware.com> <20120425181103.GA88418@elstar.local> Organization: SPARTA X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.8; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/JuH3RVPL9_I6lTYjZ2biTFu"; protocol="application/pgp-signature" Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Hypervisor MIB document X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 19:45:51 -0000 --Sig_/JuH3RVPL9_I6lTYjZ2biTFu Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Wed, 25 Apr 2012 20:11:03 +0200 Juergen wrote: JS> For the matching of physical CPUs and their features, I am not even JS> sure a MIB is the right tool. There usually is a software tool that JS> controls migration of VMs and I am not sure such a tool will rely on JS> information provided by MIBs. In fact, I see the MIBs more as a way to JS> do scalable monitoring and hence I am not sure we really have to solve JS> the CPU matching problem. I don't care about solving the matching problem either.. Just reporting the CPU type for each VM is fine with me.. Robert -- Senior Software Engineer SPARTA, Inc., a Parsons Company --Sig_/JuH3RVPL9_I6lTYjZ2biTFu Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) iEYEARECAAYFAk+YVGQACgkQ7/fVLLY1mnggewCfesXwQX9Q28PCg9eIV6C0QG+e KvoAnAyJXmtFFyeiYO1DQS3Q0YpKb/Y8 =e8vu -----END PGP SIGNATURE----- --Sig_/JuH3RVPL9_I6lTYjZ2biTFu-- From mrm@vmware.com Wed Apr 25 17:45:49 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C71EB11E8091 for ; Wed, 25 Apr 2012 17:45:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.599 X-Spam-Level: X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z97PgWntILa8 for ; Wed, 25 Apr 2012 17:45:49 -0700 (PDT) Received: from smtp-outbound-2.vmware.com (smtp-outbound-2.vmware.com [208.91.2.13]) by ietfa.amsl.com (Postfix) with ESMTP id 3DEA311E8076 for ; Wed, 25 Apr 2012 17:45:49 -0700 (PDT) Received: from sc9-mailhost1.vmware.com (sc9-mailhost1.vmware.com [10.113.161.71]) by smtp-outbound-2.vmware.com (Postfix) with ESMTP id 1C92528285; Wed, 25 Apr 2012 17:45:49 -0700 (PDT) Received: from zimbra-prod-mta-1.vmware.com (zimbra-prod-mta-1.vmware.com [10.113.160.173]) by sc9-mailhost1.vmware.com (Postfix) with ESMTP id 17FDF1852D; Wed, 25 Apr 2012 17:45:49 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by zimbra-prod-mta-1.vmware.com (Postfix) with ESMTP id 1345689FD8; Wed, 25 Apr 2012 17:45:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra-prod-mta-1.vmware.com Received: from zimbra-prod-mta-1.vmware.com ([127.0.0.1]) by localhost (zimbra-prod-mta-1.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A60Y5v0qtUIF; Wed, 25 Apr 2012 17:45:49 -0700 (PDT) Received: from zimbra-prod-mbox-2.vmware.com (zimbra-prod-mbox-2.vmware.com [10.113.160.202]) by zimbra-prod-mta-1.vmware.com (Postfix) with ESMTP id 01EF989FCC; Wed, 25 Apr 2012 17:45:49 -0700 (PDT) Date: Wed, 25 Apr 2012 17:45:48 -0700 (PDT) From: Michael MacFaden To: Juergen Schoenwaelder Message-ID: <1068654648.2726192.1335401148898.JavaMail.root@zimbra-prod-mbox-2.vmware.com> In-Reply-To: <20120425181103.GA88418@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.113.60.13] X-Mailer: Zimbra 7.1.3_GA_3374 (ZimbraWebClient - GC18 (Mac)/7.1.3_GA_3346) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Hypervisor MIB document X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2012 00:45:49 -0000 js writes: > There usually is a software tool that > controls migration of VMs and I am not sure such a tool will rely on > information provided by MIBs. Sure. Then strike a use case where an operator would use this mib module as a means 'after the fact' to understand why vm migration between two hosts failed or for collecting an inventory report the physical host systems to obtain this data. FWIW, here is the DMTF standard for reporting on CPUS: http://software.intel.com/sites/manageability/AMT_Implementation_and_Reference_Guide/DOCS/Implementation%20and%20Reference%20Guide/default.htm?turl=HTMLDocuments%2FWS-Management_Class_Reference%2FCIM_Processor.htm >In fact, I see the MIBs more as a way to do scalable monitoring and >hence I am not sure we really have to solve the CPU matching problem. Please define what scaleable monitoring trying to accomplish? who is using this mib module and what are they using it for needs to be expressly stated at some point. Mike From panda@hongo.wide.ad.jp Wed Apr 25 19:56:58 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3B82F21E809D for ; Wed, 25 Apr 2012 19:56:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.904 X-Spam-Level: X-Spam-Status: No, score=0.904 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_JP=1.244, HOST_EQ_JP=1.265, RELAY_IS_203=0.994] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sHHFT-3z7HTv for ; Wed, 25 Apr 2012 19:56:57 -0700 (PDT) Received: from mail.hongo.wide.ad.jp (mail.hongo.wide.ad.jp [203.178.135.13]) by ietfa.amsl.com (Postfix) with ESMTP id 331F221E801B for ; Wed, 25 Apr 2012 19:56:57 -0700 (PDT) Received: from dhcp-236-239.elab.ic.i.u-tokyo.ac.jp (dhcp-236-239.elab.ic.i.u-tokyo.ac.jp [133.11.236.239]) by mail.hongo.wide.ad.jp (Postfix) with ESMTPSA id 33158108DF47; Thu, 26 Apr 2012 11:56:55 +0900 (JST) Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: text/plain; charset=us-ascii From: Hirochika Asai In-Reply-To: <20120425093245.GA86754@elstar.local> Date: Thu, 26 Apr 2012 11:56:54 +0900 Content-Transfer-Encoding: quoted-printable Message-Id: <6E7E2A8C-AB20-40F7-BBAB-F2FE3DD4FE5A@hongo.wide.ad.jp> References: <17694FEE-3C81-4A9E-9526-A64E39B2B47D@hongo.wide.ad.jp> <20120424185354.GA84350@elstar.local> <20120425093245.GA86754@elstar.local> To: Juergen Schoenwaelder X-Mailer: Apple Mail (2.1257) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Yet another hypervisor MIB X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2012 02:56:58 -0000 On Apr 25, 2012, at 6:32 PM, Juergen Schoenwaelder wrote: > On Tue, Apr 24, 2012 at 08:53:54PM +0200, Juergen Schoenwaelder wrote: >> On Wed, Apr 25, 2012 at 03:44:02AM +0900, Hirochika Asai wrote: >>=20 >>> As I previously posted, we have implemented and been testing a >>> prototype code of yet another hypervisor MIB. Now you can browse >>> the current version of code in python and MIB here: >>> . >>=20 >> I will look at this in more detail tomorrow but it is clear already >> that this is not a valid MIB module. SMIv2 does not allow SEQUENCES >> inside of SEQUENCES. Anyway, I will try to figure out what the >> modeling differences are. >=20 > Ignoring a number of technical issues, I believe the major differences > are that >=20 > a) your MIB duplicate objects already found in other MIBs (SNMPv2-MIB, > IF-MIB) which I tried to avoid >=20 > b) your MIB exports information about each virtual CPU assigned to a > VM - I have no clear opinion whether that is valuable or not, in > particular since CPU states may change pretty quickly (it would > probably then make more sense how much type a CPU has spent in the > various states) >=20 > c) your MIB explicitely models a notion of disks as storage units > which is probably a good thing >=20 > Do you agree? Is this is fair summary? Yes, and thank you for your review. I have comments about b). As for virtual CPUs, I agree that the state of each vCPUs is not much valuable for SNMP because CPU state frequently changes. I defined this because it may be valuable for local operation tools like the "top" command in case that this MIB object is locally used through a lighter interface, not SNMP, but I also think it's not much valuable. I think CPU time of each virtual CPU is important to monitor its CPU utilization. In our cloud frontend system, we monitor the virtual CPU time to provide CPU utilization graph like = . --=20 Hirochika Asai , The University of Tokyo From panda@hongo.wide.ad.jp Wed Apr 25 20:00:41 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5630F21E801B for ; Wed, 25 Apr 2012 20:00:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.904 X-Spam-Level: X-Spam-Status: No, score=0.904 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_JP=1.244, HOST_EQ_JP=1.265, RELAY_IS_203=0.994] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qSfqQsz+I6Eb for ; Wed, 25 Apr 2012 20:00:41 -0700 (PDT) Received: from mail.hongo.wide.ad.jp (mail.hongo.wide.ad.jp [203.178.135.13]) by ietfa.amsl.com (Postfix) with ESMTP id D153121E8018 for ; Wed, 25 Apr 2012 20:00:40 -0700 (PDT) Received: from dhcp-236-239.elab.ic.i.u-tokyo.ac.jp (dhcp-236-239.elab.ic.i.u-tokyo.ac.jp [133.11.236.239]) by mail.hongo.wide.ad.jp (Postfix) with ESMTPSA id 4023F108E8AA; Thu, 26 Apr 2012 12:00:40 +0900 (JST) Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: text/plain; charset=us-ascii From: Hirochika Asai In-Reply-To: <20120425130244.GA86934@elstar.local> Date: Thu, 26 Apr 2012 12:00:39 +0900 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20120424185354.GA84350@elstar.local> <1848448462.2607134.1335302315758.JavaMail.root@zimbra-prod-mbox-2.vmware.com> <20120425071057.GA85712@elstar.local> <20120425130244.GA86934@elstar.local> To: Juergen Schoenwaelder X-Mailer: Apple Mail (2.1257) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Yet another hypervisor MIB X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2012 03:00:41 -0000 On Apr 25, 2012, at 10:02 PM, Juergen Schoenwaelder wrote: > On Wed, Apr 25, 2012 at 08:04:02PM +0900, Hirochika Asai wrote: >>=20 >> Only one exception is the case that a hypervisor bypasses "PCI" = (i.e., >> physical NIC) to a VM using IOMMU. In this case, hypervisors cannot >> recognize any packets passing through the NIC. >>=20 >=20 > Yes, but in this case the hypervisor can't be expected to report about > traffic it never sees. Hence in this case, a management application > has to read the statistics directly from the virtual machine that is > directly controlling the physical NIC. I want to say the same. Sorry that my expression wasn't good. The management domain of NICs in this case is completely separated from = hypervisors, and consequently, we don't have to care about it. It's VM's operators' = job. --=20 Hirochika Asai , The University of Tokyo From j.schoenwaelder@jacobs-university.de Thu Apr 26 00:44:18 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0C5CF21F87B9 for ; Thu, 26 Apr 2012 00:44:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.175 X-Spam-Level: X-Spam-Status: No, score=-103.175 tagged_above=-999 required=5 tests=[AWL=0.074, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id llyYxKd0nxyv for ; Thu, 26 Apr 2012 00:44:17 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id E77E821F87A9 for ; Thu, 26 Apr 2012 00:44:16 -0700 (PDT) Received: from localhost (demetrius2.jacobs-university.de [212.201.44.47]) by hermes.jacobs-university.de (Postfix) with ESMTP id C3E8920DD5; Thu, 26 Apr 2012 09:44:15 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius2.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id 17OXMHAzEG_q; Thu, 26 Apr 2012 09:44:15 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 397E520DAF; Thu, 26 Apr 2012 09:44:14 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id E5B451EA6A7C; Thu, 26 Apr 2012 09:44:15 +0200 (CEST) Date: Thu, 26 Apr 2012 09:44:15 +0200 From: Juergen Schoenwaelder To: Michael MacFaden Message-ID: <20120426074414.GA89882@elstar.local> Mail-Followup-To: Michael MacFaden , opsawg@ietf.org References: <20120425181103.GA88418@elstar.local> <1068654648.2726192.1335401148898.JavaMail.root@zimbra-prod-mbox-2.vmware.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1068654648.2726192.1335401148898.JavaMail.root@zimbra-prod-mbox-2.vmware.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Hypervisor MIB document X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2012 07:44:18 -0000 On Wed, Apr 25, 2012 at 05:45:48PM -0700, Michael MacFaden wrote: > js writes: > > There usually is a software tool that > > controls migration of VMs and I am not sure such a tool will rely on > > information provided by MIBs. > > Sure. Then strike a use case where an operator would use this mib > module as a means 'after the fact' to understand why vm migration > between two hosts failed or for collecting an inventory report the > physical host systems to obtain this data. If I attempt to migrate a VM, would the software tool not report a suitable error explaining why the migration can't be carried out? Are tools really written in such a way that operators need to investigate SNMP MIBs what potential reasons for not migrating a VM were? > FWIW, here is the DMTF standard for reporting on CPUS: > http://software.intel.com/sites/manageability/AMT_Implementation_and_Reference_Guide/DOCS/Implementation%20and%20Reference%20Guide/default.htm?turl=HTMLDocuments%2FWS-Management_Class_Reference%2FCIM_Processor.htm Thanks. There is lots of stuff - so let me try to see what specifically is relevant in this context. I see: Family - An enueration of processor families. Stepping - A free-form string that indicates the revision level of the Processor within the Processor And then there is a collection of clock speeds and states. I do not see something that addresses the CPU matching problem. Anyway, my current thinking is that what we are discussing here really boils down to an extension of the ENTITY-MIB, i.e. an ENTITY-CPU-MIB providing an entPhyCPUTable, sparsely augmenting the entPhysicalTable for entities with entPhysicalClass = cpu. > >In fact, I see the MIBs more as a way to do scalable monitoring and > >hence I am not sure we really have to solve the CPU matching problem. > > Please define what scaleable monitoring trying to accomplish? > who is using this mib module and what are they using it for > needs to be expressly stated at some point. Such text is difficult to write and usually use cases evolve during the work on the module. For me, the prime interest is to enable monitoring and thresholding of virtual machines using existing SNMP tools out there including tracking of resource usage to support longer term load balancing and planning etc. But I am sure there are other usages that people find important and that's fine as well. As we move forward, we will start to understand what is possible and feasible. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From mrm@vmware.com Thu Apr 26 11:41:55 2012 Return-Path: X-Original-To: opsawg@ietfa.amsl.com Delivered-To: opsawg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1B41221E813B for ; Thu, 26 Apr 2012 11:41:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.599 X-Spam-Level: X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y+mFIdPXPw3R for ; Thu, 26 Apr 2012 11:41:54 -0700 (PDT) Received: from smtp-outbound-1.vmware.com (smtp-outbound-1.vmware.com [208.91.2.12]) by ietfa.amsl.com (Postfix) with ESMTP id 3803E21E815D for ; Thu, 26 Apr 2012 11:41:54 -0700 (PDT) Received: from sc9-mailhost1.vmware.com (sc9-mailhost1.vmware.com [10.113.161.71]) by smtp-outbound-1.vmware.com (Postfix) with ESMTP id 7D4E72827C; Thu, 26 Apr 2012 11:41:53 -0700 (PDT) Received: from zimbra-prod-mta-1.vmware.com (zimbra-prod-mta-1.vmware.com [10.113.160.173]) by sc9-mailhost1.vmware.com (Postfix) with ESMTP id 6B751182CC; Thu, 26 Apr 2012 11:41:53 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by zimbra-prod-mta-1.vmware.com (Postfix) with ESMTP id 62E90C17AF; Thu, 26 Apr 2012 11:41:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra-prod-mta-1.vmware.com Received: from zimbra-prod-mta-1.vmware.com ([127.0.0.1]) by localhost (zimbra-prod-mta-1.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GlHxVKTty60w; Thu, 26 Apr 2012 11:41:53 -0700 (PDT) Received: from zimbra-prod-mbox-2.vmware.com (zimbra-prod-mbox-2.vmware.com [10.113.160.202]) by zimbra-prod-mta-1.vmware.com (Postfix) with ESMTP id 4D306C17A0; Thu, 26 Apr 2012 11:41:53 -0700 (PDT) Date: Thu, 26 Apr 2012 11:41:53 -0700 (PDT) From: Michael MacFaden To: Juergen Schoenwaelder Message-ID: <271246897.2783902.1335465713180.JavaMail.root@zimbra-prod-mbox-2.vmware.com> In-Reply-To: <20120426074414.GA89882@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.113.60.13] X-Mailer: Zimbra 7.1.3_GA_3374 (ZimbraWebClient - GC18 (Mac)/7.1.3_GA_3346) Cc: opsawg@ietf.org Subject: Re: [OPSAWG] Hypervisor MIB document X-BeenThere: opsawg@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OPSA Working Group Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2012 18:41:55 -0000 js writes: > If I attempt to migrate a VM, would the software tool not report a > suitable error explaining why the migration can't be carried out? Yes it would. > Are tools really written in such a way that operators need to investigate > SNMP MIBs what potential reasons for not migrating a VM were? Depends on the problem that causes the failure. While operator configuration errors tend to account for the bulk of IT systems failures, networking issues, hardware health issues, software issues (OS/cpu, memory etc contribute on average something like 10-15% of the failure cases. SNMP MIB modules excel at providing near real-time diagnostics, generalized configuration/inventory and performance state. So if one can't query via SNMP mib modules the physical inventory pertinent to running virtual machines then I think that needs to be addressed at some point in a separate mib 4133 or 2970 and discussed in in your front matter of the draft. > Anyway, my current thinking is that what we are discussing here > really boils down to an extension of the ENTITY-MIB, i.e. an ENTITY-CPU-MIB > providing an entPhyCPUTable, sparsely augmenting the entPhysicalTable > for entities with entPhysicalClass = cpu. Ok, sounds good. > For me, the prime interest is to enable monitoring and thresholding of > virtual machines using existing SNMP tools out there including tracking > of resource usage to support longer term load balancing and planning etc. I get monitoring of VMs on a hypervisor and reporting basic resource usage. So I think your needs align with operators deploying hypervisors that do not use clustering protocols though. Today's commercial hypervisors are running with clustering protocols that use a wooly set of internal/possibly proprietary metrics that provide the input to the load balancing algorithms running distributed across a set(cluster) of hypervisors. In addition to load balancing vms they deal with hypervisors that die suddenly or get disconnected(split-brain). Hypervisors under control of clustering software then automatically will shut down vms a or power up a copy of a given vm lost from another system automatically as configured. Further like BGP meds are to network admins, host admins teak lots of knobs on resource pools at a cluster level not hypervisor level. There's another bit of software system (call it XVCX) controlling the various hypervisors and this XVCX may also have its own SNMP agent. Here's how large datacenters are setup today -- Operators now build datacenters by ordering racks of servers at a time. A truck delivers a full 46RU high rack pre-loaded and wired up with servers. Admins simply connect the built-in (top-of-rack or TOR) ethernet switch to their network and supply power. The x86 systems PXE boot, get their hypervisor, configuration and then connect to XVCX and join their respective clusters, optionally connect to their san storage and vms being powering up or w/out power up are migrated to the storage of the new hardware, and all controlled by the clustering protocols which are controlled from the XVCX. > But I am sure there are other > usages that people find important and that's fine as well. As we move > forward, we will start to understand what is possible and feasible. Ok, let's just be clear to summarize the discussion and document why something is there or not either in the draft or otherwise. Mike