From nobody Tue Jul 2 05:49:34 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 21FFB120273 for ; Tue, 2 Jul 2019 05:49:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NXcRVpAraJKX for ; Tue, 2 Jul 2019 05:49:18 -0700 (PDT) Received: from fgont.go6lab.si (fgont.go6lab.si [91.239.96.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3A8EF120098 for ; Tue, 2 Jul 2019 05:49:18 -0700 (PDT) Received: from [192.168.1.114] (unknown [160.176.57.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by fgont.go6lab.si (Postfix) with ESMTPSA id BA08D93F84; Tue, 2 Jul 2019 14:49:15 +0200 (CEST) To: IETF SecDispatch , "pearg@irtf.org" From: Fernando Gont Openpgp: preference=signencrypt Autocrypt: addr=fgont@si6networks.com; prefer-encrypt=mutual; keydata= mQINBE5so2gBEACzBQBLUy8nzgAzSZn6ViXT6TmZBFNYNqTpPRvTVtUqF6+tkI+IEd9N2E8p pXUXCd0W4dkxz6o7pagnK63m4QSueggvp881RVVHOF8oTSHOdnGxLfLeLNJFKE1FOutU3vod GK/wG/Fwzkv9MebdXpMlLV8nnJuAt66XGl/lU1JrNfrKO4SoYQi4TsB/waUQcygh7OR/PEO0 EttiU8kZUbZNv58WH+PAj/rdZCrgUSiGXiWUQQKShqKnJxLuAcTcg5YRwL8se/V6ciW0QR9i /sr52gSmLLbW5N3hAoO+nv1V/9SjJAUvzXu43k8sua/XlCXkqU7uLj41CRR72JeUZ4DQsYfP LfNPC98ZGTVxbWbFtLXxpzzDDT8i3uo7w1LJ2Ij/d5ezcARqw01HGljWWxnidUrjbTpxkJ9X EllcsH94mer728j/HKzC9OcTuz6WUBP3Crgl6Q47gY5ZIiF0lsmd9/wxbaq5NiJ+lGuBRZrD v0dQx9KmyI0/pH2AF8cW897/6ypvcyD/1/11CJcN+uAGIrklwJlVpRSbKbFtGC6In592lhu7 wnK8cgyP5cTU+vva9+g6P1wehi4bylXdlKc6mMphbtSA+T3WBNP557+mh3L62l4pGaEGidcZ DLYT2Ud18eAJmxU3HnM8P3iZZgeoK7oqgb53/eg96vkONXNIOwARAQABtCVGZXJuYW5kbyBH b250IDxmZ29udEBzaTZuZXR3b3Jrcy5jb20+iQJBBBMBAgArAhsjBQkSzAMABgsJCAcDAgYV CAIJCgsEFgIDAQIeAQIXgAUCTmylpQIZAQAKCRCuJQ1VHU50kv7wD/9fuNtTfxSLk3B3Hs3p ixTy8YXVjdkVwWlnJjFd7BOWmg7sI+LDhpjGfT6+ddOiwkumnvUZpObodj4ysH0i8c7P4C5t F9yu7WjklSlrB5Rth2CGChg5bKt541z2WHkFFxys9qBLmCSYDeKQkzLqhCjIUJizY2kOJ2GI MnSFDzJjhSFEh//oW830Y8fel1xnf/NVF+lBVtRMtMOfoWUqDjvP3sJ1G4zgkDCnF0CfncLx +hq2Mv26Uq9OTzvLH9aSQQ/f067BOkKAJKsfHdborX4E96ISTz57/4xECRSMr5dVsKVm4Y// uVIsb+L5z+a32FaiBZIAKDgnJO7Z8j6CV5e5yfuBTtX52Yi9HjYYqnYJGSDxYd6igD4bWu+7 xmJPHjkdqZgGV6dQIgiUfqkU+s5Cv350vK48CMaT/ZLo2BdsMhWsmaHmb+waePUMyq6E4E9x 9Js+EJb9ZiCfxS9exgieZQpet1L36IvhiwByvkQM009ywfa30JeMOltUtfLi5V06WQWsTzPL 5C+4cpkguSuAJVDTctjCA0moIeVDOpJ8WH9voQ4IeWapQnX35OIoj1jGJqqYdx65gc1ygbyx b8vw+pJ9E5GLse5TQnYifOWpXzX9053dtbwp/2OVhU4KLlzfCPCEsoTyfu9nIZxdI2PMwiL5 M85BfjX4NmwBLmPGoLkCDQRObKNoARAAqqXCkr250BchRDmi+05F5UQFgylUh10XTAJxBeaQ UNtdxZiZRm6jgomSrqeYtricM9t9K0qb4X2ZXmAMW8o8AYW3RrQHTjcBwMnAKzUIEXXWaLfG cid/ygmvWzIHgMDQKP+MUq1AGQrnvt/MRLvZLyczAV1RTXS58qNaxtaSpc3K/yrDozh/a4pu WcUsVvIkzyx43sqcwamDSBb6U8JFoZizuLXiARLLASgyHrrCedNIZdWSx0z0iHEpZIelA2ih AGLiSMtmtikVEyrJICgO81DkKNCbBbPg+7fi23V6M24+3syHk3IdQibTtBMxinIPyLFF0byJ aGm0fmjefhnmVJyCIl/FDkCHprVhTme57G2/WdoGnUvnT7mcwDRb8XY5nNRkOJsqqLPemKjz kx8mXdQbunXtX9bKyVgd1gIl+LLsxbdzRCch773UBVoortPdK3kMyLtZ4uMeDX3comjx+6VL bztUdJ1Zc9/njwVG8fgmQ+0Kj5+bzQfUY+MmX0HTXIx3B4R1I1a8QoOwi1N+iZNdewV5Zfq+ 29NlQLnVPjCRCKbaz9k6RJ2oIti55YUI6zSsL3lmlOXsRbXN5bRswFczkNSCJxJMlDiyAUIC WOay7ymzvgzPa+BY/mYn94vRaurDQ4/ljOfj6oqgfjts+dJev4Jj89vp8MQI3KJpZPEAEQEA AYkCJQQYAQIADwUCTmyjaAIbDAUJEswDAAAKCRCuJQ1VHU50km4xEACho45PZrUjY4Zl2opR DFNo5a6roTOPpgwO9PcBb3I5F8yX2Dnew+9OhgWXbBhAFq4DCx+9Gjs43Bn60qbZTDbLGJ/m 8N4PwEiq0e5MKceYcbetEdEUWhm5L6psU9ZZ82GR3UGxPXYe+oifEoJjOXQ39avf9S8p3yKP Diil0E79rn7LbJjMcgMLyjFg9SDoJ6pHLtniJoDhEAaSSgeV7Y745+gyMIdtQmrFHfqrFdjq D6G0HE+Z68ywc5KN67YxhvhBmSycs1ZSKAXv1zLDlXdmjHDHkU3xMcB+RkuiTba8yRFYwb/n j62CC4NhFTuIKOc4ta3dJsyXTGh/hO9UjWUnmAGfd0fnzTBZF8Qlnw/8ftx5lt4/O+eqY1EN RITScnPzXE/wMOlTtdkddQ+QN6xt6jyR2XtAIi7aAFHypIqA3lLI9hF9x+lj4UQ2yA9LqpoX 6URpPOd13JhAyDe47cwsP1u9Y+OBvQTVLSvw7Liu2b4KjqL4lx++VdBi7dXsjJ6kjIRjI6Lb WVpxe8LumMCuVDepTafBZ49gr7Fgc4F9ZSCo6ChgQNLn6WDzIkqFX+42KuHz90AHWhuW+KZR 1aJylERWeTcMCGUSBptd48KniWmD6kPKpzwoMkJtEXTuO2lVuborxzwuqOTNuYg9lWDl7zKt wPI9brGzquUHy4qRrA== Cc: =?UTF-8?Q?Iv=c3=a1n_Arce_=28Quarkslab=29?= Message-ID: <3ef209f1-19f4-1ae9-d04b-a9bfa94810d4@si6networks.com> Date: Tue, 2 Jul 2019 13:49:08 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Archived-At: Subject: [Pearg] Predictable numeric IDs (here we go again) X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jul 2019 12:49:24 -0000 Folks, The issue of predictable numeric IDs keeps popping up. This time: * In 6man, wrt embedding MAC addresses in Interface Identifiers: https://mailarchive.ietf.org/arch/msg/ipv6/U3ZCkirsTIs6uvacyo4WsZPdPI0 and, * In NTP, wrt to employing predictable port numbers unnecessarily. See: https://mailarchive.ietf.org/arch/browse/static/ntp/2019-05/ We authored an I-D (https://tools.ietf.org/html/draft-gont-predictable-numeric-ids) to proactively address these issues, and even provided a split version of it: * https://tools.ietf.org/html/draft-gont-numeric-ids-history * https://tools.ietf.org/html/draft-gont-numeric-ids-generation * https://tools.ietf.org/html/draft-gont-numeric-ids-sec-considerations We did presentations at the sec area (back in Buenos Aires, IIRC) and secdispatch -- in different timeframes, but this still seems to be stalled somewhere. We hope the IETF gets to work on this, instead of having to re-hash the same discussions over and over again every time, for different protocols, in different wgs, and then having to patch specs where applicable. Thanks, -- Fernando Gont SI6 Networks e-mail: fgont@si6networks.com PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492 From nobody Mon Jul 8 00:56:11 2019 Return-Path: X-Original-To: pearg@irtf.org Delivered-To: pearg@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 7FF8C120091; Sun, 7 Jul 2019 17:03:20 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: pearg@irtf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.98.3 Auto-Submitted: auto-generated Precedence: bulk Reply-To: pearg@irtf.org Message-ID: <156254420044.4995.7471139515518776754@ietfa.amsl.com> Date: Sun, 07 Jul 2019 17:03:20 -0700 Archived-At: X-Mailman-Approved-At: Mon, 08 Jul 2019 00:56:10 -0700 Subject: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-00.txt X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2019 00:03:21 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Privacy Enhancements and Assessments Research Group RG of the IRTF. Title : Guidelines for Performing Safe Measurement on the Internet Author : Iain R. Learmonth Filename : draft-irtf-pearg-safe-internet-measurement-00.txt Pages : 8 Date : 2019-07-07 Abstract: Researchers from industry and academia often use Internet measurements as part of their work. While these measurements can give insight into the functioning and usage of the Internet, they can come at the cost of user privacy. This document describes guidelines for ensuring that such measurements can be carried out safely. Note Comments are solicited and should be addressed to the research group's mailing list at pearg@irtf.org and/or the author(s). The sources for this draft are at: https://github.com/irl/draft-safe-internet-measurement The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-irtf-pearg-safe-internet-measurement/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-irtf-pearg-safe-internet-measurement-00 https://datatracker.ietf.org/doc/html/draft-irtf-pearg-safe-internet-measurement-00 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Mon Jul 8 06:08:53 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F0964120199 for ; Mon, 8 Jul 2019 06:08:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=messagingengine.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ydl6_ZZU7dIu for ; Mon, 8 Jul 2019 06:08:39 -0700 (PDT) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 401E01201CA for ; Mon, 8 Jul 2019 06:08:39 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 310DE21D25 for ; Mon, 8 Jul 2019 09:08:37 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Mon, 08 Jul 2019 09:08:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=qimG0E AfQ3HJkE7BqfkN+/NQqWiPpRQ7+ccHn0t+gJY=; b=R+weGo1f+BIgpI4aV56GIt WGPghgSsAfbXbhpqT8Z7ndiG0B4fcWVFTb6Jaq7ujfPesDA2tXPoALB1LG/MSHGV HpA7m/+rIj+yAnCxFHrQ3YOdTqvYwkhV0G+HcwRpIIxu43SSbOx43SXLK0oISOjT LQZgQK5F84/fFTDViTLoau3KWaNYvWhVo0aUNYAkVypGINWFEkbkvWgvqZUIAhXn GsFViY4MnjcLZ9wJ0zb2d3zcRDN0X9kgJn6cnwxCJcGm1fh1cnPDfI8WXjlWQYim WVK82iFesRDUMbqo7nmyas6MjSVLrp5AfbocngQrQ5/jPVBM40+W0cJdTStHOYHw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrgedtgdeivdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepvfhfhffuohfkffgfgggjtgesghdtre fotdefjeenucfhrhhomhepkfgrihhnucfnvggrrhhmohhnthhhuceoihhrlhesthhorhhp rhhojhgvtghtrdhorhhgqeenucffohhmrghinhepihgvthhfrdhorhhgnecukfhppeekge drledvrdeguddrvddtudenucfrrghrrghmpehmrghilhhfrhhomhepihhrlhesthhorhhp rhhojhgvtghtrdhorhhgnecuvehluhhsthgvrhfuihiivgeptd X-ME-Proxy: Received: from [172.22.152.150] (shiftout.plus.com [84.92.41.201]) by mail.messagingengine.com (Postfix) with ESMTPA id 790F580060 for ; Mon, 8 Jul 2019 09:08:36 -0400 (EDT) To: pearg@irtf.org References: <156254420044.4995.7471139515518776754@ietfa.amsl.com> From: Iain Learmonth Openpgp: preference=signencrypt Autocrypt: addr=irl@torproject.org; prefer-encrypt=mutual; keydata= mQINBFZp8zEBEACxOYriD+tEuc3Wpnbh+GGnyiaLEMABBrfn6JlDQphbBq/YTz9M9OPkttjx hLL/yrxlM1nD69XbGKQ9gIL3LEgOz9+OdivPbN+Q5iNMqk/WCQUqd3bCFbbsn1yvoTumFy9S 9kYX45Db3jRJoN/Nye6Stf7KKPxHxot14iY+PUR/5Gx5KbeWVKfDtQejGnhxQD73KjrX4wds BAaxnQ7KbjQyUf+IxE+8qSDcyTP+pPqxspVzx+eFqsW5+kK1eJMHxJmY/KsAs6IsGf5lvyDJ JECc2iE0mFS6vc14lGcD7BAYMPRnvlK3OcDlbdJS3ZU0LQu3/AplM7cNcesq2Btm06OUTsbj 10ZiyLi7Q0WZRuUbn7t3jOQVyOlNfjUpJhKPMMobBL2R0KzcptJbUrKc08wZD/TPaXuHKWAE JuA6kFMXtHhV8Qhxz5/d2KUA8ex+zpVd2xSR6q4llcYu1w8zHZtLN+YKSmjjKs+AjiTrCMYs OYxt4cwxuaIIhBNvCC9WqZOxHX7YHmpVcSV6K9Wwhk9mVIU3Ii0G2HWs6OQ0vIueCDGMEdVk ig/a7cVlfXNz7WuaXuhOJmHz6d6Yk4dFn5mLbEY9cZhBxf5hjCwtp9b6v+ueuptfcnOd+38G 9KH6NyHKZyS4jcd3E6Dp0+9Isbl/EohjPCujevoW3/DlT08OKQARAQABtDRJYWluIFIuIExl YXJtb250aCAoVG9yIFByb2plY3QpIDxpcmxAdG9ycHJvamVjdC5vcmc+iQJUBBMBCgA+AhsD BQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAFiEEqPe6UEHhMzOcuhaWdtWAk/VAq80FAlxe8QQF CQbiS9IACgkQdtWAk/VAq80isw//QGWLLb6wjPW9K/seio9dSVM3gDk6jU7/FhlUuz7oq/6H gPrl7jrxH9PY5jAlkWCK5iPFWpFczUKC+AzzuZQt8VXQp5iYllm1KRrZA6mxpx085Yo3cSLp H9B55IHENo5RFdXMgXZ2aNc6ENG31vB6IZDHXjw9fUxtqyB7fCbIhFtO9pCqwnH/WipISdrl P5/gllUJKsvZiQj8vFfOe2rEHfldhqXxN0ZFvQtQMy0SipqfY2eM47HoDLZ2L5U1iK3iG1fa UWN059Y92zcc9B/AFqHK9xFMObUlWEvqhCGKgullDp4mVF4/OPAC/l02IS85BeFxWbV/HxTY AeoEzO9qrWVitCPTuFC4TQxJq4TbVr/jriFugmbvlE3IIXz5OjqTpyJZw+A8jJ3lwF+V5z21 LdJpEbh+fBBz6tl99Hnn7rjN9WhhsH+BvSC9fCnxs21h+oiGw9zom6blFsLW1yP7mNX2ZRJR DmQs+7kPL4ZS8XekNN82UHXE2pd9M2fSEi5dHK0R9xKJmMSNi/cnEZp1SRx+e+FFDZwzxiFO R30sDEEf+Z1MB/7o5uQXtw59W5Q7p0KycJMfacnb3dulOhFF4Qu1w1K3Iqefq8cQpnlmIGV0 dYT6juPyDlRW19LQVQSTejkLAbkBuXVE0T6dAYVyudJ2lVa3aDvQV5PqL1Y5tdK5Ag0EWmt/ jQEQANhwgR5fq1Slk0T0Sh7Tkc5LjanQH1VuNTxOE7wzcYXrgva2ic4zdhLc2QHP9G4kz0AY /oLxtw5Sj/IMtdzBHKDBXgiqBmvkrz7mOZSQQh6K/JBcKau4MorzB80Z/Z6XQcfFKQh61+8e Im/I7AnJFUELxDe8CYmkJYKVJS9b+i9RNFvXAsamOkClcyXqPYBB9bBI8QlZHweTWDsXJqul xjHLjCWOQyJxfl9xFmlSJtskrLEmprw/PaOtXglrz/2vadn2lL3ack9V1ux9ALa5q8Oc6dnx vA/W3palFpdBoz75FckhRfliYNfdCpgo86w00bJvbJ1f6XfBIBYvsAvrIRWIkMYEmhTYm4xG gEDWz5CvGuuzkF7kJwfPdj1RgVe80JHr9OJc8ZrafBb/p35ldLxmhDi16j2VOj58CKwpi3En nMOMPteDxmCZrWeYKhfwgVAP4/zc4+9vmTp5Wpze00DnQ9MEinmJ2bxf6TyQP36Bf/8zgJVs rpqMOaumbwwmcUh85Q1PXneVWX2ryEv1I7xVMpW4APZDeOMiiHb+EpXPLTPBvXTerd/Pwzb7 WLKNpZI4tl/vh88INdrs3xZd174skDJNaNSnqzUP2kNYcxWZSH/FP4AH1IXvIxwIr9g0RpvY oX55ALOSQOdZ9ioCjf9x9mvnBsehcoEPrgozljzpABEBAAGJBHIEGAEKACYCGwIWIQSo97pQ QeEzM5y6FpZ21YCT9UCrzQUCXF7xLQUJAuC/nwJAwXQgBBkBCgAdFiEE/ps4MHN0fw3t6Aud F4mIfdjVvF0FAlprf40ACgkQF4mIfdjVvF0NJBAAuAVw1aY3IUtp8ZRI5xuhieVFDbye7z86 gP5KW8cAMKTpuowD7E3Rm31R2Pb5dX+vZlunSTBPuzRVnVNd6Kj7CR8BR426L48ogSUfIh4V IWiYxXn4DqqNGow+saPI8VwJ/y80NSb2v+qUdv7us052vs9UBBnI/zl8XrnUNix+B0g+jekT ENEVD9fNJ+9YE9cn80wLGKBx047id1IqgOeJShjE13WRj7sa+LhB8FtoOUeXduQ1sJBZTVJq XW9v0e2PNc37cc8VATrA+3HNSQJY1PETDxjJ1TW8gALpZQrfR+FDu/d2vALdULw/djCeIpl0 dr79isd6mINBKTkEcR8XPQjDUnrnO1U18YETo71N5Z6HpvaV5/XWLUl81wovddpVRvwbwuFP bJE506UiQJrthPKi6s670VBV6u5I+5+h6D+CHp1xp0Bq8PLz0EwqjZwOiLAByBT/ryD2ZY8+ 2rP9gphnEBcCtyp6Fz6Q0KyutpCPv94b4OU0eUTvl3Nroq2GWGGHAV+8r1uXpqdNSHxYmQgy b3H1dvObmE37L3BWrVNzeFb0USNiV04up/55QUrqrz7kLdBO0ZVMVLU3j7e48KcbJG2yzNHZ dnQa+ZMMIRFtm8vzSt+jEIwodi6nOeBgEiqODF8TZPbTc6183ErFoYc/FuhkUh0aR0MLZ92Q bPEJEHbVgJP1QKvNZ94QAK9KrF8hpdlSLTdlQptQ+pfKH2AyC1zyyrSxoxh56inKMgETXLVJ WmEYc1OCLWktUU0qQkCzo6umR7TNVZjx86L6jKKlFTptulQyFVaeaOU8DgFuiqnDXWn5E1cu bp1KCVU9TpBqIC7hN4ifJeY1lVoRGR+x+PCDeYgT+birtIanK5mqKzx74pbLeTTQZALqD8Hr UouxO41NyOGMv4BgZrr9z2ATIEIHZDLihKMEbF6LMjC6oCOgf51x6gV8doVw4ySNc3i/mk6h ztLQBhEc1q5SWfNgXcPDnzzoYsVJMSmV6yMCw3OEwtfZ0BYPQ6hihShOBKVQEHVnGbMIT4md V1S/axXDsLPFLkq1CxZ1L7N8Y5n2lUH00iSo+yWs7lhvt9WKl7sTewjXtibgrAC89uKvIyHe bA0GO9wojlwGAQboCMxZkkADgpHkDNvY675QLWaEs7+JJq4iWwojFh6QrDuOZmR3e9a5xZDb nQScTuPWvhZUtq7ZNZxmjPDdSUomSHe/ahBd+FHNKbgumN3PuXitFTg7LozIZV05oVv9arMs 9KyKGOS6rbim5C3Nx+MitMdr2Vh3tRfHugy0iWFSX5ITkG/RH4fMOzJMgMz/SftZMwMOU4J/ RBiPRyeaMBMqwomg/7o+NjulWHrUlAX+DAsyVtV+K9OLkglikNQtkG0MuQINBFprf6sBEADO ANf22so7uoGcvok2TM/T8BHI5+TqHEc4hVe+JGGJ1ZnWlgtGmpOs0fOQj3WAgGI0ZmTqMuoz KF/K9ljbjaMXsLD+JIBTD4rINy60VX2zHhmWhNaOcJvq+wbuHx0tMbhqsTStGnSkvRhH61nc MqVqlTTTLVQQSxKl9D2l7ZGwEPLHRFlydTOOix+F+Y1ehxYLVaPkaycs8wvgjYsDLo3T8Tmu OL+rcEfvxJ6lT2V5I51xqievqoBazAfXvA8FW/0G+Z9LUJmViOVluWg3xjP8okKYgOkOeX00 vMBCVaiEA08oaxY0ebS7uBEgppjWSwn+WAhB+6spd67d4W+DmAnM262lxFMhVYhXpfeV9zyg ULQOofdE6xtFkaxr/y8xQ4Bf7zX8ko6X9aFQFB/vc+zUtjzjg4VaQvWrThjaHlbEKR55MDxJ u2T9S7g5bR4zxZNv36gwlIdmx33a1AeR1nGcWa/7OtoS53+lUwyFVWLOnucqKh71Y38AAMd5 L9Fsb+ArQem71knEUTC+HvBGkPb2Y2PzGnnzhZyC8zgE8AjVD0wB+RMDNI3+fIW6biKAHDqr S6ZCVkzJ1R9nOjXMHRYZ5qlG+rCOeu6Jp4yNwp46z4PqiiLJ9NtmdNttLCEn5PDVF3g9g811 JcadvFVH1ZELoDGWMg3Q+QOHQBFYj7cj2QARAQABiQI8BBgBCgAmAhsMFiEEqPe6UEHhMzOc uhaWdtWAk/VAq80FAlxe8S0FCQLgv4EACgkQdtWAk/VAq81y0Q//YAbM50BQSjWAyVk52AMv PDEalSOau9jd1zllgurs1+BD4eWkNOJtN9BKaR5pd5VsjYPhL6ypUv/lRsyDPM3h7knuCZmI XYSBkloDqutZ020kR1jNhIZLL3Nma1xqh+oMsB7M+1AcT+Nlez74WgYjNyb7uuFWr99pUBuJ KbVjBq772Jjz3U0q7sa35wQvLaMC+AG/8L+e/fos6jgJhvIq2QeVLuVyeKVlEEO6tpe07Q+e mdCNu4l5eNbWAuvOgvtbVCLz5C4nLNZzr27u0thXzVhZ8ovAzdQTx6d6X4YI3fwBhPxXpqkg eTo3+B8D5lKUgkSZ9xTo+JP9yjCWwytezTL3oElL5LBVlYxdnpDWwOx8rCIgi5OlfVwU03/P O7IrEkqU5jvhZYT4c5/ktCx99kJcYwPUbT4wuhI1JnXyILrtmfC9a0vK8hpIUP4HU9GJTnVH Bl/XZZ6OJWEmSlJWm8KbageOcfLewc3BdWFnY6k5TmrzsFbamaBOgZmJgkNgylEyTjxVnLTk i5wciAp+N9K+tOP5FvgR943RSESZrxHo+XAV+BAK6K6Oae/jlrzGzNLAyJKWjefhyGL3AOru EBhSQaRemp/IUmyKREfowz3f8Lw2NFyJuWGzDgo9/1fmqmZ1JegtfWw0uPHB/rooajODBaol obxU65Qt1SOk2Ws= Organization: Tor Project Message-ID: <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> Date: Mon, 8 Jul 2019 14:08:34 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: <156254420044.4995.7471139515518776754@ietfa.amsl.com> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="7kHbH3IksT4VLrmBtZStWmtajSTCv1mWA" Archived-At: Subject: Re: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-00.txt X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2019 13:08:51 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --7kHbH3IksT4VLrmBtZStWmtajSTCv1mWA Content-Type: multipart/mixed; boundary="GK9GoEmoTevQZvciB1arOewy15xiHHW3R"; protected-headers="v1" From: Iain Learmonth To: pearg@irtf.org Message-ID: <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> Subject: Re: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-00.txt References: <156254420044.4995.7471139515518776754@ietfa.amsl.com> In-Reply-To: <156254420044.4995.7471139515518776754@ietfa.amsl.com> --GK9GoEmoTevQZvciB1arOewy15xiHHW3R Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Hi All, I wasn't sure that I'd have the time today to work on this, and I wanted to make sure the chairs had time to approve the rename before the cut-off, so I've uploaded two versions in quick succession. Sorry if you've already read the new -00 only to find that a -01 has already appeared. If you'd like to diff the new -01 against the previous individual draft, see: https://tools.ietf.org/rfcdiff?url1=3Ddraft-learmonth-pearg-safe-internet= -measurement-02.txt&url2=3Ddraft-irtf-pearg-safe-internet-measurement-01.= txt Big changes are the inclusion of "case studies" in to help describe situations in which differing levels of consent might be considered acceptable for measurements to be performed, and also expansion of the data minimalization section. Instead of redefining anonymity, psuedonymity and the thresholds needed to say that a system can achieve these things, it may be useful to reference draft-irtf-hrpc-anonymity depending on how that draft develops.= The guidelines on where consent is required do belong in this draft, but the definition of informed consent, proxy consent and implied consent may instead be in an HRPC draft. I've not looked extensively through those drafts yet. As always, comments are welcome either directly to me or via the PEARG li= st. I won't be at this IETF meeting but am happy to chat by mail or set up a call with anyone on this topic. Thanks, Iain. --GK9GoEmoTevQZvciB1arOewy15xiHHW3R-- --7kHbH3IksT4VLrmBtZStWmtajSTCv1mWA Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEE/ps4MHN0fw3t6AudF4mIfdjVvF0FAl0jQFIACgkQF4mIfdjV vF3ilQ//TQQhrXuuydVtUQ+DdujUzq2UztHZPFhxxb8Za+IuMB6TVbneqNoJnC56 y8chtFYHnM8u27f1mcR16jFDfU7LGzn352aq/ctRqt+TEh/V9POkPvXFEOPS43f3 HyL6HNq1eK6LyKrKlRTlx531yQ3P2vEQewN88QkcChiVjCzZ2BlcWEsCPHXESj6+ QBhhfleb6/5iPm7ZwQZN0pzsDHVxYgeskTUpv/iKQpt/nWIcfFf7Q52CVMy7zaV3 WoieDE7yL+rMTdq0UpNwG6ueSYZVTUoVtjNaCrDdlBJR+ZF3gm4faDdUfUjp+9A1 wBBrweiivasDJVdRZTcDrO7WY9+NLE9ZMYt2rBsOSyp8JwSjeQvB5HFdcHjrKqoR DMEwCRInwW+Rc93/aoq+1sALfNKQbiuNuyYpcUOqE01ofGYqlnbFj6L8n5ztkYVO LrMjRXE/+8jgH4RUdrESzrwQ4khwG1aEuobzqM2K5GDJOr5CFRU8YonokYz+bzZY ila56BIZjTlgae2CfTmvzz1OWmf0SsWLAi57uJQbHXvz8nOl+bptHXuTuuv96otk X0Zzx+1QJitrLPXxEuoSMRNOUfMnrGc3rDGZyffaMZMmCOhwjKfBTGo34uRYdXA/ Ke2A5h9P2jcfYnd8a9tqVxcNPYpjbC6SR3XlU/a1CVzmfwaj7NA= =RO7k -----END PGP SIGNATURE----- --7kHbH3IksT4VLrmBtZStWmtajSTCv1mWA-- From nobody Mon Jul 8 06:30:52 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 788A2120182 for ; Mon, 8 Jul 2019 06:30:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.601 X-Spam-Level: X-Spam-Status: No, score=-0.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, PDS_NO_HELO_DNS=1.295, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=rtfm-com.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7UuRoe5J-3T8 for ; Mon, 8 Jul 2019 06:30:48 -0700 (PDT) Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com [IPv6:2a00:1450:4864:20::12b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2A2C3120178 for ; Mon, 8 Jul 2019 06:30:48 -0700 (PDT) Received: by mail-lf1-x12b.google.com with SMTP id c9so9191655lfh.4 for ; Mon, 08 Jul 2019 06:30:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rtfm-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=pbgeKtyjxXASsaZyJGCnrYBvYaYCohH6Z2egU8zU62U=; b=dE2/yY2WvhLdLBoRGEGYSi3/GtJtFQT3jZK6U22JYvQlng7RqXOurf8bU9lF/bDruH yf72jeoFsE+kGVMgFSudrrUyo80BKyw7ie/3+OA/DpPa4KrtgNeQIb2bYKjNkdQ03akn YBSs9KPbA01s2UZwJ0eSZyTBV5wcSKsCe6vZf78BsOulk/SRHo1ubWljbhw67+eihtFL cE0nwGqH98Qk4oQPIvE8B6joMgIQ4nsMBPXutvCgujc4o196FZSo21KJmAnfgyLWSrKB lhlpoPRSQenKWt9gL48yJO699fQ11yOx9b6KNK/ezKX6Mtf5OdYyzvWHN+nf9pVBz2Ty 23BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=pbgeKtyjxXASsaZyJGCnrYBvYaYCohH6Z2egU8zU62U=; b=cjyCYA+4NYWi7h1EVpD3xcwgxa/YeL1pEEr9FBBkSlW/V8W3Prs8Akb1kBonn0j6hM 2wCkczJsu1dsJwr00N60UDw6WI1JW8ijpFUrFqNyOFycylf8DZDItH8e+VJke5snYbQe CxkrPXDtE+SclQrfaaCmvzJIkBhougb3vfQAQnH7Q1jwmpDmDh7dUXGgWeXUi3cy8EEe zvKVAvQsIwX7QVf+sqilfLNonunoCrQ3bov7ugK1/wA4uReYAz3EbCqlQc4pRAOdsibd Tl00YLdIaHwfgjF+eubp+b1CubT4OPMrYiapOBPwJwebuX/djSdsAOadBJiYVjr1vLsE uMjQ== X-Gm-Message-State: APjAAAUihdoxnxl5ht2ixMa7pOM+4yz1HigWV4iMaX8pvqInyg/EztU9 J4G6nKNSjrFJcW/PXESFb7JPVj8/3PrhW/UZkrY5vQ== X-Google-Smtp-Source: APXvYqxzuP5ZEwaAzwfyo9UtwxSj+FDJw7cm1uMOwo578WpCJC6q+Bpm4vX5/4QzTjSJclZt4v721TKORaN8Udzv0M4= X-Received: by 2002:ac2:51a3:: with SMTP id f3mr7072033lfk.94.1562592646270; Mon, 08 Jul 2019 06:30:46 -0700 (PDT) MIME-Version: 1.0 References: <156254420044.4995.7471139515518776754@ietfa.amsl.com> <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> In-Reply-To: <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> From: Eric Rescorla Date: Mon, 8 Jul 2019 06:30:09 -0700 Message-ID: To: Iain Learmonth Cc: pearg@irtf.org Content-Type: multipart/alternative; boundary="0000000000009c1ffb058d2b7307" Archived-At: Subject: Re: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-00.txt X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2019 13:30:51 -0000 --0000000000009c1ffb058d2b7307 Content-Type: text/plain; charset="UTF-8" After a quick look, I see.... Document: draft-irtf-pearg-safe-internet-measurement-01.txt The reduced impact should not be used as an excuse for pushing higher risk updates, only updates that could be considered appropriate to push to all users should be A/B tested. This may just be wordsmithing, but as written, this text is entirely unrealistic. One of the major reasons that one does A/B testing is that you are concerned about risk in the Treatment group (e.g., that there will be a higher risk of failures, crashes, etc.) and the reason you are doing an A/B test is to mitigate that risk. I'm not sure how to phrase this (internally, we usually talk about the code having been appropriately reviewed and landed according to policy), but the current text doesn't work. -Ekr On Mon, Jul 8, 2019 at 6:10 AM Iain Learmonth wrote: > Hi All, > > I wasn't sure that I'd have the time today to work on this, and I wanted > to make sure the chairs had time to approve the rename before the > cut-off, so I've uploaded two versions in quick succession. Sorry if > you've already read the new -00 only to find that a -01 has already > appeared. > > If you'd like to diff the new -01 against the previous individual draft, > see: > > > https://tools.ietf.org/rfcdiff?url1=draft-learmonth-pearg-safe-internet-measurement-02.txt&url2=draft-irtf-pearg-safe-internet-measurement-01.txt > > Big changes are the inclusion of "case studies" in to help describe > situations in which differing levels of consent might be considered > acceptable for measurements to be performed, and also expansion of the > data minimalization section. > > Instead of redefining anonymity, psuedonymity and the thresholds needed > to say that a system can achieve these things, it may be useful to > reference draft-irtf-hrpc-anonymity depending on how that draft develops. > > The guidelines on where consent is required do belong in this draft, but > the definition of informed consent, proxy consent and implied consent > may instead be in an HRPC draft. I've not looked extensively through > those drafts yet. > > As always, comments are welcome either directly to me or via the PEARG > list. > > I won't be at this IETF meeting but am happy to chat by mail or set up a > call with anyone on this topic. > > Thanks, > Iain. > > -- > Pearg mailing list > Pearg@irtf.org > https://www.irtf.org/mailman/listinfo/pearg > --0000000000009c1ffb058d2b7307 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
After a quick look, I see....

Document: draft-irtf-pearg-safe-internet-measurement-01.txt

=C2=A0= =C2=A0The reduced impact should not be used as an excuse for pushing highe= r
=C2=A0 =C2=A0risk updates, only updates that could be considered appro= priate to
=C2=A0 =C2=A0push to all users should be A/B tested.

Th= is may just be wordsmithing, but as written, this text is entirely
unrea= listic. One of the major reasons that one does A/B testing is
that you a= re concerned about risk in the Treatment group (e.g., that
there will be= a higher risk of failures, crashes, etc.) and the
reason you are doing = an A/B test is to mitigate that risk.

I'm not sure how to phrase= this (internally, we usually talk about
the code having been appropriat= ely reviewed and landed according
to policy), but the current text doesn= 't work.

-Ekr



On Mon, Jul 8, 2019 at 6:10 AM Ia= in Learmonth <irl@torproject.org> wrote:
Hi= All,

I wasn't sure that I'd have the time today to work on this, and I w= anted
to make sure the chairs had time to approve the rename before the
cut-off, so I've uploaded two versions in quick succession. Sorry if you've already read the new -00 only to find that a -01 has already
appeared.

If you'd like to diff the new -01 against the previous individual draft= ,
see:

https://tools.ietf.org= /rfcdiff?url1=3Ddraft-learmonth-pearg-safe-internet-measurement-02.txt&= url2=3Ddraft-irtf-pearg-safe-internet-measurement-01.txt

Big changes are the inclusion of "case studies" in to help descri= be
situations in which differing levels of consent might be considered
acceptable for measurements to be performed, and also expansion of the
data minimalization section.

Instead of redefining anonymity, psuedonymity and the thresholds needed
to say that a system can achieve these things, it may be useful to
reference draft-irtf-hrpc-anonymity depending on how that draft develops.
The guidelines on where consent is required do belong in this draft, but the definition of informed consent, proxy consent and implied consent
may instead be in an HRPC draft. I've not looked extensively through those drafts yet.

As always, comments are welcome either directly to me or via the PEARG list= .

I won't be at this IETF meeting but am happy to chat by mail or set up = a
call with anyone on this topic.

Thanks,
Iain.

--
Pearg mailing list
Pearg@irtf.org
https://www.irtf.org/mailman/listinfo/pearg
--0000000000009c1ffb058d2b7307-- From nobody Mon Jul 8 06:47:49 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6358812003F for ; Mon, 8 Jul 2019 06:47:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_NONE=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=messagingengine.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EGXpoIc9tPeC for ; Mon, 8 Jul 2019 06:47:45 -0700 (PDT) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 83FF7120196 for ; Mon, 8 Jul 2019 06:47:45 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 9CB7921BF9; Mon, 8 Jul 2019 09:47:44 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Mon, 08 Jul 2019 09:47:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=OvYBHd U1v1QkeCVwR5hfHQN8jjMvgohlzQJ2DbHybJI=; b=tO2hw/aeuaFHa0ALpiw59m BoiXphkzEM2WHQNnOq7PCV1UZfx85WP4BG7r1gtDhrrR39TBSpkpuPHRgPTMKlcI i2yvVwpHTPfltg7TZxXPtkn5ZCZ0Fn/nhU1wnn4+CiwKUcTQDxmBlLlxWYhcxYfe 3z822JtN5QucdsbslI+iZNeZ03/qPz9xAmjBxLR/dsspxpa3sdVU0A7IVL4KXEK5 14L7561aQrKYgjeipbA/6ewxeA9n7COZGASeSTOvtcl2gZN1zUpo7FUa071CqeqO q/JCZRp0+SCdC5r8N0D5bTgEnabiql+vfDlU91peGVkMm0LBVPNnHZxw0PSS7ddw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrgedtgdejudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepvfhfhfhouffkffgfgggjtgesghdtre fotdefjeenucfhrhhomhepkfgrihhnucfnvggrrhhmohhnthhhuceoihhrlhesthhorhhp rhhojhgvtghtrdhorhhgqeenucfkphepkeegrdelvddrgedurddvtddunecurfgrrhgrmh epmhgrihhlfhhrohhmpehirhhlsehtohhrphhrohhjvggtthdrohhrghenucevlhhushht vghrufhiiigvpedt X-ME-Proxy: Received: from [172.22.152.150] (shiftout.plus.com [84.92.41.201]) by mail.messagingengine.com (Postfix) with ESMTPA id 8E3A580066; Mon, 8 Jul 2019 09:47:43 -0400 (EDT) To: Eric Rescorla Cc: pearg@irtf.org References: <156254420044.4995.7471139515518776754@ietfa.amsl.com> <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> From: Iain Learmonth Openpgp: preference=signencrypt Autocrypt: addr=irl@torproject.org; prefer-encrypt=mutual; keydata= mQINBFZp8zEBEACxOYriD+tEuc3Wpnbh+GGnyiaLEMABBrfn6JlDQphbBq/YTz9M9OPkttjx hLL/yrxlM1nD69XbGKQ9gIL3LEgOz9+OdivPbN+Q5iNMqk/WCQUqd3bCFbbsn1yvoTumFy9S 9kYX45Db3jRJoN/Nye6Stf7KKPxHxot14iY+PUR/5Gx5KbeWVKfDtQejGnhxQD73KjrX4wds BAaxnQ7KbjQyUf+IxE+8qSDcyTP+pPqxspVzx+eFqsW5+kK1eJMHxJmY/KsAs6IsGf5lvyDJ JECc2iE0mFS6vc14lGcD7BAYMPRnvlK3OcDlbdJS3ZU0LQu3/AplM7cNcesq2Btm06OUTsbj 10ZiyLi7Q0WZRuUbn7t3jOQVyOlNfjUpJhKPMMobBL2R0KzcptJbUrKc08wZD/TPaXuHKWAE JuA6kFMXtHhV8Qhxz5/d2KUA8ex+zpVd2xSR6q4llcYu1w8zHZtLN+YKSmjjKs+AjiTrCMYs OYxt4cwxuaIIhBNvCC9WqZOxHX7YHmpVcSV6K9Wwhk9mVIU3Ii0G2HWs6OQ0vIueCDGMEdVk ig/a7cVlfXNz7WuaXuhOJmHz6d6Yk4dFn5mLbEY9cZhBxf5hjCwtp9b6v+ueuptfcnOd+38G 9KH6NyHKZyS4jcd3E6Dp0+9Isbl/EohjPCujevoW3/DlT08OKQARAQABtDRJYWluIFIuIExl YXJtb250aCAoVG9yIFByb2plY3QpIDxpcmxAdG9ycHJvamVjdC5vcmc+iQJUBBMBCgA+AhsD BQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAFiEEqPe6UEHhMzOcuhaWdtWAk/VAq80FAlxe8QQF CQbiS9IACgkQdtWAk/VAq80isw//QGWLLb6wjPW9K/seio9dSVM3gDk6jU7/FhlUuz7oq/6H gPrl7jrxH9PY5jAlkWCK5iPFWpFczUKC+AzzuZQt8VXQp5iYllm1KRrZA6mxpx085Yo3cSLp H9B55IHENo5RFdXMgXZ2aNc6ENG31vB6IZDHXjw9fUxtqyB7fCbIhFtO9pCqwnH/WipISdrl P5/gllUJKsvZiQj8vFfOe2rEHfldhqXxN0ZFvQtQMy0SipqfY2eM47HoDLZ2L5U1iK3iG1fa UWN059Y92zcc9B/AFqHK9xFMObUlWEvqhCGKgullDp4mVF4/OPAC/l02IS85BeFxWbV/HxTY AeoEzO9qrWVitCPTuFC4TQxJq4TbVr/jriFugmbvlE3IIXz5OjqTpyJZw+A8jJ3lwF+V5z21 LdJpEbh+fBBz6tl99Hnn7rjN9WhhsH+BvSC9fCnxs21h+oiGw9zom6blFsLW1yP7mNX2ZRJR DmQs+7kPL4ZS8XekNN82UHXE2pd9M2fSEi5dHK0R9xKJmMSNi/cnEZp1SRx+e+FFDZwzxiFO R30sDEEf+Z1MB/7o5uQXtw59W5Q7p0KycJMfacnb3dulOhFF4Qu1w1K3Iqefq8cQpnlmIGV0 dYT6juPyDlRW19LQVQSTejkLAbkBuXVE0T6dAYVyudJ2lVa3aDvQV5PqL1Y5tdK5Ag0EWmt/ jQEQANhwgR5fq1Slk0T0Sh7Tkc5LjanQH1VuNTxOE7wzcYXrgva2ic4zdhLc2QHP9G4kz0AY /oLxtw5Sj/IMtdzBHKDBXgiqBmvkrz7mOZSQQh6K/JBcKau4MorzB80Z/Z6XQcfFKQh61+8e Im/I7AnJFUELxDe8CYmkJYKVJS9b+i9RNFvXAsamOkClcyXqPYBB9bBI8QlZHweTWDsXJqul xjHLjCWOQyJxfl9xFmlSJtskrLEmprw/PaOtXglrz/2vadn2lL3ack9V1ux9ALa5q8Oc6dnx vA/W3palFpdBoz75FckhRfliYNfdCpgo86w00bJvbJ1f6XfBIBYvsAvrIRWIkMYEmhTYm4xG gEDWz5CvGuuzkF7kJwfPdj1RgVe80JHr9OJc8ZrafBb/p35ldLxmhDi16j2VOj58CKwpi3En nMOMPteDxmCZrWeYKhfwgVAP4/zc4+9vmTp5Wpze00DnQ9MEinmJ2bxf6TyQP36Bf/8zgJVs rpqMOaumbwwmcUh85Q1PXneVWX2ryEv1I7xVMpW4APZDeOMiiHb+EpXPLTPBvXTerd/Pwzb7 WLKNpZI4tl/vh88INdrs3xZd174skDJNaNSnqzUP2kNYcxWZSH/FP4AH1IXvIxwIr9g0RpvY oX55ALOSQOdZ9ioCjf9x9mvnBsehcoEPrgozljzpABEBAAGJBHIEGAEKACYCGwIWIQSo97pQ QeEzM5y6FpZ21YCT9UCrzQUCXF7xLQUJAuC/nwJAwXQgBBkBCgAdFiEE/ps4MHN0fw3t6Aud F4mIfdjVvF0FAlprf40ACgkQF4mIfdjVvF0NJBAAuAVw1aY3IUtp8ZRI5xuhieVFDbye7z86 gP5KW8cAMKTpuowD7E3Rm31R2Pb5dX+vZlunSTBPuzRVnVNd6Kj7CR8BR426L48ogSUfIh4V IWiYxXn4DqqNGow+saPI8VwJ/y80NSb2v+qUdv7us052vs9UBBnI/zl8XrnUNix+B0g+jekT ENEVD9fNJ+9YE9cn80wLGKBx047id1IqgOeJShjE13WRj7sa+LhB8FtoOUeXduQ1sJBZTVJq XW9v0e2PNc37cc8VATrA+3HNSQJY1PETDxjJ1TW8gALpZQrfR+FDu/d2vALdULw/djCeIpl0 dr79isd6mINBKTkEcR8XPQjDUnrnO1U18YETo71N5Z6HpvaV5/XWLUl81wovddpVRvwbwuFP bJE506UiQJrthPKi6s670VBV6u5I+5+h6D+CHp1xp0Bq8PLz0EwqjZwOiLAByBT/ryD2ZY8+ 2rP9gphnEBcCtyp6Fz6Q0KyutpCPv94b4OU0eUTvl3Nroq2GWGGHAV+8r1uXpqdNSHxYmQgy b3H1dvObmE37L3BWrVNzeFb0USNiV04up/55QUrqrz7kLdBO0ZVMVLU3j7e48KcbJG2yzNHZ dnQa+ZMMIRFtm8vzSt+jEIwodi6nOeBgEiqODF8TZPbTc6183ErFoYc/FuhkUh0aR0MLZ92Q bPEJEHbVgJP1QKvNZ94QAK9KrF8hpdlSLTdlQptQ+pfKH2AyC1zyyrSxoxh56inKMgETXLVJ WmEYc1OCLWktUU0qQkCzo6umR7TNVZjx86L6jKKlFTptulQyFVaeaOU8DgFuiqnDXWn5E1cu bp1KCVU9TpBqIC7hN4ifJeY1lVoRGR+x+PCDeYgT+birtIanK5mqKzx74pbLeTTQZALqD8Hr UouxO41NyOGMv4BgZrr9z2ATIEIHZDLihKMEbF6LMjC6oCOgf51x6gV8doVw4ySNc3i/mk6h ztLQBhEc1q5SWfNgXcPDnzzoYsVJMSmV6yMCw3OEwtfZ0BYPQ6hihShOBKVQEHVnGbMIT4md V1S/axXDsLPFLkq1CxZ1L7N8Y5n2lUH00iSo+yWs7lhvt9WKl7sTewjXtibgrAC89uKvIyHe bA0GO9wojlwGAQboCMxZkkADgpHkDNvY675QLWaEs7+JJq4iWwojFh6QrDuOZmR3e9a5xZDb nQScTuPWvhZUtq7ZNZxmjPDdSUomSHe/ahBd+FHNKbgumN3PuXitFTg7LozIZV05oVv9arMs 9KyKGOS6rbim5C3Nx+MitMdr2Vh3tRfHugy0iWFSX5ITkG/RH4fMOzJMgMz/SftZMwMOU4J/ RBiPRyeaMBMqwomg/7o+NjulWHrUlAX+DAsyVtV+K9OLkglikNQtkG0MuQINBFprf6sBEADO ANf22so7uoGcvok2TM/T8BHI5+TqHEc4hVe+JGGJ1ZnWlgtGmpOs0fOQj3WAgGI0ZmTqMuoz KF/K9ljbjaMXsLD+JIBTD4rINy60VX2zHhmWhNaOcJvq+wbuHx0tMbhqsTStGnSkvRhH61nc MqVqlTTTLVQQSxKl9D2l7ZGwEPLHRFlydTOOix+F+Y1ehxYLVaPkaycs8wvgjYsDLo3T8Tmu OL+rcEfvxJ6lT2V5I51xqievqoBazAfXvA8FW/0G+Z9LUJmViOVluWg3xjP8okKYgOkOeX00 vMBCVaiEA08oaxY0ebS7uBEgppjWSwn+WAhB+6spd67d4W+DmAnM262lxFMhVYhXpfeV9zyg ULQOofdE6xtFkaxr/y8xQ4Bf7zX8ko6X9aFQFB/vc+zUtjzjg4VaQvWrThjaHlbEKR55MDxJ u2T9S7g5bR4zxZNv36gwlIdmx33a1AeR1nGcWa/7OtoS53+lUwyFVWLOnucqKh71Y38AAMd5 L9Fsb+ArQem71knEUTC+HvBGkPb2Y2PzGnnzhZyC8zgE8AjVD0wB+RMDNI3+fIW6biKAHDqr S6ZCVkzJ1R9nOjXMHRYZ5qlG+rCOeu6Jp4yNwp46z4PqiiLJ9NtmdNttLCEn5PDVF3g9g811 JcadvFVH1ZELoDGWMg3Q+QOHQBFYj7cj2QARAQABiQI8BBgBCgAmAhsMFiEEqPe6UEHhMzOc uhaWdtWAk/VAq80FAlxe8S0FCQLgv4EACgkQdtWAk/VAq81y0Q//YAbM50BQSjWAyVk52AMv PDEalSOau9jd1zllgurs1+BD4eWkNOJtN9BKaR5pd5VsjYPhL6ypUv/lRsyDPM3h7knuCZmI XYSBkloDqutZ020kR1jNhIZLL3Nma1xqh+oMsB7M+1AcT+Nlez74WgYjNyb7uuFWr99pUBuJ KbVjBq772Jjz3U0q7sa35wQvLaMC+AG/8L+e/fos6jgJhvIq2QeVLuVyeKVlEEO6tpe07Q+e mdCNu4l5eNbWAuvOgvtbVCLz5C4nLNZzr27u0thXzVhZ8ovAzdQTx6d6X4YI3fwBhPxXpqkg eTo3+B8D5lKUgkSZ9xTo+JP9yjCWwytezTL3oElL5LBVlYxdnpDWwOx8rCIgi5OlfVwU03/P O7IrEkqU5jvhZYT4c5/ktCx99kJcYwPUbT4wuhI1JnXyILrtmfC9a0vK8hpIUP4HU9GJTnVH Bl/XZZ6OJWEmSlJWm8KbageOcfLewc3BdWFnY6k5TmrzsFbamaBOgZmJgkNgylEyTjxVnLTk i5wciAp+N9K+tOP5FvgR943RSESZrxHo+XAV+BAK6K6Oae/jlrzGzNLAyJKWjefhyGL3AOru EBhSQaRemp/IUmyKREfowz3f8Lw2NFyJuWGzDgo9/1fmqmZ1JegtfWw0uPHB/rooajODBaol obxU65Qt1SOk2Ws= Organization: Tor Project Message-ID: <279a7516-a08d-12a6-1693-b49c94c3c2e5@torproject.org> Date: Mon, 8 Jul 2019 14:47:41 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="qPRflXRBrbhoqq8muWHtb46Zo4Y9IcDWk" Archived-At: Subject: Re: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-00.txt X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2019 13:47:48 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --qPRflXRBrbhoqq8muWHtb46Zo4Y9IcDWk Content-Type: multipart/mixed; boundary="GJnOj88XMcBMtwS3PFbmMTXNOQTeQh2x2"; protected-headers="v1" From: Iain Learmonth To: Eric Rescorla Cc: pearg@irtf.org Message-ID: <279a7516-a08d-12a6-1693-b49c94c3c2e5@torproject.org> Subject: Re: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-00.txt References: <156254420044.4995.7471139515518776754@ietfa.amsl.com> <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> In-Reply-To: --GJnOj88XMcBMtwS3PFbmMTXNOQTeQh2x2 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Hi Eric, On 08/07/2019 14:30, Eric Rescorla wrote: > After a quick look, I see.... >=20 > Document: draft-irtf-pearg-safe-internet-measurement-01.txt >=20 > =C2=A0 =C2=A0The reduced impact should not be used as an excuse for pus= hing higher > =C2=A0 =C2=A0risk updates, only updates that could be considered approp= riate to > =C2=A0 =C2=A0push to all users should be A/B tested. >=20 > This may just be wordsmithing, but as written, this text is entirely > unrealistic. One of the major reasons that one does A/B testing is > that you are concerned about risk in the Treatment group (e.g., that > there will be a higher risk of failures, crashes, etc.) and the > reason you are doing an A/B test is to mitigate that risk. The point that I'd like to put across here is that it is not an excuse to be reckless or careless. A/B testing can mitigate risk to reputation perhaps, and sure it can reduce the risk that any individual user is affected by a bad update, but it doesn't mitigate the impact for the users that are affected. > I'm not sure how to phrase this (internally, we usually talk about > the code having been appropriately reviewed and landed according > to policy), but the current text doesn't work. How about: The reduced impact should not be used as an excuse for pushing updates without the same careful consideration and review that would be applied to any software update pushed to all users. and perhaps also adding: In cases which may have high impact for any individual user, simulations or emulations should be carried out in a testing environment before pushing the updates to users via automated channels. Thanks, Iain. --GJnOj88XMcBMtwS3PFbmMTXNOQTeQh2x2-- --qPRflXRBrbhoqq8muWHtb46Zo4Y9IcDWk Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEE/ps4MHN0fw3t6AudF4mIfdjVvF0FAl0jSX0ACgkQF4mIfdjV vF1lEA/7BsR9ni2NUrBu/t592m5+cWZL7v/tHozVtkiyGAbEUignyR12CoYclaKN rQ6SL1WxlrTz6AuSOx1rA3DLJy7mFDDkh/EGMphyza2TFk12vhWGy5meGcvpVn7Q Jk/9gd9ci1o9Ln/f4WipzZ19pnpskAjOcftm5ILjFo30iac+nixVvcmtT8HAeOlN h6qop+Y2NeVNKDGWDP4cMS58pzkVLAD6mzLT6Qiugr4jX3RO318fKFfVXxsUXCJE gJP2vNBNMJN3uJlZ6ShKUB2+STywawqYRxfwX7vOmiZJTFVnrQY7lZ45rQIzFoSG UrdXVaqrA5S9BX6GqDRKkgdxF+bYw70DXBs3EbL55zLR9SNgYR4D1VAcP06yRfIV WXycjgxWfVr1W27PSUw0IESscYON696LhvrjPFgCucfjx2nMT49519LB36St18QX mXCH5vtRmjzj2v25D6GmeyiZY0XBDRrRxhFzHFMQ2TKgQ7sIwIeuNH5HQzwvJdVa 3c9UU+dH5/TpQuTiVXRln/DK13Bq8uJoPIopkidP5sfk8D0Tu9pbKD6B2Qfe7bul H3lcnP1R9xGojn7k6R5r6+9xuO1K+IybwBWMD8ntQDo2we1Uz9BIxNmYL9Ze9dkn Vsu8iFlCYL7JuXeqZ9Ngnsp0BHpYve72k+yIvvtB5u7eToq0rj8= =oC/Q -----END PGP SIGNATURE----- --qPRflXRBrbhoqq8muWHtb46Zo4Y9IcDWk-- From nobody Mon Jul 8 07:16:29 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C6F35120279 for ; Mon, 8 Jul 2019 07:16:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.601 X-Spam-Level: X-Spam-Status: No, score=-0.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, PDS_NO_HELO_DNS=1.295, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=rtfm-com.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ElncGIgvYhfg for ; Mon, 8 Jul 2019 07:16:17 -0700 (PDT) Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [IPv6:2a00:1450:4864:20::12a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0B1EF120273 for ; Mon, 8 Jul 2019 07:16:17 -0700 (PDT) Received: by mail-lf1-x12a.google.com with SMTP id q26so11054986lfc.3 for ; Mon, 08 Jul 2019 07:16:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rtfm-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=6NuEk1AjKGLGlP8OvQdmXQ5PUj0OKKNvmpfUCZso8zg=; b=u4Tz7MxaruI4cQ/0Nibi3G153OOkN3za5sq1P6CoBShM6ttZBhOQy0oKim/0+rPOLo wM2LkbkdsOJfF+2vNymK0y5vOHTylhkv3+N+Dl1uX+o79cmw9NsbamadbGUCrJrvz9sb muKaQ3ogc6stjOGpuImFk5CM5I2m/v0D37spU3ZUo3CXwp/gYieIaulbU9b9dWzx8y3J O5tYj57VbUKG2ZjTjeZ3yq8orvvUNkhG6vq2Z6UurtWlNZFyGjohptFJz456PPivD8so zdB183hR5JKX2lgz7iLCinHX3bEEVJrXbTaXPZm3d1Owv4/PywOjoBG6ieV84Z1Yh8Pw GipA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=6NuEk1AjKGLGlP8OvQdmXQ5PUj0OKKNvmpfUCZso8zg=; b=dnrSFzRFIMiW9g3kXdwIOnrllvHBiTTRc8bf4sEjHreWrPB3ECsYgHclmErFdRWQw/ V5DOCZ+Q01VMzsbmcjc3J9dGkGlMxO5dDb9x5sapEx7ha4mPlTil5AIex1txRSveladd GSsFhNBzKqKw5oBxVitZJRxD9pSfu78ugHktbg/YObEDziQO9wEPuXK5dc0dWxHBmW/p voLXr9F8YIkHkODnpXWOSZe70LstChj4JISFTMN78qtHFVcKN/jlWcDjzINqsasxAufj UYwcnC6kOcL5dVuUOsrvlTETzEptVTVDU1DRirmbL8YCRzWi+JuukFoQHgmXtS0QlAR7 +jEw== X-Gm-Message-State: APjAAAWsdLjs6fu/OgATZOU01m01EonMlvabC0wZEhCvEqt7SdQc+Y/E 4k/uuV3SyFDxlKJOIKZGDM4jAMKfxg+HSW2wFmyNZw== X-Google-Smtp-Source: APXvYqzEMTogcB+6YL+LkNNL/YZrSpGZBPNPejmWNh8uJ6591Bb6PbDpQoQddwCdM3hhiOVohBN4iF8iVQof2aMJC4c= X-Received: by 2002:a19:f24e:: with SMTP id d14mr9315095lfk.184.1562595375170; Mon, 08 Jul 2019 07:16:15 -0700 (PDT) MIME-Version: 1.0 References: <156254420044.4995.7471139515518776754@ietfa.amsl.com> <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> <279a7516-a08d-12a6-1693-b49c94c3c2e5@torproject.org> In-Reply-To: <279a7516-a08d-12a6-1693-b49c94c3c2e5@torproject.org> From: Eric Rescorla Date: Mon, 8 Jul 2019 07:15:39 -0700 Message-ID: To: Iain Learmonth Cc: pearg@irtf.org Content-Type: multipart/alternative; boundary="00000000000043dc16058d2c1632" Archived-At: Subject: Re: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-00.txt X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2019 14:16:28 -0000 --00000000000043dc16058d2c1632 Content-Type: text/plain; charset="UTF-8" On Mon, Jul 8, 2019 at 6:47 AM Iain Learmonth wrote: > Hi Eric, > > On 08/07/2019 14:30, Eric Rescorla wrote: > > After a quick look, I see.... > > > > Document: draft-irtf-pearg-safe-internet-measurement-01.txt > > > > The reduced impact should not be used as an excuse for pushing higher > > risk updates, only updates that could be considered appropriate to > > push to all users should be A/B tested. > > > > This may just be wordsmithing, but as written, this text is entirely > > unrealistic. One of the major reasons that one does A/B testing is > > that you are concerned about risk in the Treatment group (e.g., that > > there will be a higher risk of failures, crashes, etc.) and the > > reason you are doing an A/B test is to mitigate that risk. > > The point that I'd like to put across here is that it is not an excuse > to be reckless or careless. A/B testing can mitigate risk to reputation > perhaps, and sure it can reduce the risk that any individual user is > affected by a bad update, but it doesn't mitigate the impact for the > users that are affected. > This seems like the kind of product question that is well out of scope for PEARG. Software vendors have a wide variety of processes for determining whether a given piece of code is suitable for shipping to their users, ranging (at least) from "some developer thought it was good" to "multiple detailed code reviews". I think we can all agree that defining that is out of scope, but without that, defining what you have to do before you ship to a fraction of your users is largely irrelevant. -Ekr --00000000000043dc16058d2c1632 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Mon, Jul 8, 2019 at 6:47 AM Iain L= earmonth <irl@torproject.org&g= t; wrote:
Hi Eri= c,

On 08/07/2019 14:30, Eric Rescorla wrote:
> After a quick look, I see....
>
> Document: draft-irtf-pearg-safe-internet-measurement-01.txt
>
> =C2=A0 =C2=A0The reduced impact should not be used as an excuse for pu= shing higher
> =C2=A0 =C2=A0risk updates, only updates that could be considered appro= priate to
> =C2=A0 =C2=A0push to all users should be A/B tested.
>
> This may just be wordsmithing, but as written, this text is entirely > unrealistic. One of the major reasons that one does A/B testing is
> that you are concerned about risk in the Treatment group (e.g., that > there will be a higher risk of failures, crashes, etc.) and the
> reason you are doing an A/B test is to mitigate that risk.

The point that I'd like to put across here is that it is not an excuse<= br> to be reckless or careless. A/B testing can mitigate risk to reputation
perhaps, and sure it can reduce the risk that any individual user is
affected by a bad update, but it doesn't mitigate the impact for the users that are affected.

This seems lik= e the kind of product question that is well out of scope for PEARG. Softwar= e vendors have a wide variety of processes for determining whether a given = piece of code is suitable for shipping to their users, ranging (at least) f= rom "some developer thought it was good" to "multiple detail= ed code reviews". I think we can all agree that defining that is out o= f scope, but without that, defining what you have to do before you ship to = a fraction of your users is largely irrelevant.

-Ekr

--00000000000043dc16058d2c1632-- From nobody Mon Jul 8 07:41:59 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0EA3D12018A for ; Mon, 8 Jul 2019 07:41:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id njbrYRYEqM6f for ; Mon, 8 Jul 2019 07:41:55 -0700 (PDT) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 700B2120199 for ; Mon, 8 Jul 2019 07:41:54 -0700 (PDT) Received: from smtp.greenhost.nl ([213.108.110.112]) by smarthost1.greenhost.nl with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1hkUpV-0006bE-CE for pearg@irtf.org; Mon, 08 Jul 2019 16:41:51 +0200 To: pearg@irtf.org References: <156254420044.4995.7471139515518776754@ietfa.amsl.com> <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> <279a7516-a08d-12a6-1693-b49c94c3c2e5@torproject.org> From: Niels ten Oever Openpgp: preference=signencrypt Autocrypt: addr=lists@digitaldissidents.org; prefer-encrypt=mutual; keydata= mQINBFgpcR0BEACnfvNwTMlN+pyZT0AFYhWqxG3N4AoPIeNfbxLQH7dk8ZL7Ls05xtORfnu9 ovoaRrZpDufkMviUFidNYePbQNdgf63vWVgwpQR7utluwWraetcmZOu6tayJuyBK2b6d2Z23 MJAQxfa2/GMlN3QkvobaoyKtgbc8rOCgNla7WwkgtiVJ89xbAUHXPFpKWZluVRjaFh4p5C5r 7E5OvUiEGLQ5Cn2ir2PGIyIVqjB+hLTyaI6dIGCz2jtL0RATjmsmYUX7UkU/pz8MPPC2BJ5P KU9pdXMRBhAStxcph8vCo2ze9xSi3+1/5A2ULVtvO4s0hZ+exbTfMxMg3H5CCRFEEJXlQEXa Cd0ZHvqcv5xq8n9w/Ccd0CqYWATIwyP8Jlzd+BY3QGTWnWlgoAbs3Guh/pFYhEFNuuAF5Jk1 k5OlNGsRE/LQJmbT5SE7AtLJLbWewcHlEyIH+K6J8uVa4ExLXmRy+eRkFaxjGy3fLlUpy1Ee 1kU7VsQ/TZ8g8ujsMzxqsdB6y0TD/kVlWaDqPL6F+b+pm3lAuCBGWM1YZROTG58R6pD7sNVm i0ift4dIttAsg+2KoShm9A8kQ3tACXZDgNPC0l7VOqnVayjnF0RmjGeiX7PjOcLQCZ9a5wAH 5mrXMaKvfszqAVkP9HSrk1QVZOipF6vEimL43Czy7Rp1aUaUwwARAQABtC1OaWVscyB0ZW4g T2V2ZXIgPGxpc3RzQGRpZ2l0YWxkaXNzaWRlbnRzLm9yZz6JAj8EEwEIACkFAlgvB3YCGyMF CQlmAYAHCwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRAO2D86RorIs56yD/44BSJvKnjH ex0nhPDI9nIJlzlnypa4qsniy0obG5GRbVRikT1E1xaz7VBoPs39hCywoIWd6p0hs1PG1Tcj WV0GwNKRt90PPEh6iNJSGjV2Aq3IlME/aUViD9008yfbRSqfsnPXLW1kpCoZNaOSNzpURoM9 OkVU/z4LSLD61SfFFByBne/GkJKt96/fcspBif1GPC//63ZKFrDqQ9JFR6dECAmsKv7baayz MTv3wrTcqpuHcqJIv4vTm8IPx1QiGgEvrMwsPZz/vx8bMdxxxHWgCcbrt+0b3tRzq9ATZwG3 xDiwnJgKd+ioZOC/b5sY69721sqwBmWYyXWVqtqt01xIgNZjr/wixam+l1bTGUgj0rwPWJWx +7Whe25ff+mNNW/UQeCBjZlxoxAJWSr1Pp3n+SQKQ4TLs8wIwHZtcVCffepfHd47CEbnR8Kc Tjm3tlKzSWq4zcUy6BaxHfgn9+HaAM7fwLqx9/WAtSfdmLXJTN+Swy0w/slakD75jl2o7U3e ETjoYQWt+306X2Uly/0ge7VEQ4ySmmbru6U5ainGE95gjsc++s+hvKmMuGYL3h4ijE1RSe/k wgM6/Z1B6JosssdX+KRuuk2A4FHGbcee8LUIJ3C36qyI7s6PJBXi6SjIPN0wpx30P/DUf/Lr o5lmHF03qQ5eeqI8lDwIobWlJbkCDQRYKXEdARAAxYOE3/AFmEfQ0SVVFujYFhZKX+BGXolY ytC2a1soZogVYTIIlypxkRtN+ljteFAY3xX/El7cx5Fxj+uXvLKAm9xQRI/DCug7/NGULMk9 bDK5bzSGw817cyiL5Kb+0RkWj2Y5ArOAK6XPGBZWZTHwyIawsSCN9AhDXZQWVRqkR1QXcq3I YKl+OHWMO7+1VfixCSakNf7T/Kiq46rQEPW8Eghk6CVOBR8xUCBbyk5aRW4VSGO6pUD3H21u r+5fTLsVyan1NHhxNNiXfnEJKr+JI5dXSkj7WqA5n8ITaNdFSAttkdT56wAQpxE2h8zaOmBa FUWQ4D8SdXDVymP5QMtLG+ItMMiNV6kXgsRFugAKM5yZtPP9gIX+ic8QO5iuct37bRXJU/rm rH54Ab0kyAeeRE7oSsfTZPKvgtUh7VLAUEw/wy6TORJHE8JMaX0yYT6h4PGRS3mNM4bka8hj dfcrexI0zSqFOl2I22zQlG3YqSzIvVh98W67hxfAIaCVaTfJLFPEru3drxNwi6ogdkRmcLGK qqTgeYItrvITyFvzqbrcO2exp0KKEK3cDIZypqHHUf4+uPlDtuExehLsNOMpjP8qhZpFtyLe DS07qunbvstcyvR30wOJ3DyAbHGzq739UyDcO9Jt5jwODyVwk3MK5Em4pJ0+IAJx+F6gta0B k2MAEQEAAYkCJQQYAQgADwUCWClxHQIbDAUJCWYBgAAKCRAO2D86RorIs0ykD/4t151SZG9M beKRVKbs9Ecjady9bO0L3oBos4rhqY12ha8smFlsUzvbgB4CtkBuXQlq+plOBWv+rFEThOzy 3bezgEDjlxycoO1W2wJD6E7Fo9fkHT6UOm9fQBkuKRqK83OGnfM02qP1Ky8d7EoZz+nTSMf/ DJgWw1YRKrXkMHBwKD83lCENsmePWE5AjMqk8cojPv9Oy1wWy6fHjwx3r+wQSokBNfxgQyAF onmgBbhlic/pZUYRSIcldyUlaomrjFfr4egzmNE7aWDvLwOUYKevBIeJJcqTyfAn3TtJbPCE HOC2+lP6EcmPFyhQdiia+RqOClumqbWOPeQ2VM8j7NWvKKmBNBB5OJ/rmHogbNU+wWPJ723q MBoOp1jIwFNkQhx01W6v55VMwLr+IuBKY1ggJ2BhwQiGpWv4tMc5oB/qVh3my1VO65ErcJ3S 9blpwJdDj5/YDOU7BKEmpRUP+xkaryNzH2x7FzrOOHzJBX6jeYZabGvnTicQlBAzfGpblFqV 3YN6EhCF2AHmGLTZ/DrjGYToIsW8cXlEMqN4u8ODEUY0OhbnytnopKJKk99bwMoCqDkfQvT3 LKDWtZj9NzFndfuoKXsVpwAitrG0mau0/16DKDyVWdtJ9DYmtE40zO6g70VVxUj+dKt2hbJT y/KQTb7Ijhw7wZrGp/P7nhbVyA== Message-ID: <62cbf8ac-8ed2-913b-07b6-cafaf8250468@digitaldissidents.org> Date: Mon, 8 Jul 2019 16:41:19 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Authenticated-As-Hash: 29cc722430e8f1f6ed904119444c0d49b0f3ee91 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Scan-Signature: 4c3c8b3d32e7d0cfaf2d58264fc1daa3 Archived-At: Subject: Re: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-00.txt X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2019 14:41:58 -0000 On 7/8/19 4:15 PM, Eric Rescorla wrote: > > > On Mon, Jul 8, 2019 at 6:47 AM Iain Learmonth > wrote: > > Hi Eric, > > On 08/07/2019 14:30, Eric Rescorla wrote: > > After a quick look, I see.... > > > > Document: draft-irtf-pearg-safe-internet-measurement-01.txt > > > >    The reduced impact should not be used as an excuse for pushing higher > >    risk updates, only updates that could be considered appropriate to > >    push to all users should be A/B tested. > > > > This may just be wordsmithing, but as written, this text is entirely > > unrealistic. One of the major reasons that one does A/B testing is > > that you are concerned about risk in the Treatment group (e.g., that > > there will be a higher risk of failures, crashes, etc.) and the > > reason you are doing an A/B test is to mitigate that risk. > > The point that I'd like to put across here is that it is not an excuse > to be reckless or careless. A/B testing can mitigate risk to reputation > perhaps, and sure it can reduce the risk that any individual user is > affected by a bad update, but it doesn't mitigate the impact for the > users that are affected. > > > This seems like the kind of product question that is well out of scope for PEARG. Software vendors have a wide variety of processes for determining whether a given piece of code is suitable for shipping to their users, ranging (at least) from "some developer thought it was good" to "multiple detailed code reviews". I think we can all agree that defining that is out of scope, but without that, Why could we not document best practices for this? Seems quite useful to me. > defining what you have to do before you ship to a fraction of your users is largely irrelevant. > Why ? Niels > -Ekr > > -- Niels ten Oever Researcher and PhD Candidate Datactive Research Group University of Amsterdam PGP fingerprint 2458 0B70 5C4A FD8A 9488 643A 0ED8 3F3A 468A C8B3 From nobody Mon Jul 8 07:48:40 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D8EEA120276 for ; Mon, 8 Jul 2019 07:48:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.601 X-Spam-Level: X-Spam-Status: No, score=-0.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, PDS_NO_HELO_DNS=1.295, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=rtfm-com.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vvtpw7jBSl5W for ; Mon, 8 Jul 2019 07:48:27 -0700 (PDT) Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com [IPv6:2a00:1450:4864:20::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 91544120219 for ; Mon, 8 Jul 2019 07:48:16 -0700 (PDT) Received: by mail-lj1-x22e.google.com with SMTP id v24so16199275ljg.13 for ; Mon, 08 Jul 2019 07:48:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rtfm-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3SnVUTQWcaD+nOK0R26yb4hvwMY+MGYoy2Bp1noiMdM=; b=RCALRxUgCazhRM+dC6jHSk3ZllLGgLgS6qCbZiZTAHTGO0CcpkqSPMn2jK1B5h3MDV /xjyH/U+hwXkxkZiIIo2r8S8J/My3AeN+p1W1QgWiaJtyCLBEPp51rX57pByJdQNvJWw m0zFcfu8trmoNboAlWgOKRXSzGEpuxqpDYuhsSwSllpkXAJXH4hC+LjLMw8r94Oz4SbO ar8BJRTfQIoIK14nFN83yFOJwQkeUb9AOoUo4lfhxHRpaMvQVjXZxTlII2Zx7tktpsDt wyKtlztsVxjkjb4InFPk4LOhW543RYG2A5dm9yhLNtRDc8i48qqB+U4LG6OipWlMvSOa jo7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3SnVUTQWcaD+nOK0R26yb4hvwMY+MGYoy2Bp1noiMdM=; b=RK/MKGnnsRmUAQeH0rKgJzqEhg2pnrxHfMuTiD3/XhR/e1rK6E2A0V9vMPs2uehEjm v1KdEdzg6Xqj+nufz7KeZcbj5X49JMvWrkho3dwsV9mNkRsZZ1eFUxkoUYshvy10Dc+S i/F1NPSB7yiy0U6/dROzpARcGmyebyKUN0j8JneDYErkzWsdx7ph+EXTSkv0mRclUEyS fRP9NR1CVyoPKmXI60pAgc7/S48Pee65kI5uv498Wez5jOZktDHV7pglbSU8zXIF93NH pGIjezFv4qngZ1CqBfq6mAAXEK8A/adP6eWYabQglbdVLSvs6SmvrZGkKL5WE9M3BeAm c2kg== X-Gm-Message-State: APjAAAXKNSMJSt++PfGC7xAv+e6XASK+73CvN5rGXknpdIJxaWvJyPYT qYaYKkZpbNsgUMy1asP9mDkxTHrpKwMkyKiQzgkMUw== X-Google-Smtp-Source: APXvYqzB3gcXQNJ5omSz6u+suDoYo8DCIG8f4+L5TBE3vdops8ZHf5jN0AxCeW3iGOu7OMfhO6dF2c+zRdkjXu5XFic= X-Received: by 2002:a2e:3008:: with SMTP id w8mr10918358ljw.13.1562597294816; Mon, 08 Jul 2019 07:48:14 -0700 (PDT) MIME-Version: 1.0 References: <156254420044.4995.7471139515518776754@ietfa.amsl.com> <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> <279a7516-a08d-12a6-1693-b49c94c3c2e5@torproject.org> <62cbf8ac-8ed2-913b-07b6-cafaf8250468@digitaldissidents.org> In-Reply-To: <62cbf8ac-8ed2-913b-07b6-cafaf8250468@digitaldissidents.org> From: Eric Rescorla Date: Mon, 8 Jul 2019 07:47:38 -0700 Message-ID: To: Niels ten Oever Cc: pearg@irtf.org Content-Type: multipart/alternative; boundary="000000000000af4e78058d2c88da" Archived-At: Subject: Re: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-00.txt X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2019 14:48:39 -0000 --000000000000af4e78058d2c88da Content-Type: text/plain; charset="UTF-8" On Mon, Jul 8, 2019 at 7:42 AM Niels ten Oever wrote: > > > On 7/8/19 4:15 PM, Eric Rescorla wrote: > > > > > > On Mon, Jul 8, 2019 at 6:47 AM Iain Learmonth > wrote: > > > > Hi Eric, > > > > On 08/07/2019 14:30, Eric Rescorla wrote: > > > After a quick look, I see.... > > > > > > Document: draft-irtf-pearg-safe-internet-measurement-01.txt > > > > > > The reduced impact should not be used as an excuse for pushing > higher > > > risk updates, only updates that could be considered appropriate > to > > > push to all users should be A/B tested. > > > > > > This may just be wordsmithing, but as written, this text is > entirely > > > unrealistic. One of the major reasons that one does A/B testing is > > > that you are concerned about risk in the Treatment group (e.g., > that > > > there will be a higher risk of failures, crashes, etc.) and the > > > reason you are doing an A/B test is to mitigate that risk. > > > > The point that I'd like to put across here is that it is not an > excuse > > to be reckless or careless. A/B testing can mitigate risk to > reputation > > perhaps, and sure it can reduce the risk that any individual user is > > affected by a bad update, but it doesn't mitigate the impact for the > > users that are affected. > > > > > > This seems like the kind of product question that is well out of scope > for PEARG. Software vendors have a wide variety of processes for > determining whether a given piece of code is suitable for shipping to their > users, ranging (at least) from "some developer thought it was good" to > "multiple detailed code reviews". I think we can all agree that defining > that is out of scope, but without that, > > Why could we not document best practices for this? Seems quite useful to > me. > The question isn't useful, but rather what this group is chartered to do. Do you have some charter text that would support such an activity? -Ekr > defining what you have to do before you ship to a fraction of your users > is largely irrelevant. > > > > Why ? > > Niels > > > -Ekr > > > > > > -- > Niels ten Oever > Researcher and PhD Candidate > Datactive Research Group > University of Amsterdam > > PGP fingerprint 2458 0B70 5C4A FD8A 9488 > 643A 0ED8 3F3A 468A C8B3 > > -- > Pearg mailing list > Pearg@irtf.org > https://www.irtf.org/mailman/listinfo/pearg > --000000000000af4e78058d2c88da Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Mon, Jul 8, 2019 at 7:42 AM Niels = ten Oever <lists@digitald= issidents.org> wrote:


On 7/8/19 4:15 PM, Eric Rescorla wrote:
>
>
> On Mon, Jul 8, 2019 at 6:47 AM Iain Learmonth <irl@torproject.org <mailto:irl@torproject.org&g= t;> wrote:
>
>=C2=A0 =C2=A0 =C2=A0Hi Eric,
>
>=C2=A0 =C2=A0 =C2=A0On 08/07/2019 14:30, Eric Rescorla wrote:
>=C2=A0 =C2=A0 =C2=A0> After a quick look, I see....
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> Document: draft-irtf-pearg-safe-internet-measu= rement-01.txt
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> =C2=A0 =C2=A0The reduced impact should not be = used as an excuse for pushing higher
>=C2=A0 =C2=A0 =C2=A0> =C2=A0 =C2=A0risk updates, only updates that c= ould be considered appropriate to
>=C2=A0 =C2=A0 =C2=A0> =C2=A0 =C2=A0push to all users should be A/B t= ested.
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> This may just be wordsmithing, but as written,= this text is entirely
>=C2=A0 =C2=A0 =C2=A0> unrealistic. One of the major reasons that one= does A/B testing is
>=C2=A0 =C2=A0 =C2=A0> that you are concerned about risk in the Treat= ment group (e.g., that
>=C2=A0 =C2=A0 =C2=A0> there will be a higher risk of failures, crash= es, etc.) and the
>=C2=A0 =C2=A0 =C2=A0> reason you are doing an A/B test is to mitigat= e that risk.
>
>=C2=A0 =C2=A0 =C2=A0The point that I'd like to put across here is t= hat it is not an excuse
>=C2=A0 =C2=A0 =C2=A0to be reckless or careless. A/B testing can mitigat= e risk to reputation
>=C2=A0 =C2=A0 =C2=A0perhaps, and sure it can reduce the risk that any i= ndividual user is
>=C2=A0 =C2=A0 =C2=A0affected by a bad update, but it doesn't mitiga= te the impact for the
>=C2=A0 =C2=A0 =C2=A0users that are affected.
>
>
> This seems like the kind of product question that is well out of scope= for PEARG. Software vendors have a wide variety of processes for determini= ng whether a given piece of code is suitable for shipping to their users, r= anging (at least) from "some developer thought it was good" to &q= uot;multiple detailed code reviews". I think we can all agree that def= ining that is out of scope, but without that,

Why could we not document best practices for this? Seems quite useful to me= .

The question isn't useful, but r= ather what this group is chartered to do. Do you have some charter text tha= t would support such an activity?

-Ekr
<= br>
> defining what you have to do before you ship to a fraction of your use= rs is largely irrelevant.
>

Why ?

Niels

> -Ekr
>
>

--
Niels ten Oever
Researcher and PhD Candidate
Datactive Research Group
University of Amsterdam

PGP fingerprint=C2=A0 =C2=A0 2458 0B70 5C4A FD8A 9488=C2=A0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0643A 0= ED8 3F3A 468A C8B3

--
Pearg mailing list
Pearg@irtf.org
https://www.irtf.org/mailman/listinfo/pearg
--000000000000af4e78058d2c88da-- From nobody Mon Jul 8 08:01:09 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CCF03120294 for ; Mon, 8 Jul 2019 08:00:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_NONE=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=messagingengine.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WAsRgx5ISZxG for ; Mon, 8 Jul 2019 08:00:48 -0700 (PDT) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A61A4120250 for ; Mon, 8 Jul 2019 08:00:48 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id D80F2220AA; Mon, 8 Jul 2019 11:00:47 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Mon, 08 Jul 2019 11:00:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=hV11R0 lSsLxszFkz+z6lB7lvT/dTuWa7K2aoP1UDXOM=; b=TOeTm1iLdIdL8fS69z/+KY HW65JvPXJnPdmZ89rsJ1Lpe0ReDWxXjEUI/MLGlcYHtloBbCbYdrtD5K7M+0L5IV qxswhx+h6NrIUieODriDPhO2OIjLgVnMfqF44oFAQYfo0EzuUcABTTlFQcoEcnUR 6S3rKHRCeBDmLT70E2YzeMisU1icGzgyEnkQPJg31Z+72U1ahbljNF+BbmWFj8lo 8G1s14fc/Y9pqM2Yf7/EkIe82TTm6HGCn0NkSURtOzpG5+AZhu+kaLkmQ2HpKue8 zjMD4dhVP5oMyqFUZlUlJqOnEY+Nr4FsMZKuRGJzskEYhyVF1sCmWcQsLrrRIUQw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrgedtgdekiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepvfhfhfhouffkffgfgggjtgesghdtre fotdefjeenucfhrhhomhepkfgrihhnucfnvggrrhhmohhnthhhuceoihhrlhesthhorhhp rhhojhgvtghtrdhorhhgqeenucfkphepkeegrdelvddrgedurddvtddunecurfgrrhgrmh epmhgrihhlfhhrohhmpehirhhlsehtohhrphhrohhjvggtthdrohhrghenucevlhhushht vghrufhiiigvpedt X-ME-Proxy: Received: from [172.22.152.150] (shiftout.plus.com [84.92.41.201]) by mail.messagingengine.com (Postfix) with ESMTPA id C96D380069; Mon, 8 Jul 2019 11:00:46 -0400 (EDT) To: Eric Rescorla Cc: pearg@irtf.org References: <156254420044.4995.7471139515518776754@ietfa.amsl.com> <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> <279a7516-a08d-12a6-1693-b49c94c3c2e5@torproject.org> From: Iain Learmonth Openpgp: preference=signencrypt Autocrypt: addr=irl@torproject.org; prefer-encrypt=mutual; keydata= mQINBFZp8zEBEACxOYriD+tEuc3Wpnbh+GGnyiaLEMABBrfn6JlDQphbBq/YTz9M9OPkttjx hLL/yrxlM1nD69XbGKQ9gIL3LEgOz9+OdivPbN+Q5iNMqk/WCQUqd3bCFbbsn1yvoTumFy9S 9kYX45Db3jRJoN/Nye6Stf7KKPxHxot14iY+PUR/5Gx5KbeWVKfDtQejGnhxQD73KjrX4wds BAaxnQ7KbjQyUf+IxE+8qSDcyTP+pPqxspVzx+eFqsW5+kK1eJMHxJmY/KsAs6IsGf5lvyDJ JECc2iE0mFS6vc14lGcD7BAYMPRnvlK3OcDlbdJS3ZU0LQu3/AplM7cNcesq2Btm06OUTsbj 10ZiyLi7Q0WZRuUbn7t3jOQVyOlNfjUpJhKPMMobBL2R0KzcptJbUrKc08wZD/TPaXuHKWAE JuA6kFMXtHhV8Qhxz5/d2KUA8ex+zpVd2xSR6q4llcYu1w8zHZtLN+YKSmjjKs+AjiTrCMYs OYxt4cwxuaIIhBNvCC9WqZOxHX7YHmpVcSV6K9Wwhk9mVIU3Ii0G2HWs6OQ0vIueCDGMEdVk ig/a7cVlfXNz7WuaXuhOJmHz6d6Yk4dFn5mLbEY9cZhBxf5hjCwtp9b6v+ueuptfcnOd+38G 9KH6NyHKZyS4jcd3E6Dp0+9Isbl/EohjPCujevoW3/DlT08OKQARAQABtDRJYWluIFIuIExl YXJtb250aCAoVG9yIFByb2plY3QpIDxpcmxAdG9ycHJvamVjdC5vcmc+iQJUBBMBCgA+AhsD BQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAFiEEqPe6UEHhMzOcuhaWdtWAk/VAq80FAlxe8QQF CQbiS9IACgkQdtWAk/VAq80isw//QGWLLb6wjPW9K/seio9dSVM3gDk6jU7/FhlUuz7oq/6H gPrl7jrxH9PY5jAlkWCK5iPFWpFczUKC+AzzuZQt8VXQp5iYllm1KRrZA6mxpx085Yo3cSLp H9B55IHENo5RFdXMgXZ2aNc6ENG31vB6IZDHXjw9fUxtqyB7fCbIhFtO9pCqwnH/WipISdrl P5/gllUJKsvZiQj8vFfOe2rEHfldhqXxN0ZFvQtQMy0SipqfY2eM47HoDLZ2L5U1iK3iG1fa UWN059Y92zcc9B/AFqHK9xFMObUlWEvqhCGKgullDp4mVF4/OPAC/l02IS85BeFxWbV/HxTY AeoEzO9qrWVitCPTuFC4TQxJq4TbVr/jriFugmbvlE3IIXz5OjqTpyJZw+A8jJ3lwF+V5z21 LdJpEbh+fBBz6tl99Hnn7rjN9WhhsH+BvSC9fCnxs21h+oiGw9zom6blFsLW1yP7mNX2ZRJR DmQs+7kPL4ZS8XekNN82UHXE2pd9M2fSEi5dHK0R9xKJmMSNi/cnEZp1SRx+e+FFDZwzxiFO R30sDEEf+Z1MB/7o5uQXtw59W5Q7p0KycJMfacnb3dulOhFF4Qu1w1K3Iqefq8cQpnlmIGV0 dYT6juPyDlRW19LQVQSTejkLAbkBuXVE0T6dAYVyudJ2lVa3aDvQV5PqL1Y5tdK5Ag0EWmt/ jQEQANhwgR5fq1Slk0T0Sh7Tkc5LjanQH1VuNTxOE7wzcYXrgva2ic4zdhLc2QHP9G4kz0AY /oLxtw5Sj/IMtdzBHKDBXgiqBmvkrz7mOZSQQh6K/JBcKau4MorzB80Z/Z6XQcfFKQh61+8e Im/I7AnJFUELxDe8CYmkJYKVJS9b+i9RNFvXAsamOkClcyXqPYBB9bBI8QlZHweTWDsXJqul xjHLjCWOQyJxfl9xFmlSJtskrLEmprw/PaOtXglrz/2vadn2lL3ack9V1ux9ALa5q8Oc6dnx vA/W3palFpdBoz75FckhRfliYNfdCpgo86w00bJvbJ1f6XfBIBYvsAvrIRWIkMYEmhTYm4xG gEDWz5CvGuuzkF7kJwfPdj1RgVe80JHr9OJc8ZrafBb/p35ldLxmhDi16j2VOj58CKwpi3En nMOMPteDxmCZrWeYKhfwgVAP4/zc4+9vmTp5Wpze00DnQ9MEinmJ2bxf6TyQP36Bf/8zgJVs rpqMOaumbwwmcUh85Q1PXneVWX2ryEv1I7xVMpW4APZDeOMiiHb+EpXPLTPBvXTerd/Pwzb7 WLKNpZI4tl/vh88INdrs3xZd174skDJNaNSnqzUP2kNYcxWZSH/FP4AH1IXvIxwIr9g0RpvY oX55ALOSQOdZ9ioCjf9x9mvnBsehcoEPrgozljzpABEBAAGJBHIEGAEKACYCGwIWIQSo97pQ QeEzM5y6FpZ21YCT9UCrzQUCXF7xLQUJAuC/nwJAwXQgBBkBCgAdFiEE/ps4MHN0fw3t6Aud F4mIfdjVvF0FAlprf40ACgkQF4mIfdjVvF0NJBAAuAVw1aY3IUtp8ZRI5xuhieVFDbye7z86 gP5KW8cAMKTpuowD7E3Rm31R2Pb5dX+vZlunSTBPuzRVnVNd6Kj7CR8BR426L48ogSUfIh4V IWiYxXn4DqqNGow+saPI8VwJ/y80NSb2v+qUdv7us052vs9UBBnI/zl8XrnUNix+B0g+jekT ENEVD9fNJ+9YE9cn80wLGKBx047id1IqgOeJShjE13WRj7sa+LhB8FtoOUeXduQ1sJBZTVJq XW9v0e2PNc37cc8VATrA+3HNSQJY1PETDxjJ1TW8gALpZQrfR+FDu/d2vALdULw/djCeIpl0 dr79isd6mINBKTkEcR8XPQjDUnrnO1U18YETo71N5Z6HpvaV5/XWLUl81wovddpVRvwbwuFP bJE506UiQJrthPKi6s670VBV6u5I+5+h6D+CHp1xp0Bq8PLz0EwqjZwOiLAByBT/ryD2ZY8+ 2rP9gphnEBcCtyp6Fz6Q0KyutpCPv94b4OU0eUTvl3Nroq2GWGGHAV+8r1uXpqdNSHxYmQgy b3H1dvObmE37L3BWrVNzeFb0USNiV04up/55QUrqrz7kLdBO0ZVMVLU3j7e48KcbJG2yzNHZ dnQa+ZMMIRFtm8vzSt+jEIwodi6nOeBgEiqODF8TZPbTc6183ErFoYc/FuhkUh0aR0MLZ92Q bPEJEHbVgJP1QKvNZ94QAK9KrF8hpdlSLTdlQptQ+pfKH2AyC1zyyrSxoxh56inKMgETXLVJ WmEYc1OCLWktUU0qQkCzo6umR7TNVZjx86L6jKKlFTptulQyFVaeaOU8DgFuiqnDXWn5E1cu bp1KCVU9TpBqIC7hN4ifJeY1lVoRGR+x+PCDeYgT+birtIanK5mqKzx74pbLeTTQZALqD8Hr UouxO41NyOGMv4BgZrr9z2ATIEIHZDLihKMEbF6LMjC6oCOgf51x6gV8doVw4ySNc3i/mk6h ztLQBhEc1q5SWfNgXcPDnzzoYsVJMSmV6yMCw3OEwtfZ0BYPQ6hihShOBKVQEHVnGbMIT4md V1S/axXDsLPFLkq1CxZ1L7N8Y5n2lUH00iSo+yWs7lhvt9WKl7sTewjXtibgrAC89uKvIyHe bA0GO9wojlwGAQboCMxZkkADgpHkDNvY675QLWaEs7+JJq4iWwojFh6QrDuOZmR3e9a5xZDb nQScTuPWvhZUtq7ZNZxmjPDdSUomSHe/ahBd+FHNKbgumN3PuXitFTg7LozIZV05oVv9arMs 9KyKGOS6rbim5C3Nx+MitMdr2Vh3tRfHugy0iWFSX5ITkG/RH4fMOzJMgMz/SftZMwMOU4J/ RBiPRyeaMBMqwomg/7o+NjulWHrUlAX+DAsyVtV+K9OLkglikNQtkG0MuQINBFprf6sBEADO ANf22so7uoGcvok2TM/T8BHI5+TqHEc4hVe+JGGJ1ZnWlgtGmpOs0fOQj3WAgGI0ZmTqMuoz KF/K9ljbjaMXsLD+JIBTD4rINy60VX2zHhmWhNaOcJvq+wbuHx0tMbhqsTStGnSkvRhH61nc MqVqlTTTLVQQSxKl9D2l7ZGwEPLHRFlydTOOix+F+Y1ehxYLVaPkaycs8wvgjYsDLo3T8Tmu OL+rcEfvxJ6lT2V5I51xqievqoBazAfXvA8FW/0G+Z9LUJmViOVluWg3xjP8okKYgOkOeX00 vMBCVaiEA08oaxY0ebS7uBEgppjWSwn+WAhB+6spd67d4W+DmAnM262lxFMhVYhXpfeV9zyg ULQOofdE6xtFkaxr/y8xQ4Bf7zX8ko6X9aFQFB/vc+zUtjzjg4VaQvWrThjaHlbEKR55MDxJ u2T9S7g5bR4zxZNv36gwlIdmx33a1AeR1nGcWa/7OtoS53+lUwyFVWLOnucqKh71Y38AAMd5 L9Fsb+ArQem71knEUTC+HvBGkPb2Y2PzGnnzhZyC8zgE8AjVD0wB+RMDNI3+fIW6biKAHDqr S6ZCVkzJ1R9nOjXMHRYZ5qlG+rCOeu6Jp4yNwp46z4PqiiLJ9NtmdNttLCEn5PDVF3g9g811 JcadvFVH1ZELoDGWMg3Q+QOHQBFYj7cj2QARAQABiQI8BBgBCgAmAhsMFiEEqPe6UEHhMzOc uhaWdtWAk/VAq80FAlxe8S0FCQLgv4EACgkQdtWAk/VAq81y0Q//YAbM50BQSjWAyVk52AMv PDEalSOau9jd1zllgurs1+BD4eWkNOJtN9BKaR5pd5VsjYPhL6ypUv/lRsyDPM3h7knuCZmI XYSBkloDqutZ020kR1jNhIZLL3Nma1xqh+oMsB7M+1AcT+Nlez74WgYjNyb7uuFWr99pUBuJ KbVjBq772Jjz3U0q7sa35wQvLaMC+AG/8L+e/fos6jgJhvIq2QeVLuVyeKVlEEO6tpe07Q+e mdCNu4l5eNbWAuvOgvtbVCLz5C4nLNZzr27u0thXzVhZ8ovAzdQTx6d6X4YI3fwBhPxXpqkg eTo3+B8D5lKUgkSZ9xTo+JP9yjCWwytezTL3oElL5LBVlYxdnpDWwOx8rCIgi5OlfVwU03/P O7IrEkqU5jvhZYT4c5/ktCx99kJcYwPUbT4wuhI1JnXyILrtmfC9a0vK8hpIUP4HU9GJTnVH Bl/XZZ6OJWEmSlJWm8KbageOcfLewc3BdWFnY6k5TmrzsFbamaBOgZmJgkNgylEyTjxVnLTk i5wciAp+N9K+tOP5FvgR943RSESZrxHo+XAV+BAK6K6Oae/jlrzGzNLAyJKWjefhyGL3AOru EBhSQaRemp/IUmyKREfowz3f8Lw2NFyJuWGzDgo9/1fmqmZ1JegtfWw0uPHB/rooajODBaol obxU65Qt1SOk2Ws= Organization: Tor Project Message-ID: <4c853f4c-2103-527a-e1c2-b5b8c92884cd@torproject.org> Date: Mon, 8 Jul 2019 16:00:45 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="m4X1ZkShPljuPIBtc8LYaOYWhTy17Swa5" Archived-At: Subject: Re: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-00.txt X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2019 15:00:59 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --m4X1ZkShPljuPIBtc8LYaOYWhTy17Swa5 Content-Type: multipart/mixed; boundary="ytW2frCwH9famw2htl9iofODywnZy0RT4"; protected-headers="v1" From: Iain Learmonth To: Eric Rescorla Cc: pearg@irtf.org Message-ID: <4c853f4c-2103-527a-e1c2-b5b8c92884cd@torproject.org> Subject: Re: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-00.txt References: <156254420044.4995.7471139515518776754@ietfa.amsl.com> <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> <279a7516-a08d-12a6-1693-b49c94c3c2e5@torproject.org> In-Reply-To: --ytW2frCwH9famw2htl9iofODywnZy0RT4 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Hi Eric, On 08/07/2019 15:15, Eric Rescorla wrote: > This seems like the kind of product question that is well out of scope > for PEARG. Software vendors have a wide variety of processes for > determining whether a given piece of code is suitable for shipping to > their users, ranging (at least) from "some developer thought it was > good" to "multiple detailed code reviews". The point is not so much to define what make code suitable for shipping, but to say that you should not use A/B testing as an excuse to ship code that you would not have otherwise shipped. Your criteria for shipping to some users should be the same as the criteria for shipping to all users. The code quality should be the same, it should have undergone the same pre-shipping testing, etc. The only difference should be the feature that is enabled/disabled. This is also needed to make the experiment into good science. If you've got other variables like relaxed review processes then you've introduced other factors that don't make it a fair comparison between the A/B groups= =2E This all comes back to implied consent, and when it is appropriate to presume that users will be expecting something. If you typically have stringent review processes, users will expect this from future updates. You violate the user's expectations, and also their trust, when you push a shoddy update to them. This is true even if you've only pushed it to a tiny fraction of your users. Thanks, Iain. --ytW2frCwH9famw2htl9iofODywnZy0RT4-- --m4X1ZkShPljuPIBtc8LYaOYWhTy17Swa5 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEE/ps4MHN0fw3t6AudF4mIfdjVvF0FAl0jWp0ACgkQF4mIfdjV vF34Cw//cP06+2JJPCq8IXbR+Wo8DPHF0H8bTui0BkFrWU67zmUjqfQqabOHPeN6 RueUx4CZuf2DR1QNM/x0tHrqFZySV8NQ3JibVB02GN5ixoNkO9ShX3nY1pvU5N9D CilJlDWK1xj+15GV4bE/nKSysFGolVfgYsPICBb47Vs32/axuR9XKUeOutG7l7E0 ew+iPTQa93Z+kPQw9GCeg0Pm7TUIDJudIEHRbkW4Gc9s+5UlU6ean6b/nSoclnEI xMQPW4z7DnJslVMnsuGZhesOa1V5K/TpJvcGgaK34YFKfh8Z1CNsErurwhD1ZH/z jjh1nq0MJSWYMu8aLgj6E50PR3Ozb4Bd9Zj5gbgjylo9VuWfYOkg34nEZ9+yQ41w mawn/Ppi8H5udGiHNV9BeOgJ3Dq1U+Ogc1Wn5DL/zjkkIhsdPxS9uEXw41lt6VaJ UAT/wqbb4G4HuYx75M1q+9zPBi/MqYisvIWUy7bqvRVEeENQJJLWqsoWLqCBvTD3 5M3cP7g6+Uez4s7oUdHO9amdEY+hI7po9f6phwhgnD2ukHV0zV6X6KM/SQosTTBX qT36GnFKcK6AYnzbfholIYvM4lKaaiOdpc9hJlR90Y/C6lrmT1J3Ef+KItnbIl7y rTuOlDfFerfMTtQJrkpMsHxrD9tknV8OCiX0uqmDrz9l6mda4jg= =9APW -----END PGP SIGNATURE----- --m4X1ZkShPljuPIBtc8LYaOYWhTy17Swa5-- From nobody Mon Jul 8 08:27:16 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2DF9A1202D3 for ; Mon, 8 Jul 2019 08:27:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.601 X-Spam-Level: X-Spam-Status: No, score=-0.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, PDS_NO_HELO_DNS=1.295, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=rtfm-com.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rTqXePE6CJJE for ; Mon, 8 Jul 2019 08:26:59 -0700 (PDT) Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com [IPv6:2a00:1450:4864:20::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 020211202C5 for ; Mon, 8 Jul 2019 08:25:41 -0700 (PDT) Received: by mail-lj1-x22c.google.com with SMTP id v24so16338453ljg.13 for ; Mon, 08 Jul 2019 08:25:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rtfm-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=SW31GyEynzeWG9cuo24ePuZS60LXg97ymZHt3ZGwyAI=; b=YEyaSUYhu3h4f0nW/jFnQhmA9zFYfhPgmMPLSMocFEZgAiRVyo8CdUTDO0w57gHc3n xPLAijT2TOeeOqVkyFy6IqNufr2gDzHBnHXVmVUuZPUFnF/UpmVuO6ulUJy8nz0ZZjql IuAqZQciJYql8chxMRYrdmsPKzG6K55vy+uRdy2SmYybdvKJQNWKaeI2MylIkHqqHPgJ PyU5lbTl70/GRNRMoxNZwLPDjK52CKOWiOHSeVJgce9sWF9ClszVHMojecLMW7UHhb0v w20yiAHYINUOapmOccA8eNx3ekO+TAH+/+ruf3zFzgmrY3CPY6zIUS9WPrd3dNlcf44u Wrng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=SW31GyEynzeWG9cuo24ePuZS60LXg97ymZHt3ZGwyAI=; b=kEesedKhDjTtbhWRVuUf5bK49Mf4QIVsqhTA/VhYYbUbnl6icAtVKrWbkNm+Qdz/Td JAFO+ldgXgLZ6EporbhLMsIS6cPb2p3ssPWLWlo7XYtbf03q9UfsDOvx6BlWLWwwjLh8 U7mK2o8HjBhwIrWLg/H0CYO5/Z11hGrZe9MRU+x5Gr4auKuKsJDHByNzRq58qXiCYFJo DCmBexiepvggi1OGaEEFBlPK9hhZwEgCRqREDe8MKM4WmoimPufs14KbfFmcJSUxt1BL cMXOht9/qGPbUvP5vUQM9v9LsTOvxudPEpnVoAGPMasC916VkPmWQ4EzeMNbqrSCjN90 qtcQ== X-Gm-Message-State: APjAAAWWajUZZbhAisUmcGYjUEZ5lV9n4Jp1TNRGvKYYmHuayDxnRPNT mpR1vpKMj/v9QFJuxlmpsM4eCg2xuQXb4WDzPJslAg== X-Google-Smtp-Source: APXvYqxWS8yngRFLiLNmr8KxbLpTGXK0HD7+Pa9pPsPTfuF3poWKVYQpCKYH8dLE5ol/hsQHxKv1ncqb9fLiRR9NPSI= X-Received: by 2002:a2e:96d0:: with SMTP id d16mr10856877ljj.14.1562599539096; Mon, 08 Jul 2019 08:25:39 -0700 (PDT) MIME-Version: 1.0 References: <156254420044.4995.7471139515518776754@ietfa.amsl.com> <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> <279a7516-a08d-12a6-1693-b49c94c3c2e5@torproject.org> <4c853f4c-2103-527a-e1c2-b5b8c92884cd@torproject.org> In-Reply-To: <4c853f4c-2103-527a-e1c2-b5b8c92884cd@torproject.org> From: Eric Rescorla Date: Mon, 8 Jul 2019 08:25:02 -0700 Message-ID: To: Iain Learmonth Cc: pearg@irtf.org Content-Type: multipart/alternative; boundary="000000000000745df4058d2d0e10" Archived-At: Subject: Re: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-00.txt X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2019 15:27:09 -0000 --000000000000745df4058d2d0e10 Content-Type: text/plain; charset="UTF-8" On Mon, Jul 8, 2019 at 8:00 AM Iain Learmonth wrote: > Hi Eric, > > On 08/07/2019 15:15, Eric Rescorla wrote: > > This seems like the kind of product question that is well out of scope > > for PEARG. Software vendors have a wide variety of processes for > > determining whether a given piece of code is suitable for shipping to > > their users, ranging (at least) from "some developer thought it was > > good" to "multiple detailed code reviews". > > The point is not so much to define what make code suitable for shipping, > but to say that you should not use A/B testing as an excuse to ship code > that you would not have otherwise shipped. > But again, that's precisely the point of much A/B testing: to verify that the code is good enough to ship. It's *part* of the shipping process. Your criteria for shipping to some users should be the same as the > criteria for shipping to all users. The code quality should be the same, > it should have undergone the same pre-shipping testing, etc. The only > difference should be the feature that is enabled/disabled. > > This is also needed to make the experiment into good science. If you've > got other variables like relaxed review processes then you've introduced > other factors that don't make it a fair comparison between the A/B groups. > > This all comes back to implied consent, and when it is appropriate to > presume that users will be expecting something. If you typically have > stringent review processes, users will expect this from future updates. > You violate the user's expectations, and also their trust, when you push > a shoddy update to them. This is true even if you've only pushed it to a > tiny fraction of your users. > As I said to Niels, the question of precisely what code to ship to users seems well outside the charter of this group, which is focused strictly on privacy. Can you please point to some charter text which would authorize the group to work on this? -Ekr > Thanks, > Iain. > > --000000000000745df4058d2d0e10 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Mon, Jul 8, 2019 at 8:00 AM Iain L= earmonth <irl@torproject.org&g= t; wrote:
Hi Eri= c,

On 08/07/2019 15:15, Eric Rescorla wrote:
> This seems like the kind of product question that is well out of scope=
> for PEARG. Software vendors have a wide variety of processes for
> determining whether a given piece of code is suitable for shipping to<= br> > their users, ranging (at least) from "some developer thought it w= as
> good" to "multiple detailed code reviews".

The point is not so much to define what make code suitable for shipping, but to say that you should not use A/B testing as an excuse to ship code that you would not have otherwise shipped.

<= div>But again, that's precisely the point of much A/B testing: to verif= y that the
code is good enough to ship. It's *part* of the sh= ipping process.


Your criteria for shipping to some users should be the same as the
criteria for shipping to all users. The code quality should be the same, it should have undergone the same pre-shipping testing, etc. The only
difference should be the feature that is enabled/disabled.

This is also needed to make the experiment into good science. If you've=
got other variables like relaxed review processes then you've introduce= d
other factors that don't make it a fair comparison between the A/B grou= ps.

This all comes back to implied consent, and when it is appropriate to
presume that users will be expecting something. If you typically have
stringent review processes, users will expect this from future updates.
You violate the user's expectations, and also their trust, when you pus= h
a shoddy update to them. This is true even if you've only pushed it to = a
tiny fraction of your users.

As I said = to Niels, the question of precisely what code to ship to users
se= ems well outside the charter of this group, which is focused strictly
=
on privacy. Can you please point to some charter text which would
authorize the group to work on this?

-Ek= r





Thanks,
Iain.

--000000000000745df4058d2d0e10-- From nobody Mon Jul 8 09:23:02 2019 Return-Path: X-Original-To: pearg@irtf.org Delivered-To: pearg@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id E7A2A1201B4; Mon, 8 Jul 2019 05:49:15 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: pearg@irtf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.98.3 Auto-Submitted: auto-generated Precedence: bulk Reply-To: pearg@irtf.org Message-ID: <156259015587.1035.9063218224806367909@ietfa.amsl.com> Date: Mon, 08 Jul 2019 05:49:15 -0700 Archived-At: X-Mailman-Approved-At: Mon, 08 Jul 2019 09:23:00 -0700 Subject: [Pearg] I-D Action: draft-irtf-pearg-safe-internet-measurement-01.txt X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2019 12:49:29 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Privacy Enhancements and Assessments Research Group RG of the IRTF. Title : Guidelines for Performing Safe Measurement on the Internet Author : Iain R. Learmonth Filename : draft-irtf-pearg-safe-internet-measurement-01.txt Pages : 9 Date : 2019-07-08 Abstract: Researchers from industry and academia often use Internet measurements as part of their work. While these measurements can give insight into the functioning and usage of the Internet, they can come at the cost of user privacy. This document describes guidelines for ensuring that such measurements can be carried out safely. Note Comments are solicited and should be addressed to the research group's mailing list at pearg@irtf.org and/or the author(s). The sources for this draft are at: https://github.com/irl/draft-safe-internet-measurement The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-irtf-pearg-safe-internet-measurement/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-irtf-pearg-safe-internet-measurement-01 https://datatracker.ietf.org/doc/html/draft-irtf-pearg-safe-internet-measurement-01 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-irtf-pearg-safe-internet-measurement-01 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Mon Jul 8 09:57:23 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D820E12027D for ; Mon, 8 Jul 2019 09:53:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.126 X-Spam-Level: X-Spam-Status: No, score=-0.126 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.247, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, PDS_NO_HELO_DNS=1.295, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xXfwPDbj-U9C for ; Mon, 8 Jul 2019 09:53:51 -0700 (PDT) Received: from mail-qt1-f177.google.com (mail-qt1-f177.google.com [209.85.160.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3BBD1120347 for ; Mon, 8 Jul 2019 09:52:30 -0700 (PDT) Received: by mail-qt1-f177.google.com with SMTP id 44so14844452qtg.11 for ; Mon, 08 Jul 2019 09:52:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=Ohf9sVrXyf8FrXxOTlJTG+yayJl6oDebM5iUMOKOLDU=; b=MlBhDrRSbFEN8luuCcBmcV/Q9Y5oz9BUfOFzvpVGSDcHwuG9T+gkf6Lf86JzLBfS7Y 4Ftg0Z4ucU1wwqyD+ZnO9T7g5t7axfhNn0ToiH1otxfsBl0p1C7lao++Za5k+Eis8oZM OvAHq/EDO/iKYtDFhvWuygSl8Zi7rnzNFy8O3n5vZQbL4rYiU3qkhcXhaABiQ7cmHoLe uHurWs3boZfGOqRGmyRVzhl93zn36pibPbmUHoM0FbGx3UaPiwRct5MXyt1+DwsijFT5 ePE5dDCNGoa9+Xi5LbKjSITfmT2tux5uepC5uL6PTf+vqmZOrDEXFPMTorlUx3ElGuzD haxw== X-Gm-Message-State: APjAAAWdd6vXoLlckmDSb3d9Zx3WzwvtG7XsQfnsORvLLTeRzALNzNDi /vbn+ZozvzlrBsReiDzvTw1FADEGdUKPAWXKglCGMNrv X-Google-Smtp-Source: APXvYqwX+HblrzNGSVeYrMBDOPyBGImj4q7IdkaMTkwpWadnNQbjLV7/3SB0qIeSnoaXOhPI22wYWUT/UwjXPyYC2dE= X-Received: by 2002:aed:220e:: with SMTP id n14mr15270540qtc.388.1562604748848; Mon, 08 Jul 2019 09:52:28 -0700 (PDT) MIME-Version: 1.0 References: <156260285253.826.12680990574277231497.idtracker@ietfa.amsl.com> In-Reply-To: <156260285253.826.12680990574277231497.idtracker@ietfa.amsl.com> From: David Oliver Date: Mon, 8 Jul 2019 12:52:18 -0400 Message-ID: To: pearg@irtf.org Content-Type: multipart/alternative; boundary="000000000000fac538058d2e44c0" Archived-At: X-Mailman-Approved-At: Mon, 08 Jul 2019 09:57:22 -0700 Subject: [Pearg] Pluggable Transports: draft-oliver-pluggable-transports-00.txt submitted X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2019 16:53:55 -0000 --000000000000fac538058d2e44c0 Content-Type: text/plain; charset="UTF-8" With welcome assistance from Shivan Kaul and Brandon Wiley of Operation Foundation, I have submitted a new Internet Draft to the IETF covered the work on "Pluggable Transports" as a means of privacy protection. The submission is based, broadly, on the work of the community that grew up around this work but specifically on the specification authored by Brandon. The ID lives at: https://www.ietf.org/id/draft-oliver-pluggable-transports-00.txt Once again, thanks to the PT community, Brandon and Shivan for their incredible work in this emerging area. David Oliver, Guardian Project David M. Oliver | david@g uardianproject.info | http://g uardianproject.info | @davidmoliver | +1 970 368 2366 ---------- Forwarded message --------- From: Date: Mon, Jul 8, 2019 at 12:20 PM Subject: New Version Notification for draft-oliver-pluggable-transports-00.txt To: David M. Oliver , Brandon Wiley < brandon@operatorfoundation.org> A new version of I-D, draft-oliver-pluggable-transports-00.txt has been successfully submitted by David M. Oliver and posted to the IETF repository. Name: draft-oliver-pluggable-transports Revision: 00 Title: Enabling Network Traffic Obfuscation - Pluggable Transports Document date: 2019-07-08 Group: Individual Submission Pages: 12 URL: https://www.ietf.org/internet-drafts/draft-oliver-pluggable-transports-00.txt Status: https://datatracker.ietf.org/doc/draft-oliver-pluggable-transports/ Htmlized: https://tools.ietf.org/html/draft-oliver-pluggable-transports-00 Htmlized: https://datatracker.ietf.org/doc/html/draft-oliver-pluggable-transports Abstract: Pluggable Transports (PTs) are a mechanism enabling the rapid development and deployment of network traffic obfuscation techniques used to circumvent surveillance and censorship. This specification does not define or limit the techniques themselves, but rather focuses on the startup, shutdown, and inter-process communication mechanisms required to make these technologies interoperable with applications. This document is based heavily on [PT2.1]. Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. The IETF Secretariat --000000000000fac538058d2e44c0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
With welcome assistance from Shivan Kaul and=C2=A0Brandon Wiley of Operation Foundation, I have submitted a new Internet Draft to the I= ETF covered the work on "Pluggable Transports" as a means of priv= acy protection.=C2=A0 The submission is based, broadly, on the work of the = community that grew up around this work but specifically on the specificati= on authored by Brandon. =C2=A0

Once again, thanks to the PT community, Brandon and Shiva= n for their incredible work in this emerging area.
<= font color=3D"#000000">
David Oliver, Guardian Project=

Dav= id M. Oliver |=C2=A0david@guar= dianproject.info=C2=A0|=C2=A0http://guardianproject.info=C2=A0| @davidmoliver |=C2=A0+1 970 368 2366


---------- Forwarded message ---------
From: = <internet-drafts@ietf.org>
Date: Mon, Jul 8, 2019 at 12:20 PM
Subject: New Version = Notification for draft-oliver-pluggable-transports-00.txt
To: David M. O= liver <david@guardianproje= ct.info>, Brandon Wiley <brandon@operatorfoundation.org>



A new version of I-D, draft-oliver-pluggable-transports-00.txt
has been successfully submitted by David M. Oliver and posted to the
IETF repository.

Name:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0draft-oliver-pluggable-transp= orts
Revision:=C2=A0 =C2=A0 =C2=A0 =C2=A000
Title:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Enabling Network Traffic Obfuscati= on - Pluggable Transports
Document date:=C2=A0 2019-07-08
Group:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Individual Submission
Pages:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 12
URL:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 https://www.ietf.org/internet-drafts/draft-oliver-pl= uggable-transports-00.txt
Status:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0https://datatracker.ietf.org/doc/draft-oliver-pluggable-transports/=
Htmlized:=C2=A0 =C2=A0 =C2=A0 =C2=A0= https://tools.ietf.org/html/draft-oliver-pluggable-transports-00
Htmlized:=C2=A0 =C2=A0 =C2=A0 =C2=A0https://datatracker.ietf.org/doc/html/draft-oliver-pluggable-transpo= rts


Abstract:
=C2=A0 =C2=A0Pluggable Transports (PTs) are a mechanism enabling the rapid<= br> =C2=A0 =C2=A0development and deployment of network traffic obfuscation tech= niques
=C2=A0 =C2=A0used to circumvent surveillance and censorship.=C2=A0 This spe= cification
=C2=A0 =C2=A0does not define or limit the techniques themselves, but rather=
=C2=A0 =C2=A0focuses on the startup, shutdown, and inter-process communicat= ion
=C2=A0 =C2=A0mechanisms required to make these technologies interoperable w= ith
=C2=A0 =C2=A0applications.

=C2=A0 =C2=A0This document is based heavily on [PT2.1].




Please note that it may take a couple of minutes from the time of submissio= n
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat
--000000000000fac538058d2e44c0-- From nobody Mon Jul 8 11:33:22 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E821512036C for ; Mon, 8 Jul 2019 11:33:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZqZiqiqlyVER for ; Mon, 8 Jul 2019 11:33:16 -0700 (PDT) Received: from fgont.go6lab.si (fgont.go6lab.si [IPv6:2001:67c:27e4::14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D2106120478 for ; Mon, 8 Jul 2019 11:33:00 -0700 (PDT) Received: from [192.168.1.16] (unknown [41.143.218.184]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by fgont.go6lab.si (Postfix) with ESMTPSA id 6243584B04; Mon, 8 Jul 2019 20:32:49 +0200 (CEST) References: <156260942108.1051.14578810787202221256.idtracker@ietfa.amsl.com> To: "pearg@irtf.org" From: Fernando Gont Openpgp: preference=signencrypt Autocrypt: addr=fgont@si6networks.com; prefer-encrypt=mutual; keydata= mQINBE5so2gBEACzBQBLUy8nzgAzSZn6ViXT6TmZBFNYNqTpPRvTVtUqF6+tkI+IEd9N2E8p pXUXCd0W4dkxz6o7pagnK63m4QSueggvp881RVVHOF8oTSHOdnGxLfLeLNJFKE1FOutU3vod GK/wG/Fwzkv9MebdXpMlLV8nnJuAt66XGl/lU1JrNfrKO4SoYQi4TsB/waUQcygh7OR/PEO0 EttiU8kZUbZNv58WH+PAj/rdZCrgUSiGXiWUQQKShqKnJxLuAcTcg5YRwL8se/V6ciW0QR9i /sr52gSmLLbW5N3hAoO+nv1V/9SjJAUvzXu43k8sua/XlCXkqU7uLj41CRR72JeUZ4DQsYfP LfNPC98ZGTVxbWbFtLXxpzzDDT8i3uo7w1LJ2Ij/d5ezcARqw01HGljWWxnidUrjbTpxkJ9X EllcsH94mer728j/HKzC9OcTuz6WUBP3Crgl6Q47gY5ZIiF0lsmd9/wxbaq5NiJ+lGuBRZrD v0dQx9KmyI0/pH2AF8cW897/6ypvcyD/1/11CJcN+uAGIrklwJlVpRSbKbFtGC6In592lhu7 wnK8cgyP5cTU+vva9+g6P1wehi4bylXdlKc6mMphbtSA+T3WBNP557+mh3L62l4pGaEGidcZ DLYT2Ud18eAJmxU3HnM8P3iZZgeoK7oqgb53/eg96vkONXNIOwARAQABtCVGZXJuYW5kbyBH b250IDxmZ29udEBzaTZuZXR3b3Jrcy5jb20+iQJBBBMBAgArAhsjBQkSzAMABgsJCAcDAgYV CAIJCgsEFgIDAQIeAQIXgAUCTmylpQIZAQAKCRCuJQ1VHU50kv7wD/9fuNtTfxSLk3B3Hs3p ixTy8YXVjdkVwWlnJjFd7BOWmg7sI+LDhpjGfT6+ddOiwkumnvUZpObodj4ysH0i8c7P4C5t F9yu7WjklSlrB5Rth2CGChg5bKt541z2WHkFFxys9qBLmCSYDeKQkzLqhCjIUJizY2kOJ2GI MnSFDzJjhSFEh//oW830Y8fel1xnf/NVF+lBVtRMtMOfoWUqDjvP3sJ1G4zgkDCnF0CfncLx +hq2Mv26Uq9OTzvLH9aSQQ/f067BOkKAJKsfHdborX4E96ISTz57/4xECRSMr5dVsKVm4Y// uVIsb+L5z+a32FaiBZIAKDgnJO7Z8j6CV5e5yfuBTtX52Yi9HjYYqnYJGSDxYd6igD4bWu+7 xmJPHjkdqZgGV6dQIgiUfqkU+s5Cv350vK48CMaT/ZLo2BdsMhWsmaHmb+waePUMyq6E4E9x 9Js+EJb9ZiCfxS9exgieZQpet1L36IvhiwByvkQM009ywfa30JeMOltUtfLi5V06WQWsTzPL 5C+4cpkguSuAJVDTctjCA0moIeVDOpJ8WH9voQ4IeWapQnX35OIoj1jGJqqYdx65gc1ygbyx b8vw+pJ9E5GLse5TQnYifOWpXzX9053dtbwp/2OVhU4KLlzfCPCEsoTyfu9nIZxdI2PMwiL5 M85BfjX4NmwBLmPGoLkCDQRObKNoARAAqqXCkr250BchRDmi+05F5UQFgylUh10XTAJxBeaQ UNtdxZiZRm6jgomSrqeYtricM9t9K0qb4X2ZXmAMW8o8AYW3RrQHTjcBwMnAKzUIEXXWaLfG cid/ygmvWzIHgMDQKP+MUq1AGQrnvt/MRLvZLyczAV1RTXS58qNaxtaSpc3K/yrDozh/a4pu WcUsVvIkzyx43sqcwamDSBb6U8JFoZizuLXiARLLASgyHrrCedNIZdWSx0z0iHEpZIelA2ih AGLiSMtmtikVEyrJICgO81DkKNCbBbPg+7fi23V6M24+3syHk3IdQibTtBMxinIPyLFF0byJ aGm0fmjefhnmVJyCIl/FDkCHprVhTme57G2/WdoGnUvnT7mcwDRb8XY5nNRkOJsqqLPemKjz kx8mXdQbunXtX9bKyVgd1gIl+LLsxbdzRCch773UBVoortPdK3kMyLtZ4uMeDX3comjx+6VL bztUdJ1Zc9/njwVG8fgmQ+0Kj5+bzQfUY+MmX0HTXIx3B4R1I1a8QoOwi1N+iZNdewV5Zfq+ 29NlQLnVPjCRCKbaz9k6RJ2oIti55YUI6zSsL3lmlOXsRbXN5bRswFczkNSCJxJMlDiyAUIC WOay7ymzvgzPa+BY/mYn94vRaurDQ4/ljOfj6oqgfjts+dJev4Jj89vp8MQI3KJpZPEAEQEA AYkCJQQYAQIADwUCTmyjaAIbDAUJEswDAAAKCRCuJQ1VHU50km4xEACho45PZrUjY4Zl2opR DFNo5a6roTOPpgwO9PcBb3I5F8yX2Dnew+9OhgWXbBhAFq4DCx+9Gjs43Bn60qbZTDbLGJ/m 8N4PwEiq0e5MKceYcbetEdEUWhm5L6psU9ZZ82GR3UGxPXYe+oifEoJjOXQ39avf9S8p3yKP Diil0E79rn7LbJjMcgMLyjFg9SDoJ6pHLtniJoDhEAaSSgeV7Y745+gyMIdtQmrFHfqrFdjq D6G0HE+Z68ywc5KN67YxhvhBmSycs1ZSKAXv1zLDlXdmjHDHkU3xMcB+RkuiTba8yRFYwb/n j62CC4NhFTuIKOc4ta3dJsyXTGh/hO9UjWUnmAGfd0fnzTBZF8Qlnw/8ftx5lt4/O+eqY1EN RITScnPzXE/wMOlTtdkddQ+QN6xt6jyR2XtAIi7aAFHypIqA3lLI9hF9x+lj4UQ2yA9LqpoX 6URpPOd13JhAyDe47cwsP1u9Y+OBvQTVLSvw7Liu2b4KjqL4lx++VdBi7dXsjJ6kjIRjI6Lb WVpxe8LumMCuVDepTafBZ49gr7Fgc4F9ZSCo6ChgQNLn6WDzIkqFX+42KuHz90AHWhuW+KZR 1aJylERWeTcMCGUSBptd48KniWmD6kPKpzwoMkJtEXTuO2lVuborxzwuqOTNuYg9lWDl7zKt wPI9brGzquUHy4qRrA== Cc: =?UTF-8?Q?Iv=c3=a1n_Arce_=28Quarkslab=29?= X-Forwarded-Message-Id: <156260942108.1051.14578810787202221256.idtracker@ietfa.amsl.com> Message-ID: <95cd483c-8178-63f3-d50d-2a8fa80631ba@si6networks.com> Date: Mon, 8 Jul 2019 19:22:31 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: <156260942108.1051.14578810787202221256.idtracker@ietfa.amsl.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Archived-At: Subject: [Pearg] Unfortunate History of Transient Numeric Identifiers (Fwd: New Version Notification for draft-gont-numeric-ids-history-05.txt) X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2019 18:33:20 -0000 Folks, We have posted a rev of our I-D. which now incorporates the timeline of yet more numeric IDs with a sad story of flawed generation. The new revision is available at: https://www.ietf.org/internet-drafts/draft-gont-numeric-ids-history-05.txt We have also revised a companion document, entitled " On the Generation of Transient Numeric Identifiers", which analyzes numeric IDs and proposes algorithms that generate numeric IDs while mitigating possible security and privacy implications. It is available at: https://www.ietf.org/id/draft-gont-numeric-ids-generation-04.txt We would like to hear your comments, and also hope that PEARG is willing to work on them. P.S.: We are also pursuing (at the IETF) a third companion document entitled "Security Considerations for Transient Numeric Identifiers Employed in Network Protocols" that requires spec authors to do the warranted interoperability, security, and privacy analysis of the transient numeric IDs they employ. Such document is available at: https://www.ietf.org/id/draft-gont-numeric-ids-generation-04.txt We hope this work helps to change the sad story we have had with numeric identifiers in our protocols. Thanks, Ivan & Fernando -------- Forwarded Message -------- Subject: New Version Notification for draft-gont-numeric-ids-history-05.txt Date: Mon, 08 Jul 2019 11:10:21 -0700 From: internet-drafts@ietf.org To: Fernando Gont , Ivan Arce A new version of I-D, draft-gont-numeric-ids-history-05.txt has been successfully submitted by Fernando Gont and posted to the IETF repository. Name: draft-gont-numeric-ids-history Revision: 05 Title: Unfortunate History of Transient Numeric Identifiers Document date: 2019-07-08 Group: Individual Submission Pages: 25 URL: https://www.ietf.org/internet-drafts/draft-gont-numeric-ids-history-05.txt Status: https://datatracker.ietf.org/doc/draft-gont-numeric-ids-history/ Htmlized: https://tools.ietf.org/html/draft-gont-numeric-ids-history-05 Htmlized: https://datatracker.ietf.org/doc/html/draft-gont-numeric-ids-history Diff: https://www.ietf.org/rfcdiff?url2=draft-gont-numeric-ids-history-05 Abstract: This document analyzes the timeline of the specification of different types of "numeric identifiers" used in IETF protocols, and how the security and privacy implications of such protocols has been affected as a result of it. It provides concrete evidence that advice in this area is warranted. Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. The IETF Secretariat From nobody Wed Jul 10 09:56:31 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7E3F51203E3 for ; Wed, 10 Jul 2019 09:56:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.7 X-Spam-Level: X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=heapingbits.net header.b=WUHITZTy; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=kBjVdHYq Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id boU_qwMSybgs for ; Wed, 10 Jul 2019 09:56:15 -0700 (PDT) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9CC231202BF for ; Wed, 10 Jul 2019 09:55:19 -0700 (PDT) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 9913421F41 for ; Wed, 10 Jul 2019 12:55:18 -0400 (EDT) Received: from imap4 ([10.202.2.54]) by compute6.internal (MEProxy); Wed, 10 Jul 2019 12:55:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heapingbits.net; h=mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type:content-transfer-encoding; s=fm1; bh=ENE1A 5viB4dyaPNHacXpx6rQze8MLtQPqmSGH62yj9E=; b=WUHITZTyenCwtFUfSwhSs t/bveeSexbb0XBj/jsgWLcgVF8NxW2hPDmwOIAEL6eSktdxqIWhS1e0ZYKj/9FpF ln0cWc/Pu/ud8FBlalAJqm7fUniDKMVt6+4n292oNF382X/SOI4G7+BkNdCu/iC9 wrddRqi0AJOt3KPiWL1FBfIAo4x2EuDi9orljGouniN3q+k2A6NQ8Cd44E/n9od1 g8GT08zDHzc6EBZ17X07FqzFGRetf3z5DoWKDQ5sj7/17seFWioNnUHBd5HCGeAu wKWfpUtT0tn81cLoGs6iw6Rg8qlyPlMRT+jP7kT08nGZnWotYU1GuVv0u+E8Y8CQ g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=ENE1A5viB4dyaPNHacXpx6rQze8MLtQPqmSGH62yj 9E=; b=kBjVdHYqsrSyvmxvm+Y1yEXH96HEbYQdYmPCa5Ur3hcBw87BlqPaz69Yg OY7uAvPp/lofwA59ypRn/ksRC635+tRQpPnRSv7uewwJETnDW3Za19uUgG2Opcbx RwFDJRlNbhUAJIXeGFOFeI2ZRXwJ9U7HYhfkPYy1gm0TQZUOqW+m8y96ufLNuoR+ yoj7p7dRIME/qjq3CjYaElfzGrZGtIKeET+bEIoRX+QB3EPejK10pM7YUJDomS0e OWgvLLfCLP7PuSfcB/W3zOAXciohsVtWmwWSKkOaLAviTPRS/DVOeVydctGDC3zm rKAyAkztaqasx69AArREKxjoe9EOA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrgeeigddutdekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtgfesth hqredtreerjeenucfhrhhomhepfdevhhhrihhsthhophhhvghrucghohhougdfuceotggr fieshhgvrghpihhnghgsihhtshdrnhgvtheqnecuffhomhgrihhnpehirhhtfhdrohhrgh enucfrrghrrghmpehmrghilhhfrhhomheptggrfieshhgvrghpihhnghgsihhtshdrnhgv thenucevlhhushhtvghrufhiiigvpedt X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 0B11B3C00A1; Wed, 10 Jul 2019 12:55:18 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.1.6-731-g19d3b16-fmstable-20190627v1 Mime-Version: 1.0 Message-Id: <63de9f42-96c2-4ef9-bd7d-e1a9c62a3c6e@www.fastmail.com> In-Reply-To: References: <156254420044.4995.7471139515518776754@ietfa.amsl.com> <240d826f-1d7a-834a-919a-f4d5aa9fed58@torproject.org> <279a7516-a08d-12a6-1693-b49c94c3c2e5@torproject.org> <4c853f4c-2103-527a-e1c2-b5b8c92884cd@torproject.org> Date: Wed, 10 Jul 2019 09:55:17 -0700 From: "Christopher Wood" To: pearg@irtf.org Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Pearg] =?utf-8?q?I-D_Action=3A_draft-irtf-pearg-safe-internet-m?= =?utf-8?q?easurement-00=2Etxt?= X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2019 16:56:23 -0000 Hi Ian, As Eric points out, criteria for shipping software is well outside the s= cope of this RG. Thus, it should not be used as a basis for specifying m= easurement practices. More generally, as a document describing safe Inte= rnet measurements in a privacy-oriented RG, issues such as code quality = are out of scope. While it=E2=80=99s true that software bugs can lead to= privacy issues, criteria for preventing bugs is not a topic this group = is chartered to work on. That said, being mindful of privacy when conduc= ting A/B testing is good (though not very actionable) advice. Also, for what it=E2=80=99s worth, I think it=E2=80=99s prudent to consi= der the advice and recommendations from folks who have done these experi= ments at scale.=20 Best, Chris, on behalf of the chairs=20 On Mon, Jul 8, 2019, at 8:27 AM, Eric Rescorla wrote: >=20 >=20 > On Mon, Jul 8, 2019 at 8:00 AM Iain Learmonth wro= te: > > Hi Eric, > >=20 > > On 08/07/2019 15:15, Eric Rescorla wrote: > > > This seems like the kind of product question that is well out of = scope > > > for PEARG. Software vendors have a wide variety of processes for > > > determining whether a given piece of code is suitable for shippin= g to > > > their users, ranging (at least) from "some developer thought it w= as > > > good" to "multiple detailed code reviews". > >=20 > > The point is not so much to define what make code suitable for ship= ping, > > but to say that you should not use A/B testing as an excuse to ship= code > > that you would not have otherwise shipped. >=20 > But again, that's precisely the point of much A/B testing: to verify t= hat the > code is good enough to ship. It's *part* of the shipping process. >=20 >=20 > > Your criteria for shipping to some users should be the same as the > > criteria for shipping to all users. The code quality should be the = same, > > it should have undergone the same pre-shipping testing, etc. The on= ly > > difference should be the feature that is enabled/disabled. > >=20 > > This is also needed to make the experiment into good science. If yo= u've > > got other variables like relaxed review processes then you've intro= duced > > other factors that don't make it a fair comparison between the A/B = groups. > >=20 > > This all comes back to implied consent, and when it is appropriate = to > > presume that users will be expecting something. If you typically ha= ve > > stringent review processes, users will expect this from future upda= tes. > > You violate the user's expectations, and also their trust, when you= push > > a shoddy update to them. This is true even if you've only pushed it= to a > > tiny fraction of your users. >=20 > As I said to Niels, the question of precisely what code to ship to use= rs > seems well outside the charter of this group, which is focused strictl= y > on privacy. Can you please point to some charter text which would > authorize the group to work on this? >=20 > -Ekr >=20 >=20 >=20 >=20 >=20 > >=20 > > Thanks, > > Iain. > >=20 > --=20 > Pearg mailing list > Pearg@irtf.org > https://www.irtf.org/mailman/listinfo/pearg > From nobody Fri Jul 12 08:34:59 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4B59C1206BB for ; Fri, 12 Jul 2019 08:34:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.704 X-Spam-Level: X-Spam-Status: No, score=-0.704 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, PDS_NO_HELO_DNS=1.295, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=salesforce.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8gfkViDNQ4m1 for ; Fri, 12 Jul 2019 08:34:56 -0700 (PDT) Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0D4CB1206AF for ; Fri, 12 Jul 2019 08:34:34 -0700 (PDT) Received: by mail-io1-xd2e.google.com with SMTP id q22so21210355iog.4 for ; Fri, 12 Jul 2019 08:34:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=salesforce.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=NF3gMJQ5gbXUQiJNFJPDB4LpBUK736v/zYwn+iMC8nA=; b=OfqeoNMHSbtjPuHfaXZnr89ZthMKEqeugFcLSY6IkIvqKEybMUKF6m0ZB2L+3UR20I sZlYi8m6/s1z5B52YJHTv0wpIOqQv/uQQVC5+wuP8p3vBYFSQ5uDNeySBef5lh+od/+U xDKPHXwkUVW8B49eXhKC2NgrywW+3SHzUDphc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=NF3gMJQ5gbXUQiJNFJPDB4LpBUK736v/zYwn+iMC8nA=; b=PujPa7k/FLKHVUKZhpDdVgTh4+Z1UHEYDsNvjDB89wC+G+E2BWxfkoCxGS44x1BKNG P+/0elBtMGBjgRgHdBwlrvxX+jdXofR9WvpglWiLf4wuLwPS2xa3oTXb527qnZdLnkgQ y1a6IM+IIcONmDtbw8SOg3qz6/wbH3OYf//OYylIwr5zJ2v6hx+FBUD1mGTUQTQO7v2f ZSbDsIuCF5KyqHugTNepNsCD5jGuM7WbTzFv/0n3PtdUeBwVJwljGtkGtQAW9Hdv6p58 VO0ebs/FIXI9PmPa/keauVjZCy2ROjfXFo3rlU6gtLWm+KXTincTzP5CLX+n0byxU5Ac Zyww== X-Gm-Message-State: APjAAAXot9pBY83J16dAfYRqbFiFud4YcdUg0qcmUz4U26mP9ohEe5bD zf7fL62MT1xZnmPhevRxaJ/4WAYm5byy2TDRA8CdrIPxEr71MQ== X-Google-Smtp-Source: APXvYqyRKDjZ/aPcz6PR4rjtKpv5ZiS8Q/wI52uTMaAZo/AqTrEtavDkxZxGFBU+p4MFh+sz7LoGBFixmB42lNtGK64= X-Received: by 2002:a02:b713:: with SMTP id g19mr12119903jam.77.1562945673065; Fri, 12 Jul 2019 08:34:33 -0700 (PDT) MIME-Version: 1.0 From: Shivan Sahib Date: Fri, 12 Jul 2019 08:34:21 -0700 Message-ID: To: pearg@irtf.org Content-Type: multipart/alternative; boundary="000000000000a5c91f058d7da554" Archived-At: Subject: [Pearg] PEARG agenda for IETF 105 X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jul 2019 15:34:58 -0000 --000000000000a5c91f058d7da554 Content-Type: text/plain; charset="UTF-8" ... is up: https://datatracker.ietf.org/meeting/105/materials/agenda-105-pearg-02.html We're meeting 1.30 - 3.30 pm Wed July 24 at Laurier. Packed agenda: 4 research presentations and 3 Internet-Drafts. Cheers, Shivan --000000000000a5c91f058d7da554 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
... is up:=C2=A0https://datatracker.ietf.org/= meeting/105/materials/agenda-105-pearg-02.html

W= e're meeting 1.30 - 3.30 pm Wed July 24 at Laurier.=C2=A0
Pac= ked agenda: 4 research presentations and 3 Internet-Drafts.

<= /div>
Cheers,
Shivan
--000000000000a5c91f058d7da554-- From nobody Thu Jul 18 17:26:58 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 117461202DF for ; Thu, 18 Jul 2019 07:14:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.558 X-Spam-Level: X-Spam-Status: No, score=-1.558 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.091, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id duvdqOSq4WIw for ; Thu, 18 Jul 2019 07:14:43 -0700 (PDT) Received: from mail-qk1-f177.google.com (mail-qk1-f177.google.com [209.85.222.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6B57F1202DE for ; Thu, 18 Jul 2019 07:14:39 -0700 (PDT) Received: by mail-qk1-f177.google.com with SMTP id t8so20569230qkt.1 for ; Thu, 18 Jul 2019 07:14:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=0Le1iibFAxbGsj/Bp6tQNwiluKE6ml32pWrZTmKAhaQ=; b=fzYFZQETd8pAntUMIuliggMqjVQR+pYadAX9KUiiFptSeAE2QVBoACL7ax6ewkR+s4 1vIS5befB0H1n3dl9Azn6UszQ1VYyMpk/muXZqtxT9bjoySb767tSS5HZKl5reItpeOv GG8EXJq4AY/HmhA2asefjiMNQbgDcovQUcDDIRZ08LvvOJELNMUl2EaYURzsvwlmR2AH Svvtl2+TurjqCtCtgwUZ6BLaykRAMmmhwSTmLeAGj7u2IbsFyFIEUmIa3ufyFWwBpGX+ MPTwEAdCJ7Lk4KThJ/KU+trI4LssevpEg6XWKQzZQYex+13jWjOA31BtZMXlnYf9J6Po JefQ== X-Gm-Message-State: APjAAAVJska68HZd6lCryg3GCqduXK0Wf4QNn1GNQ4QROp8+P05r6TJx ZIAgPXMXLE8Y8VQJ7CfsJnfJA32ZQ5HP7bNO44JrxA== X-Google-Smtp-Source: APXvYqzZCAIBqoO4XN6GwH+JFd9+nKdCUUMKlPD6liGhQseFx6VUdEDwYU7GjOgXcfJp2WKr1sSNTwsKA4zBhqBdbxs= X-Received: by 2002:a37:464a:: with SMTP id t71mr31200530qka.436.1563459278090; Thu, 18 Jul 2019 07:14:38 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: David Oliver Date: Thu, 18 Jul 2019 10:14:27 -0400 Message-ID: To: pearg@irtf.org Content-Type: multipart/alternative; boundary="000000000000e434b6058df53a19" Archived-At: X-Mailman-Approved-At: Thu, 18 Jul 2019 17:26:57 -0700 Subject: [Pearg] Fwd: [105attendees] Africa Update session@IETF105 X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jul 2019 14:14:45 -0000 --000000000000e434b6058df53a19 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Anyone attending IETF105 in Montreal? This session seems like an excellent opportunity to interject input into the development of the Internet in Africa and ways in which privacy and access can be respected. We at Guardian Project have special interest in Africa with a new project targeting low-cost, low-feature devices and networks of lower-speed and higher intermittency. I'd appreciate hearing from anyone who attends this event as to outcomes and discussion. Thank you! David Oliver David M. Oliver | david@g uardianproject.info | http://g uardianproject.info | @davidmoliver | +1 970 368 2366 ---------- Forwarded message --------- From: Nabil Benamar Date: Thu, Jul 18, 2019 at 3:38 AM Subject: [105attendees] Africa Update session@IETF105 To: <105attendees@ietf.org> Cc: Kevin Chege , Dawit Bekele Dear All, There are various activities taking place in Africa, to raise the awareness of the IETF, encourage more engineers/academics to participate in the IETF, and to build communities of practice in open standards development. There will be an open session at the IETF 105 Meeting in Montreal, Canada on Tuesday the 23rd of July from 12:10 to 13:20 where updates will be provided on the activities being carried out in the region, and to discuss lessons learned and opportunities for improvement and scaling of these activities. If you are interested in learning about the activities and how one can contribute, kindly register for the event. Engineers and academics from Africa are highly encouraged to attend. NOTE: Pre-registration is required for this event to facilitate adequate seating and refreshment arrangements. Below is the registration URL. https://internetsociety.wufoo.com/forms/s1misahm0hkafmn/ Best regards Nabil Benamar ------------------- =D9=86=D8=A8=D9=8A=D9=84 =D8=A8=D9=86=D8=B9=D9=85=D8=B1=D9=88 --=20 105attendees mailing list 105attendees@ietf.org https://www.ietf.org/mailman/listinfo/105attendees --000000000000e434b6058df53a19 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Anyone attending IETF105 in Montreal?=C2=A0 This sess= ion seems like an excellent opportunity to interject input into the develop= ment of the Internet in Africa and ways in which privacy and access can be = respected.=C2=A0 We at Guardian Project have special interest in Africa wit= h a new project targeting low-cost, low-feature devices and networks of low= er-speed and higher intermittency.=C2=A0 I'd appreciate hearing from an= yone who attends this event as to outcomes and discussion.

Thank you!

David Oliver


David M. Oliver |=C2=A0david@guardianproject.info=C2=A0|=C2=A0http://guardianproject.info=C2=A0| @davidm= oliver |=C2=A0+1 970 368 2366


---------- Forwarded message -------= --
From: Nabil Benamar <benam= ar73@gmail.com>
Date: Thu, Jul 18, 2019 at 3:38 AM
Subj= ect: [105attendees] Africa Update session@IETF105
To: <105attendees@ietf.org>
Cc: Kevin Cheg= e <chege@isoc.org>, Dawit Bekel= e <bekele@isoc.org>
<= br>
Dear All,



There are various activities taking place in Africa, to raise the awaren= ess of the IETF, encourage more engineers/academics to participate in the I= ETF, and to build communities of practice in open standards development. Th= ere will be an open session at the IETF 105 Meeting in Montreal, Canada on = Tuesday the 23rd of July from 12:10 to 13:20 where updates will be provided= on the activities being carried out in the region, and to discuss lessons = learned and opportunities for improvement and scaling of these activities. =

If you are interested in learning about the activities and how one = can contribute, kindly register for the event. Engineers and academics from= Africa are highly encouraged to attend.

NOTE: Pre-registration is r= equired for this event to facilitate adequate seating and refreshment arran= gements.

Below is the registration URL.

https://i= nternetsociety.wufoo.com/forms/s1misahm0hkafmn/
=

<= div dir=3D"ltr">
Best regards
Nabil Benamar
----------= ---------
= =D9=86=D8=A8=D9=8A=D9=84 =D8=A8=D9=86=D8=B9=D9=85=D8=B1=D9=88




<= /div>
--
105attendees mailing list
105attendees@iet= f.org
https://www.ietf.org/mailman/listinfo/105attendees=
--000000000000e434b6058df53a19-- From nobody Mon Jul 22 08:32:26 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D95B9120280 for ; Mon, 22 Jul 2019 08:32:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s6UWzK6sHano for ; Mon, 22 Jul 2019 08:32:22 -0700 (PDT) Received: from softronics.hoeneisen.ch (softronics.hoeneisen.ch [62.2.86.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4420512012B for ; Mon, 22 Jul 2019 08:32:21 -0700 (PDT) Received: from localhost ([127.0.0.1]) by softronics.hoeneisen.ch with esmtp (Exim 4.86_2) (envelope-from ) id 1hpaIV-0001JU-J4 for pearg@irtf.org; Mon, 22 Jul 2019 17:32:19 +0200 Date: Mon, 22 Jul 2019 17:32:19 +0200 (CEST) From: Bernie Hoeneisen X-X-Sender: bhoeneis@softronics.hoeneisen.ch To: pearg@irtf.org In-Reply-To: Message-ID: References: <01eb4d14-822c-3806-9989-8788364c5d14@pep.foundation> User-Agent: Alpine 2.20 (DEB 67 2015-01-07) MIME-Version: 1.0 Content-Type: multipart/mixed; BOUNDARY="37663318-1349928394-1563809070=:2515" Content-ID: X-SA-Exim-Connect-IP: 127.0.0.1 X-SA-Exim-Mail-From: bernie@ietf.hoeneisen.ch X-SA-Exim-Scanned: No (on softronics.hoeneisen.ch); SAEximRunCond expanded to false Archived-At: Subject: [Pearg] FYI: [Medup] MEDUP meeting @ IETF-105 in room C2 (21st floor), 14:50-16:50 EDT X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jul 2019 15:32:25 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --37663318-1349928394-1563809070=:2515 Content-Type: text/plain; CHARSET=UTF-8; FORMAT=flowed Content-Transfer-Encoding: 8BIT Content-ID: Dear PEARG participants Please be informed about the MEDUP non-WG meeting during IETF-105 on Wed 14:50-16:50 EDT. Parts of MEDUP are also dicussed in PEARG. We have a request for presentation on the latter aspect that is currently pending with the PEARG chairs. Unfortunately, the 1st half of the MEDUP meeting slot is conflicting with PEARG. We optimize the agenda in a way that topics in particular interest to PEARG are scheduled to the 2nd half. Please let us know what these topics are, in case you can make it only to the 2nd half. You can find the more information / whereabouts below. cheers Bernie > On Mon, 22 Jul 2019, Hernâni Marques (p≡p foundation) wrote: > >> Dear folks >> >> >> Following our meeting @ IETF-104 in Prague [1][2], we will have our >> second non-WG meeting @ IETF-105 in Montreal, taking place this >> Wednesday (July 24th) from 14:50-16:50 EDT in room C2 (21st floor). >> >> To know how many people to expect, it would be nice you could sign up >> here (with some name): >> >> https://du7f.koalatux.ch/SrtXRHemfNY-xM3k-QWGKA >> >> Please note that this time we will also provide a Jitsi/WebRTC remote >> participation channel (by best effort): >> >> https://meet.digitale-gesellschaft.ch/medup >> >> The current agenda is here: >> >> >> https://pep.foundation/dev/repos/internet-drafts/file/tip/medup/ietf-105/agenda.txt >> >> In that respect, if you want to present an approach / a missing piece in >> scope of MEDUP or suggest other changes to the agenda, please reach out! >> >> >> Cu soon & thanks! >> >> Greets >> >> Hernani >> >> >> [1] Minutes from IETF-104: >> >> https://pep.foundation/dev/repos/internet-drafts/file/tip/medup/ietf-104/notes-ietf-104-medup-00.txt >> [2] Meeting materials and video podcast: >> https://pep.foundation/docs/ietf/104/medup/ >> >> -- >> p≡p foundation: https://pep.foundation/ > --37663318-1349928394-1563809070=:2515-- From nobody Tue Jul 23 16:31:40 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ADF0112096F for ; Tue, 23 Jul 2019 16:31:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.998 X-Spam-Level: X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cdt.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T1zZBooLnCbF for ; Tue, 23 Jul 2019 16:31:35 -0700 (PDT) Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 41D491201B5 for ; Tue, 23 Jul 2019 16:31:35 -0700 (PDT) Received: by mail-io1-xd2d.google.com with SMTP id s7so85470750iob.11 for ; Tue, 23 Jul 2019 16:31:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cdt.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=8ClYS2wNhUHq7oWKBZlIU7SaQwgv93MM4tuvIDd0W8s=; b=j4lMgFCvqeWIrGNR1gKzUSNX4J4CdAvnShV0x3Aljt0VxX0fgR908oDOpiF1AXzHBe bV94Sob/UtQ134oClJz5fFRggc1dMCGlNLiKK/AxuMbmJ5tYCDpGu42zEaouy6svwavs QfTy3qjmtqJOwR1nSPCZhlBNva1Nj9N8OPsZI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8ClYS2wNhUHq7oWKBZlIU7SaQwgv93MM4tuvIDd0W8s=; b=DtdpIbuRt2ODH9KjX70ntWIQQaolflSu2/1SnSxQub6vaMl13a8l/hO/tG6qnME3B3 HMANhS6ck4fSLgodonQQdjsBC/9Y/m6azWdGaRK9As0NWPszOU/Hou15GfO4v3L+lFgl hBHJz1N5i0vMPTf6aWTjCs84Unoiw9tGjGfq9iu99zo4sCa2MB/dfb4ZRUOkXGDLT+il W07hkubkwLmExaPsQ5GcR9tbHGkJzX2+C/FTnN86S3SJTEHz7svd6f5ValLh9ML9pONm wCk+FUwN6IMY1x2ZxzFOexkDL2K1CeHD0pTPWyygBB1rzQqZpnw1zLwFJ+V1PJazdbyK GC8A== X-Gm-Message-State: APjAAAWRM7U/mFuyXqiQrVWiHFO+BIJVS1X1Zx+2NbbIxNRdvZRCcsfw wTv8edBle6Zh06pGeGVmTvdTsmnESmRlFFVpJTtl/w== X-Google-Smtp-Source: APXvYqxw2DhPKBXO3+MVQo+Gy1uXylr543B30G53Gxvc1KkRS5JMMdjF63UAUcS2OyrhEVhxvYj2QLkz7Z5j9Wi5aoM= X-Received: by 2002:a5e:d507:: with SMTP id e7mr1827954iom.284.1563924694139; Tue, 23 Jul 2019 16:31:34 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Joseph Lorenzo Hall Date: Tue, 23 Jul 2019 19:31:22 -0400 Message-ID: To: Shivan Sahib Cc: Stephen Farrell , Stan Adams , Nick Feamster , pearg@irtf.org Content-Type: multipart/alternative; boundary="000000000000d99f23058e6197da" Archived-At: Subject: Re: [Pearg] descriptive censorship work: draft-hall-censorship-tech X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jul 2019 23:31:38 -0000 --000000000000d99f23058e6197da Content-Type: text/plain; charset="UTF-8" On Fri, May 17, 2019 at 5:00 PM Shivan Sahib wrote: > Hi Stephen, > > On Fri, May 17, 2019 at 12:58 AM Stephen Farrell < > stephen.farrell@cs.tcd.ie> wrote: > >> >> ... >> >> -1 to extending the scope. That could be done in other >> documents, each of which would likely take some time. FWIW, >> personally, I'd recommend just getting this one to the >> RFC editor queue, pocketing the progress, and then the RG >> can move on to new documents documenting other mechanisms >> for and countering censorship. >> >> Cheers, >> S. >> > > The suggestion to include censorship-countering techniques came up because > of the concern that as the draft stands, it would not fall under PEARG's > charter. Happy to hear disagreements about that though. > Heya Shivan, I'd love to hear more elaboration here tomorrow about this (or on the list if you have time before then). I looked at the Charter and like Stephen I'm not necessarily seeing where it would require a descriptive draft about techniques also talk about mitigations. I suspect that RFC 6973's structure of "privacy threats" and "threat mitigations" is where the linking between threats and mitigations might come from in your mind? (So it's an implicit part of the Charter by reference to 6973?) Or maybe I'm missing a more direct link in the Charter text? I can take a look at the text and see how heavy of a lift that would be to add mitigations... but my initial sense is that a quick addition to each technique subsection that listed mitigations -- instead of a new big section divorced from the specific techniques -- that have been known to work might not be so hard. E.g., "Configuring different DNS endpoints manually can get around some basic forms of DNS-based censorship". What I do worry about is that mitigations change faster than censorship techniques themselves (do people agree?), so this might be a place where there could be a lot of cruft (if we keep things) or document churn (if we remove outdated things)... for example, how should this document talk about domain fronting now that that's increasingly not a thing? Should it talk about it as an effective mitigation at one time? Or since it's not so effective anymore, should it drop discussion of fronting altogether? (of course, some of this is wrapped up in how often this kind of document might need to be updated...) Anyway, --000000000000d99f23058e6197da Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Fri, May 17, 2019 at 5:00 PM Shiva= n Sahib <ssahib@salesforce.com<= /a>> wrote:
<= div dir=3D"ltr">
Hi Stephen,

On Fri, May 17, 2019 at 12:58 AM Stephen F= arrell <s= tephen.farrell@cs.tcd.ie> wrote:

...

-1 to extending the scope. That could be done in other
documents, each of which would likely take some time. FWIW,
personally, I'd recommend just getting this one to the
RFC editor queue, pocketing the progress, and then the RG
can move on to new documents documenting other mechanisms
for and countering censorship.

Cheers,
S.

The suggestion to include censorship= -countering techniques came up because of the concern that as the draft sta= nds, it would not fall under PEARG's charter. Happy to hear disagreemen= ts about that though.=C2=A0

Heya Shivan,

I'd love to hear more ela= boration here tomorrow about this (or on the list if you have time before t= hen).

I looked at the Charter and like Stephen I&#= 39;m not necessarily seeing where it would require a descriptive draft abou= t techniques also talk about mitigations. I suspect that RFC 6973's str= ucture of "privacy threats" and "threat mitigations" is= where the linking between threats and mitigations might come from in your = mind? (So it's an implicit part of the Charter by reference to 6973?) O= r maybe I'm missing a more direct link in the Charter text?
<= br>
I can take a look at the text and see how heavy of a lift tha= t would be to add mitigations... but my initial sense is that a quick addit= ion to each technique subsection that listed mitigations -- instead of a ne= w big section divorced from the specific techniques -- that have been known= to work might not be so hard. E.g., "Configuring different DNS endpoi= nts manually can get around some basic forms of DNS-based censorship".=

What I do worry about is that mitigations ch= ange faster than censorship techniques themselves (do people agree?), so th= is might be a place where there could be a lot of cruft (if we keep things)= or document churn (if we remove outdated things)... for example, how shoul= d this document talk about domain fronting now that that's increasingly= not a thing? Should it talk about it as an effective mitigation at one tim= e? Or since it's not so effective anymore, should it drop discussion of= fronting altogether?

(of course, some of this is = wrapped up in how often this kind of document might need to be updated...)<= br>

Anyway,
--000000000000d99f23058e6197da-- From nobody Tue Jul 23 20:02:33 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 340DB1209E9 for ; Tue, 23 Jul 2019 20:02:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=salesforce.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 13XMs-OrgQN8 for ; Tue, 23 Jul 2019 20:02:24 -0700 (PDT) Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 200801209CA for ; Tue, 23 Jul 2019 20:02:24 -0700 (PDT) Received: by mail-io1-xd2c.google.com with SMTP id s7so86284557iob.11 for ; Tue, 23 Jul 2019 20:02:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=salesforce.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=aQQwEcED53Kw2/cpbhnctvT3Vc3xjMy9M3IFw41FdVI=; b=h9c0NwPD6miU03/z8v1Y1waUMJsX+g37iIA/sOMoLHgBZCGuWMPYfr+zACykzQiBE/ P0LntIulpCBzjUZvHZl1xIip62TS5GTdMBoC1aNnHe6tdzzB98ahgs0t/r0371Lg5yJv Yw6TpzPpIPandyP0KuPjLZBkao1lK3noScJeM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=aQQwEcED53Kw2/cpbhnctvT3Vc3xjMy9M3IFw41FdVI=; b=kT+ZFwiBunvhaGukeN+Ko6khfja2o8rhc5pqjnDX03vsyqxIIQKqJn4SMj0ePQ/eUN WgfYF+4LWadSYSNWYf5Sxu8r6VtzvGIWcCLN7TEgyumidmScAjl01u8+d5BOzGDs6ZHN FzSeBbset3mIb6t37X/xPDHd7qnVXo9B1oP6aN9nKg1kZW60YyS31U35oa+/VD8r127/ /3GUHoE3w0sm/o+wVJS8FAeWv/M4DOJ5Co+2b2mZgCgAt1yFwR2GhYogLyHqbJZejxMs LmFfP34xlXugcDrf52o/skIo6mnbiYSChVAL4UbnbyUV49+6ni3mHafSCWzLnA3urjuH pGZA== X-Gm-Message-State: APjAAAV2VD0+5oOEdd24qHBu2qMkAdOntZMAoEPQfH78nGBrf5blyw3N Vxrbk1konSiTKECG0xzjRztRyHfzRkq6lrOH3nWEag== X-Google-Smtp-Source: APXvYqxsIsGtBKOnqFHbLJ2A+WN+xnVvQFIg/KGLTKkHU8xwJA0ACCz8t2lAYbflNBdIvHtNxcEbGbYZIqiP+y7MYOA= X-Received: by 2002:a6b:dc08:: with SMTP id s8mr28187718ioc.209.1563937343223; Tue, 23 Jul 2019 20:02:23 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Shivan Sahib Date: Tue, 23 Jul 2019 23:02:16 -0400 Message-ID: To: Joseph Lorenzo Hall Cc: Stephen Farrell , Stan Adams , Nick Feamster , pearg@irtf.org Content-Type: multipart/alternative; boundary="000000000000cb7665058e6489f5" Archived-At: Subject: Re: [Pearg] descriptive censorship work: draft-hall-censorship-tech X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2019 03:02:32 -0000 --000000000000cb7665058e6489f5 Content-Type: text/plain; charset="UTF-8" Hi Joe! >>> -1 to extending the scope. That could be done in other >>> documents, each of which would likely take some time. FWIW, >>> personally, I'd recommend just getting this one to the >>> RFC editor queue, pocketing the progress, and then the RG >>> can move on to new documents documenting other mechanisms >>> for and countering censorship. >>> >>> Cheers, >>> S. >>> >> >> The suggestion to include censorship-countering techniques came up >> because of the concern that as the draft stands, it would not fall under >> PEARG's charter. Happy to hear disagreements about that though. >> > > Heya Shivan, > > I'd love to hear more elaboration here tomorrow about this (or on the list > if you have time before then). > > I looked at the Charter and like Stephen I'm not necessarily seeing where > it would require a descriptive draft about techniques also talk about > mitigations. I suspect that RFC 6973's structure of "privacy threats" and > "threat mitigations" is where the linking between threats and mitigations > might come from in your mind? (So it's an implicit part of the Charter by > reference to 6973?) Or maybe I'm missing a more direct link in the Charter > text? > I mentioned the concerns mainly because it was brought up by someone to me - it seems though that the group is in agreement that the document is worth pursuing as-is (and I think so too), plus I agree that we want to avoid scope-creep. Thanks, Shivan --000000000000cb7665058e6489f5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Joe= !


-1 to extending the = scope. That could be done in other
documents, each of which would likely= take some time. FWIW,
personally, I'd recommend just getting this o= ne to the
RFC editor queue, pocketing the progress, and then the RG
c= an move on to new documents documenting other mechanisms
for and counter= ing censorship.

Cheers,
S.

Th= e suggestion to include censorship-countering techniques came up because of= the concern that as the draft stands, it would not fall under PEARG's = charter. Happy to hear disagreements about that though.=C2=A0=C2=A0

Heya Shivan,

<= /div>
I'd love to hear more elaboration here tomorrow about this (o= r on the list if you have time before then).

I loo= ked at the Charter and like Stephen I'm not necessarily seeing where it= would require a descriptive draft about techniques also talk about mitigat= ions. I suspect that RFC 6973's structure of "privacy threats"= ; and "threat mitigations" is where the linking between threats a= nd mitigations might come from in your mind? (So it's an implicit part = of the Charter by reference to 6973?) Or maybe I'm missing a more direc= t link in the Charter text?

I mentioned the concerns mainly because it was brought= up by someone to me - it seems though that the group is in agreement that = the document is worth pursuing as-is (and I think so too), plus I agree tha= t we want to avoid scope-creep.=C2=A0

Thanks,
Shivan
--000000000000cb7665058e6489f5-- From nobody Wed Jul 24 07:38:46 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ECDED12037B for ; Wed, 24 Jul 2019 07:38:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.998 X-Spam-Level: X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cdt.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2j_LYLoRL9Cd for ; Wed, 24 Jul 2019 07:38:40 -0700 (PDT) Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8F9CD120357 for ; Wed, 24 Jul 2019 07:38:40 -0700 (PDT) Received: by mail-io1-xd36.google.com with SMTP id j5so85912715ioj.8 for ; Wed, 24 Jul 2019 07:38:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cdt.org; s=google; h=mime-version:from:date:message-id:subject:to; bh=1Vs+NtzkHdoM2tC8BctWGsCQs/SOcVqDYwYPGXsF274=; b=rhGM3Fcd/Gylu+W1gYkr52gNur4Xce9j1gGxC85LGSX2EIl/RoH/g9Rz/yKqUUeZaP Hy9sJzW6cC+CCWzipBVFjyOMKeFoQ7az5M5kfYro4xUGg/3ShCsWYAJQB6mYfuCt2Una Fwz4Fhlt0WK1DLwED+Uq/wV7xe/ijO3nCoBAo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=1Vs+NtzkHdoM2tC8BctWGsCQs/SOcVqDYwYPGXsF274=; b=bNepIUHugojmUxjn1OFCrXrnsdjD6QXD1TiksyB18Yjk/jPEUozrfajAbHKhw/YRH1 g23WFYZSNqV1ZQpAXsrpBuOWpC+eg1eo12ICZ25pJ2I1esjLvtzL4r8T0OlUMJNy/488 CXvrFu40M7VRSi3KH4GqJoneq6yGG5VByb+ANkA4fXUHc8KNQMAkWS4JTyl355pOi/mB dc9WRfnMya62bMPR213j4zVJ5+b2NPTvNnWUUncGVmmoIZef9MBrOYCh/8JOrV0WjPar oVNZ8VXrI7GG6SyXUkoQ7ZsEl4Db2WEmfSoOi+s8mi0KfsXjbPNsOgaTgsNypSy2rXG+ mgGw== X-Gm-Message-State: APjAAAVsy3oM69/w5QEAfd8Fz+S4/XTmvyATfWe8GpeyjgWTY5f1DhWV jOtmWIy1L+lZejm0Ca0ml7mLrwIYTTILb67hg23d2aSv9X9bjN2A X-Google-Smtp-Source: APXvYqwKr/4ZfjqOLwZNCXfhVdM/vvQvq1/g370MmIr7f8hm6g6R+p9SGjeKMBeR2XU44vUJ26oEvYbf/Ua0mURJTnA= X-Received: by 2002:a5e:9506:: with SMTP id r6mr12011786ioj.219.1563979119409; Wed, 24 Jul 2019 07:38:39 -0700 (PDT) MIME-Version: 1.0 From: Joseph Lorenzo Hall Date: Wed, 24 Jul 2019 10:38:28 -0400 Message-ID: To: pearg@irtf.org Content-Type: multipart/alternative; boundary="000000000000d95449058e6e435f" Archived-At: Subject: [Pearg] new reidentification result using GANs... X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2019 14:38:44 -0000 --000000000000d95449058e6e435f Content-Type: text/plain; charset="UTF-8" https://www.nature.com/articles/s41467-019-10933-3 (PDF: https://www.nature.com/articles/s41467-019-10933-3.pdf ) # Estimating the success of re-identifications in incomplete datasets using generative models Luc Rocher, Julien M. Hendrickx & Yves-Alexandre de Montjoye Abstract: While rich medical, behavioral, and socio-demographic data are key to modern data-driven research, their collection and use raise legitimate privacy concerns. Anonymizing datasets through de-identification and sampling before sharing them has been the main tool used to address those concerns. We here propose a generative copula-based method that can accurately estimate the likelihood of a specific person to be correctly re-identified, even in a heavily incomplete dataset. On 210 populations, our method obtains AUC scores for predicting individual uniqueness ranging from 0.84 to 0.97, with low false-discovery rate. Using our model, we find that 99.98% of Americans would be correctly re-identified in any dataset using 15 demographic attributes. Our results suggest that even heavily sampled anonymized datasets are unlikely to satisfy the modern standards for anonymization set forth by GDPR and seriously challenge the technical and legal adequacy of the de-identification release-and-forget model. -- Joseph Lorenzo Hall Chief Technologist, Center for Democracy & Technology [https://www.cdt.org] 1401 K ST NW STE 200, Washington DC 20005-3497 e: joe@cdt.org, p: 202.407.8825, pgp: https://josephhall.org/gpg-key Fingerprint: 3CA2 8D7B 9F6D DBD3 4B10 1607 5F86 6987 40A9 A871 --000000000000d95449058e6e435f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
# Estimating the success of re-identifications in incomplete d= atasets using generative models

Luc Rocher, Julien= M. Hendrickx & Yves-Alexandre de Montjoye

Ab= stract: While rich medical, behavioral, and socio-demographic data are key = to modern data-driven research, their collection and use raise legitimate p= rivacy concerns. Anonymizing datasets through de-identification and samplin= g before sharing them has been the main tool used to address those concerns= . We here propose a generative copula-based method that can accurately esti= mate the likelihood of a specific person to be correctly re-identified, eve= n in a heavily incomplete dataset. On 210 populations, our method obtains A= UC scores for predicting individual uniqueness ranging from 0.84 to 0.97, w= ith low false-discovery rate. Using our model, we find that 99.98% of Ameri= cans would be correctly re-identified in any dataset using 15 demographic a= ttributes. Our results suggest that even heavily sampled anonymized dataset= s are unlikely to satisfy the modern standards for anonymization set forth = by GDPR and seriously challenge the technical and legal adequacy of the de-= identification release-and-forget model.

--
Joseph Lorenz= o Hall
Chief Technologist, Center for Democracy & Technology [https://www.cdt.org]
140= 1 K ST NW STE 200, Washington DC 20005-3497
e: joe@cdt.org, p: 202.407.8825, pgp: https://josephhall.org/gpg= -key
Fingerprint: 3CA2 8D7B 9F6D DBD3 4B10 =C2=A01607 5F86 6987 40A9= A871

--000000000000d95449058e6e435f-- From nobody Sun Jul 28 09:03:39 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 29F5A120025 for ; Sun, 28 Jul 2019 09:03:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=salesforce.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6uDsvj7XgN60 for ; Sun, 28 Jul 2019 09:03:34 -0700 (PDT) Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E284012002E for ; Sun, 28 Jul 2019 09:03:33 -0700 (PDT) Received: by mail-io1-xd34.google.com with SMTP id e20so84450494iob.9 for ; Sun, 28 Jul 2019 09:03:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=salesforce.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=ET43H9AD66NE7pvGbYEM79kUkCFEHXeaJpNB6VsKlTk=; b=gwT4hbFrrM/JwThBkH4KLn1J+MHfqXZz/KdEKtGf/XvfuHhUIQMdcJ/Q0m/DF4EN6D 8JX2NlPIlLz6et6AVjtXGJCfI06TWl/Fx1Qy6SurmyeqgwNvwztteVjIGkP7sAk3xdD2 llDR9EUN8pwgynniqCsGutxykc3e51q3G569Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=ET43H9AD66NE7pvGbYEM79kUkCFEHXeaJpNB6VsKlTk=; b=PklMjaNGu/KnL6/iUmzcNrP1MxSypA4ELEoV6RhxMAtVy4PB08qQObnkMOubhyjjiZ J32RKzH/SwLx5EowxKdwI8D+zCbSEdR/5RnMBAWQNVTSh2NYlH7tyKPVpgBS6wztEqVY ifKAGSadpE+6KPrJbLYNMwmoU4OkhdOoVGaW+oy2NGr6vTz2skwn4oLemtDKpzJe1yu7 4TRfEb4PnHRvk0a0w+KSTvlalZjC3QOVHyCc7KsBPjz/QCELGVNU8v4PAJ+O0TMR9R4U zGCBC00EdQOOvNsjQpjYBRepSAuiP3NDFf7IHulP+fC+MHAS1Ire5cJnJoHzqiW6JC8P pEdg== X-Gm-Message-State: APjAAAVkpX0wjs3tp0yFy+C9ylimWfifQRX6Lzk4tptLfpa5UEQf0ZLZ MtBjDmFQfvDlCZ8rqkSjLaj7rWlPv6VxFUZEEADEfmqjhl8= X-Google-Smtp-Source: APXvYqzBfLVMU3D/E9qCfahNM0UBw8kinq9K959qZVUNCgRxQUsrBBYYbU1fIErvUGu7x0QcIgDZaBqx+ut1O3VrIyM= X-Received: by 2002:a6b:6d08:: with SMTP id a8mr87263408iod.191.1564329812825; Sun, 28 Jul 2019 09:03:32 -0700 (PDT) MIME-Version: 1.0 From: Shivan Sahib Date: Sun, 28 Jul 2019 12:03:22 -0400 Message-ID: To: "pearg@irtf.org" Content-Type: multipart/alternative; boundary="000000000000ce3be1058ebfeae6" Archived-At: Subject: [Pearg] Minutes for PEARG @ IETF 105 X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Jul 2019 16:03:37 -0000 --000000000000ce3be1058ebfeae6 Content-Type: text/plain; charset="UTF-8" Hi all, the meeting minutes are up: https://datatracker.ietf.org/doc/minutes-105-pearg/ Please let us know if there are any corrections/edits. Thanks to Ben Schwartz for the notes! --000000000000ce3be1058ebfeae6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi all, the meeting minutes are up:=C2=A0

Please let us know if there are any correc= tions/edits. Thanks to Ben Schwartz for the notes!
--000000000000ce3be1058ebfeae6-- From nobody Mon Jul 29 14:50:11 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7F98912006F; Mon, 29 Jul 2019 14:49:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CKxI2l_3EY4E; Mon, 29 Jul 2019 14:48:58 -0700 (PDT) Received: from balrog.mythic-beasts.com (balrog.mythic-beasts.com [IPv6:2a00:1098:0:82:1000:0:2:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CC8B312004E; Mon, 29 Jul 2019 14:48:58 -0700 (PDT) Received: from [81.187.2.149] (port=34207 helo=[192.168.0.81]) by balrog.mythic-beasts.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1hsDVl-00006l-Dd; Mon, 29 Jul 2019 22:48:57 +0100 From: Colin Perkins Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Message-Id: <24106542-A3AF-495F-99BF-B800B6E2FED6@csperkins.org> Date: Mon, 29 Jul 2019 22:48:50 +0100 Cc: pearg@irtf.org To: irtf-announce@irtf.org X-Mailer: Apple Mail (2.3445.104.11) X-BlackCat-Spam-Score: 4 Archived-At: Subject: [Pearg] PEARG approved as a full IRTF Research Group X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2019 21:49:01 -0000 Newly chartered IRTF research groups meet for one year as a Proposed = Research Group, then are subject to a review to decide whether they will = continue. The review for the Privacy Enhancements and Assessments = Research Group (PEARG) took place last week, during the IETF 105 = meeting, with the IRTF chair and IAB meeting with the RG chairs. I=E2=80=99m pleased to announce that the review was a success, and that = PEARG has been approved as a full IRTF research group. Colin Perkins IRTF Chair From nobody Mon Jul 29 15:46:41 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E7D2B1200E3 for ; Mon, 29 Jul 2019 15:38:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.896 X-Spam-Level: X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, UNPARSEABLE_RELAY=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=mjmontpetit-com.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0tgugwGrTVdg for ; Mon, 29 Jul 2019 15:38:27 -0700 (PDT) Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 375DF1200B4 for ; Mon, 29 Jul 2019 15:38:27 -0700 (PDT) Received: by mail-io1-xd29.google.com with SMTP id i10so10358108iol.13 for ; Mon, 29 Jul 2019 15:38:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mjmontpetit-com.20150623.gappssmtp.com; s=20150623; h=from:in-reply-to:references:mime-version:date:message-id:subject:to :cc; bh=Al1ua0KUrsuO890ecihbncs0oO4z0Pw+p8KIq5rwuxo=; b=I8+pgLSm9Fmz4lKj8R5nsGAbVo7XJtfjm9hxEB5kNofhJZRQfkwInxt17Jo02wRtyl 4e73fPpa18QHti6FPwFrObg40F9ZXE1figNYL8pZ/PBX218L5ZR+ea0gzSCxSKIuxwpc RAwsqM8yR71npVwcRwAPkLJwKMv8SCLDmm4mJk1iMCMt6XN3jNCAXpeOl8fbTRJfe8Wa yE1i41WbSxfGoBfEV98F167Nt64/70dJ5bFPZsHgYtfbfeNn10uwiS/Z+Spbi3hIMsaP vNO370mGPAJpAYdWHjOiN4tiiyw6Vlc25GUtyyDnurCBIQiE0jZI6JoleictgsTQRqC2 YV2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:in-reply-to:references:mime-version:date :message-id:subject:to:cc; bh=Al1ua0KUrsuO890ecihbncs0oO4z0Pw+p8KIq5rwuxo=; b=kD/5DHuCoBhvQEDUj6Dzff3h/rCNFYxGCjJ/jWlq5/MznR0rpBcqMS6+czwd+joDjw E7hQg7O8E4IkZIlaQPEVC9+fKhlF5mAi34puJGJi3yUBdYWrU21ErVH9vpJhYa1WJiN7 DulIrDvYChwc1rYAO7Y40JlpFpwB7EtBXT5RlgBPO6bNnvNAfInuipKn/wIocDCvnTfQ exSWS7XLGyyLy5zXDS5wnVknCNIKYaAf26VNpOVy4Ri+STRi8Sld+II7hJjM1N5JZ8YU cjIQ4yOKs0NREhDezkwWochCaoarrJwKCVgu07gNPyYMOFVNZKgWr58sYKZa2zdiAyj5 te9A== X-Gm-Message-State: APjAAAUykvQsAjvibIeqYihkUQyd6CF7lDbEAzkawqF33Czgpzv2aEuP 6JFJi+3XQqRx0UVCKnYRetevpasWmL0aguG5OO4= X-Google-Smtp-Source: APXvYqwyO12d5pAgYKf7oLfKhSFLXMoQQbly8VPDUMaAyZ0WuNxCNCVkvitoMrY2fVXg+xDfgVHmcLVMOVAqwDyMcjE= X-Received: by 2002:a5d:915a:: with SMTP id y26mr104685850ioq.207.1564439906434; Mon, 29 Jul 2019 15:38:26 -0700 (PDT) Received: from 1058052472880 named unknown by gmailapi.google.com with HTTPREST; Mon, 29 Jul 2019 15:38:24 -0700 From: Marie-Jose Montpetit In-Reply-To: <24106542-A3AF-495F-99BF-B800B6E2FED6@csperkins.org> References: <24106542-A3AF-495F-99BF-B800B6E2FED6@csperkins.org> MIME-Version: 1.0 Date: Mon, 29 Jul 2019 15:38:24 -0700 Message-ID: To: Colin Perkins , irtf-announce@irtf.org Cc: pearg@irtf.org Content-Type: multipart/alternative; boundary="000000000000e555ac058ed98c57" Archived-At: X-Mailman-Approved-At: Mon, 29 Jul 2019 15:46:40 -0700 Subject: Re: [Pearg] [IRTF-Announce] PEARG approved as a full IRTF Research Group X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2019 22:38:29 -0000 --000000000000e555ac058ed98c57 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Congrats to the team! mjm Marie-Jos=C3=A9 Montpetit, Ph.D. Research Affiliate, MIT Media Laboratory mariejose@mjmontpetit.com mariejo@mit.edu On July 29, 2019 at 5:49:52 PM, Colin Perkins (csp@csperkins.org) wrote: Newly chartered IRTF research groups meet for one year as a Proposed Research Group, then are subject to a review to decide whether they will continue. The review for the Privacy Enhancements and Assessments Research Group (PEARG) took place last week, during the IETF 105 meeting, with the IRTF chair and IAB meeting with the RG chairs. I=E2=80=99m pleased to announce that the review was a success, and that PEA= RG has been approved as a full IRTF research group. Colin Perkins IRTF Chair --000000000000e555ac058ed98c57 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =
Cong= rats to the team!

m= jm

Marie-Jos=C3=A9 Montpetit, Ph.= D.
Research Affiliate, MIT Media Laboratory

On July 29, 2019 at 5:49:52 PM, Colin Perkins (csp@csperkins.org) wrote:

Newly chartere= d IRTF research groups meet for one year as a Proposed Research Group, then= are subject to a review to decide whether they will continue. The review f= or the Privacy Enhancements and Assessments Research Group (PEARG) took pla= ce last week, during the IETF 105 meeting, with the IRTF chair and IAB meet= ing with the RG chairs.

I=E2=80=99m pleased to announce that the review was a success, and that= PEARG has been approved as a full IRTF research group.

Colin Perkins
IRTF Chair

--000000000000e555ac058ed98c57-- From nobody Mon Jul 29 16:13:07 2019 Return-Path: X-Original-To: pearg@ietfa.amsl.com Delivered-To: pearg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 61EBE1200A4; Mon, 29 Jul 2019 16:13:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.3 X-Spam-Level: X-Spam-Status: No, score=-4.3 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cs.tcd.ie Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZGIi_i0_MhjD; Mon, 29 Jul 2019 16:13:02 -0700 (PDT) Received: from mercury.scss.tcd.ie (mercury.scss.tcd.ie [134.226.56.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DBB4112004E; Mon, 29 Jul 2019 16:13:01 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mercury.scss.tcd.ie (Postfix) with ESMTP id 6CC39BE24; Tue, 30 Jul 2019 00:12:58 +0100 (IST) X-Virus-Scanned: Debian amavisd-new at scss.tcd.ie Received: from mercury.scss.tcd.ie ([127.0.0.1]) by localhost (mercury.scss.tcd.ie [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JXnf13OGJZzD; Tue, 30 Jul 2019 00:12:56 +0100 (IST) Received: from [10.244.2.138] (95-45-153-252-dynamic.agg2.phb.bdt-fng.eircom.net [95.45.153.252]) by mercury.scss.tcd.ie (Postfix) with ESMTPSA id 8186BBE20; Tue, 30 Jul 2019 00:12:56 +0100 (IST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cs.tcd.ie; s=mail; t=1564441976; bh=QdXxcFbirVYxxgoxsR4EWnhblxUJJUuRHesfXjH/oE4=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=bhmqVPVJVyMsgFyFrRUMHb57tkkSn9UmDgC+luGJh7y1pKCJwMv9QTVfQtzuh0ohJ 2Ikd+2D4bOCN0ZCkRpJLOcZznJgkZCwFx5yLMIoTJ6u3jT9UsOfRr64Zt4BYqEZlB5 tYhPWltYQW+gjjji7zJ3/SbeOd0nZiSssBRVBNc4= To: Marie-Jose Montpetit , Colin Perkins , irtf-announce@irtf.org Cc: pearg@irtf.org References: <24106542-A3AF-495F-99BF-B800B6E2FED6@csperkins.org> From: Stephen Farrell Openpgp: id=5BB5A6EA5765D2C5863CAE275AB2FAF17B172BEA; url= Autocrypt: addr=stephen.farrell@cs.tcd.ie; prefer-encrypt=mutual; keydata= mQINBFo9UDIBEADUH4ZPcUnX5WWRWO4kEkHea5Y5eEvZjSwe/YA+G0nrTuOU9nemCP5PMvmh 5Cg8gBTyWyN4Z2+O25p9Tja5zUb+vPMWYvOtokRrp46yhFZOmiS5b6kTq0IqYzsEv5HI58S+ QtaFq978CRa4xH9Gi9u4yzUmT03QNIGDXE37honcAM4MOEtEgvw4fVhVWJuyy3w//0F2tzKr EMjmL5VGuD/Q9+G/7abuXiYNNd9ZFjv4625AUWwy+pAh4EKzS1FE7BOZp9daMu9MUQmDqtZU bUv0Q+DnQAB/4tNncejJPz0p2z3MWCp5iSwHiQvytYgatMp34a50l6CWqa13n6vY8VcPlIqO Vz+7L+WiVfxLbeVqBwV+4uL9to9zLF9IyUvl94lCxpscR2kgRgpM6A5LylRDkR6E0oudFnJg b097ZaNyuY1ETghVB5Uir1GCYChs8NUNumTHXiOkuzk+Gs4DAHx/a78YxBolKHi+esLH8r2k 4LyM2lp5FmBKjG7cGcpBGmWavACYEa7rwAadg4uBx9SHMV5i33vDXQUZcmW0vslQ2Is02NMK 7uB7E7HlVE1IM1zNkVTYYGkKreU8DVQu8qNOtPVE/CdaCJ/pbXoYeHz2B1Nvbl9tlyWxn5Xi HzFPJleXc0ksb9SkJokAfwTSZzTxeQPER8la5lsEEPbU/cDTcwARAQABtDJTdGVwaGVuIEZh cnJlbGwgKDIwMTcpIDxzdGVwaGVuLmZhcnJlbGxAY3MudGNkLmllPokCQAQTAQgAKgIbAwUJ CZQmAAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAUCWj6jdwIZAQAKCRBasvrxexcr6o7QD/9m x9DPJetmW794RXmNTrbTJ44zc/tJbcLdRBh0KBn9OW/EaAqjDmgNJeCMyJTKr1ywaps8HGUN hLEVkc14NUpgi4/Zkrbi3DmTp25OHj6wXBS5qVMyVynTMEIjOfeFFyxG+48od+Xn7qg6LT7G rHeNf+z/r0v9+8eZ1Ip63kshQDGhhpmRMKu4Ws9ZvTW2ACXkkTFaSGYJj3yIP4R6IgwBYGMz DXFX6nS4LA1s3pcPNxOgrvCyb60AiJZTLcOk/rRrpZtXB1XQc23ZZmrlTkl2HaThL6w3YKdi Ti1NbuMeOxZqtXcUshII45sANm4HuWNTiRh93Bn5bN6ddjgsaXEZBKUBuUaPBl7gQiQJcAlS 3MmGgVS4ZoX8+VaPGpXdQVFyBMRFlOKOC5XJESt7wY0RE2C8PFm+5eywSO/P1fkl9whkMgml 3OEuIQiP2ehRt/HVLMHkoM9CPQ7t6UwdrXrvX+vBZykav8x9U9M6KTgfsXytxUl6Vx5lPMLi 2/Jrsz6Mzh/IVZa3xjhq1OLFSI/tT2ji4FkJDQbO+yYUDhcuqfakDmtWLMxecZsY6O58A/95 8Qni6Xeq+Nh7zJ7wNcQOMoDGj+24di2TX1cKLzdDMWFaWzlNP5dB5VMwS9Wqj1Z6TzKjGjru q8soqohwb2CK9B3wzFg0Bs1iBI+2RuFnxLkCDQRaPVAyARAA+g3R0HzGr/Dl34Y07XqGqzq5 SU0nXIu9u8Ynsxj7gR5qb3HgUWYEWrHW2jHOByXnvkffucf5yzwrsvw8Q8iI8CFHiTYHPpey 4yPVn6R0w/FOMcY70eTIu/k6EEFDlDbs09DtKcrsT9bmN0XoRxITlXwWTufYqUnmS+YkAuk+ TLCtUin7OdaS2uU6Ata3PLQSeM2ZsUQMmYmHPwB9rmf+q2I005AJ9Q1SPQ2KNg/8xOGxo13S VuaSqYRQdpV93RuCOzg4vuXtR+gP0KQrus/P2ZCEPvU9cXF/2MIhXgOz207lv3iE2zGyNXld /n8spvWk+0bH5Zqd9Wcba/rGcBhmX9NKKDARZqjkv/zVEP1X97w1HsNYeUFNcg2lk9zQKb4v l1jx/Uz8ukzH2QNhU4R39dbF/4AwWuSVkGW6bTxHJqGs6YimbfdQqxTzmqFwz3JP0OtXX5q/ 6D4pHwcmJwEiDNzsBLl6skPSQ0Xyq3pua/qAP8MVm+YxCxJQITqZ8qjDLzoe7s9X6FLLC/DA L9kxl5saVSfDbuI3usH/emdtn0NA9/M7nfgih92zD92sl1yQXHT6BDa8xW1j+RU4P+E0wyd7 zgB2UeYgrp2IIcfG+xX2uFG5MJQ/nYfBoiALb0+dQHNHDtFnNGY3Oe8z1M9c5aDG3/s29QbJ +w7hEKKo9YMAEQEAAYkCJQQYAQgADwUCWj1QMgIbDAUJCZQmAAAKCRBasvrxexcr6qwvD/9b Rek3kfN8Q+jGrKl8qwY8HC5s4mhdDJZI/JP2FImf5J2+d5/e8UJ4fcsT79E0/FqX3Z9wZr6h sofPqLh1/YzDsYkZDHTYSGrlWGP/I5kXwUmFnBZHzM3WGrL3S7ZmCYMdudhykxXXjq7M6Do1 oxM8JofrXGtwBTLv5wfvvygJouVCVe87Ge7mCeY5vey1eUi4zSSF1zPpR6gg64w2g4TXM5qt SwkZVOv1g475LsGlYWRuJV8TA67yp1zJI7HkNqCo8KyHX0DPOh9c+Sd9ZX4aqKfqH9HIpnCL AYEgj7vofeix7gM3kQQmwynqq32bQGQBrKJEYp2vfeO30VsVx4dzuuiC5lyjUccVmw5D72J0 FlGrfEm0kw6D1qwyBg0SAMqamKN6XDdjhNAtXIaoA2UMZK/vZGGUKbqTgDdk0fnzOyb2zvXK CiPFKqIPAqKaDHg0JHdGI3KpQdRNLLzgx083EqEc6IAwWA6jSz+6lZDV6XDgF0lYqAYIkg3+ 6OUXUv6plMlwSHquiOc/MQXHfgUP5//Ra5JuiuyCj954FD+MBKIj8eWROfnzyEnBplVHGSDI ZLzL3pvV14dcsoajdeIH45i8DxnVm64BvEFHtLNlnliMrLOrk4shfmWyUqNlzilXN2BTFVFH 4MrnagFdcFnWYp1JPh96ZKjiqBwMv/H0kw== Message-ID: Date: Tue, 30 Jul 2019 00:12:55 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ftGoz4euh6ar5mvGm6vCtxjN1tmXqa0Qt" Archived-At: Subject: Re: [Pearg] [IRTF-Announce] PEARG approved as a full IRTF Research Group X-BeenThere: pearg@irtf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Privacy Enhancements and Assessment Proposed RG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2019 23:13:05 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ftGoz4euh6ar5mvGm6vCtxjN1tmXqa0Qt Content-Type: multipart/mixed; boundary="SuMxI1ymJmdyJ4rUbqtCUpHpYzvNt0l8L"; protected-headers="v1" From: Stephen Farrell To: Marie-Jose Montpetit , Colin Perkins , irtf-announce@irtf.org Cc: pearg@irtf.org Message-ID: Subject: Re: [Pearg] [IRTF-Announce] PEARG approved as a full IRTF Research Group References: <24106542-A3AF-495F-99BF-B800B6E2FED6@csperkins.org> In-Reply-To: --SuMxI1ymJmdyJ4rUbqtCUpHpYzvNt0l8L Content-Type: multipart/mixed; boundary="------------644650695B07ADB23CA74A9C" Content-Language: en-GB This is a multi-part message in MIME format. --------------644650695B07ADB23CA74A9C Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 29/07/2019 23:38, Marie-Jose Montpetit wrote: > Congrats to the team! Indeed! But congrats also to Colin for encouraging good work - that's also an important part of all this. I look forward to seeing more pearg stuff. Sad to say that I missed most of the ietf-105 session due to other things;-( But I did really like the DNS privacy talk [1] and I'm sure others were also good. I hope to see [1] develop into some concrete guidelines for DNS privacy - if that happens, it'd be a really good "win" for the RG. Cheers, S. [1] https://datatracker.ietf.org/meeting/105/materials/slides-105-pearg-encry= pted-dns-privacy-a-traffic-analysis-perspective-01 >=20 > mjm >=20 > Marie-Jos=C3=A9 Montpetit, Ph.D. > Research Affiliate, MIT Media Laboratory > mariejose@mjmontpetit.com > mariejo@mit.edu >=20 > On July 29, 2019 at 5:49:52 PM, Colin Perkins (csp@csperkins.org) wrote= : >=20 > Newly chartered IRTF research groups meet for one year as a Proposed > Research Group, then are subject to a review to decide whether they wil= l > continue. The review for the Privacy Enhancements and Assessments Resea= rch > Group (PEARG) took place last week, during the IETF 105 meeting, with t= he > IRTF chair and IAB meeting with the RG chairs. >=20 > I=E2=80=99m pleased to announce that the review was a success, and that= PEARG has > been approved as a full IRTF research group. >=20 > Colin Perkins > IRTF Chair >=20 >=20 --------------644650695B07ADB23CA74A9C Content-Type: application/pgp-keys; name="0x5AB2FAF17B172BEA.asc" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0x5AB2FAF17B172BEA.asc" -----BEGIN PGP PUBLIC KEY BLOCK----- mQINBFo9UDIBEADUH4ZPcUnX5WWRWO4kEkHea5Y5eEvZjSwe/YA+G0nrTuOU9nem CP5PMvmh5Cg8gBTyWyN4Z2+O25p9Tja5zUb+vPMWYvOtokRrp46yhFZOmiS5b6kT q0IqYzsEv5HI58S+QtaFq978CRa4xH9Gi9u4yzUmT03QNIGDXE37honcAM4MOEtE gvw4fVhVWJuyy3w//0F2tzKrEMjmL5VGuD/Q9+G/7abuXiYNNd9ZFjv4625AUWwy +pAh4EKzS1FE7BOZp9daMu9MUQmDqtZUbUv0Q+DnQAB/4tNncejJPz0p2z3MWCp5 iSwHiQvytYgatMp34a50l6CWqa13n6vY8VcPlIqOVz+7L+WiVfxLbeVqBwV+4uL9 to9zLF9IyUvl94lCxpscR2kgRgpM6A5LylRDkR6E0oudFnJgb097ZaNyuY1ETghV B5Uir1GCYChs8NUNumTHXiOkuzk+Gs4DAHx/a78YxBolKHi+esLH8r2k4LyM2lp5 FmBKjG7cGcpBGmWavACYEa7rwAadg4uBx9SHMV5i33vDXQUZcmW0vslQ2Is02NMK 7uB7E7HlVE1IM1zNkVTYYGkKreU8DVQu8qNOtPVE/CdaCJ/pbXoYeHz2B1Nvbl9t lyWxn5XiHzFPJleXc0ksb9SkJokAfwTSZzTxeQPER8la5lsEEPbU/cDTcwARAQAB tCFTdGVwaGVuIEZhcnJlbGwgPHN0ZXBoZW5AamVsbC5pZT6JAj0EEwEIACcFAlo9 UYwCGwMFCQmUJgAFCwkIBwIGFQgJCgsCBBYCAwECHgECF4AACgkQWrL68XsXK+qG CxAApYHWYgGOIL3G6/OpkejdAkQoCVQAK8LJUSf6vzwost4iVfxIKcKW/3RqKNKk rRl8beJ7j1CWXAz9+VXAOsE9+zNxXIDgGA7HlvJnhffl+qwibVgiHgUcJFhCSbBr sjC+1uULaTU8zYEyET//GOGPLF+X+degkE/sesh4zcEAjF7fGPnlncdCCH3tvPZZ sdTcjwOCRVonKsDgQzBTCMz/RPBfEFX44HZx4g1UQAcCA4xlucY8QkJEyCrSNGpG nvGK8DcGSmnstl1/a9fnlhpdFxieX3oY2phJ1WKkYTn6Advrek3UP71CKxpgtPmk d3iUUz/VZa0Cv6YxQXskspRDVEvdCMYSQBtJPQ4y2+5UxVR9GIQXenwYp9AP2niv Voh+ITsDWWeWnnvYMq07rSDjq0nGdj41MJkNX+Yb2PXVyXItcj5ybE3T2+y3pSBG FEZYJGuaL4NwtBJFMOdOtBmUOPbetS2971EL3Izxb7ibOZWDwexv+8R6SWYfP1wV N3p46RyBQuXqJV8ccE11m6vtZTGSYgnLUUFZMRQYH+0hwuYe0T3AA18xDdSYsa8v ovCCd3l5S4UNzIM2PMChqGrEzKapUpZg7+8ACcxRU3b9Ihd7WYjJ+pQPCoWYKozv tEvenbNpE/govO/ED3B14e+R2yevRPjRrsN7PJzSf15fQLuJARwEEAEIAAYFAlo9 UqAACgkQLzyHNoBfjaLrSwf+MIHbFRQ4O5cmLYR5sIByWelN3SuRN/gW8rpKo9Ok Cz6An8uV/iCXy5tNMLzzi0BFl8f22DwBcC5qy9qnlIAdogWam1qWoTAoAD8veEqm uKhYrqJsCcAyNrKYmK0hP3rpHxx1LySDmKYXmw/8qtBXKHTouMm+5tSsznhykRMT AAr2p7PSaHgo+hIVaW/rKSspHjDhhZS+G9mtOZad1IH29M6G1Q1NCO0Ywe8krKLQ IAQlFxtgvOqpPOZNzeKBa/+KbE8TGgMWrkOhC8OeEM5PVzdDhlhD9kPzB/pCKDF5 DofJ/ZRqnDpbKPQ0bsW38AOig3kOc0A27awiBEw3urqR1YkCMwQQAQgAHRYhBH4X CgRchM9GDit5oBDvedn9g1MSBQJbtyScAAoJEBDvedn9g1MSI/oP/0A9J9nrnBMq Zpm857lfYWw+rshLK+tyeP4OQeOqnDFvs9jePpcyJLG3DF2r6VbVKPQq+AE6Uf5h cJBDEN6BjEhRPSbLcqG3A1cz/nNwm8rPmNp+oKhmaBBQGxwciMLmzgynsDydnjPp MyEs04zvsbsl4vrp2095o105l8KcrrxQrioFjbwveGwHQK9bxJKhx9D+gIk+MouB ur45UDKTZkMZrr9FGrtkyXCGAxvKdcNC5Oa8z9sj1rcUJfG/OpVAMWhArdlZbFUQ yoX6pU2Zb1CR2qpWAVerGSfBhmfCyStjARqaKxlftjO+Bj3Jj73Cr5eqej3qB5+V 4BCsPjr4RLvVbYUCPsRdxWc+nBLlfVYkRURu21g1hFm5KFPjgUkyo1s4vjUOY8Dy I+xLGF7f/IhUBG6l+Vswhpwu7ydalZkeFiPx5xna5NfbEYxvsIf71DvipGvIOaHv X4egWoFgm8n/9c3rcMxJtpwHPSsUt5dgLsyu6VE0IbvOAc3dN7CWJ355DVFJq9Zg 2YVf0izSpyyzJeGsgkfjW6xpmdvZxuT2UcN4BTcm6vYqueASGrb3lfhzC5gpeVsc /MoSjTS65vNWbpzONZWMZuLEFraxWJzC0JrDK3NCd0VN3kstqGkVbUIiYOnUm8Vu 4zoVMLlGWzHLIGoPRG2nRezn1YyNfyb5iQGcBBABCgAGBQJbxcflAAoJEGo7ETk8 pK1gE7QL/ApC5P68W5DrI1787WJVZv1u4t/g39vTr7Xer3UMTVQg10vpa7pmqOGh jIDzDMg3Pe3K3M7fVzfAlUA1qw6ne4RCueVoRKpubeF4AlYbMr0K6hNCPjt5uAxm bBVuejKTc6pru5rv5gKL0nDbr+Snft5xt7juBLSSimw0/41sZnkjCxo9rF/RA/v6 +uWyK171RKmsEYu8fFtw1eqUNt/Xj792TUixE3pxXheNtQtZGk/9P3W83ChhG4Fh 5EQsn0pIh9wZIAbMRLpgRKyW87fWHZC8/YH8h7afarvn9Thl5pFUldCe22mNJj6K LChn2aEHQd+PdY1GBpZEcmNEUPuovwzatM0h64hCzTm41eDqRfihZVBT7TbfXQnv 8rywa42Mk756RGzzEZcQEhwQXZcMQUfxIQQ2VyJo0zG36VdZTQF7TF/4Lz7/3cJ5 6jOIm+dwPXtu+C2wAQuD4USOLt4JWPYpqzDfHYJIND/497P9Z9SuQeahr2ez3DRB g3qsHEjBV7QyU3RlcGhlbiBGYXJyZWxsICgyMDE3KSA8c3RlcGhlbi5mYXJyZWxs QGNzLnRjZC5pZT6JAkAEEwEIACoCGwMFCQmUJgAFCwkIBwIGFQgJCgsCBBYCAwEC HgECF4AFAlo+o3cCGQEACgkQWrL68XsXK+qO0A//ZsfQzyXrZlu/eEV5jU620yeO M3P7SW3C3UQYdCgZ/TlvxGgKow5oDSXgjMiUyq9csGqbPBxlDYSxFZHNeDVKYIuP 2ZK24tw5k6duTh4+sFwUualTMlcp0zBCIzn3hRcsRvuPKHfl5+6oOi0+xqx3jX/s /69L/fvHmdSKet5LIUAxoYaZkTCruFrPWb01tgAl5JExWkhmCY98iD+EeiIMAWBj Mw1xV+p0uCwNbN6XDzcToK7wsm+tAIiWUy3DpP60a6WbVwdV0HNt2WZq5U5Jdh2k 4S+sN2CnYk4tTW7jHjsWarV3FLISCOObADZuB7ljU4kYfdwZ+WzenXY4LGlxGQSl AblGjwZe4EIkCXAJUtzJhoFUuGaF/PlWjxqV3UFRcgTERZTijguVyREre8GNERNg vDxZvuXssEjvz9X5JfcIZDIJpdzhLiEIj9noUbfx1SzB5KDPQj0O7elMHa1671/r wWcpGr/MfVPTOik4H7F8rcVJelceZTzC4tvya7M+jM4fyFWWt8Y4atTixUiP7U9o 4uBZCQ0GzvsmFA4XLqn2pA5rVizMXnGbGOjufAP/efEJ4ul3qvjYe8ye8DXEDjKA xo/tuHYtk19XCi83QzFhWls5TT+XQeVTMEvVqo9Wek8yoxo67qvLKKqIcG9givQd 8MxYNAbNYgSPtkbhZ8SJARwEEAEIAAYFAlo9UqAACgkQLzyHNoBfjaLzHAgAlWT6 NXEGtw/r1miKNGcopzvzILQ9oB8rKI9U9EL6tOf/y2V5oYee/GyQDb3ZdoPxxYYc Jf+RyiH1nMoqUIZiZJaf3bJXinDZ5+AdfE++UR2NBvqaNyC6u3r24jo1B/sagKbY tWgsYtRqHLD4IWi37MZrVyjBuF7u14Q07+uhjq6mX2O/tHpCYw/Q82tbeTRPyUf1 WQOAfD1kfBpW9PvAva5Iw9FWeXpCXRzwxnCZhYfGfqtuSw6CPBYLdbikqML6FZ7E DuTBb/8um1wK7Y9bgeIQC+CYjhYB5RXa1tDJRab2Js4luCvSR0w/CgHw26293tlv e2Q6UTrmHxP5U22DlokCPQQTAQgAJwUCWj1QMgIbAwUJCZQmAAULCQgHAgYVCAkK CwIEFgIDAQIeAQIXgAAKCRBasvrxexcr6tJpD/4rrILH+meP07vrx8wW5eYuqCiP GYnh/CXxIF8eLrfbe5d4QRgtq+w6UeQPMyzKRIRiCoBXB2oJLBZHyxBPxZlg33dT MrEGn8QWKx2iNuz9rZMXyOSWFetuO01d/aUPd5BnbLbIyK5of8xCQlXM6KH8bc+9 gQ7edR9mfLTdvBf2FR522hg8BRBM1imKc3vO8v39+qIHHRjuiwxBBCAOhHtHRsZX ripS0uFA07dM46Oi/E8osjx6fQt/lH5z/PN+2adxYSrLSAXfr1oD3RxYNhuWgyGF L64/VCQb1YGjf0Z5MBPnWm9jgUoOY5K9eNSS0L83WeJjlF5+Q/WOgB+rb49Prm2D Feo9+S9f2V53Llz1WIspXJg6f+n9lmHE94MfQj1GAHCzI0FeL19lvM+LhD8jJSCb hrC3+yobyy/AUOs5Z3E+njjX1FF/VCVAs6iOa6i+XG+Y1hh3ir2y1kckJ5auT10M SU8GEZu9ayU4M3o3N9yxOjaoP0NuQ4MMLL/n/u4u94AeZaHPNBXn/hVfVRRmpRXt GKvJtFAEppGEYezB+bLKIm6XlpPkhnwYzleLZ7AMEco2C6QM8QPB3g3JpS3sqRhA 5rEP4lL16BmijmF+CHoPE/zwgKZbKpyVDqvIW5IDgvfIC2X4pbZDRvGIUKaGSB4+ ksZgUUnNyvfQr2p7jokCMwQQAQgAHRYhBH4XCgRchM9GDit5oBDvedn9g1MSBQJb tySbAAoJEBDvedn9g1MSeKkQAJm44jt1kwHgQgeDBKdjdvl0AjE0xVEQxriZ6lP/ l//34YT0auFfzsYIrChSpQXAEtobBAr4Ohw1Us+BZe+H5P8vm6LRuPwozC3SjwfX 4Iec8+9ot6tIVg4sbedDSgb/CCFVjsmIGcQ1P73JLJTBJ6mxYCV/gn3QC6bwDOFo 7kD9FDHCjRN8XfhHQ4Q9cYyt06uF31qG/aumgWYC9geCGgAwiHgwxNYb9GoJ0iZj CROwbYvLTcQgsVUW2bTmsVR13UVKDsdl02sRV7qcVYW6R0a3Ra8KudX+nt25H5DR Gd382KZ5W8pydsy/viTvD9z6v0ulChBYxAedIvGIClrhbxlLEPmIg4ImVOLGqsUg Vm32J95WOjEkk4PEZ12xSDBtwhSJqmJNboWlfmw43KdIbY8zNhffIO3N6O7FsdGx mqyHeLoTpqY+ySVUPpbuyW8ujnI/J//+6hdTZ9dQsEJQlWngKuWOQ5ma58MPSN88 zllsqhZAFQjNxqnkSzL6ZQ+v/jvuRRe16B80AeO55DsmbWsMv/YLLD1mSi7+Khy2 EtMBhgojWwrGMvdLN6X3mnzNJEscYyLxM9tSk+iySP2sLthK0BVgpAzBSdaf/ezI z60P+neHDzteNFf8Mn7lmgYk1amvZoJ29s5+n2HwxyRL5dVMyMdyQmntubbctfqr Z0tIiQGcBBABCgAGBQJbxcflAAoJEGo7ETk8pK1gnCYMAJY4FeIYjlIXGghFWzsB 4fYwK1+iaFpU3fSto5qcrqVtVPjXpwqczqBWeXGyQxiB0kan4OVAXydIeaP8EAuF CA7paP3s9STLJBO3KurkwyRkPW5zo0X7xVqaVToRsX2Ul98KVJoHYQD1KdezEtwl vpNwiiBr42AYR751Vm6JBVAbQXuFpB3c8bUV0OkkRxNFtL8/2PieHar58n5dntGk bPlPkztahsFqktgacIgXHX5vaT+7YeeZ1DWLOYjGO0wNhkOSeroCmxwJUikU7joB p823L7r5KfpqWTPpSCzVstQKZUGmmoE1qCswY/Ud5wvp9SccpIILkRXj0rZRtfnE 5MpL3hjmtNzfDd9qIsJtBJlSB2hZwAsVm1l+EWN9hG3tqyA43niUMy2n6q690of3 berSiQ+kvY/aC9Hx8I+bKzOV9/J2VUTqfaPZa4Uy2rVX5Q2p69n/PMj7mEer0rCL 3j9V16J9c+s0BSkXoKdtYdB0TWVhBgUybd9qtYcwHWvhP7QuU3RlcGhlbiBGYXJy ZWxsIDxzdGVwaGVuQHRvbGVyYW50bmV0d29ya3MuY29tPokCPQQTAQgAJwUCWj1R WgIbAwUJCZQmAAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRBasvrxexcr6jsc EADEcB0WQEZn2AkrzDs1RhL0Lp6cZi0BigofkbcGfdhJyMSs19C0dhvncrAFClVI 6/Udw3yFtDyYtOCf2W3M3A1K6/RfEizCLzTsdFIhni9gOJLlUpXViQtgrlstjk7h qVV3Ooz4BlCqS4cG7rfqf4LQQPpTAuFUEV9I28FBUB2irqC+v4gTysIgpMw0bA1y BU9sX5jE/tRkzqnuzZrkwiobDtRFJ9qp+7O2JtcY4EsVtLAsaodJKc5cF8R4OvB1 n66vxxcgg9Eh4JNWZ47xsaCmAGo1Bcb2jIY35OtgAL7gCGLRSMKTtAaPy1/fEgIq hCljJ9x40Fkn/3r2BX21WC9HFSPFTBz2RluLRzxdgxOrkYK8EiHUPoE5b1AEzZKw 2AbeXfr57f5zYsN3IqfbQLUjMYtUN1wK3Pjb+idD972wyXMWt8uOzlI7b9Ocu+nY m2whBfJv9Pmp3QYTmPz+LB9lH65VNVUSxSXVr5iWXO3qx1HtEiGEqkporMQCTh3T 5Ud3PvMSRBFFKNs9WhJ/Lxz+SV30WLwG6dr5mQqlzAhb4Phc/zekZyXRdS/oDKrB LUucS36O//49JeyRi1QvOfxnfmIqRIAf/k3PoYJmTo5E82//r5Qj3YGlRu78ba0H Arxs+ACD6AnEHHcbswpbtVEKYzlSu0Ar0Dc7vRWM/IyQdIkBHAQQAQgABgUCWj1S oAAKCRAvPIc2gF+NosIsB/9f/29FNla3BJfGIEIDnhrqGD0i9bSa89SqBd++uG06 TQgW5wsqtNcrwn81yZTq6XE6i9VtD4GKfqC0d4KZJr9bnbeD81cI64VOdL8zJWJs 0vj5EIXCobKyX74Kb4uePUyZqwT2Q74I116u/HwA9/FXsPo5isbh4ZqD4t0VHpWk mfq1FPT9a/JPyX46qKqB2Fce/7Qy+SQP1NfkuUlbhUH/JG9aSSYvk3lznNiH41x9 M+FDlL106itXOubrl3oi2fT3fsSedq7uzt+IV0DQEeNaoQAUuwEhdB8IWOMqN2wo DjGVKJftfsSWY9ilZrnDBNDrp0vRqcx33LUMkIw4d7iBiQIzBBABCAAdFiEEfhcK BFyEz0YOK3mgEO952f2DUxIFAlu3JJwACgkQEO952f2DUxJjuw/6ApHSsVTWD4a0 H6FJ23A9Ftpy+aXZ4vYlzkSrfsn2ECrEfK3lXQh/uzwjJUDYZeB1/BQsFZtcYNQO JSSHbQ49BFRLwb1J/wBZG4bbmrkLxnNbKDKQvzxEpclkMW0Dj0J6o7kGrmzIGGrh B+JJN99AcineHRug8ZSFIERRCmigxdhAKU0BFD7P+5HNHltSL3DF1c2fFOf2JrgB KVoE+9RhMZjWNbYetFFLCkjXb5Rpay9zeMm1DxfSTGAnuOwUXW6qq4hnl5+VC/48 ceDZElLLfu7RQUZv44pkSTOWZs+iQoJiHMFHk9wPqyB2Vok1yJ2a2j27WhXrJlPw nZbgJO5RyWDG3p/eVmpl5Uuc2dsfIpR17KnAuWpghK6V+cyFncDoGCl/YG2Mvool sW08FiZh3Ej4dnJjj25TZkeFG74JJDXLvMYpJfSBGnmETv4Dhcm2xPqVMuFuL1qJ lMbVLrMo2GXeo03OzNyvbs+u8WLIaGm5hC7N1CXY8wZs4jo6OJ/expvnc07dEuws 4zT3AiWv3nIouWReRStZy9QkavDocqbyPmilcdPCYk4BsOlzpwwO74hNG7iyl0Kd AlwTxGQ7y0rJou6HYa1TmRhIEr3vKvlW+JfUUrqtjXgsuacTXo4+Ira2JUErL2cY zQMq1j4r1ZyhFnuz93s7Rsx/Nw0+0YuJAZwEEAEKAAYFAlvFx+UACgkQajsROTyk rWCJqwv+NLVPE4sD4sDA2/6Ek7UsRIUkg+S39fhqWsLc4rtw/mDunv8Un61I3K04 fZ2Ry4nF9hZM0a710UvXFbStvrzRJO3EAAcdJR9LTCd19e8UeruQbIee3YT91U4N kC9JMpecfq62/teOAU2e5P3fWYaLs5ZX7zCLwWuBcW2l3SyoljQczM85HhJ3XHm+ FnwQ6D9xRle+lvWTcuC9d1yAyUb8IOospcL2lJTmy8e3r79R24hPlSB4LDe0wEN8 AXbagrcAQZjwyaHyWxjJbTwZ0b43WGdfIqZ1ElOeoffbketPGRmWvx5xUvb2ALFB BdETzV270gs5XDJgJ1SIIKOyDADxwvroTe2jD8C/841eEql5QSow3s/U3zRqk3mt tto8Qw/DN71aeh6dmYSsvd2UjsHw/vofOPRBGxZLEkKTEvMnhmMW9hiKPkPia+Qg evYE020qpKSxLEdWA8nprHwxmGiDNesCfXSC6vm1qfyj5g8HzxSckq9ZaMhKMCo7 vxflUEDuuQINBFo9UDIBEAD6DdHQfMav8OXfhjTteoarOrlJTSdci727xiezGPuB HmpvceBRZgRasdbaMc4HJee+R9+5x/nLPCuy/DxDyIjwIUeJNgc+l7LjI9WfpHTD 8U4xxjvR5Mi7+ToQQUOUNuzT0O0pyuxP1uY3RehHEhOVfBZO59ipSeZL5iQC6T5M sK1SKfs51pLa5ToC1rc8tBJ4zZmxRAyZiYc/AH2uZ/6rYjTTkAn1DVI9DYo2D/zE 4bGjXdJW5pKphFB2lX3dG4I7ODi+5e1H6A/QpCu6z8/ZkIQ+9T1xcX/YwiFeA7Pb TuW/eITbMbI1eV3+fyym9aT7Rsflmp31Zxtr+sZwGGZf00ooMBFmqOS//NUQ/Vf3 vDUew1h5QU1yDaWT3NApvi+XWPH9TPy6TMfZA2FThHf11sX/gDBa5JWQZbptPEcm oazpiKZt91CrFPOaoXDPck/Q61dfmr/oPikfByYnASIM3OwEuXqyQ9JDRfKrem5r +oA/wxWb5jELElAhOpnyqMMvOh7uz1foUssL8MAv2TGXmxpVJ8Nu4je6wf96Z22f Q0D38zud+CKH3bMP3ayXXJBcdPoENrzFbWP5FTg/4TTDJ3vOAHZR5iCunYghx8b7 Ffa4UbkwlD+dh8GiIAtvT51Ac0cO0Wc0Zjc57zPUz1zloMbf+zb1Bsn7DuEQoqj1 gwARAQABiQIlBBgBCAAPBQJaPVAyAhsMBQkJlCYAAAoJEFqy+vF7FyvqrC8P/1tF 6TeR83xD6MasqXyrBjwcLmziaF0Mlkj8k/YUiZ/knb53n97xQnh9yxPv0TT8Wpfd n3BmvqGyh8+ouHX9jMOxiRkMdNhIauVYY/8jmRfBSYWcFkfMzdYasvdLtmYJgx25 2HKTFdeOrszoOjWjEzwmh+tca3AFMu/nB++/KAmi5UJV7zsZ7uYJ5jm97LV5SLjN JIXXM+lHqCDrjDaDhNczmq1LCRlU6/WDjvkuwaVhZG4lXxMDrvKnXMkjseQ2oKjw rIdfQM86H1z5J31lfhqop+of0cimcIsBgSCPu+h96LHuAzeRBCbDKeqrfZtAZAGs okRina9947fRWxXHh3O66ILmXKNRxxWbDkPvYnQWUat8SbSTDoPWrDIGDRIAypqY o3pcN2OE0C1chqgDZQxkr+9kYZQpupOAN2TR+fM7JvbO9coKI8Uqog8CopoMeDQk d0YjcqlB1E0svODHTzcSoRzogDBYDqNLP7qVkNXpcOAXSVioBgiSDf7o5RdS/qmU yXBIeq6I5z8xBcd+BQ/n/9Frkm6K7IKP3ngUP4wEoiPx5ZE5+fPIScGmVUcZIMhk vMvem9XXh1yyhqN14gfjmLwPGdWbrgG8QUe0s2WeWIyss6uTiyF+ZbJSo2XOKVc3 YFMVUUfgyudqAV1wWdZinUk+H3pkqOKoHAy/8fST =3DYzQY -----END PGP PUBLIC KEY BLOCK----- --------------644650695B07ADB23CA74A9C-- --SuMxI1ymJmdyJ4rUbqtCUpHpYzvNt0l8L-- --ftGoz4euh6ar5mvGm6vCtxjN1tmXqa0Qt Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEW7Wm6ldl0sWGPK4nWrL68XsXK+oFAl0/fXcACgkQWrL68XsX K+ocQQ//ee9NYvtHcCIsWL9fLehIcLH88vSg8bZ5B+5JLRsh8vP6DL1Suc+Pgqrr 6JVJYJkU3fRA7HegQJ+Xja0jj08F8dgGL48kpr/lUf9Kkw5HSH00OIBcmlEibCni HmjQuOksjdaMJRlYg/ruI8bgEenuVeOMO8we+GU3HHFZt0BEF+1G1Cww0kILSbt/ vLsTwzFlUxtf9DD7VS5b4PoJiEvp6leKSpeigBbQRJSB2vcEYRzx5xyR46pThi4L sU/YnZaIyo2TNR2+z03sPlj1EN1wBLWw5h4+1Ik3Ht5mN7vIeCVT9/tlPh4XN0Pq b9WXAJnQlLuQcCtjtNLmMVqoGr9W2XZjA5RE4W9LTBaYD8KcYP6sD0+gBTyVphVy NqrftnEpWEw6nQ3jxgFTe3DVRZ1xfHrzyY5FgyIdtC9QCkVFMyVoPQFIep/xR6l+ s5WysINWT739ySDU6h6IGExQcY7jzRHfi5ut7hZcgMTs61vFqoK9IxbgGqEXtaHB Lu49Nw0eHK+oaVhg04tn5AsNXtm9Ul1iOpEsgQjTghioa/ItVJ2JORdPb4pv1YKX X0NoN6pHlLxgfYjMOgXMYyoFZis1Cy/lZvbxK1PSf1+i/C9q1fyBUlRrpqp8sVTv lwP/eOY/MqNvKJ4iUC2Pequgl+A2NpMLQSCE7IAMTzWYL6qkddw= =hi0e -----END PGP SIGNATURE----- --ftGoz4euh6ar5mvGm6vCtxjN1tmXqa0Qt--