From nobody Thu Mar 13 09:06:18 2014 Return-Path: X-Original-To: therightkey@ietfa.amsl.com Delivered-To: therightkey@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 561B71A0A0B for ; Thu, 13 Mar 2014 09:06:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.526 X-Spam-Level: X-Spam-Status: No, score=-0.526 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, RP_MATCHES_RCVD=-0.547, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ULlp8lrT_jAK for ; Thu, 13 Mar 2014 09:06:13 -0700 (PDT) Received: from mail-vc0-x229.google.com (mail-vc0-x229.google.com [IPv6:2607:f8b0:400c:c03::229]) by ietfa.amsl.com (Postfix) with ESMTP id 773311A0A0F for ; Thu, 13 Mar 2014 09:06:13 -0700 (PDT) Received: by mail-vc0-f169.google.com with SMTP id ik5so1348471vcb.14 for ; Thu, 13 Mar 2014 09:06:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=jkg2f6Uv1RaAUIRNkS0aCZt6bmt0bCTl/SBuiAzvWWw=; b=EIKav2oW2n5G6TX9rdSm9PnsmXrBxTlm5jNf00iVWl9wwAp+isMcVhkxjLyDmzSyMr uqiOSZW66w661Byds74GWr0NV18jFJLV8Y+mgq3Wk2ndjYsSAZOX2I5eLT+6IZbSI0tP A/UgXuGOy+OmPuH/cgPLwU8zkoawDlsO/wDUq6VNxt0XtPgOOBA/tW9VAg26VTprvVPi Y/Me8dIMRcO3wWb7XvK3oHwijoH+6liTNp5+FgqFnFLHjuT+ZMAzLec95XO25Az0xI4x uH8cd4QSAdG/PsfpvS3v4M+0etGD6nWRfMNmYpfQhgFqDxTK2PflBzWa6IIAc13eMdUp qBCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=jkg2f6Uv1RaAUIRNkS0aCZt6bmt0bCTl/SBuiAzvWWw=; b=OTs7Pd89xa8mkPudYSb6JFwrv8XkoTE/rLy0gtPdjJQ7Jem4sxJPM2Dc23VmVlg5UO HPkoRduo3Xh7H0LbW/lt8/tKawzTDsNfnSET0+FjEcifxCDZo3rbIf3/eJ4f3n51i2/S ULssN4PBVVCQPMEI+LTup0o4D8z8C8p47otE9Z5gebIimw0HG/OWhmOJgo+YxytxH1w8 f4O3DMernJ+Ce3bj45pEmhWdJ9Qz77ijJMG+8i5lRBq1M5jfDRrOXnflYt2tyDSxRyhj dfhS3ANVLv03xB8w3RPRNKRMyaTNUWvVB2ctcgY8R07X3ToPb2o5LwbNBRWmtsQnDTHk 83Xg== X-Gm-Message-State: ALoCoQmCTFI6p2OHW4ETnZ7A3IOw1IuYJYEbM7q317Y5zpLvvYRw6bTrj95pUmGGejDGYJNcNYHIZViMITlKOHsvBRRB3B1myvg3JdSHKKv0XcCj2CSUrS33BhZJtD0S3nmQcie/1TiUEer4V0jCeS3Bbhfxqw6Gcfjmv8zeRMCQKKTBq1lsIXJk3sO8MNUQv70gBHiVnQjv MIME-Version: 1.0 X-Received: by 10.220.69.133 with SMTP id z5mr139733vci.49.1394726766692; Thu, 13 Mar 2014 09:06:06 -0700 (PDT) Received: by 10.52.230.105 with HTTP; Thu, 13 Mar 2014 09:06:06 -0700 (PDT) Date: Thu, 13 Mar 2014 16:06:06 +0000 Message-ID: From: Ben Laurie To: "trans@ietf.org" , "therightkey@ietf.org" , "certificate-transparency@googlegroups.com" , CABFPub Content-Type: text/plain; charset=UTF-8 Archived-At: http://mailarchive.ietf.org/arch/msg/therightkey/5e8-RLujgeLz5b_fZr8hhmt413A Subject: [therightkey] What's the load on a CT log? X-BeenThere: therightkey@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 16:06:15 -0000 Several people have asked me this recently. Here's a nice way to estimate load. Let's assume a single log that takes all the load. Firstly, we see about 5,000 new certificates a day, so that's around 0.06 new certificates per second. Clearly a trivial load. Next is load from audit (i.e. from browsers that wish to validate SCTs accompanying certificates they see). Given some assumptions, we can calculate the load from audit. * Clients cache audit results. * There are approximately b = 2.5B browsers in the world (http://www.internetworldstats.com/stats.htm). * The average user visits w = 89 websites a month (http://www.creditloan.com/blog/how-the-world-spends-its-time-online/ quoting a Nielsen report). Assume these are all TLS sites. * Assume a certificate lifetime of l = 12 months. So, each user sees w / l new certificates a month. Each new certificate needs to be audited, which means in practice, three web operations (fetch STH, fetch STH consistency proof, fetch SCT inclusion proof) - it might be a good idea to create a new API to do all three in one go. So, total average load is 3 * b * w / l ~ 20,000 web fetches per second. If we optimise the API we can get that down to 7,000 qps. Each query (in the optimised case) would be around 3 kB, which gives a bandwidth of around 150 kb/s. Monitors add extra load, but should only be at around the new certificate rate - i.e. ~ .06 * number of monitors fetches per second. IMO, this is achievable on a single machine (modulo reliability), with some care. Clearly not a vast farm, however its done. In practice, no one log would have to take this full load, this is a worst case analysis. From nobody Thu Mar 13 09:32:01 2014 Return-Path: X-Original-To: therightkey@ietfa.amsl.com Delivered-To: therightkey@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CC3121A0A2D; Thu, 13 Mar 2014 09:31:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eMvGlb1ghbRL; Thu, 13 Mar 2014 09:31:58 -0700 (PDT) Received: from che.mayfirst.org (che.mayfirst.org [209.234.253.108]) by ietfa.amsl.com (Postfix) with ESMTP id C91D21A08ED; Thu, 13 Mar 2014 09:31:57 -0700 (PDT) Received: from [10.70.10.55] (unknown [38.109.115.130]) by che.mayfirst.org (Postfix) with ESMTPSA id C0FD7F984; Thu, 13 Mar 2014 12:31:49 -0400 (EDT) Message-ID: <5321DD69.2040805@fifthhorseman.net> Date: Thu, 13 Mar 2014 12:31:37 -0400 From: Daniel Kahn Gillmor User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Icedove/24.2.0 MIME-Version: 1.0 To: Ben Laurie , "trans@ietf.org" , "therightkey@ietf.org" , "certificate-transparency@googlegroups.com" , CABFPub References: In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="rfCb6m9MjfUCJVi1pAePavI0ql7enmOXT" Archived-At: http://mailarchive.ietf.org/arch/msg/therightkey/hlKDCOIwBca7S3vWf5c7nPiocno Subject: Re: [therightkey] [Trans] What's the load on a CT log? X-BeenThere: therightkey@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 16:32:00 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --rfCb6m9MjfUCJVi1pAePavI0ql7enmOXT Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 03/13/2014 12:06 PM, Ben Laurie wrote: > So, total average load is 3 * b * w / l ~ 20,000 web fetches per > second.=20 This part i follow (you're switching temporal units between months and years and seconds, but i get roughly the same final figures) > If we optimise the API we can get that down to 7,000 qps. Each > query (in the optimised case) would be around 3 kB,=20 And i agree this seems like a win. Why was the API broken into three parts instead of the complete proof originally? what (other than conceptual cleanliness) might we lose by creating the optimized API? > which gives a bandwidth of around 150 kb/s. This looks off by a few orders of magnitude to me. 7kqps and 3kB/q gives me 7000*3000*8 bits per second, which is 168Mbps. Am i missing something? Should we be considering swarm-based distribution of this kind of data, or hierarchical proxying for load distribution? --dkg --rfCb6m9MjfUCJVi1pAePavI0ql7enmOXT Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iQJ8BAEBCgBmBQJTId1pXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFQjk2OTEyODdBN0FEREUzNzU3RDkxMUVB NTI0MDFCMTFCRkRGQTVDAAoJEKUkAbEb/fpcXiUQALKKrcqkmVjA8TUIfcoVuWad NdsqHmDdpkpkAADPOPrEKpOquhiOy3vLHZcXeDuFmcypzKfpR5UGDfLApwY//ghW WIcmmQjinMNzraAf1Q2x52CAxige6Hax3A4XCNAr9p6B7xYJ76BWT3kfRmbZL0QO OJrebEDCDhU/bKVQtG0IjKOImnnvg3vmTIEUUUPRGghIsL6LJwcHw5E4bekjytJn kKwICn/mZqui27WsZ80LSQAFuOIkrouhvoKCxv6M2El20NjBA8vwfJt3OkrzaQac 6d6zJIjF1Xcy9MDzz35ncQNAu93ZXJR9QZIpvfC2jqnEQuAPEMXsa2odKKSQpFle 6co09vPObPE1nYeOFTpiFOQD3/Pq/TUNhrPXaPGJpwSAnHfSWWpOBRXG1T6tgWXS CC9ibfXsIP1cupYufSXs9yoHT0H87mVUDdaqR6YkGSB65Ga6rmAO6sly1stYyK4O 3lFmx36NLa9aIe1W3Rxk5SxxIYtE7HPoBiqcKPgJ29O4GxV5i9yiOqc4e3OuchQl zTsYKy86ohPF97v07lTSeoO053DodJgX737WqY6umE+V5gG7IQ/gOL38bz6/KksC Hux0S2KtdnITi7jXkT98yLhWP2XjsKzAubYxGSVPv99lilofb1hWb6zAdtVuTPQd MXddyXB9MYOuQIKm73ZM =zWj6 -----END PGP SIGNATURE----- --rfCb6m9MjfUCJVi1pAePavI0ql7enmOXT-- From nobody Thu Mar 13 09:51:31 2014 Return-Path: X-Original-To: therightkey@ietfa.amsl.com Delivered-To: therightkey@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 99BA11A0A3E for ; Thu, 13 Mar 2014 09:51:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.926 X-Spam-Level: X-Spam-Status: No, score=-1.926 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, RP_MATCHES_RCVD=-0.547, SPF_PASS=-0.001] autolearn=unavailable Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BTyZrKsE9H2y for ; Thu, 13 Mar 2014 09:51:27 -0700 (PDT) Received: from mail-ve0-x22f.google.com (mail-ve0-x22f.google.com [IPv6:2607:f8b0:400c:c01::22f]) by ietfa.amsl.com (Postfix) with ESMTP id 20B7D1A09CE for ; Thu, 13 Mar 2014 09:51:27 -0700 (PDT) Received: by mail-ve0-f175.google.com with SMTP id oz11so1366817veb.6 for ; Thu, 13 Mar 2014 09:51:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wdrhN3U2CPr0BzTR++RCrRQOxYvYlpymZ6NY2IivIMo=; b=FZgPHWRmchsDn0mj7izclBSqMDbAflcjY8KxBQbcTeQA4yQ4X2jDJCXPPV6qRotQVq 8m5VUzK+qNrmoOy5P3AIGwX/qoqLXRas/RH4o7iySofS03+605cKcyYIWtLKlHSSt/VT lcHCTfjdjF7crMVOu797Pz8X+5ImPNHAxkFLJ7wQjQWE5gevi520W7r7QIprBbekR7S7 JK6iUgeHVdGQ/PuT0ELgak8mC4aGCRsbGwwxg1ibW2rZ3NECjYChW0jLgH86Yr1Xf+oq mPgTb0ZKnpIGTXvFlRL26CuNrPluV/HG4pYwzMbob/yXXz9Lv45n9ZnkcSFfpzOnSUdU GJww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=wdrhN3U2CPr0BzTR++RCrRQOxYvYlpymZ6NY2IivIMo=; b=O0nGiYl+crDJZDP4ZoPdPt9hlO+bZR6QMpYo5mk+qVDfQl1620bNodux+T39fljhRM ewDH8COJjqkStOjytXRiBB7vkG/lPeQBWvQ/y/kiZyBTdbSlha7nOnanNKGFA+bcZ7AP uKrgGfeA98Mfp9ZJ91tnP4mNvDyR/3FSl6wesNlAJWZwpByhz0qHSCnY4tzrSpPG5+yX Kb0O7M/LAmYNBdAK9fjFRsiVApBqSsHh3kW5VSP0in7h6gAU9vldUD7yrg+iPLgpgkAh AH89MrsyddHmIdWhfbuOm3cbwQysRTBorXHUI+dCS45SDCX4qInIrpjveVQDY3o5ffT/ 3fzA== X-Gm-Message-State: ALoCoQla1isl3XZXdv772PqdqDt210e+IWUqbGdef53XCdchK27vLnsmY1mZunaiCUsP2WLRtLXVoM6p/AkMMq8A1t3fVRLerwKIs19M/Jx2OsQf0VnQVEIn7hV5pNlKYBEH2iI01rjB4xWUGKF60oJl5eiPcpyfpuV67oxtJR16C8S28DzOFwpDu82EO0S5Flt2ef9/LOfH MIME-Version: 1.0 X-Received: by 10.220.159.4 with SMTP id h4mr2267836vcx.1.1394729480350; Thu, 13 Mar 2014 09:51:20 -0700 (PDT) Received: by 10.52.230.105 with HTTP; Thu, 13 Mar 2014 09:51:20 -0700 (PDT) In-Reply-To: <5321DD69.2040805@fifthhorseman.net> References: <5321DD69.2040805@fifthhorseman.net> Date: Thu, 13 Mar 2014 16:51:20 +0000 Message-ID: From: Ben Laurie To: Daniel Kahn Gillmor Content-Type: text/plain; charset=UTF-8 Archived-At: http://mailarchive.ietf.org/arch/msg/therightkey/Kl1Bq8NYu5rFHoC9GiAxIVZ7kU4 Cc: "therightkey@ietf.org" , "trans@ietf.org" , "certificate-transparency@googlegroups.com" , CABFPub Subject: Re: [therightkey] [Trans] What's the load on a CT log? X-BeenThere: therightkey@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 16:51:28 -0000 On 13 March 2014 16:31, Daniel Kahn Gillmor wrote: > On 03/13/2014 12:06 PM, Ben Laurie wrote: >> So, total average load is 3 * b * w / l ~ 20,000 web fetches per >> second. > > This part i follow (you're switching temporal units between months and > years and seconds, but i get roughly the same final figures) > >> If we optimise the API we can get that down to 7,000 qps. Each >> query (in the optimised case) would be around 3 kB, > > And i agree this seems like a win. Why was the API broken into three > parts instead of the complete proof originally? what (other than > conceptual cleanliness) might we lose by creating the optimized API? > >> which gives a bandwidth of around 150 kb/s. > > This looks off by a few orders of magnitude to me. 7kqps and 3kB/q > gives me 7000*3000*8 bits per second, which is 168Mbps. Am i missing > something? Sorry, you are correct - I meant 150,000 kb/s! > Should we be considering swarm-based distribution of this kind of data, > or hierarchical proxying for load distribution? Maybe :-) One thing we're working on is distributing the proofs via DNS, which is obviously of exactly that nature - which would definitely reduce bandwidth at the servers. From nobody Thu Mar 13 10:54:26 2014 Return-Path: X-Original-To: therightkey@ietfa.amsl.com Delivered-To: therightkey@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B465E1A0A39 for ; Thu, 13 Mar 2014 10:54:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.926 X-Spam-Level: X-Spam-Status: No, score=-1.926 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, RP_MATCHES_RCVD=-0.547, SPF_PASS=-0.001] autolearn=unavailable Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fBQ45GT3tBvM for ; Thu, 13 Mar 2014 10:54:22 -0700 (PDT) Received: from mail-vc0-x22d.google.com (mail-vc0-x22d.google.com [IPv6:2607:f8b0:400c:c03::22d]) by ietfa.amsl.com (Postfix) with ESMTP id 7A7C01A0A23 for ; Thu, 13 Mar 2014 10:54:22 -0700 (PDT) Received: by mail-vc0-f173.google.com with SMTP id il7so1531160vcb.32 for ; Thu, 13 Mar 2014 10:54:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=OKCZwQ6Vz2/SdbOSNNp1Gv517JQWB7Z+vqzkQics45U=; b=pUMeqK3NnpEDjMETkxSrD0Zo3/IJIFHYjADq41oxa/Nm1u5SGIDgQXqS5DvTeFT3f+ VFLA6DuyQynr7RDVsp4OY8pc8K6eCuZ11B3H7yKVnyOTx1qoPVaJ+cqnyQct60q2Gls0 WbprwPVzGEU01650y9SwpDi3pCs5wmpkw6UpsbMWujG3fMsaa6BuD3py0amlU7s5OkiW OVjvCOklVC0+2uz0br/h+WgXfuNreSOOfYfCbjqQLZlzbzDNMkkyem8sddM4sSOO8IWP t/ODGDFVBu4yKvRAVJ8M7x1ZJ3gr43h+aqGNUA5k7ur7u2lgiiRahyn140nBEdzdklFQ jcLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=OKCZwQ6Vz2/SdbOSNNp1Gv517JQWB7Z+vqzkQics45U=; b=b3mz7DW1NK6x6gUFzfS0rK8ix2mkAnEiIZhizxPnWiZVSJi53OUQNbn6bkHrnksuw4 DjuCcWKLJj0ORrcBl8lBh/jq35a/fo+WZ5A0tSNTZBJFWTgPPnBYL6gSP+DiUMNeKlNY 8pifOUibZHQ8WXryS8MBrH0t4Bo6++ulpF74E4xO8yBkA4a3DRa7/Mrz70cvidDU/GG3 KbyK8x7LHoezV7BqOk7JEMnehIFcR5P7ngf3LJXr4wyVPcp/fAsMB4kyu2MehB1eVmal qz2wvFK8nRuAPt+7swsgnrXDi0ysrtI+zLFdAHiFmWyg8+yfN08FPzJvdSAGLvMlhhtm cfzw== X-Gm-Message-State: ALoCoQnovQuTSOaL0zr8iOFAnoEHafSIWldF6bYKtEaSSajbXqhvDLiu61jNdqBCu5lbjv6iOWhz+ewJzElV7BznA9SiHv+ItVubADp3gevP9CkZZoC62rGtH9B+2WteM+00bSgEHgxX4T0wfFavhb3Qb7C8EK4QyulyDpYbFvh4Hu39bR7W+UX/phWPMYSRPtUs6ufj8NbH MIME-Version: 1.0 X-Received: by 10.221.29.196 with SMTP id rz4mr2548925vcb.8.1394733255665; Thu, 13 Mar 2014 10:54:15 -0700 (PDT) Received: by 10.52.230.105 with HTTP; Thu, 13 Mar 2014 10:54:15 -0700 (PDT) In-Reply-To: <5321DD69.2040805@fifthhorseman.net> References: <5321DD69.2040805@fifthhorseman.net> Date: Thu, 13 Mar 2014 17:54:15 +0000 Message-ID: From: Ben Laurie To: Daniel Kahn Gillmor Content-Type: text/plain; charset=UTF-8 Archived-At: http://mailarchive.ietf.org/arch/msg/therightkey/rNqO0bsvjQlspDhvA0qc0a-xAJw Cc: "therightkey@ietf.org" , "trans@ietf.org" , "certificate-transparency@googlegroups.com" , CABFPub Subject: Re: [therightkey] [Trans] What's the load on a CT log? X-BeenThere: therightkey@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 17:54:23 -0000 On 13 March 2014 16:31, Daniel Kahn Gillmor wrote: > And i agree this seems like a win. Why was the API broken into three > parts instead of the complete proof originally? what (other than > conceptual cleanliness) might we lose by creating the optimized API? Forgot to answer this part - it was broken up purely for conceptual cleanness. I'd probably leave the independent APIs but introduce a new do-everything-at-once API. There are actually good reasons to combine them anyway when you're looking at a load-balanced setup (skew between servers may cause some calls to fail - e.g. I get an STH from one server, then try to get audit against that STH from another that is slightly behind, which will fail). From nobody Thu Mar 13 13:27:21 2014 Return-Path: X-Original-To: therightkey@ietfa.amsl.com Delivered-To: therightkey@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 40A851A0A34 for ; Thu, 13 Mar 2014 13:27:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.29 X-Spam-Level: X-Spam-Status: No, score=-1.29 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_MISMATCH_NET=0.611, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iADI1c1fPIGg for ; Thu, 13 Mar 2014 13:27:13 -0700 (PDT) Received: from ian.brad.office.comodo.net (eth5.brad-fw.brad.office.ccanet.co.uk [178.255.87.226]) by ietfa.amsl.com (Postfix) with ESMTP id 1C7121A09E0 for ; Thu, 13 Mar 2014 13:27:12 -0700 (PDT) Received: (qmail 28465 invoked by uid 1000); 13 Mar 2014 20:27:06 -0000 Received: from nigel.brad.office.comodo.net (HELO [192.168.0.58]) (192.168.0.58) (smtp-auth username rob, mechanism plain) by ian.brad.office.comodo.net (qpsmtpd/0.40) with (AES128-SHA encrypted) ESMTPSA; Thu, 13 Mar 2014 20:27:06 +0000 Message-ID: <53221499.40301@comodo.com> Date: Thu, 13 Mar 2014 20:27:05 +0000 From: Rob Stradling User-Agent: Mozilla/5.0 (X11; Linux i686; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Ben Laurie , "trans@ietf.org" , "therightkey@ietf.org" , "certificate-transparency@googlegroups.com" , CABFPub References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/therightkey/S1HjYu0OxmY_JtdYvvjUnBYrFTw Subject: Re: [therightkey] [Trans] What's the load on a CT log? X-BeenThere: therightkey@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 20:27:17 -0000 I'm not sure average load tells the whole story. Won't there be a surge in audit traffic in the aftermath of a busy site installing a new cert? On 13/03/14 16:06, Ben Laurie wrote: > Several people have asked me this recently. Here's a nice way to estimate load. > > Let's assume a single log that takes all the load. > > Firstly, we see about 5,000 new certificates a day, so that's around > 0.06 new certificates per second. Clearly a trivial load. > > Next is load from audit (i.e. from browsers that wish to validate SCTs > accompanying certificates they see). Given some assumptions, we can > calculate the load from audit. > > * Clients cache audit results. > > * There are approximately b = 2.5B browsers in the world > (http://www.internetworldstats.com/stats.htm). > > * The average user visits w = 89 websites a month > (http://www.creditloan.com/blog/how-the-world-spends-its-time-online/ > quoting a Nielsen report). Assume these are all TLS sites. > > * Assume a certificate lifetime of l = 12 months. > > So, each user sees w / l new certificates a month. Each new > certificate needs to be audited, which means in practice, three web > operations (fetch STH, fetch STH consistency proof, fetch SCT > inclusion proof) - it might be a good idea to create a new API to do > all three in one go. > > So, total average load is 3 * b * w / l ~ 20,000 web fetches per > second. If we optimise the API we can get that down to 7,000 qps. Each > query (in the optimised case) would be around 3 kB, which gives a > bandwidth of around 150 kb/s. > > Monitors add extra load, but should only be at around the new > certificate rate - i.e. ~ .06 * number of monitors fetches per second. > > IMO, this is achievable on a single machine (modulo reliability), with > some care. Clearly not a vast farm, however its done. > > In practice, no one log would have to take this full load, this is a > worst case analysis. > > _______________________________________________ > Trans mailing list > Trans@ietf.org > https://www.ietf.org/mailman/listinfo/trans > -- Rob Stradling Senior Research & Development Scientist COMODO - Creating Trust Online Office Tel: +44.(0)1274.730505 Office Fax: +44.(0)1274.730909 www.comodo.com COMODO CA Limited, Registered in England No. 04058690 Registered Office: 3rd Floor, 26 Office Village, Exchange Quay, Trafford Road, Salford, Manchester M5 3EQ This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender by replying to the e-mail containing this attachment. Replies to this email may be monitored by COMODO for operational or business reasons. Whilst every endeavour is taken to ensure that e-mails are free from viruses, no liability can be accepted and the recipient is requested to use their own virus checking software. From nobody Thu Mar 13 14:48:18 2014 Return-Path: X-Original-To: therightkey@ietfa.amsl.com Delivered-To: therightkey@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DEB181A0778 for ; Thu, 13 Mar 2014 14:48:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.926 X-Spam-Level: X-Spam-Status: No, score=-1.926 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, RP_MATCHES_RCVD=-0.547, SPF_PASS=-0.001] autolearn=unavailable Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vipfdG5Vxu9o for ; Thu, 13 Mar 2014 14:48:13 -0700 (PDT) Received: from mail-vc0-x232.google.com (mail-vc0-x232.google.com [IPv6:2607:f8b0:400c:c03::232]) by ietfa.amsl.com (Postfix) with ESMTP id 55B6E1A0A47 for ; Thu, 13 Mar 2014 14:48:12 -0700 (PDT) Received: by mail-vc0-f178.google.com with SMTP id im17so1884715vcb.9 for ; Thu, 13 Mar 2014 14:48:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=D7gr2ZgoWcm1IZcJ5Vr+mUOp2WATe33hiw3846R1PuQ=; b=P95vgRV0zyxSYp87MarKjjZRhnVyavrhjyfTpgkbhQYGDfAwf4KY9fF2d0ItimQ28x MsuuEw7zWYH8GwYURxCEBtGO5H5WH9r7/RJKmwXqYBxdUdGyCLfjPMJwtgce39rxSV6v aMK/DY7OXLJrD9NSWBTwj4c32ItVXvmpQXYa1oCsjF0q4kg4psBEaIkRdkD9BtC0Qj+q Qv2q7rG3cXiqy7EdcRorjxpTZ0q6ModtToiVCRdRYa0E7i3QGJn7aOSVtsP1DAZOMyeT DBF8yPLDLdlvtIvI/roaNz96TVpBcd5hOJeq1/MXGrqt7CRkzrkjyNapM650s1MGP3Lj lbbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=D7gr2ZgoWcm1IZcJ5Vr+mUOp2WATe33hiw3846R1PuQ=; b=N9jIT/+2WbmjGzkFTWYEquZO6dtJTB7NxHQAdSQKo5Khm0zcrc8yC3vggKByTZOwxu A9P5gyiw+j5jODQb1xU9OGfTnEbVrRUb0qxttzW73UtZbJ5mhZymJnGY1GJOXf3ooESs Q58FGcaXrFiATgCPGtvGYz7J5+wGXInpic7WCVBymRwdMzohNZUkVKdIONlwDywoJwFM rJfPIBwp0feVCnfnoFOURLu7z5qhzohsDsQHE763QC1ZKgv3Y5yfTUayg2ySelAp4xiO xuHbfvDFhsaYV49DBhfcRmfLo9E6nR7UMU+XlK3uHmQR0tWv4K0NhoIytDT+hCZnZiGh 761A== X-Gm-Message-State: ALoCoQl2Spja/G610obMKG8+4poMhOZ4VMmCoP8T80C8Dl8Aoh5/hSMZ2CTLn47af3ZFYYC09RulTs9Zp/9Tk/ZLdfrpL1JQ1y/1bOC2rWg3AUAWWkwcN0741zhB7cvYxR6/AHU5esBWZDkyP0DFGi8nZZPXMmrDRE7fS3EBK0PJVMQCUwMfJrtAZ6enexzj2xiWZbeBEhza MIME-Version: 1.0 X-Received: by 10.220.106.84 with SMTP id w20mr3284221vco.18.1394747285463; Thu, 13 Mar 2014 14:48:05 -0700 (PDT) Received: by 10.52.230.105 with HTTP; Thu, 13 Mar 2014 14:48:05 -0700 (PDT) In-Reply-To: <53221499.40301@comodo.com> References: <53221499.40301@comodo.com> Date: Thu, 13 Mar 2014 21:48:05 +0000 Message-ID: From: Ben Laurie To: Rob Stradling Content-Type: text/plain; charset=UTF-8 Archived-At: http://mailarchive.ietf.org/arch/msg/therightkey/4zp4aTWl82HDQa4dFIgW_U7Ilz4 Cc: "therightkey@ietf.org" , "trans@ietf.org" , "certificate-transparency@googlegroups.com" , CABFPub Subject: Re: [therightkey] [Trans] What's the load on a CT log? X-BeenThere: therightkey@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 21:48:15 -0000 On 13 March 2014 20:27, Rob Stradling wrote: > I'm not sure average load tells the whole story. Agreed. > Won't there be a surge in audit traffic in the aftermath of a busy site > installing a new cert? Yes. > > > On 13/03/14 16:06, Ben Laurie wrote: >> >> Several people have asked me this recently. Here's a nice way to estimate >> load. >> >> Let's assume a single log that takes all the load. >> >> Firstly, we see about 5,000 new certificates a day, so that's around >> 0.06 new certificates per second. Clearly a trivial load. >> >> Next is load from audit (i.e. from browsers that wish to validate SCTs >> accompanying certificates they see). Given some assumptions, we can >> calculate the load from audit. >> >> * Clients cache audit results. >> >> * There are approximately b = 2.5B browsers in the world >> (http://www.internetworldstats.com/stats.htm). >> >> * The average user visits w = 89 websites a month >> (http://www.creditloan.com/blog/how-the-world-spends-its-time-online/ >> quoting a Nielsen report). Assume these are all TLS sites. >> >> * Assume a certificate lifetime of l = 12 months. >> >> So, each user sees w / l new certificates a month. Each new >> certificate needs to be audited, which means in practice, three web >> operations (fetch STH, fetch STH consistency proof, fetch SCT >> inclusion proof) - it might be a good idea to create a new API to do >> all three in one go. >> >> So, total average load is 3 * b * w / l ~ 20,000 web fetches per >> second. If we optimise the API we can get that down to 7,000 qps. Each >> query (in the optimised case) would be around 3 kB, which gives a >> bandwidth of around 150 kb/s. >> >> Monitors add extra load, but should only be at around the new >> certificate rate - i.e. ~ .06 * number of monitors fetches per second. >> >> IMO, this is achievable on a single machine (modulo reliability), with >> some care. Clearly not a vast farm, however its done. >> >> In practice, no one log would have to take this full load, this is a >> worst case analysis. >> >> _______________________________________________ >> Trans mailing list >> Trans@ietf.org >> https://www.ietf.org/mailman/listinfo/trans >> > > -- > Rob Stradling > Senior Research & Development Scientist > COMODO - Creating Trust Online > Office Tel: +44.(0)1274.730505 > Office Fax: +44.(0)1274.730909 > www.comodo.com > > COMODO CA Limited, Registered in England No. 04058690 > Registered Office: > 3rd Floor, 26 Office Village, Exchange Quay, > Trafford Road, Salford, Manchester M5 3EQ > > This e-mail and any files transmitted with it are confidential and intended > solely for the use of the individual or entity to whom they are addressed. > If you have received this email in error please notify the sender by > replying to the e-mail containing this attachment. Replies to this email may > be monitored by COMODO for operational or business reasons. Whilst every > endeavour is taken to ensure that e-mails are free from viruses, no > liability can be accepted and the recipient is requested to use their own > virus checking software. From nobody Fri Mar 14 03:02:53 2014 Return-Path: X-Original-To: therightkey@ietfa.amsl.com Delivered-To: therightkey@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C81561A0100 for ; Fri, 14 Mar 2014 03:02:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.925 X-Spam-Level: X-Spam-Status: No, score=-1.925 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.547, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oUibEhR27uul for ; Fri, 14 Mar 2014 03:02:50 -0700 (PDT) Received: from mail-ob0-x232.google.com (mail-ob0-x232.google.com [IPv6:2607:f8b0:4003:c01::232]) by ietfa.amsl.com (Postfix) with ESMTP id BEF7F1A00FF for ; Fri, 14 Mar 2014 03:02:49 -0700 (PDT) Received: by mail-ob0-f178.google.com with SMTP id wp18so2345226obc.9 for ; Fri, 14 Mar 2014 03:02:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ZBMK/k3DMgeB355e7/rxW/u1c5Y3o1z9nConl1DwoGI=; b=PrvWHG+enFQaSsWPxYd4OsG7uE3nxun7iNpPPVSy0kslmZ9YULRycZxcoZ6g28nS2m jDuKycLLDxam8oxYv9zron96pCOIEJcCzBuJxjmvdw+krtzMPVym6Lg7uJwZcqaCpLs5 bqsp8Fe7h12ZQ3eN2iewtjfqwCZgYkk9chL21MM5FB7SYeB2bAU605Xrs1aB9osG0yCo efWc41bMDElLhY6H/Sm4OJ9tqxBC5egRVgtnOqW9Higi4yU7l9j6LdaeEudfW1Kc0dwT mCamN69uwYuMBfnlPCg7S2O4MMFDPn/saTsKczR/l7kNOPFWoodUINc83gVrlBTpsMGG 59tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=ZBMK/k3DMgeB355e7/rxW/u1c5Y3o1z9nConl1DwoGI=; b=Fv1r+q4asRMZBmjUFN2YP1YnICysX0UucTHy6WlC7Vd8oMJcd3R0zKyDXIolU/NIed 6hyL36e68WBojmI8AztXaBaAF+NAlvIKY/BIYDvFojuqd8tEf4UBErpx2YQMvbv8lRPE IApmhH1Fct9TE7XmN82TeBMnLH74Sq0TyozQhOPixijUSjuxBqxRuTB+H8rxRWbuj3gW /+/9LUJpiYtWgS71sCN8va6Ftv1ePoTUHE8KixeH4yh4wSlCyvirlBnRokrezqu3PTis 8yGE0pCz2cxzVnOa3H5+vT0JFceUhM8KZSjizR/F9hhwhJ8o+CRvLyBs9u3It5GioNvs tqKA== X-Gm-Message-State: ALoCoQm0YUeHZMjafKGHXJWKtXKL8A1ioBlnWku6YaeY1Mig8wTu0COBKCqHU2uSqOk98j53P9iKIFxWvzkZxYIl2bujsc30HnO6OsU/oUKb6oOmYnoliLaUOxI4oThLzyxPm7YE11m0LxpVpJx9/M8y72pmv+vCMdJosdNBYdbStKh9I+t/Gm241+hm3fSYuI6bZnAaOTDR MIME-Version: 1.0 X-Received: by 10.60.119.73 with SMTP id ks9mr195341oeb.75.1394791362875; Fri, 14 Mar 2014 03:02:42 -0700 (PDT) Received: by 10.182.142.198 with HTTP; Fri, 14 Mar 2014 03:02:42 -0700 (PDT) In-Reply-To: References: <53221499.40301@comodo.com> Date: Fri, 14 Mar 2014 10:02:42 +0000 Message-ID: From: Eran Messeri To: certificate-transparency@googlegroups.com Content-Type: multipart/alternative; boundary=047d7b33cddcb86d3e04f48e2bcc Archived-At: http://mailarchive.ietf.org/arch/msg/therightkey/dONMBa6GXNvBXESiH-MGoiUn84o Cc: "therightkey@ietf.org" , Rob Stradling , "trans@ietf.org" , CABFPub Subject: Re: [therightkey] [Trans] What's the load on a CT log? X-BeenThere: therightkey@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 10:02:52 -0000 --047d7b33cddcb86d3e04f48e2bcc Content-Type: text/plain; charset=UTF-8 On Thu, Mar 13, 2014 at 9:48 PM, Ben Laurie wrote: > On 13 March 2014 20:27, Rob Stradling wrote: > > I'm not sure average load tells the whole story. > > Agreed. > > > Won't there be a surge in audit traffic in the aftermath of a busy site > > installing a new cert? > > Yes. > This is where separate calls may be beneficial - there's a higher chance that a response to each of the individual calls will be cached rather than a single call that is a response to a client request which contains an STH. > > > > > > On 13/03/14 16:06, Ben Laurie wrote: > >> > >> Several people have asked me this recently. Here's a nice way to > estimate > >> load. > >> > >> Let's assume a single log that takes all the load. > >> > >> Firstly, we see about 5,000 new certificates a day, so that's around > >> 0.06 new certificates per second. Clearly a trivial load. > >> > >> Next is load from audit (i.e. from browsers that wish to validate SCTs > >> accompanying certificates they see). Given some assumptions, we can > >> calculate the load from audit. > >> > >> * Clients cache audit results. > >> > >> * There are approximately b = 2.5B browsers in the world > >> (http://www.internetworldstats.com/stats.htm). > >> > >> * The average user visits w = 89 websites a month > >> (http://www.creditloan.com/blog/how-the-world-spends-its-time-online/ > >> quoting a Nielsen report). Assume these are all TLS sites. > >> > >> * Assume a certificate lifetime of l = 12 months. > >> > >> So, each user sees w / l new certificates a month. Each new > >> certificate needs to be audited, which means in practice, three web > >> operations (fetch STH, fetch STH consistency proof, fetch SCT > >> inclusion proof) - it might be a good idea to create a new API to do > >> all three in one go. > >> > >> So, total average load is 3 * b * w / l ~ 20,000 web fetches per > >> second. If we optimise the API we can get that down to 7,000 qps. Each > >> query (in the optimised case) would be around 3 kB, which gives a > >> bandwidth of around 150 kb/s. > >> > >> Monitors add extra load, but should only be at around the new > >> certificate rate - i.e. ~ .06 * number of monitors fetches per second. > >> > >> IMO, this is achievable on a single machine (modulo reliability), with > >> some care. Clearly not a vast farm, however its done. > >> > >> In practice, no one log would have to take this full load, this is a > >> worst case analysis. > >> > >> _______________________________________________ > >> Trans mailing list > >> Trans@ietf.org > >> https://www.ietf.org/mailman/listinfo/trans > >> > > > > -- > > Rob Stradling > > Senior Research & Development Scientist > > COMODO - Creating Trust Online > > Office Tel: +44.(0)1274.730505 > > Office Fax: +44.(0)1274.730909 > > www.comodo.com > > > > COMODO CA Limited, Registered in England No. 04058690 > > Registered Office: > > 3rd Floor, 26 Office Village, Exchange Quay, > > Trafford Road, Salford, Manchester M5 3EQ > > > > This e-mail and any files transmitted with it are confidential and > intended > > solely for the use of the individual or entity to whom they are > addressed. > > If you have received this email in error please notify the sender by > > replying to the e-mail containing this attachment. Replies to this email > may > > be monitored by COMODO for operational or business reasons. Whilst every > > endeavour is taken to ensure that e-mails are free from viruses, no > > liability can be accepted and the recipient is requested to use their own > > virus checking software. > > -- > You received this message because you are subscribed to the Google Groups > "certificate-transparency" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to certificate-transparency+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > --047d7b33cddcb86d3e04f48e2bcc Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable



On Thu, Mar 13, 2014 at 9:48 PM, Ben Laurie <<= a href=3D"mailto:benl@google.com" target=3D"_blank">benl@google.com>= wrote:
On 13 March 2014 20:27, Rob = Stradling <rob.stradling@com= odo.com> wrote:
> I'm not sure average load tells the whole story.

Agreed.

> Won't there be a surge in audit traffic in the aftermath of a busy= site
> installing a new cert?

Yes.
This is where separate calls may be benefic= ial - there's a higher chance that a response to each of the individual= calls will be cached rather than a single call that is a response to a cli= ent request which contains an STH.


>
>
> On 13/03/14 16:06, Ben Laurie wrote:
>>
>> Several people have asked me this recently. Here's a nice way = to estimate
>> load.
>>
>> Let's assume a single log that takes all the load.
>>
>> Firstly, we see about 5,000 new certificates a day, so that's = around
>> 0.06 new certificates per second. Clearly a trivial load.
>>
>> Next is load from audit (i.e. from browsers that wish to validate = SCTs
>> accompanying certificates they see). Given some assumptions, we ca= n
>> calculate the load from audit.
>>
>> * Clients cache audit results.
>>
>> * There are approximately b =3D 2.5B browsers in the world
>> (http://www.internetworldstats.com/stats.htm).
>>
>> * The average user visits w =3D 89 websites a month
>> (http://www.creditloan.com/blog/how-the-wo= rld-spends-its-time-online/
>> quoting a Nielsen report). Assume these are all TLS sites.
>>
>> * Assume a certificate lifetime of l =3D 12 months.
>>
>> So, each user sees w / l new certificates a month. Each new
>> certificate needs to be audited, which means in practice, three we= b
>> operations (fetch STH, fetch STH consistency proof, fetch SCT
>> inclusion proof) - it might be a good idea to create a new API to = do
>> all three in one go.
>>
>> So, total average load is 3 * b * w / l ~ 20,000 web fetches per >> second. If we optimise the API we can get that down to 7,000 qps. = Each
>> query (in the optimised case) would be around 3 kB, which gives a<= br> >> bandwidth of around 150 kb/s.
>>
>> Monitors add extra load, but should only be at around the new
>> certificate rate - i.e. ~ .06 * number of monitors fetches per sec= ond.
>>
>> IMO, this is achievable on a single machine (modulo reliability), = with
>> some care. Clearly not a vast farm, however its done.
>>
>> In practice, no one log would have to take this full load, this is= a
>> worst case analysis.
>>
>> _______________________________________________
>> Trans mailing list
>> Trans@ietf.org
>> https://www.ietf.org/mailman/listinfo/trans
>>
>
> --
> Rob Stradling
> Senior Research & Development Scientist
> COMODO - Creating Trust Online
> Office Tel: +44.(0)1274.730505
> Office Fax: +44.(0)1274.730909
> www.comodo.com=
>
> COMODO CA Limited, Registered in England No. 04058690
> Registered Office:
> =C2=A0 3rd Floor, 26 Office Village, Exchange Quay,
> =C2=A0 Trafford Road, Salford, Manchester M5 3EQ
>
> This e-mail and any files transmitted with it are confidential and int= ended
> solely for the use of the individual or entity to whom they are addres= sed.
> If you have received this email in error please notify the sender by > replying to the e-mail containing this attachment. Replies to this ema= il may
> be monitored by COMODO for operational or business reasons. Whilst eve= ry
> endeavour is taken to ensure that e-mails are free from viruses, no > liability can be accepted and the recipient is requested to use their = own
> virus checking software.

--
You received this message because you are subscribed to the Google Groups &= quot;certificate-transparency" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to certificate-transparency+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--047d7b33cddcb86d3e04f48e2bcc-- From nobody Tue Mar 18 21:07:42 2014 Return-Path: X-Original-To: therightkey@ietfa.amsl.com Delivered-To: therightkey@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DABF01A0343 for ; Tue, 18 Mar 2014 21:07:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.449 X-Spam-Level: X-Spam-Status: No, score=-2.449 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.547, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7qwZEzceFWnM for ; Tue, 18 Mar 2014 21:07:38 -0700 (PDT) Received: from defuse.ca (defuse.ca [192.99.8.82]) by ietfa.amsl.com (Postfix) with ESMTP id 17A701A025C for ; Tue, 18 Mar 2014 21:07:38 -0700 (PDT) Received: from [192.168.1.102] (S01065404a6902716.cg.shawcable.net [174.0.254.229]) by defuse.ca (Postfix) with ESMTPSA id 01A65100CCB for ; Wed, 19 Mar 2014 00:05:55 -0400 (EDT) Message-ID: <532917FC.2030103@defuse.ca> Date: Tue, 18 Mar 2014 22:07:24 -0600 From: Taylor Hornby Organization: https://defuse.ca/ User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: therightkey@ietf.org X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/therightkey/CrggO6AeKeg-KsX79mFtAmMqpSI Subject: [therightkey] Malicious Exit Nodes X-BeenThere: therightkey@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 04:07:40 -0000 Hi, Something that's not considered in the DetecTor paper is what happens when one or more Tor exit nodes are assumed to be malicious. Maybe this has been discussed on this list before, but a quick search turned up nothing, so sorry if this is a duplicate. For example, a malicious Tor node could purposely MITM the SSL connection, providing an incorrect certificate, to make the client's actual connection fail (Denial of Service). It's also possible for a malicious Tor node (or even a legitamete Tor node trying to save bandwidth) to return cached certificates, hiding an attack. I don't think this would be a huge problem, but it's something to consider for future versions of the paper. -- Taylor Hornby From nobody Wed Mar 19 04:47:20 2014 Return-Path: X-Original-To: therightkey@ietfa.amsl.com Delivered-To: therightkey@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A980F1A06DD for ; Wed, 19 Mar 2014 04:47:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.926 X-Spam-Level: X-Spam-Status: No, score=-1.926 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, RP_MATCHES_RCVD=-0.547, SPF_PASS=-0.001] autolearn=unavailable Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id thxKGmfGqS2A for ; Wed, 19 Mar 2014 04:47:13 -0700 (PDT) Received: from mail-ve0-x232.google.com (mail-ve0-x232.google.com [IPv6:2607:f8b0:400c:c01::232]) by ietfa.amsl.com (Postfix) with ESMTP id 290EC1A03CA for ; Wed, 19 Mar 2014 04:47:13 -0700 (PDT) Received: by mail-ve0-f178.google.com with SMTP id jw12so8523199veb.23 for ; Wed, 19 Mar 2014 04:47:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=mmbtYuz4HGelhr379M1cC1PgZWv+VVTQJ3iTHNR0URI=; b=g1i52Ln3ErnqmI3SledIuhtwifA7iSTYoyLXJpV8FTWl9fI6ufULGnB5j9adAMVROU hS1nBgBJdYKgVH00R9d2NrbHuje99B7KSZoGkxVga96f5Nx33Z+oBCGSI6ydZsIJZpzL 0eFpOdhEbVbDfdhEP6xR+0Rf6jUI3Rr8mnYBxB4xkixDcjqeceVZqvV5yPNwgg/1RrSX crzAOeyfgBjqFvwB6ONqLiHtGA0tZo98bvJrSI2FaE2MVw2daBRuyaUL9NTuYlysuS+e aZkyRPKR9407j067ZD8Cc9XS05+rieN5DJpOe2wjKaZwnealVi8IWnFLOUjQvDGgur+M /y1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=mmbtYuz4HGelhr379M1cC1PgZWv+VVTQJ3iTHNR0URI=; b=RNc8hRCcNMWt/jgooAS460FeSOM4m+VR1SSkicSp2faJl5Egp+H4Ejo7lZxzmQyxFE Y7jakr/V/hVNlKz+ZJPpG/AbUWLCkH+h5vAWKqKTEXP+B6PNEGqA9z78NEuLhXzW2dz4 nEhvFFAWk2WScal4o7btlaSVroLBbTbvZwGO2Xw3foYAyuPMsFFf3EAwq9T6zB/mx0Sq 35mxp8mrJrBMmZmfIZAaprd4Mmr56PL2SNyciv597Bn9m9lZqCpI8z0apfxC18abf9P2 xUBsro5TUeZ3ODgZIWH9QwPgHmotwWRXV2R2AfpEwAjYxiqmEhMfzX3dR1eVLmlJ63G0 HG4g== X-Gm-Message-State: ALoCoQng7bj4wGwukcQ4q1jtb3troyVNK3bA3EQdbB1W3iDBwo8BwaGOJLrjOPU32jECPNdOAtm6IIXVTXnEHMqA0YBnGcAW7/62g3EcJogcKEb5k5gUUXfruyMnQyCUQh23HJWQi6020mGnmzpqEujAyCfgLdEm7E4k/PGAj7MP6ulQFrXfZn6akDw5F/v8njAqn+WxneOh MIME-Version: 1.0 X-Received: by 10.58.122.164 with SMTP id lt4mr29722401veb.2.1395229624260; Wed, 19 Mar 2014 04:47:04 -0700 (PDT) Received: by 10.52.230.105 with HTTP; Wed, 19 Mar 2014 04:47:04 -0700 (PDT) Date: Wed, 19 Mar 2014 11:47:04 +0000 Message-ID: From: Ben Laurie To: CABFPub , "therightkey@ietf.org" , "trans@ietf.org" , "certificate-transparency@googlegroups.com" Content-Type: text/plain; charset=UTF-8 Archived-At: http://mailarchive.ietf.org/arch/msg/therightkey/kGoJSZm-QzNzAOA0WTRmGjny2SE Subject: [therightkey] Hopefully Final Version of Google's EV/CT plan X-BeenThere: therightkey@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 11:47:14 -0000 We've updated the EV/CT plan to finalise all TBDs and to clarify some points from the previous version. The current (and hopefully final) version of the plan can be found at https://sites.google.com/site/certificatetransparency/ev-ct-plan. Ideally questions should be directed to the appropriate list(s), but feel free also to contact me directly.