Subject: Re: [sidr] Slides for "RPKI Over BitTorrent" presentation

From: Eric Osterweil <[email-masked]>
Date: Sun, 1 Apr 2012 12:26:54 -0700

On Mar 29, 2012, at 12:04 PM, Rob Austein wrote:

> At Wed, 28 Mar 2012 20:33:24 -0400, Danny McPherson wrote:
>> 
>> i don't think the rsync scale issues surprise anyone that was paying
>> attention.  If we're already considering new architectures,
>> substrates, et al., here perhaps we shouldn't be so quick on the
>> trigger for Standards Track work and move this and related
>> "investigation" to the IRTF, or at least ensure they're only
>> Experimental until broader experience is gained.
> 

<snip>

> 
> For the record, this presentation was originally targeted at the IEPG,
> as a direct follow-up to multiple suggestions received at a previous
> IEPG meeting that it might be interesting to see how BitTorrent works
> in this environment.  I presented this at the SIDR WG meeting because
> the chairs thought it might be of interest to the WG.
> 

So, I just want to wade in here very quickly: in the sidr wg we are constantly facing packed agendas, multiple sessions, and now interim meeting extravaganzas... I feel like there is no shortage of feedback that people try to give to many of the drafts (during sessions and obviously on the list), but the session agendas often seem to require the chairs to cut the mic lines (imho) way too early.  This has made feedback harder to give.  I (for one) was a little surprised to see the bittorrent work presented at both the iepg and again during the wg considering the amount of feedback that often comes from pretty much _any_ given draft on the agenda.  I am not claiming it is bad work, inappropriate, or anything like that.  Simply, an interesting scheduling choice...

With all due respect (honestly, I know this can't be easy), I really feel this is symptomatic of a disconnect between the session agendas and the level of comfort that the overall wg members have with some of the drafts.  I really think fewer/more focused sets of presentations are critical to moving forward and I think I've heard _several_ people on the list push very hard for us to return our focus to solid requirements analysis (which I think very much should include drafts like the req's draft, Steve's threats draft, etc) so that we can evaluate the nature of the work we are all trying to do separately from how implementations and designs are being fleshed out.  Getting the wg members on the same page wrt what we are trying to do would likely make a number of points of friction smoother.  I really think this does _not_ necessarily invalidate work that has already been done (though we may still need to examine some of it carefully).  For example, Danny's questions to Rob et al about the scaling requirements that the protocol was aimed at is very apropos of us proceeding in an orderly fashion towards a feasible security system.  Lessons learned from experiments can be used to inform evolving requirements.  That is, this isn't necessarily the waterfall software lifecycle ( http://en.wikipedia.org/wiki/Waterfall_model ), which only goes forward without feedback loops.  Personally, I would find it helpful to see less packed, more focused, and a return to reqs in future sessions if possible...  And, yes, we should strive for world peace too...

Thanks,

Eric

---

From: Eric Osterweil <[email-masked]>
Date: Sun, 1 Apr 2012 12:44:01 -0700

Hey Rob,

One more meta-comment on this thread: we (as designers) should not expect to be prescribing exactly how operators will run their systems.  Here in the very early stages we are seeing the early adopters deploy in ways that make _their_ operations easy.  imho, this is always going to be the bottom line/starting point.  If a solution requires a specific operational model (when others are technically correct), and the overall system scalability can suffer this way, I think we have a fundamental problem.  

More succinctly, operators shouldn't need to know why you want their certs in a hierarchy.  Rather, we should recognize that they are telling us that flat structures make their lives easier, and we should use that to reengineer what _they_ need.

Just my 0.02,

Eric

On Mar 30, 2012, at 11:10 AM, Rob Austein wrote:

> While I will admit to some astonishment that the following explanation
> could possibly be news to long-time participants in this WG (given how
> much time I've spent whining about this issue over the last five years
> or so both in public and in private), let me quote from the slides:
> 
>    * How efficient [fetching RPKI repositories using rsync] is
>      depends heavily on how the publication repositories are
>      organized.
> 
>    * In an efficiently organized repository, filesystem hierarchy
>      follows X.509 certificate hierarchy, so that one can pick up
>      significant subtrees with a single rsync connection.
> 
>    * To date, the RIRs have chosen to deploy flat hierarchies where
>      there is no relationship at all between filesystem hierarchy
>      within the repository and certificate hierarchy.
> 
> To make that more concrete, here's an example.  Let's assume we have
> the following trivial hierarchy: Bob and Betty are issued by Alice,
> Carol and Carl are issued by Bob, Dave, and Dana are issued by Carol,
> Dara is issued by Carl, and and all of these are hosted in a single
> repository.
> 
> In an inefficient, "flat" repository, the publication points for
> objects issued by these entities would look something like this:
> 
>    rsync://example.org/rpki/Alice/
>    rsync://example.org/rpki/Betty/
>    rsync://example.org/rpki/Bob/
>    rsync://example.org/rpki/Carl/
>    rsync://example.org/rpki/Carol/
>    rsync://example.org/rpki/Dana/
>    rsync://example.org/rpki/Dara/
>    rsync://example.org/rpki/Dave/
> 
> In a hierarchical repository, the same publication points would look
> more like this:
> 
>    rsync://example.org/rpki/Alice/
>    rsync://example.org/rpki/Alice/Betty/
>    rsync://example.org/rpki/Alice/Bob/
>    rsync://example.org/rpki/Alice/Bob/Carl/
>    rsync://example.org/rpki/Alice/Bob/Carl/Dara/
>    rsync://example.org/rpki/Alice/Bob/Carol/
>    rsync://example.org/rpki/Alice/Bob/Carol/Dana/
>    rsync://example.org/rpki/Alice/Bob/Carol/Dave/
> 
> Assuming top-down tree walk (the normal case), retrieving objects
> issued by this set of entities takes eight rsync connections with the
> flat repository, as opposed to one rsync connection with the
> hierarchical repository.
> 
> In practice one might want a slightly more complex structure to limit
> the size of individual directories, but it doesn't matter so long as
> the filesystem hierarchy is organized in such a way that picking up
> an issuer's publication point picks up a non-trivial number of its
> subjects' publication points automatically.   It doesn't have to be
> perfect, just has to do enough better than the flat model to amortize
> the cost of setting up and tearing down the rsync connection over a
> significantly larger number of files.
> 
> This is not about PKI, it's purely an rsync efficiency issue.
> 
> Presumably there are scaling limitations to the hierarchical approach,
> but anecdotal evidence among the people I've asked ("I tried ... and
> it worked") suggests that, if the underlying networks and filesystems
> are in good shape, a single rsync connection ought to be able to
> handle up to at least 10,000 small files, perhaps a lot more than
> that.  Note that this is just talking about rsync itself: mileage
> might vary significantly if the underlying networks or filesystems are
> seriously broken.  Also note that these anecdotal estimates have not
> been tested in any rigorous fashion as far as I know, so that's
> another entry on my list of things we ought to be measuring.
> 
> Hope this helps to clarify the change I've been suggesting.
> _______________________________________________
> sidr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sidr