Subject: [SAM] [www.ietf.org/rt #50961] The IETF rsync server is down again

From: "Glen via RT" <[email-masked]>
Date: Wed, 26 Sep 2012 10:03:04 -0700

On Wed Sep 26 09:39:37 2012, [email-masked] wrote:
> > The IETF rsync server was not, in fact, down.  All available
> connections were being used, and that
> > situation has been rectified.
> 
> Thanks. Out of curiosity, why did you set "max connections" in
> rsyncd.conf?
> 
> --Paul Hoffman

Hi Paul -

Glen here.  This has been escalated to me.

I always try to set sane limits on all processes, to prevent one runaway
from consuming the entire server.  It seems like a responsible way to
proceed; and, in this case, my guess is that the offending external site
could have used up a much larger number of processes, potentially
impacting the rest of the server, had those limits not been set.

I hope this information is helpful, let me know if you need more.

Glen
Glen Barney
IT Director
AMS (IETF Secretariat)

---

From: "Glen via RT" <[email-masked]>
Date: Wed, 26 Sep 2012 12:22:58 -0700

On Wed Sep 26 10:30:06 2012, [email-masked] wrote:
> Yeah, that makes sense for a box that has multiple services on it.
>    However, it means that you (in the future) need to monitor when the
>    limit is being hit for, say, more than an hour. And then do
>    something about it, which is harder.

Yes, I totally agree.  I'm building a comprehensive monitoring system
for stuff like this for AMS, and the IETF, so your point is well-taken
and we will definitely be adding something like this in shortly.

> Glad I only do ops as a hobby...

Ahh, I dream of such luxury!

Best regards,
Glen