Subject: RE: [Forces-protocol] Issue 2: Multiple channels/connections

From: "Khosravi, Hormuzd M" <[email-masked]>
Date: Sun, 14 Mar 2004 21:19:40 -0800

Avri,

This is a good question and one which I spent a lot of time thinking
about during FACT work. There are several ways to address this (pointed
out by yourself, Jamal, Alex), a) using heartbeats for D connection, b)
protocol level responses/acks, c) LMP-(dont know much about this?)

For now I think its good that we recognize this as something that needs
to be addressed. We should probably have a section on Error Recovery in
the protocol which will address the details of what should be done in
case of such problems. However, this discussion is something that we can
defer for now...and complete resolving the major issues. Is that fine
with you ?

Hormuzd

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of [email-masked]

I have a question.  If there is no control traffic on the data link, 
how will you know if there is a data link failure?  Do you just wait 
for a transport protocol time out? And is that sufficiently timely?  Or 
will you have some other mechanism?

a.

_______________________________________________
Forces-protocol mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/forces-protocol

---

From: Jon Maloy <[email-masked]>
Date: Mon, 15 Mar 2004 16:07:25 -0500

See below
/jon

Khosravi, Hormuzd M wrote:

Avri,

out by yourself, Jamal, Alex), a) using heartbeats for D connection,

With heartbeats people normally mean a periodic sending of small
messages
to confirm that the sender is still alive. If you have a huge network
the background
load for doing this may become significant, especially if you have more
than
one connection per FE-CE. 
I found a way to lessen this load by letting each endpoint keeping track
of incoming 
data messages, and leave a trace (an updated sequence number, but just a
flag would do)
when a message is received. 
Then, instead of just blindly sending heartbeats under any
circumstances, an endpoint 
finding that the other end has remained silent too long, send a PROBE
message, 
imlying an order to the peer to immediately respond with a PROBE_REPLY.
Of course, the peer sets its 'incoming received' flag when it receives
the
PROBE.  
This way, not  a single more message than strictly necessary is sent in 
to keep the endpoints updated about the state of the connection.

 b)

Responses/acks are only useful when you actually have traffic. In this
case
we have a situation where there is no traffic, and we must find a mean
to
find out if this is normal or due to communication failure. Probes are
useful here, acks are not. (Of course, hey may be needed for other
purposes,
eg if the channel is unreliable, but not for connection failure
detection.)

with you ?

Sure. See this as an input for that later discussion.

Hormuzd

-----Original Message-----

From:  [email-masked]
<mailto:[email-masked]> 

[ mailto:[email-masked]
<mailto:[email-masked]> ] On Behalf Of  [email-masked]
<mailto:[email-masked]> 

a.

[email-masked] <mailto:[email-masked]> 

https://www1.ietf.org/mailman/listinfo/forces-protocol
<https://www1.ietf.org/mailman/listinfo/forces-protocol> 

This communication is confidential and intended solely for the addressee(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you believe this message has been sent to you in error, please notify the sender by replying to this transmission and delete the message without disclosing it. Thank you.

E-mail including attachments is susceptible to data corruption, interruption, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof.

---

From: Jamal Hadi Salim <[email-masked]>
Date: 16 Mar 2004 08:05:48 -0500

Since this topic keeps coming up and it is more related to topic#7 i
have changed the subject.

On Mon, 2004-03-15 at 16:07, Jon Maloy wrote:
[..]
> messages to confirm that the sender is still alive. If you have a huge
> network the background load for doing this may become significant,
> especially if you have more than one connection per FE-CE.

So two things whose failures need to be detected in my opinion.

1) The CE/FE. As you say in standard setups, its the sender that
issues the heartbeats. In this case probably makes sense heartbeats are
coming out of the CE. In our case heartbeats are always sent on
multicast (insetad of unicasting to each FE) to reduce CPU and b/width
abuse. This leaves out the CE knowing about the health of the FEs while
providing good knowledge of the CEs health to FEs. 
Sending heaartbeats from FEs -> CEs is too burderning. So to
compensated, we would request for an ACK from the FEs on occasion and
such hearbeat ACKs would also be sent on multicast in case other FEs are
interested in the information. The CE could also request over unicast a 
specific FE to respond.

2) a connection going down.
Something like TCP i seasy to figure out. A physical link going down is
easy too. connectionless connections (mouthful) are harder.

>  
> I found a way to lessen this load by letting each endpoint keeping
> track of incoming data messages, and leave a trace (an updated
> sequence number, but just a flag would do) when a message is received.

I take it theres time associated with the validity of the flag as well

> circumstances, an endpoint finding that the other end has remained
> silent too long, send a PROBE message, imlying an order to the peer to
> immediately respond with a PROBE_REPLY.
>
> the PROBE.

This is a good optimization. I still think that synchronous heartbeats
from CE->FE are a good idea. FE->CE could make use of what you
described.
Seems to me these are close to what we have on netlink2.
i.e NLMSG_NOOP command with NLM_F_ECHO flag when response needed.

> eg if the channel is unreliable, but not for connection failure

Agreed.

Additionaly:
Theres still issue of redundancy and fault management associated with
finding problems. Theres still issue of decidding how much a FE
should "know" etc.
But we can defer this. 

cheers,
jamal