Subject: RE: [Forces-protocol] topic 4c: Common Header: batching

From: "Khosravi, Hormuzd M" <[email-masked]>
Date: Tue, 11 May 2004 16:28:31 -0700

Weiming, All

You have raised 2 good points. Thanks for giving this more thought.

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Weiming Wang

One more thing for batch and atomicity I'v just thought of.
Consider a scenario, in which a CE is configuring a FE using batch or
atomicity.
We used to consider that, during the batch or atomicity process, there
might be
no other messages except the batch or atomicity messages sent from CE to
FE, but
I'v just thought of that this may be not always true. For e.g., there
may be
heartbeat messages sent from CE to FE during the process, because they
will be
sent automatically and periodically. If we only use flags to indicate
the batch
or atomicity messages, such heartbeat messages may intervene into the
batch or
atomicity messages, making the batch or atomicity marked by flags
incorrectly
understood by CE TML or by FE PL.
[HK] If we use 2 bits flag for batching, we can easily solve this
problem.
E.g. 00- means no batch
     01 - means start of batch (SoB)
     10 - means middle of batch (MoB)
     11 - means end of batch (EoB)

Therefore, we may need to limit more for batch or atomicity. For e.g.,
we may
need to define that only messages for config, query, and state
maintenance are
allowed to be batched or atomically executed, that is, messages other
than that
are automatically excluded from batch or atomicity.
[HK] This is another way to further protect against this, I think we
should do both
(Use 2 bits and specify which msgs cannot be batched)..thoughts ?

A more extended thinking of this is, if we need to support parallel and
multipul
atomicity at the same time, that is, if we need to allow CE to send
messages
that interleavingly belong to different atoms? I think Jamal has raised
this
question before.
[HK] We can easily solve this using same Sequence No or lets call it
Transaction Seq No for all messages belonging to the same batch. That's
how its done in a lot of protocols...what do you guys think ?

Regards
Hormuzd

_______________________________________________
Forces-protocol mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/forces-protocol

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 12 May 2004 10:52:07 +0800

Hormuzd, and all,

Thank you for the comments.

----- Original Message -----
From: "Khosravi, Hormuzd M" <[email-masked]>

Weiming, All

-----Original Message-----

One more thing for batch and atomicity I'v just thought of.
Consider a scenario, in which a CE is configuring a FE using batch or atomicity.
We used to consider that, during the batch or atomicity process, there might be
no other messages except the batch or atomicity messages sent from CE to FE, but
I'v just thought of that this may be not always true. For e.g., there may be
heartbeat messages sent from CE to FE during the process, because they will be
sent automatically and periodically. If we only use flags to indicate the batch
or atomicity messages, such heartbeat messages may intervene into the batch or
atomicity messages, making the batch or atomicity marked by flags incorrectly
understood by CE TML or by FE PL.
[HK] If we use 2 bits flag for batching, we can easily solve this problem.
E.g. 00- means no batch
     01 - means start of batch (SoB)
     10 - means middle of batch (MoB)
     11 - means end of batch (EoB)
[Weiming] Seems reasonable. So, we may also need two bits for atomicity.

Therefore, we may need to limit more for batch or atomicity. For e.g., we may
need to define that only messages for config, query, and state maintenance are
allowed to be batched or atomically executed, that is, messages other than that
are automatically excluded from batch or atomicity.
[HK] This is another way to further protect against this, I think we should do
both
(Use 2 bits and specify which msgs cannot be batched)..thoughts ?
[Weiming]If we use two bits flags, physically we don't need to specify with msgs
can or cannot be batched. But I agree it is still of some meaning semantically
to do so, that is, e.g., a heartbeat event msg should not be batched.

A more extended thinking of this is, if we need to support parallel and multipul
atomicity at the same time, that is, if we need to allow CE to send messages
that interleavingly belong to different atoms? I think Jamal has raised this
question before.
[HK] We can easily solve this using same Sequence No or lets call it
Transaction Seq No for all messages belonging to the same batch. That's
how its done in a lot of protocols...what do you guys think ?

[Weiming] Yes,  that was also one of my thought (in the discussion with Robert).
But I just think purely using the same SN for all batched msgs or atomicity msgs
may result in difficulty and ambiguity for responses/ACKs  (then we may have
several responses/ACSs that have the same SN and make sender at a loss which to
which?). As a result, I just think we may need something like a SubSN for
batching and atomicity. We may have two schemes to use:

1) With the main SN the same, and with the SubSN different for msgs belong to
the same batch or atomicity. In this case, the SubSN should be large enough able
to contain allowed max number of batched msgs. I think we may need at least 8
bits for such SubSN.

2) With the main SN incrementally changd as normal, but with the SubSN keeing
the same for msgs in the same batch or atomicity while incrementally changed for
different batches or atomicities. In this case, the SubSN actually acts as a
counter and indicator for different parallel batches or atomicities, therefore,
the space for this SubSN need to be as large as the max allowed parallel batch
or atomicity  numbers.  I think we may only need  4 bits for such SubSN, that
is, we only allow 16 batches or atomicities parallelly processed.

Hope the above schemes make a little sense. But note that, if we decide not to
support parallel transactions/batch/atomicity, we may not need to consider the
schemes, instead, the 2 bits flags for batch or atomicity as mentioned above is
enough.

[/Weiming]

Regards
Hormuzd

Cheers,
Weiming

---

From: Jamal Hadi Salim <[email-masked]>
Date: 12 May 2004 07:10:07 -0400

Weiming, please take over the text for batching since you are already
doing a good job collecting the info. I dont like things that end with
flag like batchflag - it is already part of flag, so if you call it
batch for example, should be sufficient.

Comments on email follow

> [Weiming] Seems reasonable. So, we may also need two bits for atomicity.

I am trying to see why you need two bits. There are only two pieces of
information that need to be sent: atomic start/yes and atomic end/no.
You only need one bit for that 1 or 0.

> to do so, that is, e.g., a heartbeat event msg should not be batched.

I think if you just set the flag to off, you sould be able to
distinguish the atomic scenario easily.

> batching and atomicity.

Yes, this is the headache-causing part!

>  We may have two schemes to use:
> 

We could steal subSN from the T part of the TLV. 

> is, we only allow 16 batches or atomicities parallelly processed.

16 add route per PL message may not be so good. I am going to go and
think about this and say something more meaningful.

> schemes, instead, the 2 bits flags for batch or atomicity as mentioned above is

Agreed.

cheers,
jamal

---

From: "Weiming Wang" <[email-masked]>
Date: Wed, 12 May 2004 21:08:44 +0800

Jamal,and all,

I realize now the first thing we may need to decide is if we need to parallelly
support multi-transactions of atomicity and batch, because it affects other
definitions like the flags and the SN usage.

My opinion is, we may need to use parallel atomicity and batch, because they
seem useful when we sometimes want to simulteneously configure several LFBs with
their attributes. E.g., an CE is adding many routes to a forwarder LFB in a FE,
which we set as an atomicity and may last for many messages, therefore may also
last for a period of time. During the period, CE may also want to config other
LFBs with atomicity. If our protocol does not support multi-atomicity at the
same time, we have to let CE have the 'add route' atomicity done before begin
another atomicity to configure other LFBs.

Jamal, I suppose you tend to use parallel atomicity because you first raise
this, but not very sure. How about other guys thoughts?

Pls also see some more replies inline.

Cheers,
Weiming

----- Original Message -----
From: "Jamal Hadi Salim" <[email-masked]>
> batch for example, should be sufficient.
[WeimingRe] How about let's just discuss the techniques first, leaving the
writing the last thing to do. I think it is relatively easy thing after we'v
resolved all technically. Talking about the flag name, I suppose if we can
uniformly use a 3 letter format as the name, just like TCP did, such as names as
ACK(or RSP?) for response policy, PRO for priority, BAT(or BTH) for batch,
ATM(or ATO) for atomicity)?

atomicity.
incorrectly
> You only need one bit for that 1 or 0.
[WeimingRe]I suppose if we decide to support multi-atomicity and batch, and to
use something like SubSN, we may not need to use two bits. Only one bit is
enough.

multipul
> We could steal subSN from the T part of the TLV.
[WeimingRe]I'm thinking.
> think about this and say something more meaningful.

[Weiming]Here the 16 batches actually mean 16 parallel batches the TML can
support. Every batch or atomicity may still contain as many messages as we like.
So, the allowed routes to be added in a batch is not limited to 16.
Here, one more important thing I hope we can make clear. That is, my thought has
tent to use message chains for batch, which seems you also tend. I'm not sure
when you mention '16 add route per PL message', do you mean we may need to use
TLV chains for batch also?

---

From: [email-masked]
Date: Wed, 12 May 2004 16:01:10 +0200

> this, but not very sure. How about other guys thoughts?

I would worry about introducing great amounts of complexity to the 
protocol.

While i see value in atomic operations, i think they should be kept to 
a minimum as opposed to non-atomic batching - which is just for 
transport convenience and can be used freely as long as there is a 
sperate status per operation.  I thus think that the mechanism should 
be kept simple.

Since each transaction has its own id,  i see no reason a heartbeat or 
other command could not be dealt with separately as it arrived without 
needing special flags.  I also think multiple atomic ops could be dealt 
with in parallel without special mechanisms - the CE should not send a 
dependent op until the preceding atomic op has completed successfully.

a.

---

From: Robert Haas <[email-masked]>
Date: Fri, 14 May 2004 01:03:18 +0200

I finally see a justification for plain batching (that is, batching 
without atomicity): a batch will result in a single status: OK if all 
commands in the batch failed, and notOK if some of them failed (and 
probably we should stop processing commands after the first failure).

Is this a semantics that would be acceptable to everyone ?

-Robert

[email-masked] wrote:

> needing special flags.  I also think multiple atomic ops could be 
> dealt with in parallel without special mechanisms - the CE should not 
> send a dependent op until the preceding atomic op has completed 
> successfully.
>
> a.
>
>

-- 
Robert Haas
IBM Zurich Research Laboratory
Säumerstrasse 4
CH-8803 Rüschlikon/Switzerland
phone +41-1-724-8698  fax +41-1-724-8578  http://www.zurich.ibm.com/~rha

---

From: [email-masked]
Date: Fri, 14 May 2004 15:44:47 +0200

On 14 maj 2004, at 01.03, Robert Haas wrote:

> without atomicity): a batch will result in a single status:

[ad]  Actually I have found that problematic.  What i have found is 
that I need a error per command.  the overall error indicator can be 
used to indicate that one or more of the commands failed, but each 
command probably needs its own error indicated.

> OK if all commands in the batch failed, and notOK if some of them 
> failed (and probably we should stop processing commands after the 
> first failure).

I think that is a different type of command, the do until error batch.

> Is this a semantics that would be acceptable to everyone ?

I guess I don't think so.

a.

---

From: "Weiming Wang" <[email-masked]>
Date: Mon, 17 May 2004 22:29:17 +0800

Hi Avri,

----- Original Message -----
From: <[email-masked]>
> command probably needs its own error indicated.
[Weiming] I agreed. I suppose if following scheme can meet the requirement:

Taking a Config msg with the format as:

 CommonHeader(ConfigType): TLVforOP1:TLVforOP2:...TLVforOPn

as an example. We may allow two format of ACK/Response.

Verbose mode - the response is,
CommonHeader(ConfigReType): TLVforReOP1:TLVforReOP2:TLVforReOPn:TLVforReSUM.

Precise mode - the response is,
CommonHeader(ConfigReType): TLVforReSUM

The TLVforReSUM includes a summary for the transaction excution.

I also suggest (as in my recent reply to Jamal's summary ans also in the meeting
minutes), we may better to treat a msg as a transaction, that is, all
ops(commands) inside one msg should have the meaning of 'all or nothing', so as
to simplify our protocol definition. Could you give some comments on the
thought?

> I guess I don't think so.
Thank you very much.

BR
Weiming

---

From: Robert Haas <[email-masked]>
Date: Tue, 18 May 2004 14:15:51 +0200

Weiming,

I'd prefer not to enforce that all commands in the same message must be 
treated as a single atomic transaction. As Jamal noted, treating a few 
1000s of commands as a single atomic operation in the FE will require 
much more resources and more complex processing path. I'd like to be 
able to treat this 1000s of commands in the same message one after the 
other, without having to remember the results and cache them until all 
1000s have succeeded, in case roll-back is needed.

My proposal regarding atomicity would hence be:

 2 flags, AtomicStart (AS) and AtomicEnd (AE) in the Common Header:

 if AS=AE=1: the (multiple) commands in the message must be treated as 
atomic.

 if AS=AE=0: the (multiple) commands in the message must be treated as a 
sequence of independent commands.

 if AS=1 and AE=0: start of atomic transaction that can span multiple 
messages until a message with AS=0, AE=1, arrives.

 if AS=AE=0: either the middle of an atomic transaction, or no atomic 
transaction at all. For clarity, a redundant  AtomicMiddle (AM) flag 
could be used.

We need to identify implications on batching.

Regards,
-Robert

---

From: "Wang,Weiming" <[email-masked]>
Date: Tue, 18 May 2004 22:03:18 +0800

Robert,

Yes, I also noticed that to treat a msg as one transaction may be not efficient enough for mulitple tiny independent transactions. I see that your scheme is to treat one msg either as one transaction or as a sequence of independent transactions. I agree with this scheme well, because it is actually still simple one, with I think only one more bit of flag to indicate it (also as the flags you mentioned). Now we need to discuss more is if we need to support the scheme where there are independent transactions as well as several atomic transactions inside one msg, which I think is also what Jamal means. This truly makes things quite complex. I'l just stop here. I think you are already on the phone.

See you.
Weiming

----- Original Message ----- 
  From: Robert Haas 

  Weiming,

  I'd prefer not to enforce that all commands in the same message must be treated as a single atomic transaction. As Jamal noted, treating a few 1000s of commands as a single atomic operation in the FE will require much more resources and more complex processing path. I'd like to be able to treat this 1000s of commands in the same message one after the other, without having to remember the results and cache them until all 1000s have succeeded, in case roll-back is needed.

   if AS=AE=1: the (multiple) commands in the message must be treated as atomic.

   if AS=AE=0: the (multiple) commands in the message must be treated as a sequence of independent commands.

   if AS=1 and AE=0: start of atomic transaction that can span multiple messages until a message with AS=0, AE=1, arrives.

   if AS=AE=0: either the middle of an atomic transaction, or no atomic transaction at all. For clarity, a redundant  AtomicMiddle (AM) flag could be used.

  Regards,
  -Robert

  Weiming Wang wrote:

Hi Avri,

----- Original Message -----
From: <[email-masked]>

      

      

a.

Thank you very much.

BR
Weiming