From ssireskin@gmail.com Tue Dec 4 09:13:04 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8B9A921F8C3D for ; Tue, 4 Dec 2012 09:13:04 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.522 X-Spam-Level: X-Spam-Status: No, score=-3.522 tagged_above=-999 required=5 tests=[AWL=0.076, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ync+hKsqFhLx for ; Tue, 4 Dec 2012 09:13:04 -0800 (PST) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by ietfa.amsl.com (Postfix) with ESMTP id 0A6CB21F8C38 for ; Tue, 4 Dec 2012 09:13:04 -0800 (PST) Received: by mail-pb0-f54.google.com with SMTP id wz12so2986911pbc.13 for ; Tue, 04 Dec 2012 09:13:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=B6Pln56YiopHhHn92kw3eA97XPvL4fTLUMyu5b+t9K4=; b=QQEOEv1wv4d2IrTmEX5zAT+Ipv6e2/Xe2BXWqbQzC9Ly9FumBBFHZPybp/3RgJIyyp H+3bR1SFbYJulwHdpstVCPJmjwW8FOqrlkW5hLjZldZXkL5D62M2+16Y/rEQg8wBNkD0 p6QcZLil+aoN0NRNA0NLgPlqInPRIqLKs8GsVP06eyI5ExRoYjiBEfUuQruwO/9h8bN5 NqOauimFUHCAgwNsSpEdEDD5TWWHL+f+VU/A7jkwLCvHdQru2Uk7mH9msgHBX2ogXDrO OJ19G3/BPoVVkhNOxUwocVT17VBuTnNCuSNhOrfz204vMDLR5GZRfTQKpGvqO9R3TqhK zxQQ== MIME-Version: 1.0 Received: by 10.68.225.201 with SMTP id rm9mr756710pbc.124.1354641183850; Tue, 04 Dec 2012 09:13:03 -0800 (PST) Received: by 10.68.18.172 with HTTP; Tue, 4 Dec 2012 09:13:03 -0800 (PST) Date: Tue, 4 Dec 2012 20:13:03 +0300 Message-ID: From: "ssireskin@gmail.com" To: dtn-users@irtf.org Content-Type: multipart/alternative; boundary=047d7b2ede338fbf7104d009fa6f Subject: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Dec 2012 17:13:04 -0000 --047d7b2ede338fbf7104d009fa6f Content-Type: text/plain; charset=ISO-8859-1 Hello all! I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a problem when using netcat (nc) with dtntunnel. On the sender node I run "cat /etc/passwd | nc receiver_ip". On the receiver node I run "nc -l 9999". With the help of Iptables port 9999 gets redirected to the port 19999, which is listened by dtntunnel. The file /etc/passwd is successfully delivered to the receiver and is shown on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it sends the file. It continues to run forever. When I run nc in the opposite direction, I get the same problem - the sending nc doesn't exit. My configuration on both nodes is symmetric: iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to $LOCAL_HOST:19999 dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d dtnping works ok in both directions. Please give me any advice. With best regards, Sergey Sireskin --047d7b2ede338fbf7104d009fa6f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello all!

I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I= am having a problem when using netcat (nc) with dtntunnel.
On the sende= r node I run "cat /etc/passwd | nc receiver_ip". On the receiver = node I run "nc -l 9999". With the help
of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel.

The file /etc/passwd is successfully delivered to the = receiver and is shown on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it s= ends the file. It continues to run forever. When I
run nc in the opposi= te direction, I get the same problem - the sending nc doesn't exit.

My configuration on both nodes is symmetric:
iptables -t nat -A OUTP= UT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to $LOCAL_HOST:19999
dt= ntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d=
dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d

dtnping works o= k in both directions.

Please give me any advice.

With best r= egards,
Sergey Sireskin --047d7b2ede338fbf7104d009fa6f-- From david.a.zoller@nasa.gov Tue Dec 4 12:48:49 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2982321F875E for ; Tue, 4 Dec 2012 12:48:49 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z-6LRNyiy3V3 for ; Tue, 4 Dec 2012 12:48:47 -0800 (PST) Received: from ndjsnpf02.ndc.nasa.gov (ndjsnpf02.ndc.nasa.gov [198.117.1.122]) by ietfa.amsl.com (Postfix) with ESMTP id A448F21F85DC for ; Tue, 4 Dec 2012 12:48:47 -0800 (PST) Received: from ndmsppt05.ndc.nasa.gov (ndmsppt05.ndc.nasa.gov [198.117.0.104]) by ndjsnpf02.ndc.nasa.gov (Postfix) with ESMTP id 60C5DA8011; Tue, 4 Dec 2012 14:48:47 -0600 (CST) Received: from ndmshub03.ndc.nasa.gov (ndmshub03-pub.ndc.nasa.gov [198.117.0.42]) by ndmsppt05.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qB4Kmkc6029993; Tue, 4 Dec 2012 14:48:46 -0600 Received: from NDMSSCC05.ndc.nasa.gov ([198.117.2.174]) by ndmshub03.ndc.nasa.gov ([198.117.2.162]) with mapi; Tue, 4 Dec 2012 14:48:46 -0600 From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "ssireskin@gmail.com" , "dtn-users@irtf.org" Date: Tue, 4 Dec 2012 14:48:45 -0600 Thread-Topic: [dtn-users] A problem with dtntunnel Thread-Index: Ac3SQqKY3nmhmztyTDy4Ju5Qfp1cuQAGcTtQ Message-ID: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_04E3D99A62496240BCD6A576813E6E31E0BDBBA4FANDMSSCC05ndcn_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8185, 1.0.431, 0.0.0000 definitions=2012-12-04_06:2012-12-04, 2012-12-04, 1970-01-01 signatures=0 Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Dec 2012 20:48:49 -0000 --_000_04E3D99A62496240BCD6A576813E6E31E0BDBBA4FANDMSSCC05ndcn_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Sergey, On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not impa= ct the behavior of dtntunnel. I just ran your test without the iptables redirect and I see the same behav= ior. I believe that the sender exits if you go directly from nc to nc because th= e receiver exits when it gets an end of file. You can add a "-w 1" option to the sender nc so that it will timeout and ex= it after stdin is idle for 1 second. Best regards, DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Beh= alf Of ssireskin@gmail.com Sent: Tuesday, December 04, 2012 11:13 AM To: dtn-users@irtf.org Subject: [dtn-users] A problem with dtntunnel Hello all! I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a probl= em when using netcat (nc) with dtntunnel. On the sender node I run "cat /etc/passwd | nc receiver_ip". On the receive= r node I run "nc -l 9999". With the help of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel. The file /etc/passwd is successfully delivered to the receiver and is shown= on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it sends= the file. It continues to run forever. When I run nc in the opposite direction, I get the same problem - the sending nc d= oesn't exit. My configuration on both nodes is symmetric: iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to = $LOCAL_HOST:19999 dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc = -d dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d dtnping works ok in both directions. Please give me any advice. With best regards, Sergey Sireskin --_000_04E3D99A62496240BCD6A576813E6E31E0BDBBA4FANDMSSCC05ndcn_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Sergey= ,

On RHEL 5.7, I am runnin= g dtn-2.9.0 plus modifications that should not impact the behavior of dtntu= nnel.

I just ran your test= without the iptables redirect and I see the same behavior.

I believe that the sender exits if you go= directly from nc to nc because the receiver exits when it gets an end of f= ile.

You can add a “= -w 1” option to the sender nc so that it will timeout and exit after = stdin is idle for 1 second.

Best regards,

DZ

 

David Zoller=

COLSA Corporation<= o:p>

HOSC / C107 

(Office:= (256) 544-1820
*EMail: david.a= .zoller@nasa.gov

 

From: dtn-us= ers-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Tuesday, December 04, 2012 11:13 AMTo: dtn-users@irtf.org
Subject: [dtn-users] A problem wi= th dtntunnel

 

Hello all!

I am using dtn-2.9.0 on a RHEL 6 ba= sed Linux distro and I am having a problem when using netcat (nc) with dtnt= unnel.
On the sender node I run "cat /etc/passwd | nc receiver_ip&q= uot;. On the receiver node I run "nc -l 9999". With the help
= of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel.

The file /etc/passwd is successfully delivered to the = receiver and is shown on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it sen= ds the file. It continues to run forever. When I
run nc in the opposite= direction, I get the same problem - the sending nc doesn't exit.

My= configuration on both nodes is symmetric:
iptables -t nat -A OUTPUT -d = $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to $LOCAL_HOST:19999
dtntunne= l -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d
dt= ntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d

dtnping works ok = in both directions.

Please give me any advice.

With best reg= ards,
Sergey Sireskin

= --_000_04E3D99A62496240BCD6A576813E6E31E0BDBBA4FANDMSSCC05ndcn_-- From ssireskin@gmail.com Wed Dec 5 00:56:01 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F245C21F8C9C for ; Wed, 5 Dec 2012 00:56:00 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.548 X-Spam-Level: X-Spam-Status: No, score=-3.548 tagged_above=-999 required=5 tests=[AWL=0.050, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DAWKjnSgwKIt for ; Wed, 5 Dec 2012 00:56:00 -0800 (PST) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by ietfa.amsl.com (Postfix) with ESMTP id ACDE321F8717 for ; Wed, 5 Dec 2012 00:55:59 -0800 (PST) Received: by mail-la0-f54.google.com with SMTP id j13so5102737lah.13 for ; Wed, 05 Dec 2012 00:55:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=BJuSICDxLNjpDZSt4aMJQBHAk1+P+DHrE5b8a2BpNDQ=; b=kaqh671O9YBKcdPwqpPfDtqhk3O6EAb+gXa6Y0fk0PQcOUbdZJAejMWh1QHCUC/0hg 0hF6/3T/FLwFQG+kbkHSkKgE4BkLEwM50DRNtisoy7AwsfEGQdHzwinnyT3djJPKqDXi c+aVwjj0qFdIboXsZxofxDgo0dK4dM9R4wDpkxx3r+V/xeBNJR7S+fcKJfK88hyy/BnN XfmLPd2U0yR2Oj7piiGDlw0zLUriaQe2fwPvFze86HDLpoGpDpJteqRVEx+Y/6X0r8R0 SRQGAnxtwW0cmniOqm93T6fQqoTyR7BN7vebTCGtup4p+3c/Q5ASUaoJRvWwK38iR6Xl M6AA== MIME-Version: 1.0 Received: by 10.112.87.194 with SMTP id ba2mr7184593lbb.84.1354697758343; Wed, 05 Dec 2012 00:55:58 -0800 (PST) Received: by 10.112.68.75 with HTTP; Wed, 5 Dec 2012 00:55:58 -0800 (PST) In-Reply-To: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> Date: Wed, 5 Dec 2012 11:55:58 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/alternative; boundary=f46d0401fdffaa053604d01726ef Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Dec 2012 08:56:01 -0000 --f46d0401fdffaa053604d01726ef Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David, Thanks for your help, nc -w1 did the trick. But shouldn't dtntunnel behavior be changed? The receiving dtntunnel could signal the sending dtntunnel, that the listening nc has disconnected, so that the sending dtntunnel close its connection to the sending nc. Or is this solely the problem of nc? 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < david.a.zoller@nasa.gov> > Hi Sergey,**** > > On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not > impact the behavior of dtntunnel.**** > > I just ran your test without the iptables redirect and I see the same > behavior.**** > > I believe that the sender exits if you go directly from nc to nc because > the receiver exits when it gets an end of file.**** > > You can add a =93-w 1=94 option to the sender nc so that it will timeout = and > exit after stdin is idle for 1 second.**** > > Best regards,**** > > DZ**** > > ** ** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > ** ** > > *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *O= n > Behalf Of *ssireskin@gmail.com > *Sent:* Tuesday, December 04, 2012 11:13 AM > *To:* dtn-users@irtf.org > *Subject:* [dtn-users] A problem with dtntunnel**** > > ** ** > > Hello all! > > I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a > problem when using netcat (nc) with dtntunnel. > On the sender node I run "cat /etc/passwd | nc receiver_ip". On the > receiver node I run "nc -l 9999". With the help > of Iptables port 9999 gets redirected to the port 19999, which is listene= d > by dtntunnel. > > The file /etc/passwd is successfully delivered to the receiver and is > shown on the screen. After this the receiving nc exists. > No problem here. However, nc on the sender node doesn't exit after it > sends the file. It continues to run forever. When I > run nc in the opposite direction, I get the same problem - the sending nc > doesn't exit. > > My configuration on both nodes is symmetric: > iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --t= o > $LOCAL_HOST:19999 > dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/n= c > -d > dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d > > dtnping works ok in both directions. > > Please give me any advice. > > With best regards, > Sergey Sireskin **** > --=20 Kindest Regards Sergey Sireskin FGUP CNII EISU --f46d0401fdffaa053604d01726ef Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David,

Thanks for your help, nc -w1 did the trick. But shouldn= 9;t dtntunnel behavior be changed? The receiving dtntunnel could signal the= sending dtntunnel, that the listening nc has disconnected, so that the sen= ding dtntunnel close its connection to the sending nc. Or is this solely th= e problem of nc?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC = SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

On RHEL 5.7, I am running dtn-2.9.0 plus modific= ations that should not impact the behavior of dtntunnel.

I just ran your test with= out the iptables redirect and I see the same behavior.=

I believe that the sender= exits if you go directly from nc to nc because the receiver exits when it = gets an end of file.

You can add a =93-w 1=94 = option to the sender nc so that it will timeout and exit after stdin is idl= e for 1 second.

Best regards,

DZ=

=A0<= /p>

David Zoller

COLSA Corporation

HOSC / C107=A0

(Office: (256)=A0544-1820
*EMail: david.= a.zoller@nasa.gov

=A0<= /p>

From: dtn-users-bounce= s@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Tuesday, December 04, 2012 11:13 AM
To: dtn-users@irtf.org
Su= bject: [dtn-users] A problem with dtntunnel

=A0

Hello all!

I am using dtn-2.9.0 on a RHEL 6 based Linux di= stro and I am having a problem when using netcat (nc) with dtntunnel.
On= the sender node I run "cat /etc/passwd | nc receiver_ip". On the= receiver node I run "nc -l 9999". With the help
of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel.

The file /etc/passwd is successfully delivered to the = receiver and is shown on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it s= ends the file. It continues to run forever. When I
run nc in the opposi= te direction, I get the same problem - the sending nc doesn't exit.

My configuration on both nodes is symmetric:
iptables -t nat -A OUTP= UT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to $LOCAL_HOST:19999
dt= ntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d=
dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d

dtnping works o= k in both directions.

Please give me any advice.

With best r= egards,
Sergey Sireskin




--
Kindest Regards

Sergey Siresk= in
FGUP CNII EISU

--f46d0401fdffaa053604d01726ef-- From david.a.zoller@nasa.gov Wed Dec 5 04:51:55 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 35E3121F8973 for ; Wed, 5 Dec 2012 04:51:55 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8jzBDYfxMUT2 for ; Wed, 5 Dec 2012 04:51:52 -0800 (PST) Received: from ndjsnpf02.ndc.nasa.gov (ndjsnpf02.ndc.nasa.gov [198.117.1.122]) by ietfa.amsl.com (Postfix) with ESMTP id ADC1F21F8932 for ; Wed, 5 Dec 2012 04:51:52 -0800 (PST) Received: from ndmsppt04.ndc.nasa.gov (ndmsppt04.ndc.nasa.gov [198.117.0.103]) by ndjsnpf02.ndc.nasa.gov (Postfix) with ESMTP id CE2B7A82BB; Wed, 5 Dec 2012 06:51:51 -0600 (CST) Received: from ndmshub06.ndc.nasa.gov (ndmshub06.ndc.nasa.gov [198.117.2.165]) by ndmsppt04.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qB5CpphG021013; Wed, 5 Dec 2012 06:51:51 -0600 Received: from NDMSSCC05.ndc.nasa.gov ([198.117.2.174]) by ndmshub06.ndc.nasa.gov ([198.117.2.165]) with mapi; Wed, 5 Dec 2012 06:51:51 -0600 From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "ssireskin@gmail.com" Date: Wed, 5 Dec 2012 06:51:49 -0600 Thread-Topic: [dtn-users] A problem with dtntunnel Thread-Index: Ac3SxlkfPNmOw0OXQNGSinMNnoJ5iQAHSDGg Message-ID: <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_04E3D99A62496240BCD6A576813E6E31E0BDBBA685NDMSSCC05ndcn_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8185, 1.0.431, 0.0.0000 definitions=2012-12-05_06:2012-12-05, 2012-12-05, 1970-01-01 signatures=0 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Dec 2012 12:51:55 -0000 --_000_04E3D99A62496240BCD6A576813E6E31E0BDBBA685NDMSSCC05ndcn_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Sergey, I would not even call this a problem. netcat is designed to be extremely fl= exible and by default the sender keeps its connection open as long as the o= ther end does. On the other hand the receiver by default terminates after r= eceipt of an EOF but there is a switch to keep listening if that is the des= ired behavior. With DTN, your sending nc may do its thing while the destination node is no= t even on line and then an hour later it becomes available and completes th= e transmission to the receiving nc. In this scenario, the sender would stil= l be "hung up" possibly indefinitely waiting for a terminating signal. I am not the DTN2 authority, but, I don't see a change to dtntunnel for thi= s as it would assume a specific usage that would probably break someone els= e's usage (like mine :)). Best regards, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gmail.com] Sent: Wednesday, December 05, 2012 2:56 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Thanks for your help, nc -w1 did the trick. But shouldn't dtntunnel behavio= r be changed? The receiving dtntunnel could signal the sending dtntunnel, t= hat the listening nc has disconnected, so that the sending dtntunnel close = its connection to the sending nc. Or is this solely the problem of nc? 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not impa= ct the behavior of dtntunnel. I just ran your test without the iptables redirect and I see the same behav= ior. I believe that the sender exits if you go directly from nc to nc because th= e receiver exits when it gets an end of file. You can add a "-w 1" option to the sender nc so that it will timeout and ex= it after stdin is idle for 1 second. Best regards, DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: dtn-users-bounces@irtf.org [mailto= :dtn-users-bounces@irtf.org] On Behalf O= f ssireskin@gmail.com Sent: Tuesday, December 04, 2012 11:13 AM To: dtn-users@irtf.org Subject: [dtn-users] A problem with dtntunnel Hello all! I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a probl= em when using netcat (nc) with dtntunnel. On the sender node I run "cat /etc/passwd | nc receiver_ip". On the receive= r node I run "nc -l 9999". With the help of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel. The file /etc/passwd is successfully delivered to the receiver and is shown= on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it sends= the file. It continues to run forever. When I run nc in the opposite direction, I get the same problem - the sending nc d= oesn't exit. My configuration on both nodes is symmetric: iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to = $LOCAL_HOST:19999 dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc = -d dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d dtnping works ok in both directions. Please give me any advice. With best regards, Sergey Sireskin -- Kindest Regards Sergey Sireskin FGUP CNII EISU --_000_04E3D99A62496240BCD6A576813E6E31E0BDBBA685NDMSSCC05ndcn_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Sergey= ,

I would not even call th= is a problem. netcat is designed to be extremely flexible and by default th= e sender keeps its connection open as long as the other end does. On the ot= her hand the receiver by default terminates after receipt of an EOF but the= re is a switch to keep listening if that is the desired behavior.

 

With DTN, your sending nc may do its thing while the = destination node is not even on line and then an hour later it becomes avai= lable and completes the transmission to the receiving nc. In this scenario,= the sender would still be “hung up” possibly indefinitely wait= ing for a terminating signal.

 

I am not the D= TN2 authority, but, I don’t see a change to dtntunnel for this as it = would assume a specific usage that would probably break someone else’= s usage (like mine J).

 

Best re= gards,

DZ

 

David Zoller

COLSA Corporation

Marshall Space Flight Center=

 

From: ssireskin@gmail.com [mailto:ssireskin@gmail.com]
Sent:= Wednesday, December 05, 2012 2:56 AM
To: Zoller, David A. (MSFC-= EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org
Subjec= t: Re: [dtn-users] A problem with dtntunnel

 

Hi David,

Thanks for your help, nc -w1 did the trick. B= ut shouldn't dtntunnel behavior be changed? The receiving dtntunnel could s= ignal the sending dtntunnel, that the listening nc has disconnected, so tha= t the sending dtntunnel close its connection to the sending nc. Or is this = solely the problem of nc?

2012/12/5= Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>=

Hi Sergey,

On RHEL 5.7, I am running dtn-2.9.0 plus modifications that shoul= d not impact the behavior of dtntunnel.

= I just ran your test without the iptables redirect and I see the same behav= ior.

I believe that the sender exits if = you go directly from nc to nc because the receiver exits when it gets an en= d of file.

You can add a “-w 1R= 21; option to the sender nc so that it will timeout and exit after stdin is= idle for 1 second.

Best regards,=

DZ

 

David Zoller

<= span style=3D'font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F49= 7D'>COLSA Corporation

HOSC / C107 

(Office: (256)&= nbsp;544-1820
*EMail: david.a.zoller@nasa.gov

 

From= : dtn-us= ers-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of <= a href=3D"mailto:ssireskin@gmail.com" target=3D"_blank">ssireskin@gmail.com=
Sent: Tuesday, December 04, 2012 11:13 AM
To: dtn-users@irtf.org<= br>Subject: [dtn-users] A problem with dtntunnel

 

Hello all!

I am using= dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a problem when us= ing netcat (nc) with dtntunnel.
On the sender node I run "cat /etc/= passwd | nc receiver_ip". On the receiver node I run "nc -l 9999&= quot;. With the help
of Iptables port 9999 gets redirected to the port = 19999, which is listened by dtntunnel.

The file /etc/passwd is succe= ssfully delivered to the receiver and is shown on the screen. After this th= e receiving nc exists.
No problem here. However, nc on the sender node = doesn't exit after it sends the file. It continues to run forever. When I <= br>run nc in the opposite direction, I get the same problem - the sending n= c doesn't exit.

My configuration on both nodes is symmetric:
ipta= bles -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to $LOC= AL_HOST:19999
dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_N= ODE/dtntunnel/nc -d
dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d=

dtnping works ok in both directions.

Please give me any advi= ce.

With best regards,
Sergey Sireskin



--
Kindest Regards

Sergey Sireskin
FGUP = CNII EISU

= --_000_04E3D99A62496240BCD6A576813E6E31E0BDBBA685NDMSSCC05ndcn_-- From david.a.zoller@nasa.gov Wed Dec 5 09:20:16 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3987B21F8C05 for ; Wed, 5 Dec 2012 09:20:16 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HI1WBU82Ehyf for ; Wed, 5 Dec 2012 09:20:14 -0800 (PST) Received: from ndmsnpf03.ndc.nasa.gov (ndmsnpf03.ndc.nasa.gov [198.117.0.123]) by ietfa.amsl.com (Postfix) with ESMTP id 2486A21F8C3C for ; Wed, 5 Dec 2012 09:20:13 -0800 (PST) Received: from ndmsppt03.ndc.nasa.gov (ndmsppt03.ndc.nasa.gov [198.117.0.102]) by ndmsnpf03.ndc.nasa.gov (Postfix) with ESMTP id AAA202D8540; Wed, 5 Dec 2012 11:20:11 -0600 (CST) Received: from ndmshub02.ndc.nasa.gov (ndmshub02-pub.ndc.nasa.gov [198.117.0.161]) by ndmsppt03.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qB5HKBqJ014630; Wed, 5 Dec 2012 11:20:11 -0600 Received: from NDMSSCC05.ndc.nasa.gov ([198.117.2.174]) by ndmshub02.ndc.nasa.gov ([198.117.2.161]) with mapi; Wed, 5 Dec 2012 11:20:11 -0600 From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "carlosnevesabrantes@gmail.com" , "dtn-users@irtf.org" Date: Wed, 5 Dec 2012 11:20:10 -0600 Thread-Topic: RE: Streaming Video with dtn2 and vlc Thread-Index: Ac3TDCp4MvHXoaQ7RgqFfFa6UQLLig== Message-ID: <04E3D99A62496240BCD6A576813E6E31E0BDBBA97B@NDMSSCC05.ndc.nasa.gov> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_04E3D99A62496240BCD6A576813E6E31E0BDBBA97BNDMSSCC05ndcn_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8185, 1.0.431, 0.0.0000 definitions=2012-12-05_08:2012-12-05, 2012-12-05, 1970-01-01 signatures=0 Subject: Re: [dtn-users] Streaming Video with dtn2 and vlc X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Dec 2012 17:20:16 -0000 --_000_04E3D99A62496240BCD6A576813E6E31E0BDBBA97BNDMSSCC05ndcn_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Carlos, It looks like your message only made it to the dtn-security mailing list an= d this might be better on the dtn-users list. I have been using dtntunnel to stream video through DTN2 but I have not tri= ed it with a file as the source. I can experiment if that is how you would = like to operate. I have a video stream transmitted to the front end of the tunnel where I ha= ve dtntunnel running with these options: dtntunnel -e 864000 -u -T 12345:xxx.xxx.xxx.xxx:54321 dtn:///d= tntunnel where: -e =3D expiration time for the bundles -u =3D use UDP -T 12345:127.0.0.1:54321 =3D defines the tunnel: 12345 is the local port receiving the video stream (you mig= ht be able to put ":" in front to tap into a stream) xxx.xxx.xxx.xxx needs to be changed to the IP where VLC is = running or possibly a multicast 54321 is the port that VLC is listening on On the tail end of the tunnel I have dtntunnel running with these options: dtntunnel -u -L where: -u =3D use UDP -L =3D listening mode --local_eid can be used to override the default which= is dtn:///dtntunnel This tail dtntunnel transmits the payload to the IP/port specified in the -= T option on the front end. VLC is configured to receive a stream: udp://@:54321 If you transmit directly to the VLC IP then VLC still has to be set up to l= isten on a multicast even if it is a dummy address. Best regards, DZ David Zoller COLSA Corporation Marshall Space Flight Center * From: Carlos Neves > * To: dtn-interest at irtf.org, dtn-s= ecurity at irtf.org, dtn-users at mailma= n.irtf.org * Date: Wed, 5 Dec 2012 10:00:09 -0500 * In-reply-to: > * References: > * List-id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Sec= urity." ________________________________ Streaming Video with dtn2 and vlc Our team is trying compare dtn2 and Ibr-dtn performance in video streaming = to write a paper about this technologies. I have troubles to send a stream video in dtn2, because dtnsend does not wo= rk as i wanted for the flag "f" (file). Changing apps/dtnrecv.c, and do the /configure , make , make install again = , now i can send videos from a computer to another. You have any solution to do stream of video or sending files without changi= ng dtnrecv.c original file? I'm using dtnd version 2.9.0 To receive the stream i'm using: receiver: dtnrecv dtn://carlos.dtn/r | vlc - It works for remote communications with other computers. You have a copy above with the modifications we have made. Respects, Carlos Neves Abrantes, Andr=E9 Carvalho Communication Networks Engineering Instituto Superior T=E9cnico - Portugal --_000_04E3D99A62496240BCD6A576813E6E31E0BDBBA97BNDMSSCC05ndcn_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi Carlos,<= /o:p>

It looks like your message only made it to th= e dtn-security mailing list and this might be better on the dtn-users list.=

 

I have been using dtntunnel to stream video through DTN2 but I have not = tried it with a file as the source. I can experiment if that is how you wou= ld like to operate.

 

I have a video stream transmitted to the front end of= the tunnel where I have dtntunnel running with these options:

dtntunnel -e 864000 -u -T 12345:xxx.xxx.xxx.xxx:5432= 1 dtn://<destination>/dtntunnel

wh= ere: -e <expiration>=A0 =3D expiration time for the bundles

-u=A0=A0 =A0=3D use UD= P

-T 12345:12= 7.0.0.1:54321=A0 =3D defines the tunnel:

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 1= 2345 is the local port receiving the video stream (you might be able to put= “<multicast IP>:” in front to tap into a stream)

=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0 xxx.xxx.xxx.xxx needs to be changed to the IP w= here VLC is running or possibly a multicast

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 54321 is the port that VLC is listening on

 

On the tail end of the tun= nel I have dtntunnel running with these options:

dtntunnel -u -L

where: -u =3D u= se UDP

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0 -L =3D listening mode

=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 --local_eid <eid> can be u= sed to override the default which is dtn://<local eid>/dtntunnel

 

T= his tail dtntunnel transmits the payload to the IP/port specified in the -T= option on the front end.

 

VLC is configured to receive a stream:=A0=A0 ud= p://@<multicast>:54321

If you tran= smit directly to the VLC IP then VLC still has to be set up to listen on a = multicast even if it is a dummy address.

 

Best regards,

DZ

 

 

David = Zoller

COLSA Corporation

Marshall Space Flight Center

 

 

=

Streaming Video with dtn2 and vlc

=

Our team is trying compare dtn2 and Ibr-dtn performanc= e in video streaming to write a paper about this technologies.
I have tr= oubles to send a stream video in dtn2, because dtnsend does not work as i w= anted for the flag "f" (file).
Changing apps/dtnrecv.c, and do= the /configure , make , make install again , now i can send videos from a = computer to another.
You have any solution to do stream of video or send= ing files without changing dtnrecv.c original file?
I'm using dtnd vers= ion 2.9.0

To receive the stream i'm using:
receiver: dtnrecv dtn:= //carlos.dtn/r | vlc -
It works for remote communications with other com= puters.


You have a copy above with the modifications we have mad= e.
<dtnrecv.c>

 = ;

Respects,=

Carlos Neves Abrantes, Andr=E9 Carvalho
Communicati= on Networks Engineering
Instituto Superior T=E9cnico - Portugal

 

 

= --_000_04E3D99A62496240BCD6A576813E6E31E0BDBBA97BNDMSSCC05ndcn_-- From ltartarini90@gmail.com Thu Dec 6 10:31:41 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5127B21F86C8 for ; Thu, 6 Dec 2012 10:31:41 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.298 X-Spam-Level: X-Spam-Status: No, score=-1.298 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MANGLED_MEDS=2.3, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QdLV3x7xdjPa for ; Thu, 6 Dec 2012 10:31:40 -0800 (PST) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by ietfa.amsl.com (Postfix) with ESMTP id 8174621F869F for ; Thu, 6 Dec 2012 10:31:40 -0800 (PST) Received: by mail-ee0-f54.google.com with SMTP id c13so4662528eek.13 for ; Thu, 06 Dec 2012 10:31:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=FtForc0bb4BKtF8lvbbNIkY+NMzFVhJkVKM879YZQZw=; b=X2/Z+rFEY+TJiDh4DjH18E4D3RcHNduwkYKAoyfps9CYBJeuhcQlO7u0o+a/T5tnWB 6TxKXsb8pQ3JYq9wM5YM+l9pNVIHG96BZoH4Sf8wALLQFoMTw8izQ88Xsne18dpwyPKB wKtlaR9M+ZNj6MBRu1BNu7MHRqQw5Hcbov3GD3rCcfBtltDlLLtsKmEgeSPodPAMls/o bi62dyGbSUGsoualx7BgHzO/a60jpio6q0GutbWHOSwkuOoIaTDk1HuEK3qLwLZpsjT8 TWJmIeOrp+x/r9idV4T0Hi3UDRnGiqSFr15dcru658WyOCdgULfcTporrGbi4lxBw4MH yvWA== MIME-Version: 1.0 Received: by 10.14.0.71 with SMTP id 47mr8134306eea.19.1354818699709; Thu, 06 Dec 2012 10:31:39 -0800 (PST) Received: by 10.223.192.66 with HTTP; Thu, 6 Dec 2012 10:31:39 -0800 (PST) Date: Thu, 6 Dec 2012 19:31:39 +0100 Message-ID: From: Luca Tartarini To: dtn-users@irtf.org Content-Type: multipart/alternative; boundary=047d7b66f32954cbb504d0334f30 Subject: [dtn-users] Use of metadata with DTNSEND and DTNRECV X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Dec 2012 18:31:41 -0000 --047d7b66f32954cbb504d0334f30 Content-Type: text/plain; charset=ISO-8859-1 Dear DTN2 users, I am a student of the University of Bologna and I am carrying out a thesis on the possible use of metadata for DTN routing. I have a problem in the use of metadata with the applications DTNSEND and DTNRECV, DTN2 2.9.0 on Debian. If I try to send a bundle with metadata from vm1.dtn to vm2.dtn launching this command on vm1.dtn: vm1: dtnsend -v -s dtn://vm1.dtn/me -d dtn://vm2.dtn -t m -p "testing3" -M 1 -S "meta" while on vm2.dtn dtnrecv is running (vm2: dtnrecv -v dtn://vm2.dtn) I do not receive anything on vm2.dtn, while in the dtn.log of vm1.dtn I can see the following two errors: [1354724317.364108 /dtn/bundle/protocol error] BundleProtocol::prepare_blocks: 8-prepare returned BP_FAIL on bundle 10 [1354724317.364128 /dtn/bundle/actions error] BundleActions::queue_bundle: prepare_blocks returned NULL on bundle 10 Note that: 1) without the -M option the bundle is received correctly (of course without metadata). 2) if the source and the destination are the same, i.e. if I launch: vm1: dtnsend -v -s dtn://vm1.dtn/me -d dtn://vm1.dtn -t m -p "testing3" -M 1 -S "meta" both the bundle and metadata are received correctly. Any help will be greatly appreciated. Thanks, Luca Tartarini. --047d7b66f32954cbb504d0334f30 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Dear DTN2 users, I am a student of the University of Bologna and I=20 am=A0carrying out a=A0thesis on the possible use of metadata for DTN=20 routing. I have a problem=A0in the use of metadata with the applications=20 DTNSEND and=A0DTNRECV, DTN2 2.9.0 on Debian.
If I try to send a=20 bundle with metadata from vm1.dtn to vm2.dtn=A0launching this command on=20 vm1.dtn:=A0
vm1: dtnsend -v -s dtn://vm1.dtn/me -d dtn://vm2.dtn -t=20 m -p "testing3" -M=A01 -S "meta"
while on vm2.dtn dt= nrecv=20 is running (vm2: dtnrecv -v dtn://vm2.dtn)
I do not=20 receive anything on vm2.dtn, while=A0in the dtn.log of vm1.dtn I=20 can see the following two errors:

[1354724317.364108=20 /dtn/bundle/protocol error]
BundleProtocol::prepare_blocks:=20 8-prepare returned BP_FAIL on bundle 10
[1354724317.364128=20 /dtn/bundle/actions error]
BundleActions::queue_bundle:
prepare_block= s returned NULL on bundle 10

Note that: =A0
1) without the -M opt= ion the bundle is received correctly (of=20 course=A0without metadata).
2) if the source and the destination are the= =20 same, i.e. if I launch:
vm1: dtnsend -v -s dtn://vm1.dtn/me -d dtn://vm1= .dtn=20 -t m -p "testing3" -M=A01 -S "meta"
both the bundle = and metadata are=20 received correctly.

Any help will be greatly appreciated.
=
Thanks, Luca Tartarini.
--047d7b66f32954cbb504d0334f30-- From ssireskin@gmail.com Thu Dec 6 22:59:13 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6901521F8774 for ; Thu, 6 Dec 2012 22:59:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.353 X-Spam-Level: X-Spam-Status: No, score=-2.353 tagged_above=-999 required=5 tests=[AWL=-1.170, BAYES_40=-0.185, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, WEIRD_PORT=0.001] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gOTyCV1+xav0 for ; Thu, 6 Dec 2012 22:59:13 -0800 (PST) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by ietfa.amsl.com (Postfix) with ESMTP id E80AB21F8767 for ; Thu, 6 Dec 2012 22:59:12 -0800 (PST) Received: by mail-pa0-f54.google.com with SMTP id bi5so259608pad.13 for ; Thu, 06 Dec 2012 22:59:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=g2uIaYet7G1W8Hy3oKdgMVEfGr3vW/aDhhr0SUPdOaM=; b=q71N8PE3EdHTGvjfAqDEom+ttf8faREXTKSqsT/7EaRJ/g+8HNFEMQKy3jz2I+Clbj Ut1aNHUcMt8j1ZPY4dLmYWkMcPjPO+ZYDgxbOlfnScD143ZXiLcD06/urUPm2b5ei8yv 0qC61D+hpPbVk7I0wqMKQvf3MeDBQjkPtLAlFwkjdAo9EU73LmTSCVRqddW/Qpga+d/Z P/8ECKLn/IzDnjFyDcJA1ZD2ydc9GNbT21xTiGyc/wHxDhoJ/zhihKZtgQk06ZO9oizC WxOwntmpp7jjLeatSRAaiBJ7YoZDH13TJ9i+htvWPxKZcJX1JFxR22/YQHWrwRCQoLn4 qzqA== MIME-Version: 1.0 Received: by 10.68.235.40 with SMTP id uj8mr12878434pbc.98.1354863552501; Thu, 06 Dec 2012 22:59:12 -0800 (PST) Received: by 10.68.18.172 with HTTP; Thu, 6 Dec 2012 22:59:12 -0800 (PST) Date: Fri, 7 Dec 2012 09:59:12 +0300 Message-ID: From: "ssireskin@gmail.com" To: dtn-users@irtf.org Content-Type: multipart/alternative; boundary=047d7b33d072c4186204d03dc0d0 Subject: [dtn-users] Unable to compile DTN2 with NORM convergence layer X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Dec 2012 06:59:13 -0000 --047d7b33d072c4186204d03dc0d0 Content-Type: text/plain; charset=ISO-8859-1 Hello all, I have trouble compiling DTN2 with NORM convergence layer using gcc-4.4.5 on a RHEL6-based Linux distro. norm-svnsnap.tgz 09-Aug-2012 protolib-svnsnap.tgz 03-Oct-2012 Did anyone manage to build DTN2 with NORM-CL? What am I doing wrong? make -w -C applib make[1]: Entering directory `/usr/src/dtn2/dtn-2.9.0/applib' make[1]: Nothing to be done for `all'. make[1]: Leaving directory `/usr/src/dtn2/dtn-2.9.0/applib' make -w -C servlib make[1]: Entering directory `/usr/src/dtn2/dtn-2.9.0/servlib' g++ -g -fno-inline -fPIC -DPIC -MMD -MP -MT "conv_layers/ConvergenceLayer.o conv_layers/ConvergenceLayer.E" -DHAVE_CONFIG_H -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I.. -I.. -I/opt/dtn2/include -I/opt/dtn2/include/oasys/ext -I../servlib -I/opt/dtn2/include -Wall -W -Wcast-align -c conv_layers/ConvergenceLayer.cc -o conv_layers/ConvergenceLayer.o In file included from conv_layers/NORMConvergenceLayer.h:36, from conv_layers/ConvergenceLayer.cc:29: /opt/dtn2/include/normApi.h:616: warning: type qualifiers ignored on function return type conv_layers/ConvergenceLayer.cc: In static member function 'static void dtn::ConvergenceLayer::init_clayers()': conv_layers/ConvergenceLayer.cc:67: error: cannot allocate an object of abstract type 'dtn::NORMConvergenceLayer' conv_layers/NORMConvergenceLayer.h:245: note: because the following virtual functions are pure within 'dtn::NORMConvergenceLayer': conv_layers/ConvergenceLayer.h:224: note: virtual dtn::CLInfo* dtn::ConvergenceLayer::new_link_params() make[1]: *** [conv_layers/ConvergenceLayer.o] Error 1 make[1]: Leaving directory `/usr/src/dtn2/dtn-2.9.0/servlib' make: *** [servlib] Error 2 Best regards, Sergey Sireskin --047d7b33d072c4186204d03dc0d0 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello all,

I have trouble compiling DTN2 with NORM convergence layer= using gcc-4.4.5 on a RHEL6-based Linux distro.
norm-svnsnap.tgz 09=
-Aug-2012
protolib-svnsnap.tgz 03-Oct-2012
Did anyone manage to= build DTN2 with NORM-CL? What am I doing wrong?

make -w -C applib
make[1]: Entering directory `/usr/src/dtn2/dtn-2.9= .0/applib'
make[1]: Nothing to be done for `all'.
make[1]: Le= aving directory `/usr/src/dtn2/dtn-2.9.0/applib'
make -w -C servlib<= br> make[1]: Entering directory `/usr/src/dtn2/dtn-2.9.0/servlib'
g++ -g= -fno-inline=A0 -fPIC -DPIC -MMD -MP -MT "conv_layers/ConvergenceLayer= .o conv_layers/ConvergenceLayer.E"=A0 -DHAVE_CONFIG_H -D_LARGEFILE_SOU= RCE -D_FILE_OFFSET_BITS=3D64 -I.. -I.. -I/opt/dtn2/include -I/opt/dtn2/incl= ude/oasys/ext -I../servlib -I/opt/dtn2/include=A0=A0 -Wall -W -Wcast-align= =A0 -c conv_layers/ConvergenceLayer.cc -o conv_layers/ConvergenceLayer.o In file included from conv_layers/NORMConvergenceLayer.h:36,
=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 from conv_layers/ConvergenceLayer.c= c:29:
/opt/dtn2/include/normApi.h:616: warning: type qualifiers ignored = on function return type
conv_layers/ConvergenceLayer.cc: In static membe= r function 'static void dtn::ConvergenceLayer::init_clayers()':
conv_layers/ConvergenceLayer.cc:67: error: cannot allocate an object of abs= tract type 'dtn::NORMConvergenceLayer'
conv_layers/NORMConvergen= ceLayer.h:245: note:=A0=A0 because the following virtual functions are pure= within 'dtn::NORMConvergenceLayer':
conv_layers/ConvergenceLayer.h:224: note:=A0=A0=A0=A0=A0=A0 virtual dtn::CL= Info* dtn::ConvergenceLayer::new_link_params()
make[1]: *** [conv_layers= /ConvergenceLayer.o] Error 1
make[1]: Leaving directory `/usr/src/dtn2/d= tn-2.9.0/servlib'
make: *** [servlib] Error 2

Best regards,
Sergey Sireskin

--047d7b33d072c4186204d03dc0d0-- From elwynd@folly.org.uk Fri Dec 7 02:15:31 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 48B4221F882D for ; Fri, 7 Dec 2012 02:15:31 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.358 X-Spam-Level: X-Spam-Status: No, score=-1.358 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, SARE_LWSHORTT=1.24, WEIRD_PORT=0.001] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qxJr1v3pLGbg for ; Fri, 7 Dec 2012 02:15:30 -0800 (PST) Received: from a.painless.aa.net.uk (a.painless.aa.net.uk [IPv6:2001:8b0:0:30::51bb:1e33]) by ietfa.amsl.com (Postfix) with ESMTP id 8A06621F88C1 for ; Fri, 7 Dec 2012 02:15:30 -0800 (PST) Received: from mightyatom.folly.org.uk ([81.187.254.250]) by a.painless.aa.net.uk with esmtp (Exim 4.77) (envelope-from ) id 1Tguxl-0007IF-Op; Fri, 07 Dec 2012 10:15:21 +0000 From: Elwyn Davies To: "ssireskin@gmail.com" In-Reply-To: References: Content-Type: text/plain Organization: Folly Consulting Date: Fri, 07 Dec 2012 10:15:22 +0000 Message-Id: <1354875323.11916.830.camel@mightyatom> Mime-Version: 1.0 X-Mailer: Evolution 2.26.3 Content-Transfer-Encoding: 7bit Cc: dtn-users@irtf.org Subject: Re: [dtn-users] Unable to compile DTN2 with NORM convergence layer X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Dec 2012 10:15:31 -0000 Hi. I can reproduce this problem. I suspect I know what is going on and will look into the problem later today. It may be a consequence of some changes that were made to allow persistent link names.. I *thought* it had been tested with all the convergwence layers, but clearly not. *>sigh<*. If you want to experiment in the short term, v2.8.0 compiles cleanly with NORM. Regards, Elwyn On Fri, 2012-12-07 at 09:59 +0300, ssireskin@gmail.com wrote: > Hello all, > > I have trouble compiling DTN2 with NORM convergence layer using > gcc-4.4.5 on a RHEL6-based Linux distro. > norm-svnsnap.tgz 09-Aug-2012 > protolib-svnsnap.tgz 03-Oct-2012 > Did anyone manage to build DTN2 with NORM-CL? What am I doing wrong? > > make -w -C applib > make[1]: Entering directory `/usr/src/dtn2/dtn-2.9.0/applib' > make[1]: Nothing to be done for `all'. > make[1]: Leaving directory `/usr/src/dtn2/dtn-2.9.0/applib' > make -w -C servlib > make[1]: Entering directory `/usr/src/dtn2/dtn-2.9.0/servlib' > g++ -g -fno-inline -fPIC -DPIC -MMD -MP -MT > "conv_layers/ConvergenceLayer.o conv_layers/ConvergenceLayer.E" > -DHAVE_CONFIG_H -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I.. -I.. > -I/opt/dtn2/include -I/opt/dtn2/include/oasys/ext -I../servlib > -I/opt/dtn2/include -Wall -W -Wcast-align -c > conv_layers/ConvergenceLayer.cc -o conv_layers/ConvergenceLayer.o > In file included from conv_layers/NORMConvergenceLayer.h:36, > from conv_layers/ConvergenceLayer.cc:29: > /opt/dtn2/include/normApi.h:616: warning: type qualifiers ignored on > function return type > conv_layers/ConvergenceLayer.cc: In static member function 'static > void dtn::ConvergenceLayer::init_clayers()': > conv_layers/ConvergenceLayer.cc:67: error: cannot allocate an object > of abstract type 'dtn::NORMConvergenceLayer' > conv_layers/NORMConvergenceLayer.h:245: note: because the following > virtual functions are pure within 'dtn::NORMConvergenceLayer': > conv_layers/ConvergenceLayer.h:224: note: virtual dtn::CLInfo* > dtn::ConvergenceLayer::new_link_params() > make[1]: *** [conv_layers/ConvergenceLayer.o] Error 1 > make[1]: Leaving directory `/usr/src/dtn2/dtn-2.9.0/servlib' > make: *** [servlib] Error 2 > > Best regards, > Sergey Sireskin > > _______________________________________________ > dtn-users mailing list > dtn-users@irtf.org > https://www.irtf.org/mailman/listinfo/dtn-users From TSELIKIS.Christos@haicorp.com Mon Dec 10 23:24:14 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2AE7021F8707 for ; Mon, 10 Dec 2012 23:24:14 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 3.116 X-Spam-Level: *** X-Spam-Status: No, score=3.116 tagged_above=-999 required=5 tests=[BAYES_40=-0.185, EXTRA_MPART_TYPE=1, HTML_MESSAGE=0.001, MANGLED_MEDS=2.3] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gBgPVb4m-KVy for ; Mon, 10 Dec 2012 23:24:13 -0800 (PST) Received: from HAIRDGESRV2.haicorp.com (hairdgesrv2.haicorp.com [212.205.103.5]) by ietfa.amsl.com (Postfix) with ESMTP id 4D4D221F860A for ; Mon, 10 Dec 2012 23:24:11 -0800 (PST) Received: from HAIHUB2010SRV1.hai.gr (10.0.190.170) by HAIRDGESRV2.haicorp.com (192.168.2.4) with Microsoft SMTP Server (TLS) id 8.3.83.0; Tue, 11 Dec 2012 09:28:32 +0200 Received: from HAIMAIL2010SRV1.hai.gr ([fe80::cad:6c41:1f0b:46a7]) by HAIHUB2010SRV1.hai.gr ([::1]) with mapi id 14.02.0318.004; Tue, 11 Dec 2012 09:23:51 +0200 From: TSELIKIS.Christos To: 'Luca Tartarini' , "dtn-users@irtf.org" Thread-Topic: [dtn-users] Use of metadata with DTNSEND and DTNRECV Thread-Index: AQHN09/pjzVVfJOKME6QVJy9wx81XZgTOLvA Date: Tue, 11 Dec 2012 07:23:50 +0000 Message-ID: References: In-Reply-To: Accept-Language: el-GR, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.0.242.139] Content-Type: multipart/related; boundary="_004_A1E9924741E2C74A9ABA10C2DDBAA117013BE7HAIMAIL2010SRV1ha_"; type="multipart/alternative" MIME-Version: 1.0 Subject: Re: [dtn-users] Use of metadata with DTNSEND and DTNRECV X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 07:24:14 -0000 --_004_A1E9924741E2C74A9ABA10C2DDBAA117013BE7HAIMAIL2010SRV1ha_ Content-Type: multipart/alternative; boundary="_000_A1E9924741E2C74A9ABA10C2DDBAA117013BE7HAIMAIL2010SRV1ha_" --_000_A1E9924741E2C74A9ABA10C2DDBAA117013BE7HAIMAIL2010SRV1ha_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Luca, may be is trivial, ok with dtn2.8 (without vm): root@dtnrouter:~/dtn/usr/bin# ./dtnrecv -v dtn://192.168.1.51.dtn/a dtnrecv (pid 8945) starting up opening connection to dtn router... opened connection to dtn router... calling parse_eid_string find registration succeeded, regid 0 register succeeded, regid 11 looping forever to receive bundles dtn_recv [dtn://192.168.1.51.dtn/a]... bundle spec at 0xBFFA3C04 0 extension blocks from [dtn://192.168.1.100.dtn/me]: transit time=3D0 ms 1 metadata blocks from [dtn://192.168.1.100.dtn/me]: transit time=3D0 ms Metadata Extension Block 0: type =3D 1 flags =3D 0 0000000 6d65 7461 | meta 14 payload bytes from [dtn://192.168.1.100.dtn/me]: transit time=3D0 ms 0000000 7465 7374 696e 6733 2d63 6169 6e69 | testing3 Freeing metadata block [0].data at 0x08060070 dtn_recv [dtn://192.168.1.51.dtn/a]... With very br, ctselikis Tselikis Christos Hellenic Aerospace Industry S.A. Electronics Engineering Department e-mail: Tselikis.Christos@haicorp.com PO Box 23, GR-32009 Schimatari, Greece Tel. : +30 22620 52561 Fax: +30 22620 52910 [untitled] From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Beh= alf Of Luca Tartarini Sent: Thursday, December 06, 2012 8:32 PM To: dtn-users@irtf.org Subject: [dtn-users] Use of metadata with DTNSEND and DTNRECV Dear DTN2 users, I am a student of the University of Bologna and I am carry= ing out a thesis on the possible use of metadata for DTN routing. I have a = problem in the use of metadata with the applications DTNSEND and DTNRECV, D= TN2 2.9.0 on Debian. If I try to send a bundle with metadata from vm1.dtn to vm2.dtn launching t= his command on vm1.dtn: vm1: dtnsend -v -s dtn://vm1.dtn/me -d dtn://vm2.dtn -t m -p "testing3" -M = 1 -S "meta" while on vm2.dtn dtnrecv is running (vm2: dtnrecv -v dtn://vm2.dtn) I do not receive anything on vm2.dtn, while in the dtn.log of vm1.dtn I can= see the following two errors: [1354724317.364108 /dtn/bundle/protocol error] BundleProtocol::prepare_blocks: 8-prepare returned BP_FAIL on bundle 10 [1354724317.364128 /dtn/bundle/actions error] BundleActions::queue_bundle: prepare_blocks returned NULL on bundle 10 Note that: 1) without the -M option the bundle is received correctly (of course withou= t metadata). 2) if the source and the destination are the same, i.e. if I launch: vm1: dtnsend -v -s dtn://vm1.dtn/me -d dtn://vm1.dtn -t m -p "testing3" -M = 1 -S "meta" both the bundle and metadata are received correctly. Any help will be greatly appreciated. Thanks, Luca Tartarini. --_000_A1E9924741E2C74A9ABA10C2DDBAA117013BE7HAIMAIL2010SRV1ha_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Luca,

may be is = trivial, ok with dtn2.8 (without vm):

root@dtnro= uter:~/dtn/usr/bin# ./dtnrecv -v dtn://192.168.1.51.dtn/a=

dtnrecv (p= id 8945) starting up

opening co= nnection to dtn router...

opened con= nection to dtn router...

calling pa= rse_eid_string

find regis= tration succeeded, regid 0

register s= ucceeded, regid 11

looping fo= rever to receive bundles

dtn_recv [= dtn://192.168.1.51.dtn/a]...

bundle spe= c at 0xBFFA3C04

 = ;

0 extensio= n blocks from [dtn://192.168.1.100.dtn/me]: transit time=3D0 ms<= /span>

 = ;

1 metadata= blocks from [dtn://192.168.1.100.dtn/me]: transit time=3D0 ms

Metadata E= xtension Block 0:

 &nbs= p;            &= nbsp; type =3D 1

 &nbs= p;            &= nbsp; flags =3D 0

0000000 6d= 65 7461           &n= bsp;            = ;       |  meta

 = ;

14 payload= bytes from [dtn://192.168.1.100.dtn/me]: transit time=3D0 ms

0000000 74= 65 7374 696e 6733 2d63 6169 6e69      |  test= ing3

 = ;

 = ;

Freeing me= tadata block [0].data at 0x08060070

dtn_recv [= dtn://192.168.1.51.dtn/a]...

 = ;

With very = br,

ctselikis<= o:p>

 = ;

 = ;

Tselikis Christos

Hellenic Aerospace Industry S.A.
Electronics Engineering Department 
e-mail: Tselikis.Christos@haicorp.com

PO Box 23, GR-32009 Schimatari,= Greece

Tel. : +30 22620 52561=
Fax: +30 22620 52910

3D"untitled"

 <= /p>

From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@= irtf.org] On Behalf Of Luca Tartarini
Sent: Thursday, December 06, 2012 8:32 PM
To: dtn-users@irtf.org
Subject: [dtn-users] Use of metadata with DTNSEND and DTNRECV

 

Dear DTN2 users, I am a student of the University of= Bologna and I am carrying out a thesis on the possible use of me= tadata for DTN routing. I have a problem in the use of metadata with t= he applications DTNSEND and DTNRECV, DTN2 2.9.0 on Debian.
If I try to send a bundle with metadata from vm1.dtn to vm2.dtn launch= ing this command on vm1.dtn: 

vm1: dtnsend -v -s dt= n://vm1.dtn/me -d dtn://vm2.dtn -t m -p "testing3" -M 1 -S &= quot;meta"
while on vm2.dtn dtnrecv is running (vm2: dtnrecv -v dtn://vm2.dtn)
I do not receive anything on vm2.dtn, while in the dtn.log of vm1.dtn = I can see the following two errors:

[1354724317.364108 /dtn/bundle/protocol error]
BundleProtocol::prepare_blocks: 8-prepare returned BP_FAIL on bundle 10
[1354724317.364128 /dtn/bundle/actions error]
BundleActions::queue_bundle:
prepare_blocks returned NULL on bundle 10

Note that:  
1) without the -M option the bundle is received correctly (of course w= ithout metadata).
2) if the source and the destination are the same, i.e. if I launch:
vm1: dtnsend -v -s dtn://vm1.dtn/me -d dtn://vm1.dtn -t m -p "testing3= " -M 1 -S "meta"
both the bundle and metadata are received correctly.


Any help will be greatly appreciated.

Thanks, Luca Tartarini.

--_000_A1E9924741E2C74A9ABA10C2DDBAA117013BE7HAIMAIL2010SRV1ha_-- --_004_A1E9924741E2C74A9ABA10C2DDBAA117013BE7HAIMAIL2010SRV1ha_ Content-Type: image/png; name="image001.png" Content-Description: image001.png Content-Disposition: inline; filename="image001.png"; size=10192; creation-date="Tue, 11 Dec 2012 07:23:50 GMT"; modification-date="Tue, 11 Dec 2012 07:23:50 GMT" Content-ID: Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAAAF8AAAA8CAIAAACPTvFWAAAAAXNSR0IArs4c6QAAJ4pJREFUeF7F fAmUJddZ3q296u1Lv6X79et9m95m0+yj1bI8smWDjUEYR44DDhACAYxPFodYgoNPDpCTnOQ4Jphg bI4BOyCwZS2WbFljabSMRqNRz/SsPT29b6/77UvtVfluvZ6eHi2gITIpPT31e6/q1r3f/f/vX0uM bdvkx3cwlstgfIsQFjdhtt2IcfHBoV/Qv7w/iOMSnrgyfW8eLr3qpoNedfOxfdDtvzDNMbcf9Bum ea+bRm7exXu/eXzmx4sOFswCHUzIQ2fbvTf/3lzDjZW4RHCbE70x47dg9JZ13/jiJlCa990Ok8O8 FfHN2/1/QGdr2m9G56Ztvo6ayzh4kbfZdu/0Gwt7W7zeIiwYB5e8aTQ6yNte/jZy+uOWnU2J3ZSU t+qFt+obMsVgOVDDW1B2l/FU660o3CRfWzemN/iH0NmSOPbHjk5z5W8jz29CqvkRK2UhO+b1pb2V O5q/bH3PXldDXPkO2N8EE6YDUnuTmLzpLv9U6IACGRerdYm7RZ7XKXBLTd60KJzIGNtkYfvP15dx Q1moIGxnMw+jdyJqKsgemu+EzjaYPK388coOQyyGQBC2awoVj82Xy7gMe0PUm1LW3Ooba9w0Z5sS cBOJuPRsOsh2dtu0ADcr1g046B1vHDcR9s23oDP4p0EHk7i+wXQlm0zRhIYSh+tJu3fKpiQ019jE 6CZpgCRufU//BjQgcmju9nMhIJ68Nt892W2OtInj29iyt0DjbdV7ZdEZjx1d18WAjkP3hMOB+W1O Dd9AVrHbeGG+jGWZhOGIyxGWYwi3JTVN4cBZdAUulTq8OziDnuJJlTfG9e8dyzFxBiwTAGEJdx0L YO2yHEsR9tCxt+5MRcdxsSceI2K2mDM+0PnfEMzNvXzP0ME9cNCFMQxg2VyKhwk2DuTjuJiT7TAO ix9Zl+d4zwukWoapA08KqUN4CBMF0Jtt8+WpAlaxtfVbygGpYRlct0npOMN26Pa4DiNLclNBLct1 bLp+QWCwD5ap2wCUEB4HR91OXNWc+VuP9xKdLZFh6Z55cmQx/JYx2aJjz13VdI1KBcezLE5pmmXP ZFGeglHfftArbU9cthC7frpbaRRwN9yR4wSsliOC9xN8UAgcHc/bNQYn4Cx8bjSKQZ+vScyWbQFK SZRwZ2yeJ5k3EfZ7hk4TmqbsNKGhG+IKvEcfnlDRd9u1DadhuUZAlj1No/qGl70p1qZMTBA53XPs Ol6m4wk/I4qyN2Bz9k0wqR8cEPxb33jCh3Ck+SsnUKQ29wRXWvjJtgKcgN1paBrmqUgK9SBcp1Sq RCLR6+jc8NTfM3SwBsoz0GbXtbAuT3SxYYZO10JFGI4M0V2iO/RlMJQK7BqplesVHNVqvVqvm5Y6 ffWs4+qmaRs4dBNvGMy23HK5gpGpAkIqqBp62+EQgZdEjpdln9/vDwRCwUBIUfyiKLa1duCbSCQW iUQCgaBPDPCEynG9Xg3IisjJ2CfMjGOaCAKjJvTb7D1gfq9YGYhQGvYYZ4uAvMDSdDxl0UjFMMs1 rVhV1xt65Y2Jk4atNfRaTW3UcNTrmqZbtq74eMfBYI6JA8hYDujbcVyeF4ELVBWkD+zoYUF1Gb0O I+DapmNYJt5tChklLllURFkJBYLhSCyGV0s8FokHfcGje490tLYn4imc2Khr0EggC4wQwnjoIAy+ odTvGToYsqlcEJnm8Lqua0ZRCWqF2nJufWFtYyFfXNwoLRUqyzW1kGyLOowBFbNdCyzatB1gz0ZD A6cCAnwBtQI6jk2Jt1pp4B3oeMBRjcMfxGaIITk271qOgd9wIbWCsFycAwTB7/QGjmNi7YzEixIr l1ersWC8q7P70KFDx449MDa8EzpYqdYCgfCb0flHyM5mBsAjsE3PqunFcZTkIA6iCJMCoTXXy0sb pbkTp77TMFdVrQJ9kXyMzw81cBjebBh1xzFMXAIrYkFMbConNmeYrOtwFBocJt4dyAg1S1g7Rcdj I3NTfCA1lkZxdCycSZUOdo0BBYNxXMrUDCJ+quqeIUMQ4Sg7OvfsHtm3Z8+e9kxHNBoLh6MiL1u2 g5O3u12bHHpLmkUjA9ZVK1VBlHiJ0mGjYYsCzAXFA0bCJfWqvTg18+rklZfmFiaLtdl4qyD6bEWR ZEVgWUi+DjoxLB3UYABL1dJNSA7nOizlF5NvNEBTokftrGOBfSywD12/JyyGTr+BBmHxHMczvC0G QPOabWAQEDtIhGNswbU4xgXLsFrdgAhLghgKhfp7+seHD338vl+S+bgsU3u//bhuVa5bVrr9t8g7 FB3KW5gIhNaqa6CJAKaJ3YXrsV6Zf/38icnLJ5bzZ4lQisRJIOrqTonlzKbRBbKbETXIWcOaGeAC u+baom1xJiRCZ/z+Fk2zqtUqqLpehW3RsPPYdjgAlOsdosh8OBgJBsOwYoQ1q2ZedzXIDsvwPMPZ JmlU9EZN54gk8z5BkNOJzK6x3UODo309/ZnQgKVHRDZCtd+zbB4Z3BysbFpDahNvjXfgsGOneIm6 poZZIZwhcHAwrZnVqZOvvwRCWd2Yr6pLrlAVFEOQNEYwlICELQCY19HZdOahSvBmQTEOpMZ0dc1W VQOQbeSrpgnOIoZBpw2WFwVwGYEOwCgF/SEIqqYa9XpDa6iWzUm+qAYyV+uNRsUyDTgQoiCLvBgO trQls62J7u7s0NDA7mQ0yxJJIAHe9XPN3CO2mto+in1TG73j/8XfAavbBLPnZdhv1SXlqrN2/tLp k2+8cHnqvOk2CG9IfjYQ5iUf7qlBdXBj8DT0CO+UkGwbtoiaf5bHO9QE0qFphqqqmkq5o1wlgkAU hfh8bMDnVxQFMsJzInUHbdfUTSBIlQsjwNNkAgtzmiBGOM7V9Kqp14MBpbOjoz2TjYbiQ/3jXZkd YSktkihDFAOs4AgKLzf3pwnNlnK9N+hQd1MlcgDDFhYKZ5578W/PTP5oeW06mQmDTFkeThgriAwn 8iznRVusDG8Ylh6yA0mhVAuTY1NbbYFSYNZMDYQC3YGYgBkTqTArsLIANEVcSL0bELPtNqoNtYHz TTg4iuTD2kqlcrng8k63pgkCx7a2JYcGe9vbM4okgeqHd4ynAu0iCVjUzfFBcDgiIxBrevFgmabI UBnadNTfRESeeN0SKyP2xbQsrEQwVXfhie//6dPPfqPcmA/GOCnIiTJeEivAYSEwo9T7dflYpAWm 1nNeqGNyXYyZeqNKGEokDGsJIqsogj+gyEBFYajXAv9Ft4AFaNgy4OIi3BAl0cdzMgStXKjCu4V/ 5FqhluDYQO++of5BEG29XgdwAwND2UiWrhqSDJ/TASoyxoT+BqQgC8OFeYA/oUUs9Ve3IsS3wnPL 6FR1V1Gw7vJLbzz27af+ZHF1IpJgAhHOJjqsMOyyacE68qIiB/wIaHz1Sh1Oqee2YL5gcI5Gf9hC BqAwksyKEsMLLmIdsKgosnW1BmChfKbh2AYoCUkW0XV4teGaBqmW1EK+YmhWPJYCCp3ZwfbkaLVM bINpibd2dw61+FoR2dbqDegyxyIetqGTDIHPWGg0qtFgUiQtrisBHxotszQoa3remPM2dDZ5+tbQ wX1UR0eoV9RnvvGt/3n8hUcDMSeR8oGPoQScKEiwKJIImoAjQrnBMmqVMs8iPpYlSYJSCIJE6Zlx Q2EfoIFnzAuIqzXTatiOAX8Qukg9KQc6xZtAR2NMnbcNwbHE9Vy1WjZSLZnx8T19vYOyIGuqXVhv tCU6OzsGYoFWjoRQ0oB59Mwx/uOwpGGQ4vLqxQuXTll2tad7ZFfPMZcEAQcFB04agSdBfVFM7GZ0 KEDviA7WALakYT5PRbEZKFiM4bK6Teo/OPn4H3/1v4YirOFUGNYQZLAGS3ERBGp/ECFYdYzNc1AQ 0taKIAC2yclksuFwGJLs9yuIvKrVou1oih+huo6NhSiFgrHcWtkvRVhGggk3VAG4FPP6xmqjUrJ3 7zw8NrqvO9svS8H5ueVapZHNZlMtiWQ4BU7RHEdmw/CfIXcA3jBrMpWM6tTime89/a2737d/duG8 bfh+4n2/ypME7AJkGaralFPsHPUkPUPWpOymmf/70MHPXjxjA6lmfGDB+yPri+Wpv/v2N1985TlQ hj8o6KYaDgc9uoFrB2elbph1y1Wxd0gZJFrwCiAyxBEMBhFVYqjOzqwkC+vrq+VKHjAh82KYsFp1 6FE62ruRU7W6RVxlI1fPrdbjoexg/56Dt93d1TFSLmpTV+Z5IqXT7fFYIhoMe345RA5ijTxRMyKx fIK4vD5dKK509SQKpcVnn3v8p3/6gbn5y9emNu488FA80IeToescC1CAgAs13ERnEyAPHRqYvEMt 1BM26oY3Gg2Y1SZ76aTkMMUnTv711//8q7pRBfenMwlwIZZdqdcaCCcbFdPV/EEST/hiLYo/wEfD vlBQwVA8XGWWrdWqUPhUKoHRypViqVSC5QKh+nwyjBRjy7U8L3OxjfXy9NSyX4ofPfyB/bfdFQ91 VMvW7MxaPlfNtvfAtaNpP6TKOBrpuwSEb8AmmaQiEAvsdGV24oUTP4DaDg71jg2P/OjF54aHh1uj qe8+cXyk5+7B3j2KGKTpFEBKFdnTj63cNxWffwgdOOAAhcqLZTUFB5vrymrZufalP/2Dp595orU1 KfsRRDD5fF431HK1ASlQfGw4piTTgVR7KJUMBAIKDDjdGccyDA27AXKG+OAjdgxhTsAfgmGqVVW4 MKAAzgnMXSzpFT6VzN51xwcOj9xNiDKfW786taTW7PHRfV2ZLswEDhf1ACAAPKnrJuV1xiw2FjYK M73tKYuUnjnxnWQq0pppfeZ7z3zg2ANXLl6bX1j59E9+5pWJieKKdWDvXelEO3CxqKFAUM5zuPNW Ar6JDrWnLvfwww+/jaG/Hmpjh5tJCVhkz69jvnf8u8/+6PuWpSNnwgvOzMzVal3dKJihEEm1yUPD 2ZHRzo7OaCiMq2q63iCOqKuuTwklk2kwcK1aR0JOFJVgIOY6gtYgpi6yblhvSHPTpcnXl8b733fv 7R/7mQ//QndybGm9cvrMxXrR7mofPLT3aDQEp45omimB9gXqH20UNkIh2WVUWIsTp743tzQ5vTQR axMYHolHI5Voee306WSyvSM7MDW1tGv4ECP6rl68lk5lkNNoVm88U7XdWjWpByGW56y9Ezq4Doh4 TgELOWrSs2o3vv5XX52auQQx2cgvr63nF5ZIOkMyWTK+O9PTk+zojIVi1Nc3raLlViAqIX87jE42 2znWPY7AcG5uDg5xKtk2P7sicCHX9s/PFE6fmtpYM8ZGbv/ET/3KBw8/2J4aWlosnHptsryhjw7d Njq0uwVeA2EqpQrkBSpoWvVaPa9IjN9HNuog6POqvTa3eKF/MLOamxVEl+XdJ596SpL9g4NjIh/s jo0sFwsLy+tD/WOry+ttqdZAMIB8DtI6eKPma7tD+G7QwQVQAZA5AAL1NIPaE6+88J0nH6s2qqpW g+sKne/sIWPjocEdic6umD9MXQ3dgKIVkIQTRVuRgyKfsQ0lFIoKfimXWy8USgjHdc1KJTqvXV05 NzHDkvCRAx+8/76f27PrrnSk78wrVy+dmzd1du+uo6ND4z45BIcQjjc8FJ8PDrRjWlWO1300HlB/ 8Pzfzi2fEfyqy9XL1bWRsX5Nb0BVfUrYdaTOrpGRtn3np6dF0GJVLRarY71joiwk40leEGmKgyYG KTqw7Tfy7tvQYWDsm5p1vZhNa0BUtlwbFt0XoNRTKhdgiWuN2h9+6Yt/+Z0/lqNw7httWSXWwu8/ NMSJDcxYUqCrBmwgbodQEA4gEnam6XP1rGUGfbKCDxgQeUxkqi5emMqtFUPB1OjwgcMH7s2k+wt5 Y35mtVIwOxJ9mWRXuiUlUl+EMotnOpHU0A29LCmYu75Umq83KrbeWNuYu7z4+kMf/WcqcR597NFk Oqzr9UQicXjorr/+/uPtme62dObZHxz/yAMPgjxL+UZXR3+YC2II2DjINWIemjiif0heYWOrDIkE CHWa4MDAuzXxAY4adWoNG8EiJ7LIrbEiZ1kwzIiUKbmvV1a/8J9/84kXn0x3kr7+UHt7ayAkcDyc fJPjDZbXHVhxxgBD4q4cvBc4da5g2X7XVkK+eL3s1iru1IWFteXijoFdIzv2jO7Y2ZrqKuTrb7x2 ybL5XWO3pdq6g4HUqVcnJ0+c+a1ff6ikESSRMOtSZT0TAlqlirH20hsvLldy+ULhUx/9xNzsxcsL p95/z4f8pO/k5Cszcy899KGPXivN1kpOIjZ44vhJtVFsTbbdf8+DHIk7dpA6xzQzbbKipZlFSYBo qID78ecfh33YtXOfwscsUwgIKceSDJVwn3/kP9JkrMtgofBhQd94BzMBMw5wOXAIMT3TIur04oWX zzyd6iy393AtyZik8J5DjKwk6naQTuSjkF2HVygSAveB9imB8KDhYK6Zq4uvvny2mNP6u3ffdfTD B/bctW/XndcuLZ59/XJpQ71t1+Hd4/u7W/sUMQRJXspVX3jh5M7dR2X/puz4JaWurhChdunaxEp+ tneoq1DKwQlIpEPr5ZVILBkUeqsNo1JdTHSGV5Zyat3d0XEwEk51ZtuGh0Z9fIsFKmf8LoQbEiGA clWOQw6tdvrq8xNTL9mkslFahiRlom0CB/mUGKzColExLSRspyWQNTxJAdlvRnd5AwOtVGe++cRX /+7Jv2BFuy3b0RJHHBxhXNk2RbyIg+F8PBvUGlwxb1ZKAM0v8nFUU9Q6f+bkxRM/fOP8mcUdfaMf Pvaxuw7fm23trpaMv/nmo47FHj185wMf/MhA52DMH0fya6NYhCbt3N0bzGam8lVWIpDdag7FC9ZW /A4J6gwiUutAZsdItn1m7kJEiTVq9uJsgSOB1pZuYod/9MOJpTlV5NICiXelR7szwxEp1UAuEkEa CyPaICKSj6VcabpsrJXdXC6/tJZbuu/I+2PxyOLiolfygDGCVdFkhXD/6ZEveBEH75WCqOOIDD8K CfAAaHWFUS8vnP3eD//2meOPreSmY2nZcKo4wYvcaMqOFmgp4TvFYhGp2XA4Fg7FEQfkN6oz15Zn rq5pNacr07tn56H33/2TrB2au5YvrKtIBh49fM/e8UPRUIvMBnPFfEAJGyYJBwJlnWAur15c2qjW xobaJ05tqOUKyjBExgQt3S1dPX92/+iwKDuvTLzaPdyDnJdWEfqzBwNiHEklZDl6OkeGBw4RgnBc aZhVgZOgGJBKEKFMaav88tlnTp89sZqfgdp2dLXNLVzr6+sLR2NTl6Yz2S5ZCBAH6VceFyLeQ/4D ORlah6KJF5cmCziR00kdYffktdPfePR/P/fS43JET3XKrlhDGtRxDRSeTCTLwVLgJ6+4AncZ5SQ4 5gvzyyCOiYn5arURjybvOHTs6IEHIv7OixNLkxPz8Wj3np13Hrvvp7rbxh0iIdZB/iURzSIjrqrU bZchL4T0Du9cXCvmiuTEyTPlmhmMQL39DUK64oPQjOnFK22hZCQs5AtrAz0japWfW1qH/Pa37jmy /9hI1+0SSWmaiNSFIMAnRl0NNzKuzL1hk0KZzC0Vz3fviHOKPj17HqEhkhmXF64mlTRcjaWlFZGg dAEHltZHUYberEp79W+Gh3qCTIkBs/Pd5/7ma3/1lbOXTzKyGo5zjlCFGyooNstBwFCN1sFOcMmQ eUKcAMGDZJ4+NX32bLFRI5m26K6xvbftPcIzLVMXi+cnVm0rfmD/h+498jOKr11kWi0SqBliucb8 /n/7o5dfP1dqOIjagcuFRXtDI3ccbNNU5/K0zsKN7O5AxRDeH+OKEIdEKntldrZOquGgtHztWjSQ bE/3Id+uQvaQNeVChPhVE3X0IGwQT0RUwxqkNjF96vTZ584vvqyT9UCc6e5vC8al9dJasZZPpdLF Qp0jvlSyfWluparXvBifEgyKGhQMqlDIcUCEkCohWlFff/71Z//mu395fuq0L8woYaaira9XlnWn rOtl06y6NqheY1yUW1CsKxULuYkzcwtzKjK346Oxe+/Zu2/vbeFAsJArFwvEL/ce+8Cnf/r+Xx0b uG9uQ/v6N56cXasBCITIqq1YbJiV4rKPzZXIs68sfPkbf/EXj76I2qBkcadfP+v4FD1I7Qosh89C Slhua+9dUxvHJ17hbSc/v8hY/G079/dku/xeOdyrwnKSoNDiHbFXyvMbjeWF5emLU68lMtxrZ59C ZaxuFp86/tji6kJ3Xy+ybV0dQ1cuLeSNxvDQXkPnalWzXK+BeUEX3O888ghgQvkMXIpAAYxT1jZW crNf/MOHc8X5wbFOTjbXC/P+iKT4GaTFUb5HEh05OoFV4NehbJBb21hd0uMxtq87PTYy3JrsQHyw vlKulk3XjhzY97Hb9tzfldhjk0CuZtXrwvMvTuzefyQQ8HqeJPbowd1SKKRZ5L9/6f+UdSve3XPx 8tXbhkZdU55eWJpdXuSU6FBv0A/NgWdl1dItiTpXWdtYGupsl4gPWYB4pAdWRjNrNldVqIMkoObX 0KDZudcmngNDgWJVvfj+g3svLZ0uqhuygiyiNjqyczA9NrOwnG3tyy2XbZ0f6dyD/MlA97DIKZLg R22Ve+QLj9DOH8QbAlduFJBM0NzK5x/+rM02UAXDoAxvyApfU8uGYSqCZDV4xvEJbBiZl+XF4tJ8 ySeFdgwODg2MJ+OdsB35XD2/ZoQD7YcPHPvQkU+lYnslLqsR/olnX336+69K/tZcoSEFE+3tMkWH IefnyLe/+8LAcOf5y+stbR3v/9Dg5ORSWEiMDSdfPnVu177dJ146IQotgx1BVGOQX7OImY4k5Qg3 nhw4PXEhGhtqiXVRT44zVGsV5RJU/TZKa6ZdOX7icd3O3bnnwEYjv15YuLT8kj/MRePJvs7hazOL HZl+zSCXLyyAxRnbH/KlJDG4o3enyPu9ejSCSplHqURSOIQFik9Esg7e0Te/9We5/KLkd/1hweUM +P4oXUZjIUnmTM2N+9qJLS3NrFUr6pEj9xw7dqxUKj7z/Se1QqigIqdBWuJdu/aP9feNJOWsQ6Iq CT11/MXLVxYXlsp33f3A6K7ktcWByUsLme7I2XMrmUxrNktqJrOwTnglvLZeCJHuVCRyeepCe/sh ENxQX3cmFf/qH385IfzCHQcTFhFM4ptZnnzxxKn1vqTDR5RAsoFiGNKrrFGtr51fmi4XSD5fOvbB ++JJhKN+mzRS6cQLL6509nNDO/oaddmw+LC/9cLEjGtJPiEuMeHhwVZTJQElgnoPTf97+WZaxhGQ F0GaQKGsjqTj5fmJx576ayQZNb1i2RoIFxlxxlaMuqSVBK3on56snXulmI0f/MPf+cv/8Itf3t3x kf7MXb/00O/W1yNxZfzg2Mc/cu+/unP059rkfTzpbJh+nsg6ahhyRNXseKItqJDuvr7l9eJzL87+ 4PmXn37ueWQ3a5pWQOpv327YPrVGBtrby2YFuYrWTGz24rljB0I//4lPmrJYIuT01VWXhPtTu7qT u/IVKd4+kk2NSlxI1RvwP5ZzV6/Nnxd8elVdrWtLyVa5WlsHjcbZRFsqm4imO4Se1YVKdZ3sHr6j JdgZUVp3Dh8QSchHwlElIRAfcZClRcMaAKIuMeUdlJ1gc4q1NVkkf/ZXfzQxeTIck1uSkXKlgFg4 6A+HAwkdRqJgOw2lM7n73/767336wV8J+tP1uuuT4hFfmnWVdKJz99ihkYG9YSkDq2EaAuMGZAHR F2nr6tg51jc3X7YcfrA/DPPz6mvT3b1dPf3di0uLY+M9F87Po0qwc2fs+DMnd/YPd3WEvvPM092D e1KRQD2/MdyXGRwIJ9Pycpn8+de+lYy29KYzg11DbV2IyEZ1Nygwgu42HK56afZVuNd37L1jNb/A 8mpXZ9up0ye7e3dwXAs8z3OTr+Yb5Y1luyXc058aaU13ZVLdiUAbqqa0C4uVvUCK1iuo4Hjda7SX 0ctUuFCc87NnfvjcU4ofAYFVLOckqHssydnK0nRpY97sS+//5E/92pd+/2v7xu7giD8gRKL+JJxP U+PigczuHQeziX6JhOFDcw7Scj6JZgdoqiQikLBAMonYzJWLuGk0gpyeE/AL998eFXgXFYXhwZ7V pVnYu1hYufD6RHuU7Bzq2Vid2T2e/PjH9ysijQ7Xa6Su4q5JXYNqgNEDDGnd0Plnnj6Tr8D8xVB9 FyQezeA8YYJRefraBYEwiViqkFNlkh7vvScdG12e0Xoye9KxPuTnFRINCS2AhkN3hguRQcRpIZiC UwD3Fq4XAg0W//H7ZfQ1+Hjpqe89VqisgX1QIajC20fnRKFWWK/Hg9kH3v+Jf/mpzz34oc8gy0wb ZTSUG2gZza8ICganLTS0BAEPGi4Czdp5ySUos4iqEu2UIX0dmVo5ZyLa5kh/T2b26iW6PXZj7tr0 6EiH3iiXNshtu0ZLxTU4xQ997P6ju/tRUkSKYWpefflM7sv/41uX3pgNBVLgr3ydlC3ajMA64dOn 5+YXdK+Yp8SSqUK5lCcb4+PDuq5uVHN9fUMXz80gEkYUev+RT37sg/9i7+gdbZFeXUMji0IcAckY 2puKeiRoHVNHAwmNFgGNgZ4GmhsDVDDnCxsLp069DAligJijt6Za0HNU2CgN9o785r/+95/75S+M 9x8iSPsjoydYMJOOUysV8ihHEldHggw+NM8hf6g21PVqea2hVoEQGvTgc6JBJ+ong70p16gtzSHC g7B0Vcs5CNaukYHpqQtBH4kExNWlxc7O1mMfvg8hUZIjaQ/WvGb8+Xe+8eiTf/fpn33wI3d3DXQM vH7uytlF8yvfOnHpas2qYPwWx5Fgs1wippMZEOXCwkIUAhqJrK2tp5OdjqUUVq3iBtCMx+RuhY1B XlD2QHcD+jR8aKzDPOhW0xIEoRE1XCYUNNGdZ7LITeSr6yjh/uiF48vLq+B+BO9owZq6sNEW7/+V X/jc7/72fzkwdif6SKplHTEGz0uNBkpyFUmWIvFwKBQwdIQAtEUL8gPXwKf4g5GQzw81ppVg2lpi 6Li1jG0w6/XKOjJG6Dwo5RbhUu8dHVifm8YzR4f3DPdmYuPdQjpGIjIJ8kSvoI+Caj0nigePHE0k aJDhD4YFXzDdLZw4Pfniaxf8ISKHIihb00Yu4sZIeySQmp6anVy6VC7XVxbXA2Lk3ns/0JYJxKLE 0CCIQdOg1hoeo6ki/QsfQfJqMzRipOVSXadladoihRACNReSF4PG8VeeeuYHT9dqDk9aGC1Vz0V+ 4n2/+Nlf/INPfeS3Yjzg50RGjIRoww5tdlVCsi+MlBpttSOwegGGRciPX33IyDkuSpc04UYLsaKA eBUl0XJZDfrJjqGeS+dOAan+rNLiFy+/fj7Ms5/66AOkXLx7T++urA+V3Rb0Jal1TB9mFBKRDvqj vsTc7LIviPWTvt3MNFzfFfLbD//yMydOv3rZ1kVYRLpAH4G9Dx0c/wDv+CZeO9vXPVytgE+FiBKq 1PMoYaC9EK09SEmj1gjqRczESSLKKsgRomRDnT4U79HHi1yMg3wOaqki9xuPfEYk/Fe+9pWzk+cU KWQZrMxHfvZn/vlDD36mr3tEYHygbDC57LVn0iasZtuslyfZ9mpm1bDZXkaveYJXOUMLE48aqIwM LpmZXbh2bXp81/6wn2yslXtQ6O0IpuKxtpgCL47SU2kdrRfoIqMlXlQRbSBMZhfKy6t5ImdfeWM1 ngiXK46m2SMjIctJnL98wdArY2N9kQDdfdRsgqwv3hIcHhpsT3fOzC2xgoIsjyz6eUa29GaPwfU0 6Pbpv3k59DfaKY047drK7JXL10zDPT95Odve9fnP//YnP/lQKp5GYZ8GLRyHcrjXTW+jMtNMs777 A01r+KcJZ2dHO9StWq4CwnvuuqO7E9xCfCL9CYQOAUB+lnZOcZA+igssBi7szLZXysUzp06uLMwa DdLRnro8+QbI6+4jXbnlObAacu8YEODS0h2s21rl0sWZyQtTPKf4fVF0yaHblBoAWITmPN71gRRY 4PRr544fP5HPlX7j33z2i7/3+4dvuyPABxs6WkAhmWinhw4yyDkhTUzrf7cyOg2kkLd3bcQ8uG6g t/fQgX0hv4JJpmJMxEezopBNKp6omTMu0pPeEwAcEkg438sfAdM0upcSsfCDH7ujq52kYyHYQrBn e4zcc/uhYnGlobqoXgMXhQsIxE9scWWpdOnCXGd2qLtlEO4o2g8BHVTlXcOyeSKLAP/xbz91eP/t n/31f/e53/r33W29mLFmGT4pgHZDrwOcoU2PaMWipu7m0s+7uBuSK+iuMTUVU4tHpLtvP5yK0lGg o17/K3IwtDEgoEBe0MNlUPTpv1QHcAn+SkZJLKj4BSYbx9JJS1BijMr8VBWCPb6jC1W1SIjBwr1e N/gS0tDg7oP77n7fnQ/sHDiI3aH5wOZoiOG3SnrvYubUG1xe2GiJpv/kf339l3/+1xyTzRWQhZTg E9FCFnLoBjJiJnoHFJ+PQkN7JG7hgB1AeAwm9/vQWYSeZTuAfoQmLdl0PXh5Xg8eVKJF92ZrBPaC eqs0W0PRwVS6s21Ls1dUxLIQpQwfC0rV/BIuNOpFnjGCMnWL8BNafuCOKUykMzmUTQwiJEYeGbGl wFBg0DXm3ewWEGIl3v87D38xk+hcWVv3C6F0LFNv1EHXm/Jyc2sUisW3gM3WqY7jR1KNNuIbSChS O40mIKSzMFMLthO9FjDHaCtwBVTNaO8HRQeJW0gEtgMnDPZ0VAtrSBMAhZhE7n/f0Z3DfVixyJiN KlKsukqzxkTkFdptSlDrUgyL1RESEEVkkPxgUeOnvW/v9MTpO6yKzaSz0WACch6PJJrPvaGqiwwq av4SWpQkH1zgWrUKzwWzhwTdKjq8JKn1Gi3rum7ArzTFhDYHwPNykSjW8FHwGEEHRtSgsGjYpd5d s6zmKWBbKlourEno5kFNiJB9OzszcR5Nf6ytHti7swXuEwgZ6KAFkwfNIQkLd8OnsCGQDlBXddWP RmkJOQ70zdyC7DAGbdm7ici958CuNz9tfzKn+SV90uzdHptPPl5//gwue/MxzuZ7szH8+jne1zCN 3jm0Bc976oQGap4dOHHiVEdHRyaTEijO3qPriIbQ1eXNBdqHzDMPKJtzxKAweZsNxNgPh6MNrt5P 6IS/ubH071kMQ9vwPSKg7ttbsKBPdW4dzQeY3/XQ21a+if/WU4ceDDeehb5+F+9rLJPiQuFrAtQ8 0K/rh/H3DlQvcTEOC13T3k5SdFA+2HpY+zo6FFlkzik6Ds0Q3yI6tC5M9dHbL/r0xZaH5328+ZmT pi/1jzg2vQCgsPmiC7/+940N2HIWKCXTe2+bT8CHeJauDq3y0CNkwCE7zZ7sphPTXEVTPKhreENC mo+eehJ1i/6IJ+HNBx3epGD/CBDecok38Ntg6t3xbfR/+yOtWwB57X80BoJPZOoqLU3Qh75oVrz5 yNkND2y77HuTaS7NU+br03ib/5fBOy71/wJiiAD5npnXkAAAAABJRU5ErkJggg== --_004_A1E9924741E2C74A9ABA10C2DDBAA117013BE7HAIMAIL2010SRV1ha_-- From sickmind@lavabit.com Tue Dec 11 03:40:10 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4F88121F881C for ; Tue, 11 Dec 2012 03:40:10 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.415 X-Spam-Level: X-Spam-Status: No, score=0.415 tagged_above=-999 required=5 tests=[BAYES_40=-0.185, J_CHICKENPOX_64=0.6] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pL73r5RyhZuU for ; Tue, 11 Dec 2012 03:40:09 -0800 (PST) Received: from karen.lavabit.com (karen.lavabit.com [72.249.41.33]) by ietfa.amsl.com (Postfix) with ESMTP id 93FB421F8824 for ; Tue, 11 Dec 2012 03:40:09 -0800 (PST) Received: from a.earth.lavabit.com (a.earth.lavabit.com [192.168.111.10]) by karen.lavabit.com (Postfix) with ESMTP id B4BCB11BB91 for ; Tue, 11 Dec 2012 05:40:08 -0600 (CST) Received: from localhost (mx10.linuxwizard.ru [83.68.37.138]) by lavabit.com with ESMTP id CCZA2QLPWDEE for ; Tue, 11 Dec 2012 05:40:08 -0600 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=lavabit; d=lavabit.com; b=UFA4F78SXAszo8qSqzplz9aJoLyQuC7N+CONFJUpfaQHsSWfBp+3RjJvKnvo3857DQ9tCcQcMpU3lP0jiQIFIkNr/NtQuVnIo1stitIe4Bc86HHFUsUuRIE36GH4DQxCUV7xUfVui37qmaPThYOcg9EqjejgKDoFoVcYgGIr6IM=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type:Content-Disposition:User-Agent; Date: Tue, 11 Dec 2012 11:37:16 +0000 From: sickmind@lavabit.com To: dtn-users@irtf.org Message-ID: <20121211113715.GA24009@kasumi> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Subject: [dtn-users] dtnd segfault X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 11:40:10 -0000 Hi, For some reason EthConvergenceLayer::init_link() is missing, which causes segmentation fault when trying to add ethernet link. The following patch fixes this issue. Tested against the latest mercurial tree. diff -r 52e2b403edd4 servlib/conv_layers/EthConvergenceLayer.cc --- a/servlib/conv_layers/EthConvergenceLayer.cc Mon Nov 19 11:58:18 2012 +0000 +++ b/servlib/conv_layers/EthConvergenceLayer.cc Tue Dec 11 11:04:28 2012 +0000 @@ -218,6 +218,30 @@ } //---------------------------------------------------------------------- +bool +EthConvergenceLayer::init_link(const LinkRef& link, int argc, const char* argv[]) +{ + ASSERT(link != NULL); + ASSERT(!link->isdeleted()); + ASSERT(link->cl_info() == NULL); + + log_debug("adding %s link %s", link->type_str(), link->nexthop()); + + Params* params = new Params(defaults_); + + const char* invalid; + if(!parse_params(params, argc, argv, &invalid)) { + log_err("error parsing link options: invalid option '%s'", invalid); + delete params; + return false; + } + + link->set_cl_info(params); + + return true; +} + +//---------------------------------------------------------------------- void EthConvergenceLayer::delete_link(const LinkRef& link) { diff -r 52e2b403edd4 servlib/conv_layers/EthConvergenceLayer.h --- a/servlib/conv_layers/EthConvergenceLayer.h Mon Nov 19 11:58:18 2012 +0000 +++ b/servlib/conv_layers/EthConvergenceLayer.h Tue Dec 11 11:04:28 2012 +0000 @@ -131,6 +131,11 @@ bool close_contact(const ContactRef& contact); /** + * Create any CL-specific components of the Link. + */ + bool init_link(const LinkRef& link, int argc, const char* argv[]); + + /** * Delete any CL-specific components of the Link. */ void delete_link(const LinkRef& link); From ssireskin@gmail.com Wed Dec 12 03:10:20 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 314C021F897F for ; Wed, 12 Dec 2012 03:10:20 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.326 X-Spam-Level: X-Spam-Status: No, score=-3.326 tagged_above=-999 required=5 tests=[AWL=0.272, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W9AOu7Zi-1U7 for ; Wed, 12 Dec 2012 03:10:18 -0800 (PST) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by ietfa.amsl.com (Postfix) with ESMTP id D2E6521F894F for ; Wed, 12 Dec 2012 03:10:18 -0800 (PST) Received: by mail-pb0-f54.google.com with SMTP id wz12so454527pbc.13 for ; Wed, 12 Dec 2012 03:10:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=UCnv54pFAWMQCe0Voj4KPvwkzpNYQdY0Ak4iNBIDSik=; b=W+Z7jsfTtUwXquIeQaKNiiD7KbQoMO7WqxEhVTnq9Xa2CtI8yzF71OOYzjHHX/TRwn J+X7sScKoJ+JSo08dflRhGnKBLKwbV2JP588nzM2U+/wfysQFyU1yHSN9LRokKAKmFa7 IdItI5kAmLQGikysy4uX/5vN/2HIUYaT3x/siXmAjzvCeu4K7J1ytXRxBclTSmi2CBme U4sRfQZtn3ZFV3i3qNkynIyN90+VadobUqJxTWttLM2Y6PV5bJ0PA+XJjlVXBkj05jWS dWWZ92iCAyAkcQVtHnTNjggxEap5/QH7OCmrer8X4JnC08KzdCjTuM+OwgfvTdBK+Olq fClw== MIME-Version: 1.0 Received: by 10.68.134.232 with SMTP id pn8mr1719056pbb.47.1355310618553; Wed, 12 Dec 2012 03:10:18 -0800 (PST) Received: by 10.68.18.172 with HTTP; Wed, 12 Dec 2012 03:10:18 -0800 (PST) In-Reply-To: References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> Date: Wed, 12 Dec 2012 14:10:18 +0300 Message-ID: From: "ssireskin@gmail.com" To: dtn-users@irtf.org Content-Type: multipart/alternative; boundary=047d7b10c861faace404d0a5d727 Subject: [dtn-users] Fwd: A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 11:10:20 -0000 --047d7b10c861faace404d0a5d727 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David, I have investigated that issue a little. I have found, that no matter whether netcat is run with or without the -w1 option, something strange happens. According to netstat -ntp on the sending node, dtntunnel remains in CLOSE_WAIT state after nc finishes sending data. I have run tcpdump on both sender and receiver, and it showed that final FIN from the receiver doesn't reach the sender. This looked like a bug in dtntunnel to me. I decided to re-check with iperf, and again, tcpdump showed the same problem. Here are the last lines of the tcpdump. Remember that in my setup all outgoing TCP packets with destination port 9999 are redirected with the help of IPTables to the local dtntunnel process, listening om port 19999. That is why I run tcpdump on loopback interface. Receiver (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999 15:05:02.496366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 123995:131097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr 11163718], length 7102 15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0 15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], seq 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724], length 0 15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length 0 15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], seq 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684], length 0 15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack 2, win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0 Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999 15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 0 15:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], seq 127448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887], length 2896 15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], seq 130344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887], length 752 15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131097, win 386, options [nop,nop,TS val 11172898 ecr 11172887], length 0 15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], seq 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], length 0 15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], length 0 I believe that the cause of why the sending netcat without -w1 option doesn't exit, is that it doesn't receive the final FIN from the dtntunnel. Can this issue be fixed with a small amount of work? Best regards, Sergey 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < david.a.zoller@nasa.gov> > Hi Sergey,**** > > I would not even call this a problem. netcat is designed to be extremely > flexible and by default the sender keeps its connection open as long as t= he > other end does. On the other hand the receiver by default terminates afte= r > receipt of an EOF but there is a switch to keep listening if that is the > desired behavior. **** > > ** ** > > With DTN, your sending nc may do its thing while the destination node is > not even on line and then an hour later it becomes available and complete= s > the transmission to the receiving nc. In this scenario, the sender would > still be =93hung up=94 possibly indefinitely waiting for a terminating si= gnal. > **** > > ** ** > > I am not the DTN2 authHi ority, but, I don=92t see a change to dtntunnel = for > this as it would assume a specific usage that would probably break someon= e > else=92s usage (like mine J).**** > > ** ** > > Best regards,**** > > DZ**** > > ** ** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > ** ** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 05, 2012 2:56 AM > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > ** ** > > Hi David, > > Thanks for your help, nc -w1 did the trick. But shouldn't dtntunnel > behavior be changed? The receiving dtntunnel could signal the sending > dtntunnel, that the listening nc has disconnected, so that the sending > dtntunnel close its connection to the sending nc. Or is this solely the > problem of nc?**** > > 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not > impact the behavior of dtntunnel.**** > > I just ran your test without the iptables redirect and I see the same > behavior.**** > > I believe that the sender exits if you go directly from nc to nc because > the receiver exits when it gets an end of file.**** > > You can add a =93-w 1=94 option to the sender nc so that it will timeout = and > exit after stdin is idle for 1 second.**** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *O= n > Behalf Of *ssireskin@gmail.com > *Sent:* Tuesday, December 04, 2012 11:13 AM > *To:* dtn-users@irtf.org > *Subject:* [dtn-users] A problem with dtntunnel**** > > **** > > Hello all! > > I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a > problem when using netcat (nc) with dtntunnel. > On the sender node I run "cat /etc/passwd | nc receiver_ip". On the > receiver node I run "nc -l 9999". With the help > of Iptables port 9999 gets redirected to the port 19999, which is listene= d > by dtntunnel. > > The file /etc/passwd is successfully delivered to the receiver and is > shown on the screen. After this the receiving nc exists. > No problem here. However, nc on the sender node doesn't exit after it > sends the file. It continues to run forever. When I > run nc in the opposite direction, I get the same problem - the sending nc > doesn't exit. > > My configuration on both nodes is symmetric: > iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --t= o > $LOCAL_HOST:19999 > dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/n= c > -d > dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d > > dtnping works ok in both directions. > > Please give me any advice. > > With best regards, > Sergey Sireskin **** > > --047d7b10c861faace404d0a5d727 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Hi David,

I have investigated that issue = a little. I have found, that no matter whether netcat is run with or withou= t the -w1 option, something
strange happens. According to netstat -ntp = on the sending node, dtntunnel remains in CLOSE_WAIT state after nc finishe= s sending
data. I have run tcpdump on both sender and receiver, and it showed that fi= nal FIN from the receiver doesn't reach the sender.
This looked lik= e a bug in dtntunnel to me. I decided to re-check with iperf, and again, tc= pdump showed the same problem.

Here are the last lines of the tcpdump. Remember that in my setup all o= utgoing TCP packets with destination=20 port 9999 are redirected
with the help of IPTables to the local dtntunn= el process, listening om=20 port 19999. That is why I run tcpdump on loopback interface.

Receive= r (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.49= 6366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 123995:131= 097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr 11163718], length= 7102
15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack = 131097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0<= br>15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], = seq 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724],= length 0
15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack = 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length 0<= br>15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], = seq 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684]= , length 0
15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack = 2, win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0
Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack = 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 015:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], s= eq 127448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172= 887], length 2896
15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], se= q 130344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 111728= 87], length 752
15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.433= 60: Flags [.], ack 131097, win 386, options [nop,nop,TS val 11172898 ecr 11= 172887], length 0
15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], se= q 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], le= ngth 0
15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags= [.], ack 131098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], = length 0

I believe that the cause of why the sending netcat without -w1 option d= oesn't exit, is that it doesn't receive the final FIN from the dtnt= unnel.
Can this issue be fixed with a small amount of work?

Best regards,
Sergey

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <davi= d.a.zoller@nasa.gov>

Hi Sergey,

I would not even call this a problem. netcat is = designed to be extremely flexible and by default the sender keeps its conne= ction open as long as the other end does. On the other hand the receiver by= default terminates after receipt of an EOF but there is a switch to keep l= istening if that is the desired behavior.

=A0<= /p>

With DTN, your sending= nc may do its thing while the destination node is not even on line and the= n an hour later it becomes available and completes the transmission to the = receiving nc. In this scenario, the sender would still be =93hung up=94 pos= sibly indefinitely waiting for a terminating signal. <= /p>

=A0<= /p>

I am not the DTN= 2 authHi ority, but, I don=92t see a change to dtntunnel for this as it wou= ld assume a specific usage that would probably break someone else=92s usage= (like mine J).

=A0

Best regards,<= /u>

DZ

=A0

David Zoller

COLSA Corporation

Marshall Space Flight= Center

=A0<= /p>

From: ssireskin@gmail.com= [mailto:ssireskin= @gmail.com]
Sent: Wednesday, December 05, 2012 2:56 AM
To: Zoller, Dav= id A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org
Subject:<= /b> Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,

Thanks for your help, nc = -w1 did the trick. But shouldn't dtntunnel behavior be changed? The rec= eiving dtntunnel could signal the sending dtntunnel, that the listening nc = has disconnected, so that the sending dtntunnel close its connection to the= sending nc. Or is this solely the problem of nc?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SER= VICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

On RHEL 5.7, I am running dtn-2.9.0 pl= us modifications that should not impact the behavior of dtntunnel.

I just ran your test with= out the iptables redirect and I see the same behavior.=

I believe that the sender= exits if you go directly from nc to nc because the receiver exits when it = gets an end of file.

You can add a =93-w 1=94 = option to the sender nc so that it will timeout and exit after stdin is idl= e for 1 second.

Best regards,

DZ=

=A0<= /p>

David Zoller

COLSA Corporation=

HOSC / C107= =A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: dtn-users-bounce= s@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Tuesday, December 04, 2012 11:13 AM
To: dtn-users@irtf.org
Su= bject: [dtn-users] A problem with dtntunnel

=A0

Hel= lo all!

I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am= having a problem when using netcat (nc) with dtntunnel.
On the sender n= ode I run "cat /etc/passwd | nc receiver_ip". On the receiver nod= e I run "nc -l 9999". With the help
of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel.

The file /etc/passwd is successfully delivered to the = receiver and is shown on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it s= ends the file. It continues to run forever. When I
run nc in the opposi= te direction, I get the same problem - the sending nc doesn't exit.

My configuration on both nodes is symmetric:
iptables -t nat -A OUTP= UT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to $LOCAL_HOST:19999
dt= ntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d=
dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d

dtnping works o= k in both directions.

Please give me any advice.

With best r= egards,
Sergey Sireskin



--047d7b10c861faace404d0a5d727-- From ssireskin@gmail.com Wed Dec 12 03:14:04 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 27AFA21F8980 for ; Wed, 12 Dec 2012 03:14:04 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.372 X-Spam-Level: X-Spam-Status: No, score=-3.372 tagged_above=-999 required=5 tests=[AWL=0.226, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ofhi80PUcnvF for ; Wed, 12 Dec 2012 03:14:02 -0800 (PST) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by ietfa.amsl.com (Postfix) with ESMTP id 7129821F897B for ; Wed, 12 Dec 2012 03:14:02 -0800 (PST) Received: by mail-pa0-f54.google.com with SMTP id bi5so488051pad.13 for ; Wed, 12 Dec 2012 03:14:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=lEMKBI16N+whQI9wlDzUEyAcAV3Icv57ZMGDppBvFu4=; b=J+jU6bJtgcynI2bXYxSluL0c9uOT68bnnqfxqeXSYUun3hARYXEN3y17qs6kip7YnD DGxL6Hw9WlpDXus+RZa6AVvMhGFQiNpCM+mV511X4RzRuzAwf54xxZ2ZkfuerdBV/Qxl ZYGghQD139HE6OJ7CKs7cLGod67r0jmPbLypnclEz50Ftka7EGNF0e17TUxsviJwpEqv JYQM3pDkedfgQ/RtIVBwssJF+/3s2OQYQ34k4YUTg9tRXdXR4CSejz3UwZP40qrVid98 FSk5qXoYGCv0tFlO7ftW9xtZ49buYtsppNH74D6FrRbJGq9Z3pYgOz/+S/3ivFGmPv0y o1Sg== MIME-Version: 1.0 Received: by 10.68.190.227 with SMTP id gt3mr1795233pbc.5.1355310842139; Wed, 12 Dec 2012 03:14:02 -0800 (PST) Received: by 10.68.18.172 with HTTP; Wed, 12 Dec 2012 03:14:02 -0800 (PST) In-Reply-To: References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> Date: Wed, 12 Dec 2012 14:14:02 +0300 Message-ID: From: "ssireskin@gmail.com" To: dtn-users@irtf.org Content-Type: multipart/alternative; boundary=e89a8ff1bf664e549a04d0a5e5f0 Subject: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 11:14:04 -0000 --e89a8ff1bf664e549a04d0a5e5f0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David, I have investigated that issue a little. I have found, that no matter whether netcat is run with or without the -w1 option, something strange happens. According to netstat -ntp on the sending node, dtntunnel remains in CLOSE_WAIT state after nc finishes sending data. I have run tcpdump on both sender and receiver, and it showed that final FIN from the receiver doesn't reach the sender. This looked like a bug in dtntunnel to me. I decided to re-check with iperf, and again, tcpdump showed the same problem. Here are the last lines of the tcpdump. Remember that in my setup all outgoing TCP packets with destination port 9999 are redirected with the help of IPTables to the local dtntunnel process, listening om port 19999. That is why I run tcpdump on loopback interface. Receiver (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999 15:05:02.496366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 123995:131097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr 11163718], length 7102 15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0 15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], seq 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724], length 0 15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length 0 15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], seq 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684], length 0 15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack 2, win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0 Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999 15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 0 15:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], seq 127448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887], length 2896 15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], seq 130344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887], length 752 15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131097, win 386, options [nop,nop,TS val 11172898 ecr 11172887], length 0 15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], seq 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], length 0 15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], length 0 I believe that the cause of why the sending netcat without -w1 option doesn't exit, is that it doesn't receive the final FIN from the dtntunnel. Can this issue be fixed with a small amount of work? Best regards, Sergey 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < david.a.zoller@nasa.gov> > Hi Sergey,**** > > I would not even call this a problem. netcat is designed to be extremely > flexible and by default the sender keeps its connection open as long as t= he > other end does. On the other hand the receiver by default terminates afte= r > receipt of an EOF but there is a switch to keep listening if that is the > desired behavior. **** > > ** ** > > With DTN, your sending nc may do its thing while the destination node is > not even on line and then an hour later it becomes available and complete= s > the transmission to the receiving nc. In this scenario, the sender would > still be =93hung up=94 possibly indefinitely waiting for a terminating si= gnal. > **** > > ** ** > > I am not the DTN2 authHi ority, but, I don=92t see a change to dtntunnel = for > this as it would assume a specific usage that would probably break someon= e > else=92s usage (like mine J).**** > > ** ** > > Best regards,**** > > DZ**** > > ** ** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > ** ** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 05, 2012 2:56 AM > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > ** ** > > Hi David, > > Thanks for your help, nc -w1 did the trick. But shouldn't dtntunnel > behavior be changed? The receiving dtntunnel could signal the sending > dtntunnel, that the listening nc has disconnected, so that the sending > dtntunnel close its connection to the sending nc. Or is this solely the > problem of nc?**** > > 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not > impact the behavior of dtntunnel.**** > > I just ran your test without the iptables redirect and I see the same > behavior.**** > > I believe that the sender exits if you go directly from nc to nc because > the receiver exits when it gets an end of file.**** > > You can add a =93-w 1=94 option to the sender nc so that it will timeout = and > exit after stdin is idle for 1 second.**** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *O= n > Behalf Of *ssireskin@gmail.com > *Sent:* Tuesday, December 04, 2012 11:13 AM > *To:* dtn-users@irtf.org > *Subject:* [dtn-users] A problem with dtntunnel**** > > **** > > Hello all! > > I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a > problem when using netcat (nc) with dtntunnel. > On the sender node I run "cat /etc/passwd | nc receiver_ip". On the > receiver node I run "nc -l 9999". With the help > of Iptables port 9999 gets redirected to the port 19999, which is listene= d > by dtntunnel. > > The file /etc/passwd is successfully delivered to the receiver and is > shown on the screen. After this the receiving nc exists. > No problem here. However, nc on the sender node doesn't exit after it > sends the file. It continues to run forever. When I > run nc in the opposite direction, I get the same problem - the sending nc > doesn't exit. > > My configuration on both nodes is symmetric: > iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --t= o > $LOCAL_HOST:19999 > dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/n= c > -d > dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d > > dtnping works ok in both directions. > > Please give me any advice. > > With best regards, > Sergey Sireskin **** > > --=20 Best regards, Sergey Sireskin --e89a8ff1bf664e549a04d0a5e5f0 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Hi = David,

I have investigated that issue a little. I have found, that n= o matter whether netcat is run with or without the -w1 option, something strange happens. According to netstat -ntp on the sending node, dtntunnel= remains in CLOSE_WAIT state after nc finishes sending
data. I have run tcpdump on both sender and receiver, and it showed that fi= nal FIN from the receiver doesn't reach the sender.
This looked lik= e a bug in dtntunnel to me. I decided to re-check with iperf, and again, tc= pdump showed the same problem.

Here are the last lines of the tcpdump. Remember that in my setup all o= utgoing TCP packets with destination=20 port 9999 are redirected
with the help of IPTables to the local dtntunn= el process, listening om=20 port 19999. That is why I run tcpdump on loopback interface.

Receive= r (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.49= 6366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 123995:131= 097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr 11163718], length= 7102
15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack = 131097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0<= br>15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], = seq 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724],= length 0
15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack = 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length 0<= br>15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], = seq 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684]= , length 0
15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack = 2, win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0
Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack = 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 015:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], s= eq 127448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172= 887], length 2896
15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], se= q 130344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 111728= 87], length 752
15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.433= 60: Flags [.], ack 131097, win 386, options [nop,nop,TS val 11172898 ecr 11= 172887], length 0
15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], se= q 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], le= ngth 0
15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags= [.], ack 131098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], = length 0

I believe that the cause of why the sending netcat without -w1 option d= oesn't exit, is that it doesn't receive the final FIN from the dtnt= unnel.
Can this issue be fixed with a small amount of work?

Best regards,
Sergey


2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <davi= d.a.zoller@nasa.gov>

Hi Sergey,

I would not even call this a problem. netcat is = designed to be extremely flexible and by default the sender keeps its conne= ction open as long as the other end does. On the other hand the receiver by= default terminates after receipt of an EOF but there is a switch to keep l= istening if that is the desired behavior.

=A0<= /p>

With DTN, your sending= nc may do its thing while the destination node is not even on line and the= n an hour later it becomes available and completes the transmission to the = receiving nc. In this scenario, the sender would still be =93hung up=94 pos= sibly indefinitely waiting for a terminating signal. <= /p>

=A0<= /p>

I am not the DTN= 2 authHi ority, but, I don=92t see a change to dtntunnel for this as it wou= ld assume a specific usage that would probably break someone else=92s usage= (like mine J).

=A0

Best regards,<= /u>

DZ

=A0

David Zoller

COLSA Corporation

Marshall Space Flight= Center

=A0<= /p>

From: ssireskin@gmail.com= [mailto:ssireskin= @gmail.com]
Sent: Wednesday, December 05, 2012 2:56 AM
To: Zoller, Dav= id A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org
Subject:<= /b> Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,

Thanks for your help, nc = -w1 did the trick. But shouldn't dtntunnel behavior be changed? The rec= eiving dtntunnel could signal the sending dtntunnel, that the listening nc = has disconnected, so that the sending dtntunnel close its connection to the= sending nc. Or is this solely the problem of nc?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SER= VICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

On RHEL 5.7, I am running dtn-2.9.0 pl= us modifications that should not impact the behavior of dtntunnel.

I just ran your test with= out the iptables redirect and I see the same behavior.=

I believe that the sender= exits if you go directly from nc to nc because the receiver exits when it = gets an end of file.

You can add a =93-w 1=94 = option to the sender nc so that it will timeout and exit after stdin is idl= e for 1 second.

Best regards,

DZ=

=A0<= /p>

David Zoller

COLSA Corporation=

HOSC / C107= =A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: dtn-users-bounce= s@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Tuesday, December 04, 2012 11:13 AM
To: dtn-users@irtf.org
Su= bject: [dtn-users] A problem with dtntunnel

=A0

Hel= lo all!

I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am= having a problem when using netcat (nc) with dtntunnel.
On the sender n= ode I run "cat /etc/passwd | nc receiver_ip". On the receiver nod= e I run "nc -l 9999". With the help
of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel.

The file /etc/passwd is successfully delivered to the = receiver and is shown on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it s= ends the file. It continues to run forever. When I
run nc in the opposi= te direction, I get the same problem - the sending nc doesn't exit.

My configuration on both nodes is symmetric:
iptables -t nat -A OUTP= UT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to $LOCAL_HOST:19999
dt= ntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d=
dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d

dtnping works o= k in both directions.

Please give me any advice.

With best r= egards,
Sergey Sireskin






--
Best regards,
Sergey Sireskin
=

--e89a8ff1bf664e549a04d0a5e5f0-- From david.a.zoller@nasa.gov Wed Dec 12 07:31:09 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F19C921E804A for ; Wed, 12 Dec 2012 07:31:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i8z+3UN9nLJh for ; Wed, 12 Dec 2012 07:31:03 -0800 (PST) Received: from ndmsnpf01.ndc.nasa.gov (ndmsnpf01.ndc.nasa.gov [198.117.0.121]) by ietfa.amsl.com (Postfix) with ESMTP id 42DF621E80A3 for ; Wed, 12 Dec 2012 07:30:42 -0800 (PST) Received: from ndmsppt02.ndc.nasa.gov (ndmsppt02.ndc.nasa.gov [198.117.0.101]) by ndmsnpf01.ndc.nasa.gov (Postfix) with ESMTP id 1A14C2606A8; Wed, 12 Dec 2012 09:30:31 -0600 (CST) Received: from ndmshub06.ndc.nasa.gov (ndmshub06.ndc.nasa.gov [198.117.2.165]) by ndmsppt02.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBCFUUtm020717; Wed, 12 Dec 2012 09:30:30 -0600 Received: from NDMSSCC05.ndc.nasa.gov ([198.117.2.174]) by ndmshub06.ndc.nasa.gov ([198.117.2.165]) with mapi; Wed, 12 Dec 2012 09:30:30 -0600 From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "ssireskin@gmail.com" , "dtn-users@irtf.org" Date: Wed, 12 Dec 2012 09:30:29 -0600 Thread-Topic: [dtn-users] A problem with dtntunnel Thread-Index: Ac3YVLcoQqtPNWquR9O+gQw5+4BGUQAIaVug Message-ID: <04E3D99A62496240BCD6A576813E6E31E0BDECD834@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_04E3D99A62496240BCD6A576813E6E31E0BDECD834NDMSSCC05ndcn_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-12_05:2012-12-12, 2012-12-12, 1970-01-01 signatures=0 Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 15:31:09 -0000 --_000_04E3D99A62496240BCD6A576813E6E31E0BDECD834NDMSSCC05ndcn_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Sergey, I have duplicated what you are reporting and have to retract my earlier sta= tement saying that this is the netcat design and not a problem :) The sender netcat attempts to close the socket when it gets the [EOF] and n= ot in response to the receiver closing its end. netcat sender: cat /etc/passwd | nc x.x.x.x 12345 scenario 1 - netcat receiver with the keep listener option: nc -k -l 12345 * Sender terminates and receiver stays alive and listening for a re= -connect * FIN is sent in both directions initiated by the sender scenario 2 - dtntunnel receiver: dtntunnel -t -T 12345:0.0.0.0:54321 dtn://desteid/xxxx * Sender does not terminate * FIN is sent from sender to receiver but not the other way * Sender socket is in state FIN_WAIT2 * Receiver socket is in state CLOSE_WAIT Kill the sender and repeat... * Sender does not terminate * FIN is sent from sender to receiver but not the other way * New Sender socket is in state FIN_WAIT2 and the first one has tim= ed out and died * Both Receiver sockets are in state CLOSE_WAIT Kill the dtntunnel receiver... * 2 FINs are sent from the receiver - 1 to each of the sockets (eve= n though the other ends have expired) I believe the issue would be in the oasys IPSocket or possibly the IPClient= . I'll have a look at it unless there is a low-level socket expert out ther= e that wants to give it a go... Best regards, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gmail.com] Sent: Wednesday, December 12, 2012 4:38 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Subject: Re: [dtn-users] A problem with dtntunnel Hi David, I have investigated that issue a little. I have found, that no matter wheth= er netcat is run with or without the -w1 option, something strange happens. According to netstat -ntp on the sending node, dtntunnel r= emains in CLOSE_WAIT state after nc finishes sending data. I have run tcpdump on both sender and receiver, and it showed that fi= nal FIN from the receiver doesn't reach the sender. This looked like a bug in dtntunnel to me. I decided to re-check with iperf= , and again, tcpdump showed the same problem. Here are the last lines of the tcpdump. Remember that in my setup all outgo= ing TCP packets with destination port 9999 are redirected with the help of IPTables to the local dtntunnel process, listening om port= 19999. That is why I run tcpdump on loopback interface. Receiver (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999 15:05:02.496366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 12= 3995:131097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr 11163718]= , length 7102 15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131= 097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0 15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], seq 13= 1097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724], lengt= h 0 15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131= 098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length 0 15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], seq 1,= ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684], leng= th 0 15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack 2, = win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0 Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999 15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 127= 449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 0 15:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], seq 12= 7448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887],= length 2896 15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], seq 1= 30344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887]= , length 752 15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131= 097, win 386, options [nop,nop,TS val 11172898 ecr 11172887], length 0 15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], seq 1= 31096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], lengt= h 0 15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131= 098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], length 0 I believe that the cause of why the sending netcat without -w1 option doesn= 't exit, is that it doesn't receive the final FIN from the dtntunnel. Can this issue be fixed with a small amount of work? 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I would not even call this a problem. netcat is designed to be extremely fl= exible and by default the sender keeps its connection open as long as the o= ther end does. On the other hand the receiver by default terminates after r= eceipt of an EOF but there is a switch to keep listening if that is the des= ired behavior. With DTN, your sending nc may do its thing while the destination node is no= t even on line and then an hour later it becomes available and completes th= e transmission to the receiving nc. In this scenario, the sender would stil= l be "hung up" possibly indefinitely waiting for a terminating signal. I am not the DTN2 authHi ority, but, I don't see a change to dtntunnel for = this as it would assume a specific usage that would probably break someone = else's usage (like mine :)). Best regards, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Wednesday, December 05, 2012 2:56 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Thanks for your help, nc -w1 did the trick. But shouldn't dtntunnel behavio= r be changed? The receiving dtntunnel could signal the sending dtntunnel, t= hat the listening nc has disconnected, so that the sending dtntunnel close = its connection to the sending nc. Or is this solely the problem of nc? 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not impa= ct the behavior of dtntunnel. I just ran your test without the iptables redirect and I see the same behav= ior. I believe that the sender exits if you go directly from nc to nc because th= e receiver exits when it gets an end of file. You can add a "-w 1" option to the sender nc so that it will timeout and ex= it after stdin is idle for 1 second. Best regards, DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: dtn-users-bounces@irtf.org [mailto= :dtn-users-bounces@irtf.org] On Behalf O= f ssireskin@gmail.com Sent: Tuesday, December 04, 2012 11:13 AM To: dtn-users@irtf.org Subject: [dtn-users] A problem with dtntunnel Hello all! I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a probl= em when using netcat (nc) with dtntunnel. On the sender node I run "cat /etc/passwd | nc receiver_ip". On the receive= r node I run "nc -l 9999". With the help of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel. The file /etc/passwd is successfully delivered to the receiver and is shown= on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it sends= the file. It continues to run forever. When I run nc in the opposite direction, I get the same problem - the sending nc d= oesn't exit. My configuration on both nodes is symmetric: iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to = $LOCAL_HOST:19999 dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc = -d dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d dtnping works ok in both directions. Please give me any advice. With best regards, Sergey Sireskin -- Kindest Regards Sergey Sireskin FGUP CNII EISU -- Kindest Regards Sergey Sireskin FGUP CNII EISU --_000_04E3D99A62496240BCD6A576813E6E31E0BDECD834NDMSSCC05ndcn_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Sergey= ,

I have duplicated what y= ou are reporting and have to retract my earlier statement saying that this = is the netcat design and not a problem J

The sender netcat attempts to close the socke= t when it gets the [EOF] and not in response to the receiver closing its en= d.

 

 

netcat sender:

cat /etc/passwd | nc x.x.x.x 12345

 

<= p class=3DMsoNormal>scenario 1 - netcat receiver with the keep li= stener option:

 &= nbsp;           &nbs= p;  nc -k -l 12345

·         Sender terminates and receiver stays alive and = listening for a re-connect

·         = FIN is sent in both directions initiated by = the sender

 

scenario 2 - dtntunnel receiver= :

   &n= bsp;            dtnt= unnel -t -T 12345:0.0.0.0:54321 dtn://desteid/xxxx

·     &n= bsp;   Sender does not term= inate

·&= nbsp;        FIN is sent from sender to receiver but not the other way

·  &n= bsp;      Se= nder socket is in state FIN_WAIT2

·         = Receiver socket is in state CLOSE_WAI= T

Kill the sender and repeat...

·         <= /span>Sender does not terminate

·   &n= bsp;     FIN is s= ent from sender to receiver but not the other way

·     &nb= sp;   New Sender socket is = in state FIN_WAIT2 and the first one has timed out and died

&= middot;    = ;     Both Receiv= er sockets are in state CLOSE_WAIT

Kill the dtntunnel receiver…=

· &nbs= p;       2 FINs are sent from the receiver – 1 to each of the sockets (even = though the other ends have expired)

 

&nbs= p;

I believe the issue would be= in the oasys IPSocket or possibly the IPClient. I’ll have a look at = it unless there is a low-level socket expert out there that wants to give i= t a go…

 <= /o:p>

Best regards,=

DZ

 

David Zoller=

COLSA Corpora= tion

Marshall = Space Flight Center

 

From: ssireskin@gmai= l.com [mailto:ssireskin@gmail.com]
Sent: Wednesday, December 12,= 2012 4:38 AM
To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONT= RACT]
Subject: Re: [dtn-users] A problem with dtntunnel

 

Hi David,

I have investigated that is= sue a little. I have found, that no matter whether netcat is run with or wi= thout the -w1 option, something
strange happens. According to netstat -= ntp on the sending node, dtntunnel remains in CLOSE_WAIT state after nc fin= ishes sending
data. I have run tcpdump on both sender and receiver, and= it showed that final FIN from the receiver doesn't reach the sender.
T= his looked like a bug in dtntunnel to me. I decided to re-check with iperf,= and again, tcpdump showed the same problem.

Here are the last lines= of the tcpdump. Remember that in my setup all outgoing TCP packets with de= stination port 9999 are redirected
with the help of IPTables to the loc= al dtntunnel process, listening om port 19999. That is why I run tcpdump on= loopback interface.

Receiver (192.168.4.1): tcpdump -nn -npi lo por= t 9999 or port 19999
15:05:02.496366 IP 192.168.4.1.34284 > 192.168.4= .1.9999: Flags [P.], seq 123995:131097, ack 1, win 257, options [nop,nop,TS= val 11163723 ecr 11163718], length 7102
15:05:02.496428 IP 192.168.4.1.= 9999 > 192.168.4.1.34284: Flags [.], ack 131097, win 1154, options [nop,= nop,TS val 11163724 ecr 11163723], length 0
15:05:03.457421 IP 192.168.4= .1.34284 > 192.168.4.1.9999: Flags [F.], seq 131097, ack 1, win 257, opt= ions [nop,nop,TS val 11164684 ecr 11163724], length 0
15:05:03.497482 IP= 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131098, win 1154, = options [nop,nop,TS val 11164725 ecr 11164684], length 0
15:05:04.457762= IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], seq 1, ack 131098,= win 1154, options [nop,nop,TS val 11165685 ecr 11164684], length 0
15:0= 5:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack 2, w= in 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0

Sen= der (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02= .433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 127449, = win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 0
15:05:= 02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], seq 12744= 8:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887], le= ngth 2896
15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: F= lags [P.], seq 130344:131096, ack 1, win 46, options [nop,nop,TS val 111728= 87 ecr 11172887], length 752
15:05:02.444839 IP 192.168.4.1.9999 > 19= 2.168.1.2.43360: Flags [.], ack 131097, win 386, options [nop,nop,TS val 11= 172898 ecr 11172887], length 0
15:05:03.421915 IP 192.168.1.2.43360 >= 192.168.1.2.19999: Flags [F.], seq 131096, ack 1, win 46, options [nop,nop= ,TS val 11173875 ecr 11172898], length 0
15:05:03.461742 IP 192.168.4.1.= 9999 > 192.168.1.2.43360: Flags [.], ack 131098, win 386, options [nop,n= op,TS val 11173915 ecr 11173875], length 0

I believe that the cause = of why the sending netcat without -w1 option doesn't exit, is that it doesn= 't receive the final FIN from the dtntunnel.
Can this issue be fixed wit= h a small amount of work?

2012/12/5= Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>=

Hi Sergey,

I would not even call this a problem. netcat is designed to be ex= tremely flexible and by default the sender keeps its connection open as lon= g as the other end does. On the other hand the receiver by default terminat= es after receipt of an EOF but there is a switch to keep listening if that = is the desired behavior.

 

With DTN, your sending nc may do its thing wh= ile the destination node is not even on line and then an hour later it beco= mes available and completes the transmission to the receiving nc. In this s= cenario, the sender would still be “hung up” possibly indefinit= ely waiting for a terminating signal.

&= nbsp;

I am not the DTN2 authHi ority, bu= t, I don’t see a change to dtntunnel for this as it would assume a sp= ecific usage that would probably break someone else’s usage (like min= e J).

 

Best regards,

<= span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F= 497D'>DZ

 

David Zoller

COLSA Corporation

Marshall Space Flight Center

 

From: ssireskin@gmail.com [mai= lto:ssireskin@gmai= l.com]
Sent: Wednesday, December 05, 2012 2:56 AM
To:<= /b> Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org<= br>Subject: Re: [dtn-users] A problem with dtntunnel

 

Hi David,

Thanks for yo= ur help, nc -w1 did the trick. But shouldn't dtntunnel behavior be changed?= The receiving dtntunnel could signal the sending dtntunnel, that the liste= ning nc has disconnected, so that the sending dtntunnel close its connectio= n to the sending nc. Or is this solely the problem of nc?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <= ;david.a.zolle= r@nasa.gov>

Hi Sergey,<= o:p>

On RHEL 5.7, I am running dtn-2.9.0 plus modifi= cations that should not impact the behavior of dtntunnel.=

I just ran your test without the iptables redirect and I = see the same behavior.

I believe that th= e sender exits if you go directly from nc to nc because the receiver exits = when it gets an end of file.

You can add= a “-w 1” option to the sender nc so that it will timeout and e= xit after stdin is idle for 1 second.

Be= st regards,

DZ

 

David Zoller<= /o:p>

COLSA Corporation

HOS= C / C107 

(Office: (256) 544-1820
*EMail: david.a.zolle= r@nasa.gov

 =

From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] = On Behalf Of s= sireskin@gmail.com
Sent: Tuesday, December 04, 2012 11:13 AM<= br>To: dtn-u= sers@irtf.org
Subject: [dtn-users] A problem with dtntunnel

 

Hello all!=

I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having= a problem when using netcat (nc) with dtntunnel.
On the sender node I r= un "cat /etc/passwd | nc receiver_ip". On the receiver node I run= "nc -l 9999". With the help
of Iptables port 9999 gets redir= ected to the port 19999, which is listened by dtntunnel.

The file /e= tc/passwd is successfully delivered to the receiver and is shown on the scr= een. After this the receiving nc exists.
No problem here. However, nc o= n the sender node doesn't exit after it sends the file. It continues to run= forever. When I
run nc in the opposite direction, I get the same probl= em - the sending nc doesn't exit.

My configuration on both nodes is = symmetric:
iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999= -j DNAT --to $LOCAL_HOST:19999
dtntunnel -T $LOCAL_HOST:19999:$REMOTE_H= OST:9999 $REMOTE_NODE/dtntunnel/nc -d
dtntunnel -L --local-eid $LOCAL_NO= DE/dtntunnel/nc -d

dtnping works ok in both directions.

Pleas= e give me any advice.

With best regards,
Sergey Sireskin




--
Kinde= st Regards

Sergey Sireskin
FGUP CNII EISU



--
Kindest Regards

Sergey Sireskin
FGUP= CNII EISU

= --_000_04E3D99A62496240BCD6A576813E6E31E0BDECD834NDMSSCC05ndcn_-- From ssireskin@gmail.com Wed Dec 12 09:05:50 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0FB4B21E80C6 for ; Wed, 12 Dec 2012 09:05:50 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.103 X-Spam-Level: X-Spam-Status: No, score=-3.103 tagged_above=-999 required=5 tests=[AWL=-0.107, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_LOW=-1, WEIRD_PORT=0.001] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n1gf6AlN1Yc6 for ; Wed, 12 Dec 2012 09:05:45 -0800 (PST) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by ietfa.amsl.com (Postfix) with ESMTP id 63C0821E8084 for ; Wed, 12 Dec 2012 09:05:45 -0800 (PST) Received: by mail-pa0-f54.google.com with SMTP id bi5so736156pad.13 for ; Wed, 12 Dec 2012 09:05:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=666RKbJxtSys7x8adXjM+PVMQExzc575RFH1GVzdNwc=; b=NSa3CFgDo2QPLUyZn0SPS3AFBd8/ezQFFCjQZTWpG5uGfHqjwzmjdQQy0vz1GIP7LZ nlzstkE5To106eKtfmcK5Ed1f/h1JTVpOJz1orvem8woHsXMpjS+Lhf/XDj0JcRX0r/M wvZCiP77GkYK4s1PYebfSszF6hKJkq6SBguC8tPiI/lzcR7S47wXn+cp9zWAVUIMeWj2 DmErVSXQrmv51T74MzFz7qF93ENt7zHElQj+gOnNFihdY95ht1UIZy7GiYGZF42ROGFU +D89IU63zS5f50z7F7WFuOjiQTLmbDTIyZjlh7XFw+JlV1Ft6rhNdbzJYpOIsIs/uMR3 bRvA== MIME-Version: 1.0 Received: by 10.66.88.198 with SMTP id bi6mr4190188pab.54.1355331945013; Wed, 12 Dec 2012 09:05:45 -0800 (PST) Received: by 10.68.18.172 with HTTP; Wed, 12 Dec 2012 09:05:44 -0800 (PST) In-Reply-To: <04E3D99A62496240BCD6A576813E6E31E0BDECD834@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD834@NDMSSCC05.ndc.nasa.gov> Date: Wed, 12 Dec 2012 20:05:44 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/alternative; boundary=f46d042de42322a11504d0aacfbf Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 17:05:50 -0000 --f46d042de42322a11504d0aacfbf Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David, I added the following code at the line 492 of TCPTunnel.cc and it seems that it helped. if (sock_eof) { sock_.close(); goto done; } The senders, both nc and iperf, exit after having finished sending the data. There are no more sockets in CLOSE_WAIT state. However I am completely unsure whether my code is correct and is in correct place. Now I have encountered another problem. When iperf test is run with 100 parallel flows, some dtntunnel connections are not removed from dtntunel's connection table, and the related sockets are not closed. I suppose this happens because my code addition is far from perfect. And one more question, what does 0.0.0.0 mean in -T tunnel specification? Is it just an example host, or 0.0.0.0 has some special meaning for dtntunnel? 2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < david.a.zoller@nasa.gov> > Hi Sergey,**** > > I have duplicated what you are reporting and have to retract my earlier > statement saying that this is the netcat design and not a problem J**** > > The sender netcat attempts to close the socket when it gets the [EOF] and > not in response to the receiver closing its end.**** > > ** ** > > ** ** > > *netcat sender:* > > cat /etc/passwd | nc x.x.x.x 12345**** > > ** ** > > *scenario 1 - netcat receiver with the keep listener option:* > > nc -k -l 12345**** > > **=B7 **Sender terminates and receiver stays alive and listening > for a re-connect**** > > **=B7 **FIN is sent in both directions initiated by the sender***= * > > ** ** > > *scenario 2 - dtntunnel receiver:* > > dtntunnel -t -T 12345:0.0.0.0:54321 dtn://desteid/xxxx***= * > > **=B7 **Sender does not terminate**** > > **=B7 **FIN is sent from sender to receiver but not the other way= *** > * > > **=B7 **Sender socket is in state FIN_WAIT2**** > > **=B7 **Receiver socket is in state CLOSE_WAIT**** > > Kill the sender and repeat...**** > > **=B7 **Sender does not terminate**** > > **=B7 **FIN is sent from sender to receiver but not the other way= *** > * > > **=B7 **New Sender socket is in state FIN_WAIT2 and the first one > has timed out and died**** > > **=B7 **Both Receiver sockets are in state CLOSE_WAIT**** > > Kill the dtntunnel receiver=85**** > > **=B7 **2 FINs are sent from the receiver =96 1 to each of the > sockets (even though the other ends have expired)**** > > ** ** > > ** ** > > I believe the issue would be in the oasys IPSocket or possibly the > IPClient. I=92ll have a look at it unless there is a low-level socket exp= ert > out there that wants to give it a go=85**** > > ** ** > > Best regards,**** > > DZ**** > > ** ** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > ** ** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 12, 2012 4:38 AM > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > ** ** > > Hi David, > > I have investigated that issue a little. I have found, that no matter > whether netcat is run with or without the -w1 option, something > strange happens. According to netstat -ntp on the sending node, dtntunnel > remains in CLOSE_WAIT state after nc finishes sending > data. I have run tcpdump on both sender and receiver, and it showed that > final FIN from the receiver doesn't reach the sender. > This looked like a bug in dtntunnel to me. I decided to re-check with > iperf, and again, tcpdump showed the same problem. > > Here are the last lines of the tcpdump. Remember that in my setup all > outgoing TCP packets with destination port 9999 are redirected > with the help of IPTables to the local dtntunnel process, listening om > port 19999. That is why I run tcpdump on loopback interface. > > Receiver (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999 > 15:05:02.496366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq > 123995:131097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr > 11163718], length 7102 > 15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack > 131097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length = 0 > 15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], seq > 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724], > length 0 > 15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack > 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length = 0 > 15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], seq > 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684], > length 0 > 15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack 2= , > win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0 > > Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999 > 15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack > 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 0 > 15:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], seq > 127448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr > 11172887], length 2896 > 15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], seq > 130344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr > 11172887], length 752 > 15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack > 131097, win 386, options [nop,nop,TS val 11172898 ecr 11172887], length 0 > 15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], seq > 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], > length 0 > 15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack > 131098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], length 0 > > I believe that the cause of why the sending netcat without -w1 option > doesn't exit, is that it doesn't receive the final FIN from the dtntunnel= . > Can this issue be fixed with a small amount of work?**** > > 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > I would not even call this a problem. netcat is designed to be extremely > flexible and by default the sender keeps its connection open as long as t= he > other end does. On the other hand the receiver by default terminates afte= r > receipt of an EOF but there is a switch to keep listening if that is the > desired behavior. **** > > **** > > With DTN, your sending nc may do its thing while the destination node is > not even on line and then an hour later it becomes available and complete= s > the transmission to the receiving nc. In this scenario, the sender would > still be =93hung up=94 possibly indefinitely waiting for a terminating si= gnal. > **** > > **** > > I am not the DTN2 authHi ority, but, I don=92t see a change to dtntunnel = for > this as it would assume a specific usage that would probably break someon= e > else=92s usage (like mine J).**** > > **** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 05, 2012 2:56 AM > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > Thanks for your help, nc -w1 did the trick. But shouldn't dtntunnel > behavior be changed? The receiving dtntunnel could signal the sending > dtntunnel, that the listening nc has disconnected, so that the sending > dtntunnel close its connection to the sending nc. Or is this solely the > problem of nc?**** > > 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not > impact the behavior of dtntunnel.**** > > I just ran your test without the iptables redirect and I see the same > behavior.**** > > I believe that the sender exits if you go directly from nc to nc because > the receiver exits when it gets an end of file.**** > > You can add a =93-w 1=94 option to the sender nc so that it will timeout = and > exit after stdin is idle for 1 second.**** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *O= n > Behalf Of *ssireskin@gmail.com > *Sent:* Tuesday, December 04, 2012 11:13 AM > *To:* dtn-users@irtf.org > *Subject:* [dtn-users] A problem with dtntunnel**** > > **** > > Hello all! > > I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a > problem when using netcat (nc) with dtntunnel. > On the sender node I run "cat /etc/passwd | nc receiver_ip". On the > receiver node I run "nc -l 9999". With the help > of Iptables port 9999 gets redirected to the port 19999, which is listene= d > by dtntunnel. > > The file /etc/passwd is successfully delivered to the receiver and is > shown on the screen. After this the receiving nc exists. > No problem here. However, nc on the sender node doesn't exit after it > sends the file. It continues to run forever. When I > run nc in the opposite direction, I get the same problem - the sending nc > doesn't exit. > > My configuration on both nodes is symmetric: > iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --t= o > $LOCAL_HOST:19999 > dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/n= c > -d > dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d > > dtnping works ok in both directions. > > Please give me any advice. > > With best regards, > Sergey Sireskin **** > > > > > -- > Kindest Regards > > Sergey Sireskin > FGUP CNII EISU**** > > > > > -- > Kindest Regards > > Sergey Sireskin > FGUP CNII EISU**** > --=20 Best regards, Sergey Sireskin --f46d042de42322a11504d0aacfbf Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David,

I added the following code at the line 492 of TCPTunnel.cc= and it seems that it helped.

=A0=A0=A0 if (sock_eof) {
=A0=A0= =A0 =A0=A0=A0 sock_.close();
=A0=A0=A0 =A0=A0=A0 goto done;
=A0=A0=A0= }

The senders, both nc and iperf, exit after having finished sendin= g the data. There are
no more sockets in CLOSE_WAIT state. However I am completely unsure whether=
my code is correct and is in correct place.

Now I have encounte= red another problem. When iperf test is run with 100 parallel flows,
some dtntunnel connections are not removed from dtntunel's connection t= able, and the
related sockets are not closed. I suppose this happens bec= ause my code addition is far
from perfect.

And one more question,= what does 0.0.0.0 mean in -T tunnel specification? Is it just an
example host, or 0.0.0.0 has some special meaning for dtntunnel?

2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVIC= ES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,=

I have duplicated what yo= u are reporting and have to retract my earlier statement saying that this i= s the netcat design and not a problem J<= u>

The sender netcat attempt= s to close the socket when it gets the [EOF] and not in response to the rec= eiver closing its end.

=A0<= /p>

=A0

netcat sender:<= u>

cat /etc/passwd | nc x.x.x.x 12345

=A0<= /p>

scenario 1 - netcat= receiver with the keep listener option:

=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 nc -k -l 12345

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Sender terminates and receiver stays alive and listening for a re-= connect

=B7=A0=A0=A0=A0=A0=A0=A0=A0 FIN is sent in both directions initiated by the send= er

=A0<= /p>

scenario 2 - dtntun= nel receiver:

=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 dtntunnel -t -T 12345:0.0.0.0:54321 dtn://desteid/xxxx

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Sender does not terminate

=B7=A0=A0=A0=A0=A0=A0=A0=A0 FIN is sent from sender to receiver but not the othe= r way

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Sender socket is in state FIN_WAIT2

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Receiver socket is in state CLOSE_WAIT=

=B7=A0=A0=A0=A0= =A0=A0=A0=A0 Sender = does not terminate

=B7=A0=A0=A0=A0=A0=A0=A0=A0 FIN is sent from sender to receiver but not the othe= r way

=B7=A0=A0=A0=A0=A0=A0=A0=A0 New Sender socket is in state FIN_WAIT2 and the firs= t one has timed out and died

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Both Receiver sockets are in state CLOSE_WAIT=

=B7=A0=A0=A0=A0= =A0=A0=A0=A0 2 FINs = are sent from the receiver =96 1 to each of the sockets (even though the ot= her ends have expired)

=A0<= /p>

=A0

I believe the issue would= be in the oasys IPSocket or possibly the IPClient. I=92ll have a look at i= t unless there is a low-level socket expert out there that wants to give it= a go=85

= =A0

Best= regards,

DZ

=A0

David Zoller

COLSA Corporation

Marshall Space Flight Cente= r

=A0<= /p>

From: ssireskin@gmail.= com [mailto:ss= ireskin@gmail.com]
Sent: Wednesday, December 12, 2012 4:38 AM


To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
<= /div>
Subject: Re: [dtn-users] A problem with = dtntunnel

=A0

Hi David,
I have investigated that issue a little. I have found, that no matter whet= her netcat is run with or without the -w1 option, something
strange happens. According to netstat -ntp on the sending node, dtntunnel r= emains in CLOSE_WAIT state after nc finishes sending
data. I have run t= cpdump on both sender and receiver, and it showed that final FIN from the r= eceiver doesn't reach the sender.
This looked like a bug in dtntunnel to me. I decided to re-check with iperf= , and again, tcpdump showed the same problem.

Here are the last line= s of the tcpdump. Remember that in my setup all outgoing TCP packets with d= estination port 9999 are redirected
with the help of IPTables to the local dtntunnel process, listening om port= 19999. That is why I run tcpdump on loopback interface.

Receiver (1= 92.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.496366= IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 123995:131097,= ack 1, win 257, options [nop,nop,TS val 11163723 ecr 11163718], length 710= 2
15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack = 131097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0<= br>15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], = seq 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724],= length 0
15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack = 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length 0<= br>15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], = seq 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684]= , length 0
15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack = 2, win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0
Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack = 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 015:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], s= eq 127448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172= 887], length 2896
15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], se= q 130344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 111728= 87], length 752
15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.433= 60: Flags [.], ack 131097, win 386, options [nop,nop,TS val 11172898 ecr 11= 172887], length 0
15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], se= q 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], le= ngth 0
15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags= [.], ack 131098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], = length 0

I believe that the cause of why the sending netcat without -w1 option d= oesn't exit, is that it doesn't receive the final FIN from the dtnt= unnel.
Can this issue be fixed with a small amount of work?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SER= VICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

I would not even call this a problem. = netcat is designed to be extremely flexible and by default the sender keeps= its connection open as long as the other end does. On the other hand the r= eceiver by default terminates after receipt of an EOF but there is a switch= to keep listening if that is the desired behavior.

=A0<= /p>

With DTN, your sending= nc may do its thing while the destination node is not even on line and the= n an hour later it becomes available and completes the transmission to the = receiving nc. In this scenario, the sender would still be =93hung up=94 pos= sibly indefinitely waiting for a terminating signal. <= /p>

=A0<= /p>

I am not the DTN2 auth= Hi ority, but, I don=92t see a change to dtntunnel for this as it would ass= ume a specific usage that would probably break someone else=92s usage (like= mine J).

=A0=

Best regards,

DZ

=A0

David Zoller<= u>

COLSA Corporation=

Marshall Space Flight= Center

=A0

From: ssireskin@gmail.com [= mailto:ssireskin@g= mail.com]
Sent: Wednesday, December 05, 2012 2:56 AM
To: Zoller, Dav= id A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org
Subject:<= /b> Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,

Thanks for your help, nc = -w1 did the trick. But shouldn't dtntunnel behavior be changed? The rec= eiving dtntunnel could signal the sending dtntunnel, that the listening nc = has disconnected, so that the sending dtntunnel close its connection to the= sending nc. Or is this solely the problem of nc?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SER= VICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

On RHEL 5.7, I am running dtn-2.9.0 pl= us modifications that should not impact the behavior of dtntunnel.

I just ran your test with= out the iptables redirect and I see the same behavior.=

I believe that the sender= exits if you go directly from nc to nc because the receiver exits when it = gets an end of file.

You can add a =93-w 1=94 = option to the sender nc so that it will timeout and exit after stdin is idl= e for 1 second.

Best regards,

DZ=

=A0<= /p>

David Zoller

COLSA Corporation=

HOSC / C107= =A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: dtn-users-bounce= s@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Tuesday, December 04, 2012 11:13 AM
To: dtn-users@irtf.org
Su= bject: [dtn-users] A problem with dtntunnel

=A0

Hel= lo all!

I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am= having a problem when using netcat (nc) with dtntunnel.
On the sender n= ode I run "cat /etc/passwd | nc receiver_ip". On the receiver nod= e I run "nc -l 9999". With the help
of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel.

The file /etc/passwd is successfully delivered to the = receiver and is shown on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it s= ends the file. It continues to run forever. When I
run nc in the opposi= te direction, I get the same problem - the sending nc doesn't exit.

My configuration on both nodes is symmetric:
iptables -t nat -A OUTP= UT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to $LOCAL_HOST:19999
dt= ntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d=
dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d

dtnping works o= k in both directions.

Please give me any advice.

With best r= egards,
Sergey Sireskin




--
Kindest Regards

Sergey Sireskin
FGUP CNII EISU=




--
Kindest Regards

Sergey Sireskin
= FGUP CNII EISU

=


--
Best regards,
Sergey Sireskin

--f46d042de42322a11504d0aacfbf-- From david.a.zoller@nasa.gov Wed Dec 12 10:03:28 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 64BC921F87AD for ; Wed, 12 Dec 2012 10:03:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.297 X-Spam-Level: X-Spam-Status: No, score=-6.297 tagged_above=-999 required=5 tests=[AWL=-0.301, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_MED=-4, WEIRD_PORT=0.001] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DiAZoRgEvNG3 for ; Wed, 12 Dec 2012 10:03:22 -0800 (PST) Received: from ndmsnpf01.ndc.nasa.gov (ndmsnpf01.ndc.nasa.gov [198.117.0.121]) by ietfa.amsl.com (Postfix) with ESMTP id EEE8121F8762 for ; Wed, 12 Dec 2012 10:03:21 -0800 (PST) Received: from ndmsppt03.ndc.nasa.gov (ndmsppt03.ndc.nasa.gov [198.117.0.102]) by ndmsnpf01.ndc.nasa.gov (Postfix) with ESMTP id EC5BC2606DD; Wed, 12 Dec 2012 12:03:20 -0600 (CST) Received: from ndmshub01.ndc.nasa.gov (ndmshub01-pub.ndc.nasa.gov [198.117.0.160]) by ndmsppt03.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBCI3K2t019891; Wed, 12 Dec 2012 12:03:20 -0600 Received: from NDMSSCC05.ndc.nasa.gov ([198.117.2.174]) by ndmshub01.ndc.nasa.gov ([192.168.0.111]) with mapi; Wed, 12 Dec 2012 12:03:20 -0600 From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "ssireskin@gmail.com" Date: Wed, 12 Dec 2012 12:03:18 -0600 Thread-Topic: [dtn-users] A problem with dtntunnel Thread-Index: Ac3YivAk1wasVXhxQtGNNleyO5V1oAAAH04Q Message-ID: <04E3D99A62496240BCD6A576813E6E31E0BDECD996@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD834@NDMSSCC05.ndc.nasa.gov> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_04E3D99A62496240BCD6A576813E6E31E0BDECD996NDMSSCC05ndcn_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-12_06:2012-12-12, 2012-12-12, 1970-01-01 signatures=0 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 18:03:28 -0000 --_000_04E3D99A62496240BCD6A576813E6E31E0BDECD996NDMSSCC05ndcn_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Sergey, I was narrowing in on the same track... The FIN results in a returned lengt= h of zero on the read and errno is also zero so you have to key on the retu= rn length which is already in the code. You may have added a couple of lines of debug code and it should be inserte= d just before the end of the while(1) loop at line 491 in the latest versio= n on Sourceforge. I am seeing the sending socket stay in the TIME_WAIT mode even if I add a c= all to sock_.shutdown(SHUT_RDWR) before closing the socket; Getting close I think. The 0.0.0.0 was because I did not have anything at the other end of the tun= nel and just stuck some easy numbers in there. Thanks, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gmail.com] Sent: Wednesday, December 12, 2012 11:06 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, I added the following code at the line 492 of TCPTunnel.cc and it seems tha= t it helped. if (sock_eof) { sock_.close(); goto done; } The senders, both nc and iperf, exit after having finished sending the data= . There are no more sockets in CLOSE_WAIT state. However I am completely unsure whether my code is correct and is in correct place. Now I have encountered another problem. When iperf test is run with 100 par= allel flows, some dtntunnel connections are not removed from dtntunel's connection table= , and the related sockets are not closed. I suppose this happens because my code addi= tion is far from perfect. And one more question, what does 0.0.0.0 mean in -T tunnel specification? I= s it just an example host, or 0.0.0.0 has some special meaning for dtntunnel? 2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I have duplicated what you are reporting and have to retract my earlier sta= tement saying that this is the netcat design and not a problem :) The sender netcat attempts to close the socket when it gets the [EOF] and n= ot in response to the receiver closing its end. netcat sender: cat /etc/passwd | nc x.x.x.x 12345 scenario 1 - netcat receiver with the keep listener option: nc -k -l 12345 * Sender terminates and receiver stays alive and listening for a re= -connect * FIN is sent in both directions initiated by the sender scenario 2 - dtntunnel receiver: dtntunnel -t -T 12345:0.0.0.0:54321 d= tn://desteid/xxxx * Sender does not terminate * FIN is sent from sender to receiver but not the other way * Sender socket is in state FIN_WAIT2 * Receiver socket is in state CLOSE_WAIT Kill the sender and repeat... * Sender does not terminate * FIN is sent from sender to receiver but not the other way * New Sender socket is in state FIN_WAIT2 and the first one has tim= ed out and died * Both Receiver sockets are in state CLOSE_WAIT Kill the dtntunnel receiver... * 2 FINs are sent from the receiver - 1 to each of the sockets (eve= n though the other ends have expired) I believe the issue would be in the oasys IPSocket or possibly the IPClient= . I'll have a look at it unless there is a low-level socket expert out ther= e that wants to give it a go... Best regards, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Wednesday, December 12, 2012 4:38 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Subject: Re: [dtn-users] A problem with dtntunnel Hi David, I have investigated that issue a little. I have found, that no matter wheth= er netcat is run with or without the -w1 option, something strange happens. According to netstat -ntp on the sending node, dtntunnel r= emains in CLOSE_WAIT state after nc finishes sending data. I have run tcpdump on both sender and receiver, and it showed that fi= nal FIN from the receiver doesn't reach the sender. This looked like a bug in dtntunnel to me. I decided to re-check with iperf= , and again, tcpdump showed the same problem. Here are the last lines of the tcpdump. Remember that in my setup all outgo= ing TCP packets with destination port 9999 are redirected with the help of IPTables to the local dtntunnel process, listening om port= 19999. That is why I run tcpdump on loopback interface. Receiver (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999 15:05:02.496366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 12= 3995:131097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr 11163718]= , length 7102 15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131= 097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0 15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], seq 13= 1097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724], lengt= h 0 15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131= 098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length 0 15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], seq 1,= ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684], leng= th 0 15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack 2, = win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0 Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999 15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 127= 449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 0 15:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], seq 12= 7448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887],= length 2896 15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], seq 1= 30344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887]= , length 752 15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131= 097, win 386, options [nop,nop,TS val 11172898 ecr 11172887], length 0 15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], seq 1= 31096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], lengt= h 0 15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131= 098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], length 0 I believe that the cause of why the sending netcat without -w1 option doesn= 't exit, is that it doesn't receive the final FIN from the dtntunnel. Can this issue be fixed with a small amount of work? 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I would not even call this a problem. netcat is designed to be extremely fl= exible and by default the sender keeps its connection open as long as the o= ther end does. On the other hand the receiver by default terminates after r= eceipt of an EOF but there is a switch to keep listening if that is the des= ired behavior. With DTN, your sending nc may do its thing while the destination node is no= t even on line and then an hour later it becomes available and completes th= e transmission to the receiving nc. In this scenario, the sender would stil= l be "hung up" possibly indefinitely waiting for a terminating signal. I am not the DTN2 authHi ority, but, I don't see a change to dtntunnel for = this as it would assume a specific usage that would probably break someone = else's usage (like mine :)). Best regards, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Wednesday, December 05, 2012 2:56 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Thanks for your help, nc -w1 did the trick. But shouldn't dtntunnel behavio= r be changed? The receiving dtntunnel could signal the sending dtntunnel, t= hat the listening nc has disconnected, so that the sending dtntunnel close = its connection to the sending nc. Or is this solely the problem of nc? 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not impa= ct the behavior of dtntunnel. I just ran your test without the iptables redirect and I see the same behav= ior. I believe that the sender exits if you go directly from nc to nc because th= e receiver exits when it gets an end of file. You can add a "-w 1" option to the sender nc so that it will timeout and ex= it after stdin is idle for 1 second. Best regards, DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: dtn-users-bounces@irtf.org [mailto= :dtn-users-bounces@irtf.org] On Behalf O= f ssireskin@gmail.com Sent: Tuesday, December 04, 2012 11:13 AM To: dtn-users@irtf.org Subject: [dtn-users] A problem with dtntunnel Hello all! I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a probl= em when using netcat (nc) with dtntunnel. On the sender node I run "cat /etc/passwd | nc receiver_ip". On the receive= r node I run "nc -l 9999". With the help of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel. The file /etc/passwd is successfully delivered to the receiver and is shown= on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it sends= the file. It continues to run forever. When I run nc in the opposite direction, I get the same problem - the sending nc d= oesn't exit. My configuration on both nodes is symmetric: iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to = $LOCAL_HOST:19999 dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc = -d dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d dtnping works ok in both directions. Please give me any advice. With best regards, Sergey Sireskin -- Kindest Regards Sergey Sireskin FGUP CNII EISU -- Kindest Regards Sergey Sireskin FGUP CNII EISU -- Best regards, Sergey Sireskin --_000_04E3D99A62496240BCD6A576813E6E31E0BDECD996NDMSSCC05ndcn_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Sergey= ,

I was narrowing in on th= e same track... The FIN results in a returned length of zero on the read an= d errno is also zero so you have to key on the return length which is alrea= dy in the code.

 = ;

You may have added a couple o= f lines of debug code and it should be inserted just before the end of the = while(1) loop at line 491 in the latest version on Sourceforge.<= /span>

 

I am seeing the sending socket stay in the TIME_WAIT mo= de even if I add a call to sock_.shutdown(SHUT_RDWR) before closing the soc= ket;

Getting close I think= .

 =

The 0.0.0.0 was because I did not have anyt= hing at the other end of the tunnel and just stuck some easy numbers in the= re.

Thanks,
DZ

 

David Zoller

COLSA Corporation

Marshall Space Flight Center

 

From: ssireskin@gmail.com [mailto:ssireskin@gmail.com]
S= ent: Wednesday, December 12, 2012 11:06 AM
To: Zoller, David = A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org
= Subject: Re: [dtn-users] A problem with dtntunnel<= /p>

 

Hi David,

I added the following code at the li= ne 492 of TCPTunnel.cc and it seems that it helped.

  &nb= sp; if (sock_eof) {
        sock_.close();=
        goto done;
    = }

The senders, both nc and iperf, exit after having finished sending= the data. There are
no more sockets in CLOSE_WAIT state. However I am c= ompletely unsure whether
my code is correct and is in correct place.
Now I have encountered another problem. When iperf test is run with 10= 0 parallel flows,
some dtntunnel connections are not removed from dtntun= el's connection table, and the
related sockets are not closed. I suppose= this happens because my code addition is far
from perfect.

And o= ne more question, what does 0.0.0.0 mean in -T tunnel specification? Is it = just an
example host, or 0.0.0.0 has some special meaning for dtntunnel?=

2012/12/12 Zoller, David A. (MSFC-= EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

I have dupli= cated what you are reporting and have to retract my earlier statement sayin= g that this is the netcat design and not a problem J

=

The sender netcat attempts to close the socket when it gets t= he [EOF] and not in response to the receiver closing its end.

 

 

netcat sender:

<= p class=3DMsoNormal style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:= auto;text-indent:.5in'>cat /etc/passwd | nc x.x.x.x 12345

 

scen= ario 1 - netcat receiver with the keep listener option:

         &= nbsp;      nc -k -l 12345

·         Sender = terminates and receiver stays alive and listening for a re-connect

·         FIN is sent in both directions initiated by the sender=

 

scen= ario 2 - dtntunnel receiver:

 &= nbsp;           &nbs= p;  dtntunnel -t -T 12345:0.0.0.0:54321 dtn://desteid/xxxx

·=          Sender does = not terminate

·     &n= bsp;   FIN is sent from sender to receiver but not = the other way

·     &n= bsp;   Sender socket is in state FIN_WAIT2

·         Receiver socket is in state CLOSE_WAIT

Kill the sender and repeat...

·         Sen= der does not terminate

= ·<= /span>    = ;     FIN is sent from sender to receiver= but not the other way

= ·<= /span>    = ;     New Sender socket is in state FIN_W= AIT2 and the first one has timed out and died

·=          Both Receive= r sockets are in state CLOSE_WAIT

Kill the dtntunnel receiver…

·=          2 FINs are s= ent from the receiver – 1 to each of the sockets (even though the oth= er ends have expired)

 =

 

I believe= the issue would be in the oasys IPSocket or possibly the IPClient. I’= ;ll have a look at it unless there is a low-level socket expert out there t= hat wants to give it a go…

&n= bsp;

Best regards,

=

DZ

 

David Zoller

COLSA Corpor= ation

Marshall Space Flight Center<= o:p>

 

From: ssireskin@gmail.com<= /a> [mailto:ssires= kin@gmail.com]
Sent: Wednesday, December 12, 2012 4:38 AM


To: Zoller, David A= . (MSFC-EO60)[HOSC SERVICES CONTRACT]

Subject: Re: [dtn-users] A problem with dtntunnel=

 

Hi David,
<= br>I have investigated that issue a little. I have found, that no matter wh= ether netcat is run with or without the -w1 option, something
strange h= appens. According to netstat -ntp on the sending node, dtntunnel remains in= CLOSE_WAIT state after nc finishes sending
data. I have run tcpdump on= both sender and receiver, and it showed that final FIN from the receiver d= oesn't reach the sender.
This looked like a bug in dtntunnel to me. I d= ecided to re-check with iperf, and again, tcpdump showed the same problem.<= br>
Here are the last lines of the tcpdump. Remember that in my setup al= l outgoing TCP packets with destination port 9999 are redirected
with t= he help of IPTables to the local dtntunnel process, listening om port 19999= . That is why I run tcpdump on loopback interface.

Receiver (192.168= .4.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.496366 IP 19= 2.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 123995:131097, ack 1= , win 257, options [nop,nop,TS val 11163723 ecr 11163718], length 7102
1= 5:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 1= 31097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 015:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], s= eq 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724], = length 0
15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Fla= gs [.], ack 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684= ], length 0
15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: = Flags [F.], seq 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 e= cr 11164684], length 0
15:05:04.457856 IP 192.168.4.1.34284 > 192.168= .4.1.9999: Flags [.], ack 2, win 257, options [nop,nop,TS val 11165685 ecr = 11165685], length 0

Sender (192.168.2.1): tcpdump -nn -npi lo port 9= 999 or port 19999
15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.= 43360: Flags [.], ack 127449, win 386, options [nop,nop,TS val 11172887 ecr= 11172875], length 0
15:05:02.433776 IP 192.168.1.2.43360 > 192.168.1= .2.19999: Flags [.], seq 127448:130344, ack 1, win 46, options [nop,nop,TS = val 11172887 ecr 11172887], length 2896
15:05:02.433783 IP 192.168.1.2.4= 3360 > 192.168.1.2.19999: Flags [P.], seq 130344:131096, ack 1, win 46, = options [nop,nop,TS val 11172887 ecr 11172887], length 752
15:05:02.4448= 39 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131097, win 3= 86, options [nop,nop,TS val 11172898 ecr 11172887], length 0
15:05:03.42= 1915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], seq 131096, a= ck 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], length 0
1= 5:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 1= 31098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], length 0
I believe that the cause of why the sending netcat without -w1 option = doesn't exit, is that it doesn't receive the final FIN from the dtntunnel.<= br>Can this issue be fixed with a small amount of work?

=

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <<= a href=3D"mailto:david.a.zoller@nasa.gov" target=3D"_blank">david.a.zoller@= nasa.gov>

Hi Sergey,

I would not even call this a problem. netcat is d= esigned to be extremely flexible and by default the sender keeps its connec= tion open as long as the other end does. On the other hand the receiver by = default terminates after receipt of an EOF but there is a switch to keep li= stening if that is the desired behavior.

 

With DTN, your sending nc may d= o its thing while the destination node is not even on line and then an hour= later it becomes available and completes the transmission to the receiving= nc. In this scenario, the sender would still be “hung up” poss= ibly indefinitely waiting for a terminating signal.

<= p class=3DMsoNormal style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:= auto'> 

I am not the DTN2 au= thHi ority, but, I don’t see a change to dtntunnel for this as it wou= ld assume a specific usage that would probably break someone else’s u= sage (like mine J).

=  

Best regards,

DZ

 

David Zoller

COLSA Corp= oration

Marshall Space Flight Center=

 

<= span style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From: = ssireskin@gmail.co= m [mailto:ssir= eskin@gmail.com]
Sent: Wednesday, December 05, 2012 2:56 AM<= br>To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc= : dtn-users@irt= f.org
Subject: Re: [dtn-users] A problem with dtntunnel

 

Hi David,

Tha= nks for your help, nc -w1 did the trick. But shouldn't dtntunnel behavior b= e changed? The receiving dtntunnel could signal the sending dtntunnel, that= the listening nc has disconnected, so that the sending dtntunnel close its= connection to the sending nc. Or is this solely the problem of nc?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CON= TRACT] <dav= id.a.zoller@nasa.gov>

Hi Serge= y,

On RHEL 5.7, I am running dtn-2.9.0 p= lus modifications that should not impact the behavior of dtntunnel.<= o:p>

I just ran your test without the iptables redir= ect and I see the same behavior.

I belie= ve that the sender exits if you go directly from nc to nc because the recei= ver exits when it gets an end of file.

Y= ou can add a “-w 1” option to the sender nc so that it will tim= eout and exit after stdin is idle for 1 second.

Best regards,

DZ

 

David Zoller

COLSA Corporation

HOSC / C107 

(Office: (256) 544-1820
*EMail: david.a.zoller@nasa.gov

&nb= sp;

From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf= .org] On Behalf Of ssireskin@gmail.com
Sent: Tuesday, December 04, 2= 012 11:13 AM
To: dtn-users@irtf.org
Subject: [dtn-users] A problem wit= h dtntunnel

 

Hello all!

I am using dtn-2.9.0 on a RHEL 6 based Linux distro an= d I am having a problem when using netcat (nc) with dtntunnel.
On the se= nder node I run "cat /etc/passwd | nc receiver_ip". On the receiv= er node I run "nc -l 9999". With the help
of Iptables port 99= 99 gets redirected to the port 19999, which is listened by dtntunnel.
The file /etc/passwd is successfully delivered to the receiver and is sho= wn on the screen. After this the receiving nc exists.
No problem here. = However, nc on the sender node doesn't exit after it sends the file. It con= tinues to run forever. When I
run nc in the opposite direction, I get t= he same problem - the sending nc doesn't exit.

My configuration on b= oth nodes is symmetric:
iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp= --dport 9999 -j DNAT --to $LOCAL_HOST:19999
dtntunnel -T $LOCAL_HOST:19= 999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d
dtntunnel -L --local-= eid $LOCAL_NODE/dtntunnel/nc -d

dtnping works ok in both directions.=

Please give me any advice.

With best regards,
Sergey Sir= eskin



--
Kindest Regards

Sergey Sireskin
FGUP CNII EISU=




--
Kindest R= egards

Sergey Sireskin
FGUP CNII EISU

<= /div>



--
Best regards,
Sergey Sireskin

= --_000_04E3D99A62496240BCD6A576813E6E31E0BDECD996NDMSSCC05ndcn_-- From ssireskin@gmail.com Wed Dec 12 12:07:09 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EA02111E80BF for ; Wed, 12 Dec 2012 12:07:09 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.09 X-Spam-Level: X-Spam-Status: No, score=-3.09 tagged_above=-999 required=5 tests=[AWL=-0.093, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_LOW=-1, WEIRD_PORT=0.001] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IpjgHivGXIa6 for ; Wed, 12 Dec 2012 12:07:08 -0800 (PST) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by ietfa.amsl.com (Postfix) with ESMTP id 3BD7A11E80A5 for ; Wed, 12 Dec 2012 12:07:08 -0800 (PST) Received: by mail-pb0-f54.google.com with SMTP id wz12so829463pbc.13 for ; Wed, 12 Dec 2012 12:07:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=anCtIY7RqINwyWN/qUD8T1bpxABH6EjUI6dPuiNQJrc=; b=hvwiHOU8l5r3qxA5Xc7/xZG0uffz07jSWmJ4ho8ZWgER+NAbpObCxv5UpwhOBNKdy7 AaPaJhDSGUIWe+u20rVE42kqP1FGD7BnyipzzYmVBAzwzIF4hnQGTvNfn6YDUoxpQqEM 6ayFEwKZj6WClSgl+RY5OBtTT0enB1RcY/EOvZvx/FwSRdW2B87KTCfM3j97gb3WwDif I16TTOetBIJziROmvJbHOwFaVisxBsGuoaZrdwogIvhS5O93Xbd4AtFqux1vC0LmT2bC gUPYR3YLH+hD32EXqH0koUsVWzw3C1cjHqsad5YAjWi9L3jFsO3GtwjZf2OW743DyXtr rgHw== MIME-Version: 1.0 Received: by 10.68.225.201 with SMTP id rm9mr5802490pbc.124.1355342827850; Wed, 12 Dec 2012 12:07:07 -0800 (PST) Received: by 10.68.18.172 with HTTP; Wed, 12 Dec 2012 12:07:07 -0800 (PST) In-Reply-To: <04E3D99A62496240BCD6A576813E6E31E0BDECD996@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD834@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD996@NDMSSCC05.ndc.nasa.gov> Date: Wed, 12 Dec 2012 23:07:07 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/alternative; boundary=047d7b2ede33cd8ac104d0ad570a Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 20:07:10 -0000 --047d7b2ede33cd8ac104d0ad570a Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David, I meant exactly the same position before the end of the while loop, just described it incorrect. Please let me know if you find the solution for this problem. Best regards, Sergey 2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < david.a.zoller@nasa.gov> > Hi Sergey,**** > > I was narrowing in on the same track... The FIN results in a returned > length of zero on the read and errno is also zero so you have to key on t= he > return length which is already in the code.**** > > ** ** > > You may have added a couple of lines of debug code and it should be > inserted just before the end of the while(1) loop at line 491 in the late= st > version on Sourceforge.**** > > ** ** > > I am seeing the sending socket stay in the TIME_WAIT mode even if I add a > call to sock_.shutdown(SHUT_RDWR) before closing the socket;**** > > Getting close I think.**** > > ** ** > > The 0.0.0.0 was because I did not have anything at the other end of the > tunnel and just stuck some easy numbers in there.**** > > Thanks, > DZ**** > > ** ** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > ** ** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 12, 2012 11:06 AM > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > ** ** > > Hi David, > > I added the following code at the line 492 of TCPTunnel.cc and it seems > that it helped. > > if (sock_eof) { > sock_.close(); > goto done; > } > > The senders, both nc and iperf, exit after having finished sending the > data. There are > no more sockets in CLOSE_WAIT state. However I am completely unsure > whether > my code is correct and is in correct place. > > Now I have encountered another problem. When iperf test is run with 100 > parallel flows, > some dtntunnel connections are not removed from dtntunel's connection > table, and the > related sockets are not closed. I suppose this happens because my code > addition is far > from perfect. > > And one more question, what does 0.0.0.0 mean in -T tunnel specification? > Is it just an > example host, or 0.0.0.0 has some special meaning for dtntunnel?**** > > 2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > I have duplicated what you are reporting and have to retract my earlier > statement saying that this is the netcat design and not a problem J**** > > The sender netcat attempts to close the socket when it gets the [EOF] and > not in response to the receiver closing its end.**** > > **** > > **** > > *netcat sender:***** > > cat /etc/passwd | nc x.x.x.x 12345**** > > **** > > *scenario 1 - netcat receiver with the keep listener option:***** > > nc -k -l 12345**** > > =B7 Sender terminates and receiver stays alive and listening for = a > re-connect**** > > =B7 FIN is sent in both directions initiated by the sender**** > > **** > > *scenario 2 - dtntunnel receiver:***** > > dtntunnel -t -T 12345:0.0.0.0:54321 dtn://desteid/xxxx***= * > > =B7 Sender does not terminate**** > > =B7 FIN is sent from sender to receiver but not the other way**** > > =B7 Sender socket is in state FIN_WAIT2**** > > =B7 Receiver socket is in state CLOSE_WAIT**** > > Kill the sender and repeat...**** > > =B7 Sender does not terminate**** > > =B7 FIN is sent from sender to receiver but not the other way**** > > =B7 New Sender socket is in state FIN_WAIT2 and the first one has > timed out and died**** > > =B7 Both Receiver sockets are in state CLOSE_WAIT**** > > Kill the dtntunnel receiver=85**** > > =B7 2 FINs are sent from the receiver =96 1 to each of the socket= s > (even though the other ends have expired)**** > > **** > > **** > > I believe the issue would be in the oasys IPSocket or possibly the > IPClient. I=92ll have a look at it unless there is a low-level socket exp= ert > out there that wants to give it a go=85**** > > **** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 12, 2012 4:38 AM**** > > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]**** > > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > I have investigated that issue a little. I have found, that no matter > whether netcat is run with or without the -w1 option, something > strange happens. According to netstat -ntp on the sending node, dtntunnel > remains in CLOSE_WAIT state after nc finishes sending > data. I have run tcpdump on both sender and receiver, and it showed that > final FIN from the receiver doesn't reach the sender. > This looked like a bug in dtntunnel to me. I decided to re-check with > iperf, and again, tcpdump showed the same problem. > > Here are the last lines of the tcpdump. Remember that in my setup all > outgoing TCP packets with destination port 9999 are redirected > with the help of IPTables to the local dtntunnel process, listening om > port 19999. That is why I run tcpdump on loopback interface. > > Receiver (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999 > 15:05:02.496366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq > 123995:131097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr > 11163718], length 7102 > 15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack > 131097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length = 0 > 15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], seq > 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724], > length 0 > 15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack > 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length = 0 > 15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], seq > 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684], > length 0 > 15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack 2= , > win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0 > > Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999 > 15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack > 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 0 > 15:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], seq > 127448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr > 11172887], length 2896 > 15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], seq > 130344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr > 11172887], length 752 > 15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack > 131097, win 386, options [nop,nop,TS val 11172898 ecr 11172887], length 0 > 15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], seq > 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], > length 0 > 15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack > 131098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], length 0 > > I believe that the cause of why the sending netcat without -w1 option > doesn't exit, is that it doesn't receive the final FIN from the dtntunnel= . > Can this issue be fixed with a small amount of work?**** > > 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > I would not even call this a problem. netcat is designed to be extremely > flexible and by default the sender keeps its connection open as long as t= he > other end does. On the other hand the receiver by default terminates afte= r > receipt of an EOF but there is a switch to keep listening if that is the > desired behavior. **** > > **** > > With DTN, your sending nc may do its thing while the destination node is > not even on line and then an hour later it becomes available and complete= s > the transmission to the receiving nc. In this scenario, the sender would > still be =93hung up=94 possibly indefinitely waiting for a terminating si= gnal. > **** > > **** > > I am not the DTN2 authHi ority, but, I don=92t see a change to dtntunnel = for > this as it would assume a specific usage that would probably break someon= e > else=92s usage (like mine J).**** > > **** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 05, 2012 2:56 AM > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > Thanks for your help, nc -w1 did the trick. But shouldn't dtntunnel > behavior be changed? The receiving dtntunnel could signal the sending > dtntunnel, that the listening nc has disconnected, so that the sending > dtntunnel close its connection to the sending nc. Or is this solely the > problem of nc?**** > > 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not > impact the behavior of dtntunnel.**** > > I just ran your test without the iptables redirect and I see the same > behavior.**** > > I believe that the sender exits if you go directly from nc to nc because > the receiver exits when it gets an end of file.**** > > You can add a =93-w 1=94 option to the sender nc so that it will timeout = and > exit after stdin is idle for 1 second.**** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *O= n > Behalf Of *ssireskin@gmail.com > *Sent:* Tuesday, December 04, 2012 11:13 AM > *To:* dtn-users@irtf.org > *Subject:* [dtn-users] A problem with dtntunnel**** > > **** > > Hello all! > > I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a > problem when using netcat (nc) with dtntunnel. > On the sender node I run "cat /etc/passwd | nc receiver_ip". On the > receiver node I run "nc -l 9999". With the help > of Iptables port 9999 gets redirected to the port 19999, which is listene= d > by dtntunnel. > > The file /etc/passwd is successfully delivered to the receiver and is > shown on the screen. After this the receiving nc exists. > No problem here. However, nc on the sender node doesn't exit after it > sends the file. It continues to run forever. When I > run nc in the opposite direction, I get the same problem - the sending nc > doesn't exit. > > My configuration on both nodes is symmetric: > iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --t= o > $LOCAL_HOST:19999 > dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/n= c > -d > dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d > > dtnping works ok in both directions. > > Please give me any advice. > > With best regards, > Sergey Sireskin **** > > > > > -- > Kindest Regards > > Sergey Sireskin > FGUP CNII EISU**** > > > > > -- > Kindest Regards > > Sergey Sireskin > FGUP CNII EISU**** > > > > > -- > Best regards, > Sergey Sireskin > > **** > --=20 Best regards, Sergey Sireskin --047d7b2ede33cd8ac104d0ad570a Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David,

I meant exactly the same position before the end of the wh= ile loop, just described it incorrect.

Please let me know if you fin= d the solution for this problem.

Best regards,
Sergey

2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC= SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

I was narrowing in on the same track... The FIN = results in a returned length of zero on the read and errno is also zero so = you have to key on the return length which is already in the code.

=A0<= /p>

You may have added a c= ouple of lines of debug code and it should be inserted just before the end = of the while(1) loop at line 491 in the latest version on Sourceforge.

=A0<= /p>

I am seeing the sendin= g socket stay in the TIME_WAIT mode even if I add a call to sock_.shutdown(= SHUT_RDWR) before closing the socket;

Getting close I think.=

= =A0

The 0.0.0.0 was because I= did not have anything at the other end of the tunnel and just stuck some e= asy numbers in there.

Thanks,
DZ

= =A0

David Zoller

COLSA Corporation

Marshall Space Flight Cente= r

=A0<= /p>

From: ssireskin@gmail.= com [mailto:ss= ireskin@gmail.com]
Sent: Wednesday, December 12, 2012 11:06 AM


= To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.o= rg
Subject: Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,

I added the= following code at the line 492 of TCPTunnel.cc and it seems that it helped= .

=A0=A0=A0 if (sock_eof) {
=A0=A0=A0 =A0=A0=A0 sock_.close();
=A0= =A0=A0 =A0=A0=A0 goto done;
=A0=A0=A0 }

The senders, both nc and = iperf, exit after having finished sending the data. There are
no more so= ckets in CLOSE_WAIT state. However I am completely unsure whether
my code is correct and is in correct place.

Now I have encountered a= nother problem. When iperf test is run with 100 parallel flows,
some dtn= tunnel connections are not removed from dtntunel's connection table, an= d the
related sockets are not closed. I suppose this happens because my code addi= tion is far
from perfect.

And one more question, what does 0.0.0.= 0 mean in -T tunnel specification? Is it just an
example host, or 0.0.0.= 0 has some special meaning for dtntunnel?

2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SE= RVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

I have duplicated what you are reporti= ng and have to retract my earlier statement saying that this is the netcat = design and not a problem J

The sender netcat attempt= s to close the socket when it gets the [EOF] and not in response to the rec= eiver closing its end.

=A0<= /p>

=A0

netcat sender:<= /b>

cat /etc/passwd | nc x.x.x.x 12345<= /u>

=A0<= /p>

scenario 1 - netcat= receiver with the keep listener option:

=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 nc -k -l 12345

=B7=A0=A0=A0= =A0=A0=A0=A0=A0 Sender terminates and rec= eiver stays alive and listening for a re-connect

=B7=A0=A0=A0=A0=A0=A0=A0=A0 FIN is sent i= n both directions initiated by the sender

=A0<= /p>

scenario 2 - dtntun= nel receiver:

=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 dtntunnel -t -T 12345:0.0.0.0:54321 dtn://desteid/xxxx

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Sender does n= ot terminate

=B7=A0=A0=A0=A0=A0=A0=A0=A0 FIN is sent f= rom sender to receiver but not the other way

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Sender socket= is in state FIN_WAIT2

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Receiver sock= et is in state CLOSE_WAIT

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Sender does not terminate

=B7=A0=A0=A0=A0=A0=A0=A0=A0 FIN is sent f= rom sender to receiver but not the other way

=B7=A0=A0=A0=A0=A0=A0=A0=A0 New Sender so= cket is in state FIN_WAIT2 and the first one has timed out and died<= u>

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Both Receiver= sockets are in state CLOSE_WAIT

=B7=A0=A0=A0=A0=A0=A0=A0=A0 2 FINs are sent from the receiver =96 1 to e= ach of the sockets (even though the other ends have expired)<= u>

=A0<= /p>

=A0

I believe the issue would= be in the oasys IPSocket or possibly the IPClient. I=92ll have a look at i= t unless there is a low-level socket expert out there that wants to give it= a go=85

=A0=

Best regards,

DZ

=A0

David Zoller<= u>

COLSA Corporation=

Marshall Space Flight Cente= r

=A0

From: ssireskin@gmail.com<= /a> [mailto:ssires= kin@gmail.com]
Sent: Wednesday, December 12, 2012 4:38 AM

<= div>


To: Zoller, David A. (MSFC-EO60)[HOSC= SERVICES CONTRACT]

Subject: Re: [dtn-users] A problem with dtntunnel

<= /div>

=A0

Hi David,

I have inves= tigated that issue a little. I have found, that no matter whether netcat is= run with or without the -w1 option, something
strange happens. According to netstat -ntp on the sending node, dtntunnel r= emains in CLOSE_WAIT state after nc finishes sending
data. I have run t= cpdump on both sender and receiver, and it showed that final FIN from the r= eceiver doesn't reach the sender.
This looked like a bug in dtntunnel to me. I decided to re-check with iperf= , and again, tcpdump showed the same problem.

Here are the last line= s of the tcpdump. Remember that in my setup all outgoing TCP packets with d= estination port 9999 are redirected
with the help of IPTables to the local dtntunnel process, listening om port= 19999. That is why I run tcpdump on loopback interface.

Receiver (1= 92.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.496366= IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 123995:131097,= ack 1, win 257, options [nop,nop,TS val 11163723 ecr 11163718], length 710= 2
15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack = 131097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0<= br>15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], = seq 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724],= length 0
15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack = 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length 0<= br>15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], = seq 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684]= , length 0
15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack = 2, win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0
Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack = 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 015:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], s= eq 127448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172= 887], length 2896
15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], se= q 130344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 111728= 87], length 752
15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.433= 60: Flags [.], ack 131097, win 386, options [nop,nop,TS val 11172898 ecr 11= 172887], length 0
15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], se= q 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], le= ngth 0
15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags= [.], ack 131098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], = length 0

I believe that the cause of why the sending netcat without -w1 option d= oesn't exit, is that it doesn't receive the final FIN from the dtnt= unnel.
Can this issue be fixed with a small amount of work?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SER= VICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

I would not even call this a problem. = netcat is designed to be extremely flexible and by default the sender keeps= its connection open as long as the other end does. On the other hand the r= eceiver by default terminates after receipt of an EOF but there is a switch= to keep listening if that is the desired behavior.

=A0<= /p>

With DTN, your sending= nc may do its thing while the destination node is not even on line and the= n an hour later it becomes available and completes the transmission to the = receiving nc. In this scenario, the sender would still be =93hung up=94 pos= sibly indefinitely waiting for a terminating signal. <= /p>

=A0<= /p>

I am not the DTN2 auth= Hi ority, but, I don=92t see a change to dtntunnel for this as it would ass= ume a specific usage that would probably break someone else=92s usage (like= mine J).

=A0=

Best regards,

DZ

=A0

David Zoller<= u>

COLSA Corporation=

Marshall Space Flight= Center

=A0

From: ssireskin@gmail.com [= mailto:ssireskin@g= mail.com]
Sent: Wednesday, December 05, 2012 2:56 AM
To: Zoller, Dav= id A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org
Subject:<= /b> Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,

Thanks for your help, nc = -w1 did the trick. But shouldn't dtntunnel behavior be changed? The rec= eiving dtntunnel could signal the sending dtntunnel, that the listening nc = has disconnected, so that the sending dtntunnel close its connection to the= sending nc. Or is this solely the problem of nc?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SER= VICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

On RHEL 5.7, I am running dtn-2.9.0 pl= us modifications that should not impact the behavior of dtntunnel.

I just ran your test with= out the iptables redirect and I see the same behavior.=

I believe that the sender= exits if you go directly from nc to nc because the receiver exits when it = gets an end of file.

You can add a =93-w 1=94 = option to the sender nc so that it will timeout and exit after stdin is idl= e for 1 second.

Best regards,

DZ=

=A0<= /p>

David Zoller

COLSA Corporation=

HOSC / C107= =A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: dtn-users-bounce= s@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Tuesday, December 04, 2012 11:13 AM
To: dtn-users@irtf.org
Su= bject: [dtn-users] A problem with dtntunnel

=A0

Hel= lo all!

I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am= having a problem when using netcat (nc) with dtntunnel.
On the sender n= ode I run "cat /etc/passwd | nc receiver_ip". On the receiver nod= e I run "nc -l 9999". With the help
of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel.

The file /etc/passwd is successfully delivered to the = receiver and is shown on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it s= ends the file. It continues to run forever. When I
run nc in the opposi= te direction, I get the same problem - the sending nc doesn't exit.

My configuration on both nodes is symmetric:
iptables -t nat -A OUTP= UT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to $LOCAL_HOST:19999
dt= ntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d=
dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d

dtnping works o= k in both directions.

Please give me any advice.

With best r= egards,
Sergey Sireskin




--
Kindest Regards

Sergey Sireskin
FGUP CNII EISU=




--
Kindest Regards

Sergey Sireskin
= FGUP CNII EISU




--
B= est regards,
Sergey Sireskin




--
Best regards,
Sergey Sireskin<= br>

--047d7b2ede33cd8ac104d0ad570a-- From david.a.zoller@nasa.gov Wed Dec 12 17:23:37 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BC90121F8A92 for ; Wed, 12 Dec 2012 17:23:36 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.237 X-Spam-Level: X-Spam-Status: No, score=-6.237 tagged_above=-999 required=5 tests=[AWL=-0.241, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_MED=-4, WEIRD_PORT=0.001] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h6T4RrQjtLKO for ; Wed, 12 Dec 2012 17:23:24 -0800 (PST) Received: from ndjsnpf01.ndc.nasa.gov (ndjsnpf01.ndc.nasa.gov [198.117.1.121]) by ietfa.amsl.com (Postfix) with ESMTP id 1B99C21F8A78 for ; Wed, 12 Dec 2012 17:23:21 -0800 (PST) Received: from ndmsppt05.ndc.nasa.gov (ndmsppt05.ndc.nasa.gov [198.117.0.104]) by ndjsnpf01.ndc.nasa.gov (Postfix) with ESMTP id 38F0B328175; Wed, 12 Dec 2012 19:23:17 -0600 (CST) Received: from ndmshub06.ndc.nasa.gov (ndmshub06.ndc.nasa.gov [198.117.2.165]) by ndmsppt05.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBD1NG93002783; Wed, 12 Dec 2012 19:23:16 -0600 Received: from NDMSSCC05.ndc.nasa.gov ([198.117.2.174]) by ndmshub06.ndc.nasa.gov ([198.117.2.165]) with mapi; Wed, 12 Dec 2012 19:23:16 -0600 From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "ssireskin@gmail.com" Date: Wed, 12 Dec 2012 19:23:14 -0600 Thread-Topic: [dtn-users] A problem with dtntunnel Thread-Index: Ac3YpETfPmGFZ7X5TPCt5mbdHdv8hgAJz6lQ Message-ID: <04E3D99A62496240BCD6A576813E6E31E0BDECDC13@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD834@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD996@NDMSSCC05.ndc.nasa.gov> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_04E3D99A62496240BCD6A576813E6E31E0BDECDC13NDMSSCC05ndcn_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-12_10:2012-12-13, 2012-12-12, 1970-01-01 signatures=0 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 01:23:38 -0000 --_000_04E3D99A62496240BCD6A576813E6E31E0BDECDC13NDMSSCC05ndcn_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Sergey, Research indicates that the TIME_WAIT state is a feature of TCP to prevent = delayed packets for connection from being delivered to a later connection b= etween the same hosts. [http://tools.ietf.org/html/draft-faber-time-wait-av= oidance-00] Based on that, the added "if (sock_eof)" block can be removed and the while= condition can be changed to "while (!sock_eof)" and you get the same resul= t. But, looking at the intent of the code and at the oasys::IPSocket, it appea= rs that it is possible to establish a socket connection and then only close= the read side while keeping the write side open. I've never run across a s= ocket used in that manner but that is what it looks like to me. So, the TCP= Tunnel::Connection::run while loop allows for the read side to close and c= ontinue transmitting bundle payloads on the write side until it gets a bund= le with the EOF bit set. If there are no bundles coming back the other way = as is the case here then the while loop never exits which keeps the netcat = connection open. So, based on that, I think a better solution would be to implement an optio= n that configures dtntunnel to close the connection when the read side term= inates. (Someone in the know can weigh in) Your solution is probably suffic= ient for your test. -- As to connections not being removed from the table and sockets not being cl= osed... I set up a test to kick off 100 netcats on an 11MB file so that the connect= ion would be open for several seconds: #!/bin/bash ctr=3D0 while [ $ctr -lt 100 ]; do nc -w 10 x.x.x.x 12345 < data11mg & let ctr=3Dctr+1 done I also added some debug code to print out a connection accepted counter and= closed counter as they are accepted and closed. After 30 connections, the first close kicks in and then there is a mix of c= onnects and closes until finished. All of the connections were closed and I did not have any sockets left open= . On my first attempt I exceeded my payload quota and started seeing "error s= ending bundle: 141" messages so I don't think that is the issue you are run= ning into. You might try putting in similar debug to see if you can determine what is = happening. Hope this helps, DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: ssireskin@gmail.com [mailto:ssireskin@gmail.com] Sent: Wednesday, December 12, 2012 2:07 PM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, I meant exactly the same position before the end of the while loop, just de= scribed it incorrect. Please let me know if you find the solution for this problem. Best regards, Sergey 2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I was narrowing in on the same track... The FIN results in a returned lengt= h of zero on the read and errno is also zero so you have to key on the retu= rn length which is already in the code. You may have added a couple of lines of debug code and it should be inserte= d just before the end of the while(1) loop at line 491 in the latest versio= n on Sourceforge. I am seeing the sending socket stay in the TIME_WAIT mode even if I add a c= all to sock_.shutdown(SHUT_RDWR) before closing the socket; Getting close I think. The 0.0.0.0 was because I did not have anything at the other end of the tun= nel and just stuck some easy numbers in there. Thanks, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Wednesday, December 12, 2012 11:06 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, I added the following code at the line 492 of TCPTunnel.cc and it seems tha= t it helped. if (sock_eof) { sock_.close(); goto done; } The senders, both nc and iperf, exit after having finished sending the data= . There are no more sockets in CLOSE_WAIT state. However I am completely unsure whether my code is correct and is in correct place. Now I have encountered another problem. When iperf test is run with 100 par= allel flows, some dtntunnel connections are not removed from dtntunel's connection table= , and the related sockets are not closed. I suppose this happens because my code addi= tion is far from perfect. And one more question, what does 0.0.0.0 mean in -T tunnel specification? I= s it just an example host, or 0.0.0.0 has some special meaning for dtntunnel? 2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I have duplicated what you are reporting and have to retract my earlier sta= tement saying that this is the netcat design and not a problem :) The sender netcat attempts to close the socket when it gets the [EOF] and n= ot in response to the receiver closing its end. netcat sender: cat /etc/passwd | nc x.x.x.x 12345 scenario 1 - netcat receiver with the keep listener option: nc -k -l 12345 * Sender terminates and receiver stays alive and listening for a re= -connect * FIN is sent in both directions initiated by the sender scenario 2 - dtntunnel receiver: dtntunnel -t -T 12345:0.0.0.0:54321 d= tn://desteid/xxxx * Sender does not terminate * FIN is sent from sender to receiver but not the other way * Sender socket is in state FIN_WAIT2 * Receiver socket is in state CLOSE_WAIT Kill the sender and repeat... * Sender does not terminate * FIN is sent from sender to receiver but not the other way * New Sender socket is in state FIN_WAIT2 and the first one has tim= ed out and died * Both Receiver sockets are in state CLOSE_WAIT Kill the dtntunnel receiver... * 2 FINs are sent from the receiver - 1 to each of the sockets (eve= n though the other ends have expired) I believe the issue would be in the oasys IPSocket or possibly the IPClient= . I'll have a look at it unless there is a low-level socket expert out ther= e that wants to give it a go... Best regards, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Wednesday, December 12, 2012 4:38 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Subject: Re: [dtn-users] A problem with dtntunnel Hi David, I have investigated that issue a little. I have found, that no matter wheth= er netcat is run with or without the -w1 option, something strange happens. According to netstat -ntp on the sending node, dtntunnel r= emains in CLOSE_WAIT state after nc finishes sending data. I have run tcpdump on both sender and receiver, and it showed that fi= nal FIN from the receiver doesn't reach the sender. This looked like a bug in dtntunnel to me. I decided to re-check with iperf= , and again, tcpdump showed the same problem. Here are the last lines of the tcpdump. Remember that in my setup all outgo= ing TCP packets with destination port 9999 are redirected with the help of IPTables to the local dtntunnel process, listening om port= 19999. That is why I run tcpdump on loopback interface. Receiver (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999 15:05:02.496366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 12= 3995:131097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr 11163718]= , length 7102 15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131= 097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0 15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], seq 13= 1097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724], lengt= h 0 15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131= 098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length 0 15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], seq 1,= ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684], leng= th 0 15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack 2, = win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0 Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999 15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 127= 449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 0 15:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], seq 12= 7448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887],= length 2896 15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], seq 1= 30344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887]= , length 752 15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131= 097, win 386, options [nop,nop,TS val 11172898 ecr 11172887], length 0 15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], seq 1= 31096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], lengt= h 0 15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131= 098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], length 0 I believe that the cause of why the sending netcat without -w1 option doesn= 't exit, is that it doesn't receive the final FIN from the dtntunnel. Can this issue be fixed with a small amount of work? 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I would not even call this a problem. netcat is designed to be extremely fl= exible and by default the sender keeps its connection open as long as the o= ther end does. On the other hand the receiver by default terminates after r= eceipt of an EOF but there is a switch to keep listening if that is the des= ired behavior. With DTN, your sending nc may do its thing while the destination node is no= t even on line and then an hour later it becomes available and completes th= e transmission to the receiving nc. In this scenario, the sender would stil= l be "hung up" possibly indefinitely waiting for a terminating signal. I am not the DTN2 authHi ority, but, I don't see a change to dtntunnel for = this as it would assume a specific usage that would probably break someone = else's usage (like mine :)). Best regards, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Wednesday, December 05, 2012 2:56 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Thanks for your help, nc -w1 did the trick. But shouldn't dtntunnel behavio= r be changed? The receiving dtntunnel could signal the sending dtntunnel, t= hat the listening nc has disconnected, so that the sending dtntunnel close = its connection to the sending nc. Or is this solely the problem of nc? 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not impa= ct the behavior of dtntunnel. I just ran your test without the iptables redirect and I see the same behav= ior. I believe that the sender exits if you go directly from nc to nc because th= e receiver exits when it gets an end of file. You can add a "-w 1" option to the sender nc so that it will timeout and ex= it after stdin is idle for 1 second. Best regards, DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: dtn-users-bounces@irtf.org [mailto= :dtn-users-bounces@irtf.org] On Behalf O= f ssireskin@gmail.com Sent: Tuesday, December 04, 2012 11:13 AM To: dtn-users@irtf.org Subject: [dtn-users] A problem with dtntunnel Hello all! I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a probl= em when using netcat (nc) with dtntunnel. On the sender node I run "cat /etc/passwd | nc receiver_ip". On the receive= r node I run "nc -l 9999". With the help of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel. The file /etc/passwd is successfully delivered to the receiver and is shown= on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it sends= the file. It continues to run forever. When I run nc in the opposite direction, I get the same problem - the sending nc d= oesn't exit. My configuration on both nodes is symmetric: iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to = $LOCAL_HOST:19999 dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc = -d dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d dtnping works ok in both directions. Please give me any advice. With best regards, Sergey Sireskin -- Kindest Regards Sergey Sireskin FGUP CNII EISU -- Kindest Regards Sergey Sireskin FGUP CNII EISU -- Best regards, Sergey Sireskin -- Best regards, Sergey Sireskin --_000_04E3D99A62496240BCD6A576813E6E31E0BDECDC13NDMSSCC05ndcn_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable


-- Best regards,
Sergey Sireskin


--f46d042fd8982112f104d11b9cde-- From david.a.zoller@nasa.gov Tue Dec 18 03:58:14 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BFBA821F886B for ; Tue, 18 Dec 2012 03:58:14 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.131 X-Spam-Level: X-Spam-Status: No, score=-6.131 tagged_above=-999 required=5 tests=[AWL=-0.133, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id viL6redl-1lO for ; Tue, 18 Dec 2012 03:58:11 -0800 (PST) Received: from ndmsnpf01.ndc.nasa.gov (ndmsnpf01.ndc.nasa.gov [198.117.0.121]) by ietfa.amsl.com (Postfix) with ESMTP id 72F7521F8866 for ; Tue, 18 Dec 2012 03:58:10 -0800 (PST) Received: from ndmsppt03.ndc.nasa.gov (ndmsppt03.ndc.nasa.gov [198.117.0.102]) by ndmsnpf01.ndc.nasa.gov (Postfix) with ESMTP id 74C32260101; Tue, 18 Dec 2012 05:58:09 -0600 (CST) Received: from ndmshub04.ndc.nasa.gov (ndmshub04-pub.ndc.nasa.gov [198.117.0.43]) by ndmsppt03.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBIBw9Dn013225; Tue, 18 Dec 2012 05:58:09 -0600 Received: from NDMSSCC05.ndc.nasa.gov ([198.117.2.174]) by ndmshub04.ndc.nasa.gov ([198.117.2.163]) with mapi; Tue, 18 Dec 2012 05:58:09 -0600 From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "ssireskin@gmail.com" Date: Tue, 18 Dec 2012 05:58:07 -0600 Thread-Topic: [dtn-users] A problem with dtntunnel Thread-Index: Ac3c8u/89VL2FRwpTUSHnNjugXQrygAInmkA Message-ID: <04E3D99A62496240BCD6A576813E6E31E0BDF5CAE6@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDF5C59C@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5C999@NDMSSCC05.ndc.nasa.gov> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_04E3D99A62496240BCD6A576813E6E31E0BDF5CAE6NDMSSCC05ndcn_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-18_01:2012-12-18, 2012-12-18, 1970-01-01 signatures=0 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2012 11:58:14 -0000 --_000_04E3D99A62496240BCD6A576813E6E31E0BDF5CAE6NDMSSCC05ndcn_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Sergey, Looking at the rest of the code, there are probably a couple of memory leak= s. The biggie is after the call to "send_bundle(b_xmit" on success b_xmit s= hould be deleted before setting it to NULL (near line 432). Similarly after= the send_bundle for the "error connecting" the APIBundle b should also be = deleted before going to done (near line 342). You're welcome, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gmail.com] Sent: Tuesday, December 18, 2012 1:40 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: David Zoller; dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Now it works like a charm, thank you very much! 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Sergey, That was my bad. In the patch below I moved the "delete b_recv;" line above= the "if (recv_hdr->eof_) {" block as a short cut to delete it before the p= ossible exit. Move it back down below the if block and you should be okay. Sorry, that was one of those possible loose ends I mentioned :), DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Monday, December 17, 2012 8:05 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: David Zoller; dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel David, The problem is probably not with the reordering. Can it be, that "if (recv_= hdr->eof_)" is not the right condition to close a connection? Or maybe it i= s not a sufficient condition? Which connection is recv_hdr about? Can it be= a local TCP connection on the receiving side (dtntunnel <-> nc)? If it is,= then IMO this condition becomes true when the receiving application doesn'= t send any data to the sending application, please see line 406 of the orig= inal code. 2012/12/17 ssireskin@gmail.com > David, I added this code and a log_debug inside the if-clause. According to the lo= g, this condition doesn't get true. A have attached my logs and tcpdumps. Please, see dtntunnel_T.log - it's a = receiving dtntunnel log. There are 4 bundles after a eof bundle. This doesn't happen with the origin= al dtntunnel. -- Best regards, Sergey Sireskin 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I don't have your setup and this has not been tested but give this a try. I= n TCPTunnel::Connection::handle_bundle in the while(1) loop add the test to= see if the next_seqno_ is a match: while (1) { iter =3D reorder_table_.find(next_seqno_); if (iter =3D=3D reorder_table_.end()) { break; } // add this code: // if seqno does not match then still a bundle missing if (iter->first !=3D next_seqno_) { break; } bundle =3D iter->second; log_debug("delivering %zu byte bundle with seqno %d (from reorder t= able)", bundle->payload_.len(), next_seqno_); reorder_table_.erase(iter); next_seqno_++; queue_.push_back(bundle); } David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: dtn-users-bounces@irtf.org [mailto= :dtn-users-bounces@irtf.org] On Behalf O= f ssireskin@gmail.com Sent: Monday, December 17, 2012 6:40 AM To: David Zoller Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Thanks for your patch, it works! :) Connection is cleanly terminated now. However, there still seems to be a problem. I run dtntunnel over a link wit= h high latency and about 30% packet loss. The receiving nc sometimes gets F= IN packet before it receives all the data. If I understand it right, on a l= ossy link bundles may come out of order. In this case the EOF bundle may co= me too early. I think that the reordering procedure in dtntunnel should be = reviewed. Just for information, pure TCP performs better that DTN over this= link. 2012/12/14 David Zoller > Hi Sergey, I can't stay away from a problem that is nagging at me so I made some time = this morning. Going back to the original TCPTunnel.cc code, when a bundle i= s received that has the EOF indication, an empty bundle with the same EOF f= lag set needs to be returned to close the loop (the second FIN). There may = still be some loose ends or clean up needed but try the code below at the e= nd of the while loop. I did not do a packet capture but I saw that if the receiving nc is not run= ning when you start the sender nc then the receiving dtntunnel logs an erro= r on the failed connect and returns an EOF bundle which closes the sender n= c. The EOF bundle could possibly be enhanced to include an error message. I agree that some applications could benefit from a "pretend to be a receiv= er" option with the default working as a bidirectional end-to-end TCP conne= ction. Thanks for dragging me along kicking and screaming on this lesson in DTN an= d TCP tunneling. I've only been using dtntunnel as a one way UDP transmitte= r. I see a good fit for the TCP tunnel in an upcoming project and I'll try = to make the pitch to use it. This is a pretty cool little app. Best regards, DZ 484 delete b_recv; 485 486 if (recv_hdr->eof_) { 487 log_info("bundle had eof bit set... closing connection"= ); 488 489 if ( !sock_eof ) { 490 sock_eof =3D true; 491 // send an empty bundle back since we did not initi= ate the close 492 dtn::APIBundle* b =3D new dtn::APIBundle(); 493 hdr.eof_ =3D 1; 494 hdr.seqno_ =3D ntohl(send_seqno++); 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, sizeof(h= dr)); 496 b->payload_.set_len(sizeof(hdr)); 497 int err; 498 if ((err =3D tunnel->send_bundle(b, &dest_eid_)) != =3D DTN_SUCCESS) { 499 log_err("error sending final socket closed pack= et bundle: %s", 500 dtn_strerror(err)); 501 tcptun_->kill_connection(this); 502 exit(1); 503 } 504 delete b; 505 } 506 sock_.close(); 507 goto done; 508 } 509 } 510 } 511 512 done: -- Best regards, Sergey Sireskin -- Best regards, Sergey Sireskin -- Best regards, Sergey Sireskin --_000_04E3D99A62496240BCD6A576813E6E31E0BDF5CAE6NDMSSCC05ndcn_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Sergey= ,

Looking at the rest of t= he code, there are probably a couple of memory leaks. The biggie is after t= he call to “send_bundle(b_xmit” on success= b_xmit should be deleted before setting it to NULL (near line 432). Simila= rly after the send_bundle for the “error connecting” the APIBun= dle b should also be deleted before going to done (near line 342).

You’re welcome,

DZ

 

David = Zoller

COLSA = Corporation

Ma= rshall Space Flight Center

 

From:= ssiresk= in@gmail.com [mailto:ssireskin@gmail.com]
Sent: Tuesday, Decembe= r 18, 2012 1:40 AM
To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES= CONTRACT]
Cc: David Zoller; dtn-users@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

 

Hi David,

Now it works like a charm, thank you very much!

2012/12/17 Zoller, David A. (MSFC-EO6= 0)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Sergey,

That was my bad. In = the patch below I moved the “delete b_recv;” line above the “if (recv_hdr->eof_) {” block as a short cut to delete it before the possible exit. Move it back= down below the if block and you should be okay.

Sorry, that was one of those possible loose ends I mentioned J,

DZ

 

David Zoller=

COLSA Corporation

<= span style=3D'font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F49= 7D'>HOSC / C107 

(Office: (256) 544-1820
*EMail: david.= a.zoller@nasa.gov

 

From: ssireskin@gmail.com [mailto:ssireskin@gmail.com]
Sent: Mon= day, December 17, 2012 8:05 AM
To: Zoller, David A. (MSFC-EO60)[H= OSC SERVICES CONTRACT]
Cc: David Zoller; dtn-users@irtf.org


Subject: Re: [dtn-users] A probl= em with dtntunnel

 

David,

The problem is probably not with the reordering. Can = it be, that "if (recv_hdr->eof_)" is not the right condition t= o close a connection? Or maybe it is not a sufficient condition? Which conn= ection is recv_hdr about? Can it be a local TCP connection on the receiving= side (dtntunnel <-> nc)? If it is, then IMO this condition becomes t= rue when the receiving application doesn't send any data to the sending app= lication, please see line 406 of the original code.

2012/12/17 ssir= eskin@gmail.com <ssireskin@gmail.com>

David,

I ad= ded this code and a log_debug inside the if-clause. According to the log, t= his condition doesn't get true.
A have attached my logs and tcpdumps. Pl= ease, see dtntunnel_T.log - it's a receiving dtntunnel log.
There are 4 = bundles after a eof bundle. This doesn't happen with the original dtntunnel= .


--
Best regards,
Sergey Sireskin

2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVIC= ES CONTRACT] <david.a.zoller@nasa.gov>

 

Hi Sergey,

I don’t have your setup and this has not been te= sted but give this a try. In TCPTunnel::Connection::handle_bundle in the wh= ile(1) loop add the test to see if the next_seqno_ is a match:<= /o:p>

 

    while (1) {

        iter =3D reorder_table_.= find(next_seqno_);

        if= (iter =3D=3D reorder_table_.end()) {

    &n= bsp;       break;

  &nbs= p;     }

 

= &nb= sp;       // add this code:=

        // if seqno does not mat= ch then still a bundle missing

   &= nbsp;    if (iter->first !=3D next_seqno_) {<= /o:p>

          &n= bsp; break;

       &= nbsp;}

 

      &nb= sp; bundle =3D iter->second;

     &n= bsp;  log_debug("delivering %zu byte bundle with seqno %d (from r= eorder table)",

       &= nbsp;          bundle->payl= oad_.len(), next_seqno_);

   

 = ;       reorder_table_.erase(iter);

        next_seqno_++;

   

       &= nbsp;queue_.push_back(bundle);

    } &n= bsp;

 

 

 <= /p>

 

David Zoller

COLSA Corporation

<= span style=3D'font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F49= 7D'>HOSC / C107 

(Office: (256) 544-1820
*EMail: david.= a.zoller@nasa.gov

 

From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.o= rg] On Behalf Of ssireskin@gmail.com
Sent: Monday, December 17, 20= 12 6:40 AM
To: David Zoller


= Cc: dtn-user= s@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel<= o:p>

 

Hi David,

<= div>



Thanks for your patch, it works! :) Connection is c= leanly terminated now.

However, there still seems to be a problem. I= run dtntunnel over a link with high latency and about 30% packet loss. The= receiving nc sometimes gets FIN packet before it receives all the data. If= I understand it right, on a lossy link bundles may come out of order. In t= his case the EOF bundle may come too early. I think that the reordering pro= cedure in dtntunnel should be reviewed. Just for information, pure TCP perf= orms better that DTN over this link.

2012/12/14 David Zoller <zollerd@gmail.com>

Hi Se= rgey,

I can't stay away from a problem that= is nagging at me so I made some time this morning. Going back to the origi= nal TCPTunnel.cc code, when a bundle is received that has the EOF indicatio= n, an empty bundle with the same EOF flag set needs to be returned to close= the loop (the second FIN). There may still be some loose ends or clean up = needed but try the code below at the end of the while loop.

<= /div>

 

I did not do a pack= et capture but I saw that if the receiving nc is not running when you start= the sender nc then the receiving dtntunnel logs an error on the failed con= nect and returns an EOF bundle which closes the sender nc. The EOF bundle c= ould possibly be enhanced to include an error message.

=

 

I agree that some applic= ations could benefit from a "pretend to be a receiver" option wit= h the default working as a bidirectional end-to-end TCP connection.

 

Thanks for = dragging me along kicking and screaming on this lesson in DTN and TCP tunne= ling. I've only been using dtntunnel as a one way UDP transmitter. I see a = good fit for the TCP tunnel in an upcoming project and I'll try to make the= pitch to use it. This is a pretty cool little app.

Best regards,

DZ

 

 

 

 

 

484 &nbs= p;           delete b_recv;

485 

486      = ;       if (recv_hdr->eof_) {

487                 log_info(&= quot;bundle had eof bit set... closing connection");

488 

489     =             if ( !sock_eof ) {

=

490               &nbs= p;     sock_eof =3D true;

491 &nb= sp;                   // send = an empty bundle back since we did not initiate the close

492                 &= nbsp;   dtn::APIBundle* b =3D new dtn::APIBundle();

493                 &= nbsp;   hdr.eof_ =3D 1;

494   &nb= sp;                 hdr.seqno_ =3D = ntohl(send_seqno++);

495     &nbs= p;               memcpy(b->payload_.b= uf(sizeof(hdr)), &hdr, sizeof(hdr));

4= 96                     b-= >payload_.set_len(sizeof(hdr));

497 &nbs= p;                   int err;<= o:p>

498           &nbs= p;         if ((err =3D tunnel->send_bundle(b, &= dest_eid_)) !=3D DTN_SUCCESS) {

499   =                      = ; log_err("error sending final socket closed packet bundle: %s",<= o:p>

500           &nbs= p;                     dt= n_strerror(err));

501       =                   tcptun_->= kill_connection(this);

502     &= nbsp;                   exit(1= );

503           &= nbsp;         }

504 &nb= sp;                   delete b= ;

505           &n= bsp;     }

506     &nbs= p;           sock_.close();

507                 goto = done;

508          = ;   }

509         = }

510     }

<= div>

511 

512  done:

 



--
Best regards,
Sergey Sireskin

 




--
Best regards,
Sergey Sireskin




--
Best regards,
Sergey Sires= kin

= --_000_04E3D99A62496240BCD6A576813E6E31E0BDF5CAE6NDMSSCC05ndcn_-- From ssireskin@gmail.com Tue Dec 18 04:07:49 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BA55521F89E5 for ; Tue, 18 Dec 2012 04:07:49 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.04 X-Spam-Level: X-Spam-Status: No, score=-3.04 tagged_above=-999 required=5 tests=[AWL=-0.042, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iMkoUT4nzcjQ for ; Tue, 18 Dec 2012 04:07:48 -0800 (PST) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by ietfa.amsl.com (Postfix) with ESMTP id F2E6121F8932 for ; Tue, 18 Dec 2012 04:07:47 -0800 (PST) Received: by mail-vc0-f182.google.com with SMTP id fy27so698618vcb.41 for ; Tue, 18 Dec 2012 04:07:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Ko9DbEjO6R8iD2WLam3LytMWopQtjMTe3DSPb2kc5to=; b=iMTQjAOkUzwqyEA24vFtBTmrVhbaT4EsUn13F0dWQjNDqhfNtvqSyztXoZ1JQFsAVh ZuqcdKeoK4Glh/PzD3bMN6CttXVKXL4EvF+Y5O2rd8Q8zKE/2i4+2IGV/dkhs19PJKlS N/7XRMHHeMEV0lACElXdiFTezH42X0hWWGXjzMR9qmzyXYvfcrdTpJwmMVdWySiA5DSv wC0EUA7theTXpTguz+WUpUHmMM/8i6XokU1p9ocHIxLzUuAAuEoKcJW5TsM+pHB6w4Xv eVYmG3191Kz070m0LXZa0gidfiSougf1QOdIEs0vC4D5d+M8uO8POrC6qO8i89Z79v7n u/IQ== MIME-Version: 1.0 Received: by 10.220.157.71 with SMTP id a7mr2584061vcx.19.1355832467278; Tue, 18 Dec 2012 04:07:47 -0800 (PST) Received: by 10.58.219.65 with HTTP; Tue, 18 Dec 2012 04:07:47 -0800 (PST) In-Reply-To: <04E3D99A62496240BCD6A576813E6E31E0BDF5CAE6@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDF5C59C@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5C999@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5CAE6@NDMSSCC05.ndc.nasa.gov> Date: Tue, 18 Dec 2012 15:07:47 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/alternative; boundary=f46d043c7e98964b3b04d11f58c9 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2012 12:07:50 -0000 --f46d043c7e98964b3b04d11f58c9 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David, Thanks for the hint. I have also noticed that memory usage of dtntunnel noticeably grows while running iperf with multiple flows. Are you going to submit all the fixes to mercurial? --=20 Best regards, Sergey Sireskin 2012/12/18 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < david.a.zoller@nasa.gov> > Hi Sergey,**** > > Looking at the rest of the code, there are probably a couple of memory > leaks. The biggie is after the call to =93*send_bundle(b_xmit*=94 on succ= ess > b_xmit should be deleted before setting it to NULL (near line 432). > Similarly after the send_bundle for the =93error connecting=94 the APIBun= dle b > should also be deleted before going to done (near line 342).**** > > You=92re welcome,**** > > DZ**** > > ** ** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > ** ** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Tuesday, December 18, 2012 1:40 AM > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* David Zoller; dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > ** ** > > Hi David, > > Now it works like a charm, thank you very much!**** > > 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Sergey,**** > > That was my bad. In the patch below I moved the *=93delete b_recv;=94* li= ne > above the *=93if (recv_hdr->eof_) {=94* block as a short cut to delete it > before the possible exit. Move it back down below the if block and you > should be okay.**** > > Sorry, that was one of those possible loose ends I mentioned J,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Monday, December 17, 2012 8:05 AM > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* David Zoller; dtn-users@irtf.org**** > > > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > David, > > The problem is probably not with the reordering. Can it be, that "if > (recv_hdr->eof_)" is not the right condition to close a connection? Or > maybe it is not a sufficient condition? Which connection is recv_hdr abou= t? > Can it be a local TCP connection on the receiving side (dtntunnel <-> nc)= ? > If it is, then IMO this condition becomes true when the receiving > application doesn't send any data to the sending application, please see > line 406 of the original code.**** > > 2012/12/17 ssireskin@gmail.com **** > > David, > > I added this code and a log_debug inside the if-clause. According to the > log, this condition doesn't get true. > A have attached my logs and tcpdumps. Please, see dtntunnel_T.log - it's = a > receiving dtntunnel log. > There are 4 bundles after a eof bundle. This doesn't happen with the > original dtntunnel. **** > > > -- > Best regards, > Sergey Sireskin**** > > 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > **** > > Hi Sergey,**** > > I don=92t have your setup and this has not been tested but give this a tr= y. > In TCPTunnel::Connection::handle_bundle in the while(1) loop add the test > to see if the next_seqno_ is a match:**** > > **** > > while (1) {**** > > iter =3D reorder_table_.find(next_seqno_);**** > > if (iter =3D=3D reorder_table_.end()) {**** > > break;**** > > }**** > > **** > > // add this code:**** > > // if seqno does not match then still a bundle missing**** > > if (iter->first !=3D next_seqno_) {**** > > break;**** > > }**** > > **** > > bundle =3D iter->second;**** > > log_debug("delivering %zu byte bundle with seqno %d (from reorder > table)",**** > > bundle->payload_.len(), next_seqno_);**** > > **** > > reorder_table_.erase(iter);**** > > next_seqno_++;**** > > **** > > queue_.push_back(bundle);**** > > } **** > > **** > > **** > > **** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *O= n > Behalf Of *ssireskin@gmail.com > *Sent:* Monday, December 17, 2012 6:40 AM > *To:* David Zoller**** > > > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David,**** > > > > Thanks for your patch, it works! :) Connection is cleanly terminated now. > > However, there still seems to be a problem. I run dtntunnel over a link > with high latency and about 30% packet loss. The receiving nc sometimes > gets FIN packet before it receives all the data. If I understand it right= , > on a lossy link bundles may come out of order. In this case the EOF bundl= e > may come too early. I think that the reordering procedure in dtntunnel > should be reviewed. Just for information, pure TCP performs better that D= TN > over this link.**** > > 2012/12/14 David Zoller **** > > Hi Sergey,**** > > I can't stay away from a problem that is nagging at me so I made some tim= e > this morning. Going back to the original TCPTunnel.cc code, when a bundle > is received that has the EOF indication, an empty bundle with the same EO= F > flag set needs to be returned to close the loop (the second FIN). There m= ay > still be some loose ends or clean up needed but try the code below at the > end of the while loop.**** > > **** > > I did not do a packet capture but I saw that if the receiving nc is not > running when you start the sender nc then the receiving dtntunnel logs an > error on the failed connect and returns an EOF bundle which closes the > sender nc. The EOF bundle could possibly be enhanced to include an error > message.**** > > **** > > I agree that some applications could benefit from a "pretend to be a > receiver" option with the default working as a bidirectional end-to-end T= CP > connection.**** > > **** > > Thanks for dragging me along kicking and screaming on this lesson in DTN > and TCP tunneling. I've only been using dtntunnel as a one way UDP > transmitter. I see a good fit for the TCP tunnel in an upcoming project a= nd > I'll try to make the pitch to use it. This is a pretty cool little app.**= * > * > > Best regards,**** > > DZ**** > > **** > > **** > > **** > > **** > > **** > > 484 delete b_recv;**** > > 485 **** > > 486 if (recv_hdr->eof_) {**** > > 487 log_info("bundle had eof bit set... closing > connection");**** > > 488 **** > > 489 if ( !sock_eof ) {**** > > 490 sock_eof =3D true;**** > > 491 // send an empty bundle back since we did not > initiate the close**** > > 492 dtn::APIBundle* b =3D new dtn::APIBundle();**** > > 493 hdr.eof_ =3D 1;**** > > 494 hdr.seqno_ =3D ntohl(send_seqno++);**** > > 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, > sizeof(hdr));**** > > 496 b->payload_.set_len(sizeof(hdr));**** > > 497 int err;**** > > 498 if ((err =3D tunnel->send_bundle(b, &dest_eid_)) = !=3D > DTN_SUCCESS) {**** > > 499 log_err("error sending final socket closed > packet bundle: %s",**** > > 500 dtn_strerror(err));**** > > 501 tcptun_->kill_connection(this);**** > > 502 exit(1);**** > > 503 }**** > > 504 delete b;**** > > 505 }**** > > 506 sock_.close();**** > > 507 goto done;**** > > 508 }**** > > 509 }**** > > 510 }**** > > 511 **** > > 512 done:**** > > **** > > --f46d043c7e98964b3b04d11f58c9 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David,

Thanks for the hint. I have also noticed that memory usage= of dtntunnel noticeably grows while running iperf with multiple flows.
= Are you going to submit all the fixes to mercurial?

--
Best reg= ards,
Sergey Sireskin


2012/12/18 Zoller, Da= vid A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov<= /a>>

Hi Sergey,=

Looking at the rest of th= e code, there are probably a couple of memory leaks. The biggie is after th= e call to =93send_bundle(b_xmit=94 on success b_xmit should be deleted before setting it = to NULL (near line 432). Similarly after the send_bundle for the =93error c= onnecting=94 the APIBundle b should also be deleted before going to done (n= ear line 342).

You=92re welcome,<= u>

DZ

=A0<= /p>

David Zoller

COLSA Corporation

Marshall Space Flight= Center

=A0<= /p>

From: ssireskin@gmail.com= [mailto:ssireskin= @gmail.com]
Sent: Tuesday, December 18, 2012 1:40 AM


To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]=
Cc: David Zoller; dtn-users@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,
Now it works like a charm, thank you very much!

2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SE= RVICES CONTRACT] <david.a.zoller@nasa.gov>

Sergey,

That was my bad. In the patch below I mov= ed the =93delete b_recv;=94 line above the =93if (recv_hdr->eof= _) {=94 block as a short cut to delet= e it before the possible exit. Move it back down below the if block and you= should be okay.

Sorry, that was one of th= ose possible loose ends I mentioned J,

DZ

=A0<= u>

David Zoller<= u>

COLSA Corporation=

HOSC / C107=A0

(Office: (256)=A0544-1= 820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: ssireskin@gmail.= com [mailto:ss= ireskin@gmail.com]
Sent: Monday, December 17, 2012 8:05 AM
To: Zoller, David = A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: David Zoller; dtn-users@irtf.org


Subject: Re: [dtn-users] A prob= lem with dtntunnel

=A0

David,

The problem is probably not with the reordering. Can it be, t= hat "if (recv_hdr->eof_)" is not the right condition to close = a connection? Or maybe it is not a sufficient condition? Which connection i= s recv_hdr about? Can it be a local TCP connection on the receiving side (d= tntunnel <-> nc)? If it is, then IMO this condition becomes true when= the receiving application doesn't send any data to the sending applica= tion, please see line 406 of the original code.

2012/12/17 ssireskin@gmail.com <ssireskin@gmail.com>

David,

I added this code and a log_debug inside the if-clause. Acco= rding to the log, this condition doesn't get true.
A have attached m= y logs and tcpdumps. Please, see dtntunnel_T.log - it's a receiving dtn= tunnel log.
There are 4 bundles after a eof bundle. This doesn't happen with the or= iginal dtntunnel.


--
Best regards,
Sergey Sireskin

2012/12/17 Zoller, David A. (MSFC-EO60)[H= OSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

=A0

Hi Sergey,

I don=92t have your setup and this has not been = tested but give this a try. In TCPTunnel::Connection::handle_bundle in the = while(1) loop add the test to see if the next_seqno_ is a match:<= /u>

=A0<= /p>

=A0=A0=A0 while (1) {

=A0=A0=A0=A0=A0=A0=A0 iter =3D reorder_table_.find(next_seq= no_);

=A0=A0=A0=A0=A0=A0=A0 if (ite= r =3D=3D reorder_table_.end()) {

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;

=A0=A0=A0=A0=A0=A0=A0 }

=A0

= =A0=A0=A0=A0=A0=A0=A0 // add this code:

=A0=A0=A0=A0=A0=A0=A0 // if seqno does not ma= tch then still a bundle missing

=A0=A0=A0=A0=A0=A0=A0 if (iter->first !=3D next_seqno_) {

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;

=A0=A0=A0 =A0=A0=A0=A0}

=A0

=A0=A0=A0=A0=A0= =A0=A0 bundle =3D iter->second;

=A0=A0=A0=A0=A0=A0=A0 log_debug("delivering %zu byte b= undle with seqno %d (from reorder table)",

=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 bundle->payload_.len(), ne= xt_seqno_);

=A0=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0reorder_table_.erase(iter);<= u>

=A0=A0=A0=A0=A0=A0=A0 next_seqno_++;

=A0=A0=A0

<= span style=3D"font-size:9.0pt;font-family:"Courier New"">=A0=A0= =A0=A0=A0=A0=A0=A0queue_.push_back(bundle);

=A0=A0=A0 }=A0=A0

=A0

=A0<= /p>

=A0

=A0<= /p>

David Zoller

COLSA Corporation=

HOSC / C107= =A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: dtn-users= -bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Monday, December 17, 2012 6:40 AM
To: David Zoller


Cc: dtn-users@irtf.org
= Subject: Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,



Thanks for your patch, it works! :) Connection is cl= eanly terminated now.

However, there still seems to be a problem. I run dtntunnel over a link= with high latency and about 30% packet loss. The receiving nc sometimes ge= ts FIN packet before it receives all the data. If I understand it right, on= a lossy link bundles may come out of order. In this case the EOF bundle ma= y come too early. I think that the reordering procedure in dtntunnel should= be reviewed. Just for information, pure TCP performs better that DTN over = this link.

2012/12/14 David Zoller &= lt;zollerd@gmail.com= >

Hi Sergey,<= /u>

I can't stay away from a problem that= is nagging at me so I made some time this morning. Going back to the origi= nal TCPTunnel.cc code, when a bundle is received that has the EOF indicatio= n, an empty bundle with the same EOF flag set needs to be returned to close= the loop (the second FIN). There may still be some loose ends or clean up = needed but try the code below at the end of the while loop.

=A0

I did not do a packet capture but I saw that if the receivin= g nc is not running when you start the sender nc then the receiving dtntunn= el logs an error on the failed connect and returns an EOF bundle which clos= es the sender nc. The EOF bundle could possibly be enhanced to include an e= rror message.

=A0

I agree that some applications could benefit from a "pr= etend to be a receiver" option with the default working as a bidirecti= onal end-to-end TCP connection.

=A0

Thanks for dragging me along kicking and screaming on this l= esson in DTN and TCP tunneling. I've only been using dtntunnel as a one= way UDP transmitter. I see a good fit for the TCP tunnel in an upcoming pr= oject and I'll try to make the pitch to use it. This is a pretty cool l= ittle app.

Best regards,

DZ

=A0

=A0=

=A0

=A0

=A0

484 =A0 =A0 =A0 =A0 =A0 =A0= delete b_recv;

485=A0

486 =A0 =A0 =A0 =A0 =A0 =A0 if (recv_hdr->eof_) {

487 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 log_info("bundle had eof bit set... closing connection");=

488=A0

489 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ( !sock_eof ) {<= /u>

490 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 sock_eof =3D true;

491 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 // send an empty bundle back since we did not initiate the close=

492 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 dtn::APIBundle* b =3D new dtn::APIBundle();

493 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 hdr.eof_ =3D 1;

494 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 hdr.seqno_ =3D ntohl(send_seqno++);=

495 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 memcpy(b->payload_.buf(sizeo= f(hdr)), &hdr, sizeof(hdr));

496 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 b->payload_.set_len= (sizeof(hdr));

497 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 int= err;

498 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ((err =3D tunnel->send_bundle(b, &des= t_eid_)) !=3D DTN_SUCCESS) {

499 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 log_err("error sending final socket closed packet bundle: = %s",

500 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 dtn_strerror(err));=

501 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 tcptun_->kill_connection(this);

=

502 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = exit(1);

503 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 }

<= p class=3D"MsoNormal">504 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 delete b;=

505 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 }

506 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sock_.close();

<= div>

507 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto done;

508 =A0 =A0 =A0 =A0 =A0 = =A0 }

509 =A0 =A0 =A0 =A0 }

51= 0 =A0 =A0 }

511=A0

512 =A0done:

=A0

<= /div>

--f46d043c7e98964b3b04d11f58c9-- From david.a.zoller@nasa.gov Tue Dec 18 04:39:10 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 13E0321F87F7 for ; Tue, 18 Dec 2012 04:39:10 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.118 X-Spam-Level: X-Spam-Status: No, score=-6.118 tagged_above=-999 required=5 tests=[AWL=-0.120, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id om8GJgkixeni for ; Tue, 18 Dec 2012 04:39:05 -0800 (PST) Received: from ndjsnpf01.ndc.nasa.gov (ndjsnpf01.ndc.nasa.gov [198.117.1.121]) by ietfa.amsl.com (Postfix) with ESMTP id 677B221F8842 for ; Tue, 18 Dec 2012 04:39:05 -0800 (PST) Received: from ndmsppt03.ndc.nasa.gov (ndmsppt03.ndc.nasa.gov [198.117.0.102]) by ndjsnpf01.ndc.nasa.gov (Postfix) with ESMTP id A960FD0857; Tue, 18 Dec 2012 06:39:04 -0600 (CST) Received: from ndmshub06.ndc.nasa.gov (ndmshub06.ndc.nasa.gov [198.117.2.165]) by ndmsppt03.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBICd44b019747; Tue, 18 Dec 2012 06:39:04 -0600 Received: from NDMSSCC05.ndc.nasa.gov ([198.117.2.174]) by ndmshub06.ndc.nasa.gov ([198.117.2.165]) with mapi; Tue, 18 Dec 2012 06:39:04 -0600 From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "ssireskin@gmail.com" Date: Tue, 18 Dec 2012 06:39:02 -0600 Thread-Topic: [dtn-users] A problem with dtntunnel Thread-Index: Ac3dGExS/1mWzqIzTQqGxju4d1n7pgAAIpPQ Message-ID: <04E3D99A62496240BCD6A576813E6E31E0BDF5CB28@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDF5C59C@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5C999@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5CAE6@NDMSSCC05.ndc.nasa.gov> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_04E3D99A62496240BCD6A576813E6E31E0BDF5CB28NDMSSCC05ndcn_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-18_01:2012-12-18, 2012-12-18, 1970-01-01 signatures=0 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2012 12:39:10 -0000 --_000_04E3D99A62496240BCD6A576813E6E31E0BDF5CB28NDMSSCC05ndcn_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Sergey, You're persistence got us here so you are welcome to make a patch if you wo= uld like. I have a set of patches working their way through export approval= now. Once I get those thrown over the fence I don't mind making the additi= onal patch. I added options to override the socket buffer sizes that I only= had time to implement on the UDP side that I could look into adding to the= TCP side now that I've got some experience with it. Best regards, DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: ssireskin@gmail.com [mailto:ssireskin@gmail.com] Sent: Tuesday, December 18, 2012 6:08 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Thanks for the hint. I have also noticed that memory usage of dtntunnel not= iceably grows while running iperf with multiple flows. Are you going to submit all the fixes to mercurial? -- Best regards, Sergey Sireskin 2012/12/18 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, Looking at the rest of the code, there are probably a couple of memory leak= s. The biggie is after the call to "send_bundle(b_xmit" on success b_xmit s= hould be deleted before setting it to NULL (near line 432). Similarly after= the send_bundle for the "error connecting" the APIBundle b should also be = deleted before going to done (near line 342). You're welcome, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Tuesday, December 18, 2012 1:40 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: David Zoller; dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Now it works like a charm, thank you very much! 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Sergey, That was my bad. In the patch below I moved the "delete b_recv;" line above= the "if (recv_hdr->eof_) {" block as a short cut to delete it before the p= ossible exit. Move it back down below the if block and you should be okay. Sorry, that was one of those possible loose ends I mentioned :), DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Monday, December 17, 2012 8:05 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: David Zoller; dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel David, The problem is probably not with the reordering. Can it be, that "if (recv_= hdr->eof_)" is not the right condition to close a connection? Or maybe it i= s not a sufficient condition? Which connection is recv_hdr about? Can it be= a local TCP connection on the receiving side (dtntunnel <-> nc)? If it is,= then IMO this condition becomes true when the receiving application doesn'= t send any data to the sending application, please see line 406 of the orig= inal code. 2012/12/17 ssireskin@gmail.com > David, I added this code and a log_debug inside the if-clause. According to the lo= g, this condition doesn't get true. A have attached my logs and tcpdumps. Please, see dtntunnel_T.log - it's a = receiving dtntunnel log. There are 4 bundles after a eof bundle. This doesn't happen with the origin= al dtntunnel. -- Best regards, Sergey Sireskin 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I don't have your setup and this has not been tested but give this a try. I= n TCPTunnel::Connection::handle_bundle in the while(1) loop add the test to= see if the next_seqno_ is a match: while (1) { iter =3D reorder_table_.find(next_seqno_); if (iter =3D=3D reorder_table_.end()) { break; } // add this code: // if seqno does not match then still a bundle missing if (iter->first !=3D next_seqno_) { break; } bundle =3D iter->second; log_debug("delivering %zu byte bundle with seqno %d (from reorder t= able)", bundle->payload_.len(), next_seqno_); reorder_table_.erase(iter); next_seqno_++; queue_.push_back(bundle); } David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: dtn-users-bounces@irtf.org [mailto= :dtn-users-bounces@irtf.org] On Behalf O= f ssireskin@gmail.com Sent: Monday, December 17, 2012 6:40 AM To: David Zoller Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Thanks for your patch, it works! :) Connection is cleanly terminated now. However, there still seems to be a problem. I run dtntunnel over a link wit= h high latency and about 30% packet loss. The receiving nc sometimes gets F= IN packet before it receives all the data. If I understand it right, on a l= ossy link bundles may come out of order. In this case the EOF bundle may co= me too early. I think that the reordering procedure in dtntunnel should be = reviewed. Just for information, pure TCP performs better that DTN over this= link. 2012/12/14 David Zoller > Hi Sergey, I can't stay away from a problem that is nagging at me so I made some time = this morning. Going back to the original TCPTunnel.cc code, when a bundle i= s received that has the EOF indication, an empty bundle with the same EOF f= lag set needs to be returned to close the loop (the second FIN). There may = still be some loose ends or clean up needed but try the code below at the e= nd of the while loop. I did not do a packet capture but I saw that if the receiving nc is not run= ning when you start the sender nc then the receiving dtntunnel logs an erro= r on the failed connect and returns an EOF bundle which closes the sender n= c. The EOF bundle could possibly be enhanced to include an error message. I agree that some applications could benefit from a "pretend to be a receiv= er" option with the default working as a bidirectional end-to-end TCP conne= ction. Thanks for dragging me along kicking and screaming on this lesson in DTN an= d TCP tunneling. I've only been using dtntunnel as a one way UDP transmitte= r. I see a good fit for the TCP tunnel in an upcoming project and I'll try = to make the pitch to use it. This is a pretty cool little app. Best regards, DZ 484 delete b_recv; 485 486 if (recv_hdr->eof_) { 487 log_info("bundle had eof bit set... closing connection"= ); 488 489 if ( !sock_eof ) { 490 sock_eof =3D true; 491 // send an empty bundle back since we did not initi= ate the close 492 dtn::APIBundle* b =3D new dtn::APIBundle(); 493 hdr.eof_ =3D 1; 494 hdr.seqno_ =3D ntohl(send_seqno++); 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, sizeof(h= dr)); 496 b->payload_.set_len(sizeof(hdr)); 497 int err; 498 if ((err =3D tunnel->send_bundle(b, &dest_eid_)) != =3D DTN_SUCCESS) { 499 log_err("error sending final socket closed pack= et bundle: %s", 500 dtn_strerror(err)); 501 tcptun_->kill_connection(this); 502 exit(1); 503 } 504 delete b; 505 } 506 sock_.close(); 507 goto done; 508 } 509 } 510 } 511 512 done: --_000_04E3D99A62496240BCD6A576813E6E31E0BDF5CB28NDMSSCC05ndcn_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable = --_000_04E3D99A62496240BCD6A576813E6E31E0BDF5CB28NDMSSCC05ndcn_-- From ccaini@arces.unibo.it Tue Dec 18 07:52:08 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9E37421F8AB7 for ; Tue, 18 Dec 2012 07:52:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 1.581 X-Spam-Level: * X-Spam-Status: No, score=1.581 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_IT=0.635, HOST_EQ_IT=1.245, MANGLED_MEDS=2.3] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JmvGjtcwk1LA for ; Tue, 18 Dec 2012 07:52:06 -0800 (PST) Received: from mail.arces.unibo.it (mail.arces.unibo.it [137.204.143.6]) by ietfa.amsl.com (Postfix) with ESMTP id 79CEA21F8AE5 for ; Tue, 18 Dec 2012 07:52:05 -0800 (PST) Received: from webmail.arces.unibo.it (web.arces.unibo.it [137.204.143.9]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.arces.unibo.it (Postfix) with ESMTP id ACFEC43A0032; Tue, 18 Dec 2012 16:51:57 +0100 (CET) MIME-Version: 1.0 Date: Tue, 18 Dec 2012 16:51:57 +0100 From: ccaini To: "TSELIKIS.Christos" In-Reply-To: References: <20121217190307.8F273407F34C@mail.arces.unibo.it> Message-ID: <0de476e2191c06335745f853309c043f@arces.unibo.it> X-Sender: ccaini@arces.unibo.it User-Agent: RoundCube Webmail/0.2 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 X-Arces-MailScanner-Information: Please contact the ISP for more information X-Arces-MailScanner-ID: ACFEC43A0032.37F46 X-Arces-MailScanner: Found to be clean X-Arces-MailScanner-From: ccaini@arces.unibo.it Cc: dtn-users@irtf.org Subject: Re: [dtn-users] Use of metadata with DTNSEND and DTNRECV X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2012 15:52:08 -0000 Dear Christos, my mail was just to inform all dtn-user, and DTN2 mantainers, about the fact that the metadata problem with Dtnsend that has been discovered in DTN 2.9 by Luca Tartarini, and that you did not found in 2.8 (stable?), was already present in a 2.8+ version built with Mercurial (i.e. not from stable code). Now I can add that this 2.8+ version was actually built in December 2011. I would conclude that the problem originated between 31 August 2011 (2.8 stable released) and December 2011. I hope that this notes can be of some help to mantainers in order to have the bug fixed. As far as DTN experiments on smartphones, I suppose we had better open a different discussion topic. Therefore I will reply opening a new topic. Yours, Carlo On Tue, 18 Dec 2012 07:58:38 +0000, TSELIKIS.Christos wrote: > Dear Professor, dear all, we have no doubt about the correctness of your > results, also I have to try to send metadata with v2.9 asap. > We stay online for cooperation on relevant issues, actually now working on > the scenario with dynamic IPs, are you using a specific discovery protocol > when configuring an OPPORTUNISTIC link from DTN gateway to smartphone? > Thank you very much in advance, > With very best regards, > > Tselikis Christos > Hellenic Aerospace Industry S.A. > Electronics Engineering Department  > e-mail: Tselikis.Christos@haicorp.com > PO Box 23, GR-32009 Schimatari, Greece > Tel. : +30 22620 52561 > Fax: +30 22620 52910 > > > -----Original Message----- > From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On > Behalf Of Caini Carlo > Sent: Monday, December 17, 2012 9:03 PM > To: dtn-users@irtf.org > Subject: Re: [dtn-users] Use of metadata with DTNSEND and DTNRECV > > Dear Christos and others, > I have personally repeated the tests carried out by Luca Tartarini, a > thesis student of mine, on DTN2 2.9 and on 2.8+ (not the 2.8 stable > version, but one obtained from Mercurial), obtaining the same results of > Tartarini. > I can therefore confirm the bug, which is on the sending side, i.e. > on dtnsend. > > The bundle with metadata (-M1), if addressed to a node different from the > source is not actually transmitted, but is added to the list of pending > bundles. Viceversa, the same bundle is transmitted and received correctly > if the source and destination are the same. > Moreover, without the -M 1 option bundle are sent correctly in both cases. > > The fact that this bug is not present in stable 2.8 is an important > information, but we would like to use the new one, of course. > Yours, > Carlo Caini > > > > > > > > > At 08.23 11/12/2012, TSELIKIS.Christos wrote: >>Content-Language: en-US >>Content-Type: multipart/related; >> >>boundary="_004_A1E9924741E2C74A9ABA10C2DDBAA117013BE7HAIMAIL2010SRV1ha_"; >> type="multipart/alternative" >> >>Hi Luca, >>may be is trivial, ok with dtn2.8 (without vm): >>root@dtnrouter:~/dtn/usr/bin# ./dtnrecv -v dtn://192.168.1.51.dtn/a >>dtnrecv (pid 8945) starting up >>opening connection to dtn router... >>opened connection to dtn router... >>calling parse_eid_string >>find registration succeeded, regid 0 >>register succeeded, regid 11 >>looping forever to receive bundles >>dtn_recv [dtn://192.168.1.51.dtn/a]... >>bundle spec at 0xBFFA3C04 >> >>0 extension blocks from [dtn://192.168.1.100.dtn/me]: transit time=0 ms >> >>1 metadata blocks from [dtn://192.168.1.100.dtn/me]: transit time=0 ms >>Metadata Extension Block 0: >> type = 1 >> flags = 0 >>0000000 6d65 7461 | meta >> >>14 payload bytes from [dtn://192.168.1.100.dtn/me]: transit time=0 ms >>0000000 7465 7374 696e 6733 2d63 6169 6e69 | testing3 >> >> >>Freeing metadata block [0].data at 0x08060070 >>dtn_recv [dtn://192.168.1.51.dtn/a]... >> >>With very br, >>ctselikis >> >> >>Tselikis Christos >>Hellenic Aerospace Industry S.A. >>Electronics Engineering Department >>e-mail: >>Tselikis.Christos@haicorp.com >>PO Box 23, GR-32009 Schimatari, Greece >>Tel. : +30 22620 52561 >>Fax: +30 22620 52910 >>untitled >> >> >>From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] >>On Behalf Of Luca Tartarini >>Sent: Thursday, December 06, 2012 8:32 PM >>To: dtn-users@irtf.org >>Subject: [dtn-users] Use of metadata with DTNSEND and DTNRECV >> >>Dear DTN2 users, I am a student of the University of Bologna and I >>am carrying out a thesis on the possible use of metadata for DTN >>routing. I have a problem in the use of metadata with the >>applications DTNSEND and DTNRECV, DTN2 2.9.0 on Debian. >>If I try to send a bundle with metadata from vm1.dtn to vm2.dtn >>launching this command on vm1.dtn: >>vm1: dtnsend -v -s dtn://vm1.dtn/me -d dtn://vm2.dtn -t m -p >>"testing3" -M 1 -S "meta" >>while on vm2.dtn dtnrecv is running (vm2: dtnrecv -v dtn://vm2.dtn) >>I do not receive anything on vm2.dtn, while in the dtn.log of >>vm1.dtn I can see the following two errors: >>[1354724317.364108 /dtn/bundle/protocol error] >>BundleProtocol::prepare_blocks: 8-prepare returned BP_FAIL on bundle 10 >>[1354724317.364128 /dtn/bundle/actions error] >>BundleActions::queue_bundle: >>prepare_blocks returned NULL on bundle 10 >> >>Note that: >>1) without the -M option the bundle is received correctly (of course >>without metadata). >>2) if the source and the destination are the same, i.e. if I launch: >>vm1: dtnsend -v -s dtn://vm1.dtn/me -d dtn://vm1.dtn -t m -p >>"testing3" -M 1 -S "meta" >>both the bundle and metadata are received correctly. >> >>Any help will be greatly appreciated. >> >>Thanks, Luca Tartarini. >> >> >>_______________________________________________ >>dtn-users mailing list >>dtn-users@irtf.org >>https://www.irtf.org/mailman/listinfo/dtn-users From ssireskin@gmail.com Thu Dec 20 03:23:43 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AC61A21F84C9 for ; Thu, 20 Dec 2012 03:23:43 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.038 X-Spam-Level: X-Spam-Status: No, score=-3.038 tagged_above=-999 required=5 tests=[AWL=-0.040, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5noSjxRz2uXm for ; Thu, 20 Dec 2012 03:23:42 -0800 (PST) Received: from mail-vb0-f42.google.com (mail-vb0-f42.google.com [209.85.212.42]) by ietfa.amsl.com (Postfix) with ESMTP id EED4D21F84DE for ; Thu, 20 Dec 2012 03:23:41 -0800 (PST) Received: by mail-vb0-f42.google.com with SMTP id fa15so3585119vbb.15 for ; Thu, 20 Dec 2012 03:23:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9gLcdAr0sTR1+AS4k1a5PnURS+Nin5SvUYn/IOiGTMY=; b=SpmWY8Es44ddE7uSEjtMQqW6S60rg+vmwwfo3QOTee2UmokphbNdjpFqDNItntAmXd IadEowGDxhwvRSHoCtznxvnd5/1ene5HCPTALa4KCy44AwQGjWIJWh5zUnPoFyBPRUW/ daOxp55d3zjkN7GmPLJlAY0th8plBuw2/QRhaP2SFNvtihSdmRWUBJnEt1o3vlhdDQMb FOn8feuAApb5kIibCWZVhv8Pzp5Kh5RT8PUUVugFTFjsBux4E231DUM8F3R58pCiIs/3 aqfvXctVGsBnxDe9WcyYUgzvUXuiQsUco5dhxU72JeFB4N/SC8B7O5fbgpTssvoUCqyi F5wQ== MIME-Version: 1.0 Received: by 10.58.198.135 with SMTP id jc7mr13832838vec.51.1356002621342; Thu, 20 Dec 2012 03:23:41 -0800 (PST) Received: by 10.58.219.65 with HTTP; Thu, 20 Dec 2012 03:23:40 -0800 (PST) In-Reply-To: <04E3D99A62496240BCD6A576813E6E31E0BDF5CB28@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDF5C59C@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5C999@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5CAE6@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5CB28@NDMSSCC05.ndc.nasa.gov> Date: Thu, 20 Dec 2012 14:23:40 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/alternative; boundary=047d7b5d8cd78f42aa04d146f604 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Dec 2012 11:23:43 -0000 --047d7b5d8cd78f42aa04d146f604 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David, I am not a C/C++ programmer, could you please help me once again? Should delete(b) be added at line 336 of TCPTunnel.cc before goto done? --=20 Best regards, Sergey Sireskin 2012/12/18 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < david.a.zoller@nasa.gov> > Sergey,**** > > You=92re persistence got us here so you are welcome to make a patch if yo= u > would like. I have a set of patches working their way through export > approval now. Once I get those thrown over the fence I don=92t mind makin= g > the additional patch. I added options to override the socket buffer sizes > that I only had time to implement on the UDP side that I could look into > adding to the TCP side now that I=92ve got some experience with it. **** > > Best regards,**** > > DZ**** > > ** ** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > ** ** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Tuesday, December 18, 2012 6:08 AM > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > ** ** > > Hi David, > > Thanks for the hint. I have also noticed that memory usage of dtntunnel > noticeably grows while running iperf with multiple flows. > Are you going to submit all the fixes to mercurial? > > -- > Best regards, > Sergey Sireskin > > **** > > 2012/12/18 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > Looking at the rest of the code, there are probably a couple of memory > leaks. The biggie is after the call to =93*send_bundle(b_xmit*=94 on succ= ess > b_xmit should be deleted before setting it to NULL (near line 432). > Similarly after the send_bundle for the =93error connecting=94 the APIBun= dle b > should also be deleted before going to done (near line 342).**** > > You=92re welcome,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Tuesday, December 18, 2012 1:40 AM**** > > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* David Zoller; dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > Now it works like a charm, thank you very much!**** > > 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Sergey,**** > > That was my bad. In the patch below I moved the *=93delete b_recv;=94* li= ne > above the *=93if (recv_hdr->eof_) {=94* block as a short cut to delete it > before the possible exit. Move it back down below the if block and you > should be okay.**** > > Sorry, that was one of those possible loose ends I mentioned J,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Monday, December 17, 2012 8:05 AM > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* David Zoller; dtn-users@irtf.org**** > > > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > David, > > The problem is probably not with the reordering. Can it be, that "if > (recv_hdr->eof_)" is not the right condition to close a connection? Or > maybe it is not a sufficient condition? Which connection is recv_hdr abou= t? > Can it be a local TCP connection on the receiving side (dtntunnel <-> nc)= ? > If it is, then IMO this condition becomes true when the receiving > application doesn't send any data to the sending application, please see > line 406 of the original code.**** > > 2012/12/17 ssireskin@gmail.com **** > > David, > > I added this code and a log_debug inside the if-clause. According to the > log, this condition doesn't get true. > A have attached my logs and tcpdumps. Please, see dtntunnel_T.log - it's = a > receiving dtntunnel log. > There are 4 bundles after a eof bundle. This doesn't happen with the > original dtntunnel. **** > > > -- > Best regards, > Sergey Sireskin**** > > 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > **** > > Hi Sergey,**** > > I don=92t have your setup and this has not been tested but give this a tr= y. > In TCPTunnel::Connection::handle_bundle in the while(1) loop add the test > to see if the next_seqno_ is a match:**** > > **** > > while (1) {**** > > iter =3D reorder_table_.find(next_seqno_);**** > > if (iter =3D=3D reorder_table_.end()) {**** > > break;**** > > }**** > > **** > > // add this code:**** > > // if seqno does not match then still a bundle missing**** > > if (iter->first !=3D next_seqno_) {**** > > break;**** > > }**** > > **** > > bundle =3D iter->second;**** > > log_debug("delivering %zu byte bundle with seqno %d (from reorder > table)",**** > > bundle->payload_.len(), next_seqno_);**** > > **** > > reorder_table_.erase(iter);**** > > next_seqno_++;**** > > **** > > queue_.push_back(bundle);**** > > } **** > > **** > > **** > > **** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *O= n > Behalf Of *ssireskin@gmail.com > *Sent:* Monday, December 17, 2012 6:40 AM > *To:* David Zoller**** > > > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David,**** > > > > Thanks for your patch, it works! :) Connection is cleanly terminated now. > > However, there still seems to be a problem. I run dtntunnel over a link > with high latency and about 30% packet loss. The receiving nc sometimes > gets FIN packet before it receives all the data. If I understand it right= , > on a lossy link bundles may come out of order. In this case the EOF bundl= e > may come too early. I think that the reordering procedure in dtntunnel > should be reviewed. Just for information, pure TCP performs better that D= TN > over this link.**** > > 2012/12/14 David Zoller **** > > Hi Sergey,**** > > I can't stay away from a problem that is nagging at me so I made some tim= e > this morning. Going back to the original TCPTunnel.cc code, when a bundle > is received that has the EOF indication, an empty bundle with the same EO= F > flag set needs to be returned to close the loop (the second FIN). There m= ay > still be some loose ends or clean up needed but try the code below at the > end of the while loop.**** > > **** > > I did not do a packet capture but I saw that if the receiving nc is not > running when you start the sender nc then the receiving dtntunnel logs an > error on the failed connect and returns an EOF bundle which closes the > sender nc. The EOF bundle could possibly be enhanced to include an error > message.**** > > **** > > I agree that some applications could benefit from a "pretend to be a > receiver" option with the default working as a bidirectional end-to-end T= CP > connection.**** > > **** > > Thanks for dragging me along kicking and screaming on this lesson in DTN > and TCP tunneling. I've only been using dtntunnel as a one way UDP > transmitter. I see a good fit for the TCP tunnel in an upcoming project a= nd > I'll try to make the pitch to use it. This is a pretty cool little app.**= * > * > > Best regards,**** > > DZ**** > > **** > > **** > > **** > > **** > > **** > > 484 delete b_recv;**** > > 485 **** > > 486 if (recv_hdr->eof_) {**** > > 487 log_info("bundle had eof bit set... closing > connection");**** > > 488 **** > > 489 if ( !sock_eof ) {**** > > 490 sock_eof =3D true;**** > > 491 // send an empty bundle back since we did not > initiate the close**** > > 492 dtn::APIBundle* b =3D new dtn::APIBundle();**** > > 493 hdr.eof_ =3D 1;**** > > 494 hdr.seqno_ =3D ntohl(send_seqno++);**** > > 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, > sizeof(hdr));**** > > 496 b->payload_.set_len(sizeof(hdr));**** > > 497 int err;**** > > 498 if ((err =3D tunnel->send_bundle(b, &dest_eid_)) = !=3D > DTN_SUCCESS) {**** > > 499 log_err("error sending final socket closed > packet bundle: %s",**** > > 500 dtn_strerror(err));**** > > 501 tcptun_->kill_connection(this);**** > > 502 exit(1);**** > > 503 }**** > > 504 delete b;**** > > 505 }**** > > 506 sock_.close();**** > > 507 goto done;**** > > 508 }**** > > 509 }**** > > 510 }**** > > 511 **** > > 512 done:**** > > **** > > ** ** > --047d7b5d8cd78f42aa04d146f604 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David,

I am not a C/C++ programmer, could you please help me once= again? Should delete(b) be added at line 336 of TCPTunnel.cc before goto d= one?


--
Best regards,
Sergey Sireskin


2012/12/18 Zoller, David A. (MSFC-EO60)[= HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">

Sergey,

You=92re persistence got us here so you are welc= ome to make a patch if you would like. I have a set of patches working thei= r way through export approval now. Once I get those thrown over the fence I= don=92t mind making the additional patch. I added options to override the = socket buffer sizes that I only had time to implement on the UDP side that = I could look into adding to the TCP side now that I=92ve got some experienc= e with it.

Best regards,

DZ

=A0<= /p>

David Zoller

COLSA Corporation

HOSC / C107=A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov

=A0<= /p>

From: ssireskin@gmail.= com [mailto:ss= ireskin@gmail.com]
Sent: Tuesday, December 18, 2012 6:08 AM


To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,
Thanks for the hint. I have also noticed that memory usage of dtntunnel n= oticeably grows while running iperf with multiple flows.
Are you going to submit all the fixes to mercurial?

--
Best reg= ards,
Sergey Sireskin

2012/12/18 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nas= a.gov>

Hi Sergey,

Looki= ng at the rest of the code, there are probably a couple of memory leaks. Th= e biggie is after the call to =93send_bundle(b_xmit<= /span>=94 on success b_xmit should be delete= d before setting it to NULL (near line 432). Similarly after the send_bundl= e for the =93error connecting=94 the APIBundle b should also be deleted bef= ore going to done (near line 342).

You=92re welcome,<= u>

DZ<= /span>

=A0<= /p>

David Zoller

COLSA Corporation=

Marsha= ll Space Flight Center

=A0<= /p>

From: ssireskin@gmail.com= [mailto:ssireskin= @gmail.com]
Sent: Tuesday, December 18, 2012 1:40 AM


To: Zoller, David A. (MSFC-EO60)[H= OSC SERVICES CONTRACT]
Cc: David Zoller; dtn-users@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

<= /div>

=A0

Hi David,

Now it works= like a charm, thank you very much!

2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SE= RVICES CONTRACT] <david.a.zoller@nasa.gov>

Sergey,

That was my bad. In the patch below I mov= ed the =93delete b_recv;=94 line above the =93if (recv_hdr->eof= _) {=94 block as a short cut to delet= e it before the possible exit. Move it back down below the if block and you= should be okay.

Sorry, that was one of th= ose possible loose ends I mentioned J,

DZ

=A0<= u>

David Zoller<= u>

COLSA Corporation=

HOSC / C107=A0

(Office: (256)=A0544-1= 820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: ssireskin@gmail.= com [mailto:ss= ireskin@gmail.com]
Sent: Monday, December 17, 2012 8:05 AM
To: Zoller, David = A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: David Zoller; dtn-users@irtf.org


Subject: Re: [dtn-users] A prob= lem with dtntunnel

=A0

David,

The problem is probably not with the reordering. Can it be, t= hat "if (recv_hdr->eof_)" is not the right condition to close = a connection? Or maybe it is not a sufficient condition? Which connection i= s recv_hdr about? Can it be a local TCP connection on the receiving side (d= tntunnel <-> nc)? If it is, then IMO this condition becomes true when= the receiving application doesn't send any data to the sending applica= tion, please see line 406 of the original code.

2012/12/17 ssireskin@gmail.com <ssireskin@gmail.com>

David,

I added this code and a log_debug inside the if-clause. Acco= rding to the log, this condition doesn't get true.
A have attached m= y logs and tcpdumps. Please, see dtntunnel_T.log - it's a receiving dtn= tunnel log.
There are 4 bundles after a eof bundle. This doesn't happen with the or= iginal dtntunnel.


--
Best regards,
Sergey Sireskin

2012/12/17 Zoller, David A. (MSFC-EO60)[H= OSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

=A0

Hi Sergey,

I don=92t have your setup and this has not been = tested but give this a try. In TCPTunnel::Connection::handle_bundle in the = while(1) loop add the test to see if the next_seqno_ is a match:<= /u>

=A0<= /p>

=A0=A0=A0 while (1) {

=A0=A0=A0=A0=A0=A0=A0 iter =3D reorder_table_.find(next_seq= no_);

=A0=A0=A0=A0=A0=A0=A0 if (ite= r =3D=3D reorder_table_.end()) {

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;

=A0=A0=A0=A0=A0=A0=A0 }

=A0

= =A0=A0=A0=A0=A0=A0=A0 // add this code:

=A0=A0=A0=A0=A0=A0=A0 // if seqno does not ma= tch then still a bundle missing

=A0=A0=A0=A0=A0=A0=A0 if (iter->first !=3D next_seqno_) {

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;

=A0=A0=A0 =A0=A0=A0=A0}

=A0

=A0=A0=A0=A0=A0= =A0=A0 bundle =3D iter->second;

=A0=A0=A0=A0=A0=A0=A0 log_debug("delivering %zu byte b= undle with seqno %d (from reorder table)",

=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 bundle->payload_.len(), ne= xt_seqno_);

=A0=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0reorder_table_.erase(iter);<= u>

=A0=A0=A0=A0=A0=A0=A0 next_seqno_++;

=A0=A0=A0

<= span style=3D"font-size:9.0pt;font-family:"Courier New"">=A0=A0= =A0=A0=A0=A0=A0=A0queue_.push_back(bundle);

=A0=A0=A0 }=A0=A0

=A0

=A0<= /p>

=A0

=A0<= /p>

David Zoller

COLSA Corporation=

HOSC / C107= =A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: dtn-users= -bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Monday, December 17, 2012 6:40 AM
To: David Zoller


Cc: dtn-users@irtf.org
= Subject: Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,



Thanks for your patch, it works! :) Connection is cl= eanly terminated now.

However, there still seems to be a problem. I run dtntunnel over a link= with high latency and about 30% packet loss. The receiving nc sometimes ge= ts FIN packet before it receives all the data. If I understand it right, on= a lossy link bundles may come out of order. In this case the EOF bundle ma= y come too early. I think that the reordering procedure in dtntunnel should= be reviewed. Just for information, pure TCP performs better that DTN over = this link.

2012/12/14 David Zoller &= lt;zollerd@gmail.com= >

Hi Sergey,<= /u>

I can't stay away from a problem that= is nagging at me so I made some time this morning. Going back to the origi= nal TCPTunnel.cc code, when a bundle is received that has the EOF indicatio= n, an empty bundle with the same EOF flag set needs to be returned to close= the loop (the second FIN). There may still be some loose ends or clean up = needed but try the code below at the end of the while loop.

=A0

I did not do a packet capture but I saw that if the receivin= g nc is not running when you start the sender nc then the receiving dtntunn= el logs an error on the failed connect and returns an EOF bundle which clos= es the sender nc. The EOF bundle could possibly be enhanced to include an e= rror message.

=A0

I agree that some applications could benefit from a "pr= etend to be a receiver" option with the default working as a bidirecti= onal end-to-end TCP connection.

=A0

Thanks for dragging me along kicking and screaming on this l= esson in DTN and TCP tunneling. I've only been using dtntunnel as a one= way UDP transmitter. I see a good fit for the TCP tunnel in an upcoming pr= oject and I'll try to make the pitch to use it. This is a pretty cool l= ittle app.

Best regards,

DZ

=A0

=A0=

=A0

=A0

=A0

484 =A0 =A0 =A0 =A0 =A0 =A0= delete b_recv;

485=A0

486 =A0 =A0 =A0 =A0 =A0 =A0 if (recv_hdr->eof_) {

487 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 log_info("bundle had eof bit set... closing connection");=

488=A0

489 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ( !sock_eof ) {<= /u>

490 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 sock_eof =3D true;

491 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 // send an empty bundle back since we did not initiate the close=

492 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 dtn::APIBundle* b =3D new dtn::APIBundle();

493 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 hdr.eof_ =3D 1;

494 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 hdr.seqno_ =3D ntohl(send_seqno++);=

495 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 memcpy(b->payload_.buf(sizeo= f(hdr)), &hdr, sizeof(hdr));

496 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 b->payload_.set_len= (sizeof(hdr));

497 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 int= err;

498 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ((err =3D tunnel->send_bundle(b, &des= t_eid_)) !=3D DTN_SUCCESS) {

499 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 log_err("error sending final socket closed packet bundle: = %s",

500 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 dtn_strerror(err));=

501 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 tcptun_->kill_connection(this);

=

502 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = exit(1);

503 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 }

<= p class=3D"MsoNormal">504 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 delete b;=

505 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 }

506 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sock_.close();

<= div>

507 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto done;

508 =A0 =A0 =A0 =A0 =A0 = =A0 }

509 =A0 =A0 =A0 =A0 }

51= 0 =A0 =A0 }

511=A0

512 =A0done:

=A0

<= /div>

=A0


--047d7b5d8cd78f42aa04d146f604-- From david.a.zoller@nasa.gov Thu Dec 20 04:50:42 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5652221F8584 for ; Thu, 20 Dec 2012 04:50:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.107 X-Spam-Level: X-Spam-Status: No, score=-6.107 tagged_above=-999 required=5 tests=[AWL=-0.109, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U5c9bqvjEulS for ; Thu, 20 Dec 2012 04:50:26 -0800 (PST) Received: from ndmsnpf03.ndc.nasa.gov (ndmsnpf03.ndc.nasa.gov [198.117.0.123]) by ietfa.amsl.com (Postfix) with ESMTP id 2218A21F86BB for ; Thu, 20 Dec 2012 04:50:25 -0800 (PST) Received: from ndmsppt02.ndc.nasa.gov (ndmsppt02.ndc.nasa.gov [198.117.0.101]) by ndmsnpf03.ndc.nasa.gov (Postfix) with ESMTP id D720B182397; Thu, 20 Dec 2012 06:50:24 -0600 (CST) Received: from ndmshub02.ndc.nasa.gov (ndmshub02-pub.ndc.nasa.gov [198.117.0.161]) by ndmsppt02.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBKCoOwY014468; Thu, 20 Dec 2012 06:50:24 -0600 Received: from NDMSSCC05.ndc.nasa.gov ([198.117.2.174]) by ndmshub02.ndc.nasa.gov ([198.117.2.161]) with mapi; Thu, 20 Dec 2012 06:50:24 -0600 From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "ssireskin@gmail.com" Date: Thu, 20 Dec 2012 06:50:22 -0600 Thread-Topic: [dtn-users] A problem with dtntunnel Thread-Index: Ac3epHg8TFdP+LNcRpSETCZwEpTIjAAC6Bmw Message-ID: <04E3D99A62496240BCD6A576813E6E31E0BDFED5D3@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDF5C59C@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5C999@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5CAE6@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5CB28@NDMSSCC05.ndc.nasa.gov> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_04E3D99A62496240BCD6A576813E6E31E0BDFED5D3NDMSSCC05ndcn_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-20_04:2012-12-20, 2012-12-20, 1970-01-01 signatures=0 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Dec 2012 12:50:42 -0000 --_000_04E3D99A62496240BCD6A576813E6E31E0BDFED5D3NDMSSCC05ndcn_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Yes, that is correct. Looks like you've done very well in all of this for n= ot being a C/C++ programmer! David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gmail.com] Sent: Thursday, December 20, 2012 5:24 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, I am not a C/C++ programmer, could you please help me once again? Should de= lete(b) be added at line 336 of TCPTunnel.cc before goto done? -- Best regards, Sergey Sireskin 2012/12/18 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Sergey, You're persistence got us here so you are welcome to make a patch if you wo= uld like. I have a set of patches working their way through export approval= now. Once I get those thrown over the fence I don't mind making the additi= onal patch. I added options to override the socket buffer sizes that I only= had time to implement on the UDP side that I could look into adding to the= TCP side now that I've got some experience with it. Best regards, DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Tuesday, December 18, 2012 6:08 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Thanks for the hint. I have also noticed that memory usage of dtntunnel not= iceably grows while running iperf with multiple flows. Are you going to submit all the fixes to mercurial? -- Best regards, Sergey Sireskin 2012/12/18 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, Looking at the rest of the code, there are probably a couple of memory leak= s. The biggie is after the call to "send_bundle(b_xmit" on success b_xmit s= hould be deleted before setting it to NULL (near line 432). Similarly after= the send_bundle for the "error connecting" the APIBundle b should also be = deleted before going to done (near line 342). You're welcome, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Tuesday, December 18, 2012 1:40 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: David Zoller; dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Now it works like a charm, thank you very much! 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Sergey, That was my bad. In the patch below I moved the "delete b_recv;" line above= the "if (recv_hdr->eof_) {" block as a short cut to delete it before the p= ossible exit. Move it back down below the if block and you should be okay. Sorry, that was one of those possible loose ends I mentioned :), DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Monday, December 17, 2012 8:05 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: David Zoller; dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel David, The problem is probably not with the reordering. Can it be, that "if (recv_= hdr->eof_)" is not the right condition to close a connection? Or maybe it i= s not a sufficient condition? Which connection is recv_hdr about? Can it be= a local TCP connection on the receiving side (dtntunnel <-> nc)? If it is,= then IMO this condition becomes true when the receiving application doesn'= t send any data to the sending application, please see line 406 of the orig= inal code. 2012/12/17 ssireskin@gmail.com > David, I added this code and a log_debug inside the if-clause. According to the lo= g, this condition doesn't get true. A have attached my logs and tcpdumps. Please, see dtntunnel_T.log - it's a = receiving dtntunnel log. There are 4 bundles after a eof bundle. This doesn't happen with the origin= al dtntunnel. -- Best regards, Sergey Sireskin 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I don't have your setup and this has not been tested but give this a try. I= n TCPTunnel::Connection::handle_bundle in the while(1) loop add the test to= see if the next_seqno_ is a match: while (1) { iter =3D reorder_table_.find(next_seqno_); if (iter =3D=3D reorder_table_.end()) { break; } // add this code: // if seqno does not match then still a bundle missing if (iter->first !=3D next_seqno_) { break; } bundle =3D iter->second; log_debug("delivering %zu byte bundle with seqno %d (from reorder t= able)", bundle->payload_.len(), next_seqno_); reorder_table_.erase(iter); next_seqno_++; queue_.push_back(bundle); } David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: dtn-users-bounces@irtf.org [mailto= :dtn-users-bounces@irtf.org] On Behalf O= f ssireskin@gmail.com Sent: Monday, December 17, 2012 6:40 AM To: David Zoller Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Thanks for your patch, it works! :) Connection is cleanly terminated now. However, there still seems to be a problem. I run dtntunnel over a link wit= h high latency and about 30% packet loss. The receiving nc sometimes gets F= IN packet before it receives all the data. If I understand it right, on a l= ossy link bundles may come out of order. In this case the EOF bundle may co= me too early. I think that the reordering procedure in dtntunnel should be = reviewed. Just for information, pure TCP performs better that DTN over this= link. 2012/12/14 David Zoller > Hi Sergey, I can't stay away from a problem that is nagging at me so I made some time = this morning. Going back to the original TCPTunnel.cc code, when a bundle i= s received that has the EOF indication, an empty bundle with the same EOF f= lag set needs to be returned to close the loop (the second FIN). There may = still be some loose ends or clean up needed but try the code below at the e= nd of the while loop. I did not do a packet capture but I saw that if the receiving nc is not run= ning when you start the sender nc then the receiving dtntunnel logs an erro= r on the failed connect and returns an EOF bundle which closes the sender n= c. The EOF bundle could possibly be enhanced to include an error message. I agree that some applications could benefit from a "pretend to be a receiv= er" option with the default working as a bidirectional end-to-end TCP conne= ction. Thanks for dragging me along kicking and screaming on this lesson in DTN an= d TCP tunneling. I've only been using dtntunnel as a one way UDP transmitte= r. I see a good fit for the TCP tunnel in an upcoming project and I'll try = to make the pitch to use it. This is a pretty cool little app. Best regards, DZ 484 delete b_recv; 485 486 if (recv_hdr->eof_) { 487 log_info("bundle had eof bit set... closing connection"= ); 488 489 if ( !sock_eof ) { 490 sock_eof =3D true; 491 // send an empty bundle back since we did not initi= ate the close 492 dtn::APIBundle* b =3D new dtn::APIBundle(); 493 hdr.eof_ =3D 1; 494 hdr.seqno_ =3D ntohl(send_seqno++); 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, sizeof(h= dr)); 496 b->payload_.set_len(sizeof(hdr)); 497 int err; 498 if ((err =3D tunnel->send_bundle(b, &dest_eid_)) != =3D DTN_SUCCESS) { 499 log_err("error sending final socket closed pack= et bundle: %s", 500 dtn_strerror(err)); 501 tcptun_->kill_connection(this); 502 exit(1); 503 } 504 delete b; 505 } 506 sock_.close(); 507 goto done; 508 } 509 } 510 } 511 512 done: --_000_04E3D99A62496240BCD6A576813E6E31E0BDFED5D3NDMSSCC05ndcn_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Yes, that= is correct. Looks like you’ve done very well in all of this for not = being a C/C++ programmer!

=  

David Zoller

COLSA Corporation

Marshall Space Fli= ght Center

<= o:p> 

From: ssireskin@gmail.com [ma= ilto:ssireskin@gmail.com]
Sent: Thursday, December 20, 2012 5:24= AM
To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
<= b>Cc: dtn-users@irtf.org
Subject: Re: [dtn-users] A problem w= ith dtntunnel

 

Hi David,

I am= not a C/C++ programmer, could you please help me once again? Should delete= (b) be added at line 336 of TCPTunnel.cc before goto done?


--
Best regards,
Sergey Sireskin


2012/12/18 Zoller, David A. (MSFC-EO60)[HOSC SERVIC= ES CONTRACT] <david.a.zoller@nasa.gov>

Se= rgey,

You’re persistence got us he= re so you are welcome to make a patch if you would like. I have a set of pa= tches working their way through export approval now. Once I get those throw= n over the fence I don’t mind making the additional patch. I added op= tions to override the socket buffer sizes that I only had time to implement= on the UDP side that I could look into adding to the TCP side now that I&#= 8217;ve got some experience with it.

Be= st regards,

DZ

 

David Zoller<= o:p>

COLSA Corporation

<= span style=3D'font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F49= 7D'>HOSC / C107 

(Office: (256) 544-1820
*EMail: david.= a.zoller@nasa.gov

 

From: ssireskin@gmail.com [mailto:ssireskin@gmail.com]
Sent: Tue= sday, December 18, 2012 6:08 AM


To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]

Cc: dtn-users@irtf.org
Subjec= t: Re: [dtn-users] A problem with dtntunnel

<= div>

 

Hi David,

Thanks for the hint.= I have also noticed that memory usage of dtntunnel noticeably grows while = running iperf with multiple flows.
Are you going to submit all the fixes= to mercurial?

--
Best regards,
Sergey Sireskin

2012/12/18 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRAC= T] <david.a= .zoller@nasa.gov>

Hi Sergey,

Looking at the rest of the code, there ar= e probably a couple of memory leaks. The biggie is after the call to “= ;send_bundle(b_xmit” on success b_xmit should be= deleted before setting it to NULL (near line 432). Similarly after the sen= d_bundle for the “error connecting” the APIBundle b should also= be deleted before going to done (near line 342).

You’re welcome,

DZ<= /span>

 

= David Zoller

COLSA Corporation=

Marshall Space Flight Center<= /p>

 

From: ssireskin@gmail.com [mailto:ssireskin@gmail.com]
Sent: Tuesday, December 18, 2012 1:40 AM

&nbs= p;

Hi David,

Now it works like a charm, thank you ver= y much!

2012/12/17 Zoller, David A. (MSFC-EO60)[H= OSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Sergey,

That was my bad. In the = patch below I moved the “delete b_recv;” l= ine above the “if (recv_hdr->eof_) {” b= lock as a short cut to delete it before the possible exit. Move it back dow= n below the if block and you should be okay.

<= span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F= 497D'>Sorry, that was one of those possible loose ends I mentioned <= span style=3D'font-size:11.0pt;font-family:Wingdings;color:#1F497D'>J,

DZ

<= p class=3DMsoNormal style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:= auto'> 

David Zoller

COLSA Corporation

= HOSC / C107 

(Office:= (256) 544-1820
*EMail: david.a.zo= ller@nasa.gov

 

From: ssireskin@gmail.com [mailto:ssireskin@gmail.com]
Sent: Monday,= December 17, 2012 8:05 AM
To: Zoller, David A. (MSFC-EO60)[HOSC = SERVICES CONTRACT]
Cc: David Zoller; dtn-users@irtf.org


Subject: Re: [dtn-users] A problem with dtntunnel<= o:p>

 

David,
=
The problem is probably not with the reordering. Can it be, that "= if (recv_hdr->eof_)" is not the right condition to close a connecti= on? Or maybe it is not a sufficient condition? Which connection is recv_hdr= about? Can it be a local TCP connection on the receiving side (dtntunnel &= lt;-> nc)? If it is, then IMO this condition becomes true when the recei= ving application doesn't send any data to the sending application, please s= ee line 406 of the original code.

2012/12/17 ssireskin@gmail.com <ssireskin@gm= ail.com>

David,

I added this code and a= log_debug inside the if-clause. According to the log, this condition doesn= 't get true.
A have attached my logs and tcpdumps. Please, see dtntunnel= _T.log - it's a receiving dtntunnel log.
There are 4 bundles after a eof= bundle. This doesn't happen with the original dtntunnel.


--
Best regards,
Sergey Sireskin

2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@n= asa.gov>

 

Hi Sergey,

I don’t have your setup and this has not been tested but give this = a try. In TCPTunnel::Connection::handle_bundle in the while(1) loop add the= test to see if the next_seqno_ is a match:

<= span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F= 497D'> 

    while (1) {

 &nb= sp;      iter =3D reorder_table_.find(next_seqno_)= ;

        if (iter =3D=3D reo= rder_table_.end()) {

       &= nbsp;    break;

     &nb= sp;  }

 

   &n= bsp;    // add this code:

 &nb= sp;      // if seqno does not match then still a b= undle missing

      =   if (iter->first !=3D next_seqno_) {

<= span style=3D'font-size:9.0pt;font-family:"Courier New";color:#00B050'>&nbs= p;           break;

        }

 

        bundle =3D it= er->second;

        log_de= bug("delivering %zu byte bundle with seqno %d (from reorder table)&quo= t;,

          =         bundle->payload_.len(), next_= seqno_);

   

    =     reorder_table_.erase(iter);

  =       next_seqno_++;

<= span style=3D'font-size:9.0pt;font-family:"Courier New"'>   =

        queue_.push_ba= ck(bundle);

    }  

 

 

 

&n= bsp;

David Zoller

COLSA Corporation

HOSC / C107&nb= sp;

(Office: (= 256) 544-1820
*EMail: david.a.zoller@nasa.gov<= /a>

 

<= /div>

From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Beha= lf Of ssireski= n@gmail.com
Sent: Monday, December 17, 2012 6:40 AM
To:= David Zoller


Cc: dtn-users@irtf.org
= Subject: Re: [dtn-users] A problem with dtntunnel

 

Hi David,

<= br>
Thanks for your patch, it works! :) Connection is cleanly terminated= now.

However, there still seems to be a problem. I run dtntunnel ov= er a link with high latency and about 30% packet loss. The receiving nc som= etimes gets FIN packet before it receives all the data. If I understand it = right, on a lossy link bundles may come out of order. In this case the EOF = bundle may come too early. I think that the reordering procedure in dtntunn= el should be reviewed. Just for information, pure TCP performs better that = DTN over this link.

2012/12= /14 David Zoller <zollerd@gmail.com>

Hi Sergey,<= /p>

I can't stay away from a problem that is nagging at me= so I made some time this morning. Going back to the original TCPTunnel.cc = code, when a bundle is received that has the EOF indication, an empty bundl= e with the same EOF flag set needs to be returned to close the loop (the se= cond FIN). There may still be some loose ends or clean up needed but try th= e code below at the end of the while loop.

=  

I did not do a packet capture but I = saw that if the receiving nc is not running when you start the sender nc th= en the receiving dtntunnel logs an error on the failed connect and returns = an EOF bundle which closes the sender nc. The EOF bundle could possibly be = enhanced to include an error message.

 = ;

I agree that some applications could bene= fit from a "pretend to be a receiver" option with the default wor= king as a bidirectional end-to-end TCP connection.

 

Thanks for dragging me along= kicking and screaming on this lesson in DTN and TCP tunneling. I've only b= een using dtntunnel as a one way UDP transmitter. I see a good fit for the = TCP tunnel in an upcoming project and I'll try to make the pitch to use it.= This is a pretty cool little app.

Best reg= ards,

DZ

&= nbsp;

 

 

 

 

484       &= nbsp;     delete b_recv;

485 = ;

486           &n= bsp; if (recv_hdr->eof_) {

487   &n= bsp;             log_info("bundle had eo= f bit set... closing connection");

4= 88 

489         &n= bsp;       if ( !sock_eof ) {

490                    = sock_eof =3D true;

491      = ;               // send an empty bundle = back since we did not initiate the close

4= 92                     dt= n::APIBundle* b =3D new dtn::APIBundle();

4= 93                     hd= r.eof_ =3D 1;

494       &nbs= p;             hdr.seqno_ =3D ntohl(send_seqn= o++);

495          = ;           memcpy(b->payload_.buf(sizeof(hdr))= , &hdr, sizeof(hdr));

496    =                 b->payload_.set= _len(sizeof(hdr));

497      =               int err;

498                =     if ((err =3D tunnel->send_bundle(b, &dest_eid_)) !=3D= DTN_SUCCESS) {

499       &n= bsp;                 log_err("= error sending final socket closed packet bundle: %s",

500                =                 dtn_strerror(err))= ;

501           &n= bsp;             tcptun_->kill_connection(= this);

502         &nbs= p;               exit(1);

=

503               &nbs= p;     }

504      =               delete b;

<= /div>

505                = ; }

506           =       sock_.close();

507 &nb= sp;               goto done;<= /p>

508             }

509         }

<= /div>

510     }

511&nb= sp;

512  done:

 

 

 

= --_000_04E3D99A62496240BCD6A576813E6E31E0BDFED5D3NDMSSCC05ndcn_-- From ssireskin@gmail.com Thu Dec 20 04:54:00 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7E29C21F84ED for ; Thu, 20 Dec 2012 04:54:00 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.036 X-Spam-Level: X-Spam-Status: No, score=-3.036 tagged_above=-999 required=5 tests=[AWL=-0.038, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jEQAcGOJYbOx for ; Thu, 20 Dec 2012 04:53:59 -0800 (PST) Received: from mail-vc0-f169.google.com (mail-vc0-f169.google.com [209.85.220.169]) by ietfa.amsl.com (Postfix) with ESMTP id A251321F8570 for ; Thu, 20 Dec 2012 04:53:58 -0800 (PST) Received: by mail-vc0-f169.google.com with SMTP id gb23so3769902vcb.0 for ; Thu, 20 Dec 2012 04:53:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=6UArA4R+sv46hrbXehjjS03cht0MXuCFWHLMLAr9RxA=; b=h6NeAYWPBOoHR5qT/a48iQK2DbtGUVlFjQ1kp8oWAUK6xHqNz7nmAya3Mru1otEaup 9XcdNJHZtqFJfQPGG1rKBJw+HPnFCf5a/1wpiZ5ehihj0Lv1BvzPCUR1lZ4uou7i1RTB /0NU/+S5A/crbD08V53sB7KqznrjOEZNsKS+YjLxn3bkqCYX0T9icvcieBhcz8aW7j0e MCHIsyrtz29t+ozi9EHl1UDnAdw/tFCmf+XvalmKphz+RnEYu21qlpHS3PO8SJ7071oT lxtdHTCizsGanA5sbVCl6DtG0OV6mWjkw0CxzG8G93D4jdjYOMEowdGDFSyneKPTJgwQ RWxw== MIME-Version: 1.0 Received: by 10.58.116.212 with SMTP id jy20mr14429698veb.5.1356008037884; Thu, 20 Dec 2012 04:53:57 -0800 (PST) Received: by 10.58.219.65 with HTTP; Thu, 20 Dec 2012 04:53:57 -0800 (PST) In-Reply-To: <04E3D99A62496240BCD6A576813E6E31E0BDFED5D3@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDF5C59C@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5C999@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5CAE6@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5CB28@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDFED5D3@NDMSSCC05.ndc.nasa.gov> Date: Thu, 20 Dec 2012 15:53:57 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/alternative; boundary=047d7b6d99a46921f604d148391e Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Dec 2012 12:54:00 -0000 --047d7b6d99a46921f604d148391e Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Thanks a lot, David! I am more experienced in scripting languages, where there is no need to care about memory. 2012/12/20 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < david.a.zoller@nasa.gov> > Yes, that is correct. Looks like you=92ve done very well in all of this f= or > not being a C/C++ programmer!**** > > ** ** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > ** ** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Thursday, December 20, 2012 5:24 AM > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > ** ** > > Hi David, > > I am not a C/C++ programmer, could you please help me once again? Should > delete(b) be added at line 336 of TCPTunnel.cc before goto done? > > > -- > Best regards, > Sergey Sireskin > > > **** > > 2012/12/18 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Sergey,**** > > You=92re persistence got us here so you are welcome to make a patch if yo= u > would like. I have a set of patches working their way through export > approval now. Once I get those thrown over the fence I don=92t mind makin= g > the additional patch. I added options to override the socket buffer sizes > that I only had time to implement on the UDP side that I could look into > adding to the TCP side now that I=92ve got some experience with it. **** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Tuesday, December 18, 2012 6:08 AM**** > > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]**** > > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > Thanks for the hint. I have also noticed that memory usage of dtntunnel > noticeably grows while running iperf with multiple flows. > Are you going to submit all the fixes to mercurial? > > -- > Best regards, > Sergey Sireskin**** > > 2012/12/18 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > Looking at the rest of the code, there are probably a couple of memory > leaks. The biggie is after the call to =93*send_bundle(b_xmit*=94 on succ= ess > b_xmit should be deleted before setting it to NULL (near line 432). > Similarly after the send_bundle for the =93error connecting=94 the APIBun= dle b > should also be deleted before going to done (near line 342).**** > > You=92re welcome,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Tuesday, December 18, 2012 1:40 AM**** > > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* David Zoller; dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > Now it works like a charm, thank you very much!**** > > 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Sergey,**** > > That was my bad. In the patch below I moved the *=93delete b_recv;=94* li= ne > above the *=93if (recv_hdr->eof_) {=94* block as a short cut to delete it > before the possible exit. Move it back down below the if block and you > should be okay.**** > > Sorry, that was one of those possible loose ends I mentioned J,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Monday, December 17, 2012 8:05 AM > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* David Zoller; dtn-users@irtf.org**** > > > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > David, > > The problem is probably not with the reordering. Can it be, that "if > (recv_hdr->eof_)" is not the right condition to close a connection? Or > maybe it is not a sufficient condition? Which connection is recv_hdr abou= t? > Can it be a local TCP connection on the receiving side (dtntunnel <-> nc)= ? > If it is, then IMO this condition becomes true when the receiving > application doesn't send any data to the sending application, please see > line 406 of the original code.**** > > 2012/12/17 ssireskin@gmail.com **** > > David, > > I added this code and a log_debug inside the if-clause. According to the > log, this condition doesn't get true. > A have attached my logs and tcpdumps. Please, see dtntunnel_T.log - it's = a > receiving dtntunnel log. > There are 4 bundles after a eof bundle. This doesn't happen with the > original dtntunnel. **** > > > -- > Best regards, > Sergey Sireskin**** > > 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > **** > > Hi Sergey,**** > > I don=92t have your setup and this has not been tested but give this a tr= y. > In TCPTunnel::Connection::handle_bundle in the while(1) loop add the test > to see if the next_seqno_ is a match:**** > > **** > > while (1) {**** > > iter =3D reorder_table_.find(next_seqno_);**** > > if (iter =3D=3D reorder_table_.end()) {**** > > break;**** > > }**** > > **** > > // add this code:**** > > // if seqno does not match then still a bundle missing**** > > if (iter->first !=3D next_seqno_) {**** > > break;**** > > }**** > > **** > > bundle =3D iter->second;**** > > log_debug("delivering %zu byte bundle with seqno %d (from reorder > table)",**** > > bundle->payload_.len(), next_seqno_);**** > > **** > > reorder_table_.erase(iter);**** > > next_seqno_++;**** > > **** > > queue_.push_back(bundle);**** > > } **** > > **** > > **** > > **** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *O= n > Behalf Of *ssireskin@gmail.com > *Sent:* Monday, December 17, 2012 6:40 AM > *To:* David Zoller**** > > > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David,**** > > > > Thanks for your patch, it works! :) Connection is cleanly terminated now. > > However, there still seems to be a problem. I run dtntunnel over a link > with high latency and about 30% packet loss. The receiving nc sometimes > gets FIN packet before it receives all the data. If I understand it right= , > on a lossy link bundles may come out of order. In this case the EOF bundl= e > may come too early. I think that the reordering procedure in dtntunnel > should be reviewed. Just for information, pure TCP performs better that D= TN > over this link.**** > > 2012/12/14 David Zoller **** > > Hi Sergey,**** > > I can't stay away from a problem that is nagging at me so I made some tim= e > this morning. Going back to the original TCPTunnel.cc code, when a bundle > is received that has the EOF indication, an empty bundle with the same EO= F > flag set needs to be returned to close the loop (the second FIN). There m= ay > still be some loose ends or clean up needed but try the code below at the > end of the while loop.**** > > **** > > I did not do a packet capture but I saw that if the receiving nc is not > running when you start the sender nc then the receiving dtntunnel logs an > error on the failed connect and returns an EOF bundle which closes the > sender nc. The EOF bundle could possibly be enhanced to include an error > message.**** > > **** > > I agree that some applications could benefit from a "pretend to be a > receiver" option with the default working as a bidirectional end-to-end T= CP > connection.**** > > **** > > Thanks for dragging me along kicking and screaming on this lesson in DTN > and TCP tunneling. I've only been using dtntunnel as a one way UDP > transmitter. I see a good fit for the TCP tunnel in an upcoming project a= nd > I'll try to make the pitch to use it. This is a pretty cool little app.**= * > * > > Best regards,**** > > DZ**** > > **** > > **** > > **** > > **** > > **** > > 484 delete b_recv;**** > > 485 **** > > 486 if (recv_hdr->eof_) {**** > > 487 log_info("bundle had eof bit set... closing > connection");**** > > 488 **** > > 489 if ( !sock_eof ) {**** > > 490 sock_eof =3D true;**** > > 491 // send an empty bundle back since we did not > initiate the close**** > > 492 dtn::APIBundle* b =3D new dtn::APIBundle();**** > > 493 hdr.eof_ =3D 1;**** > > 494 hdr.seqno_ =3D ntohl(send_seqno++);**** > > 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, > sizeof(hdr));**** > > 496 b->payload_.set_len(sizeof(hdr));**** > > 497 int err;**** > > 498 if ((err =3D tunnel->send_bundle(b, &dest_eid_)) = !=3D > DTN_SUCCESS) {**** > > 499 log_err("error sending final socket closed > packet bundle: %s",**** > > 500 dtn_strerror(err));**** > > 501 tcptun_->kill_connection(this);**** > > 502 exit(1);**** > > 503 }**** > > 504 delete b;**** > > 505 }**** > > 506 sock_.close();**** > > 507 goto done;**** > > 508 }**** > > 509 }**** > > 510 }**** > > 511 **** > > 512 done:**** > > **** > > **** > > ** ** > --=20 Best regards, Sergey Sireskin --047d7b6d99a46921f604d148391e Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Thanks a lot, David!

I am more experienced in scripting languages, = where there is no need to care about memory.

2012/12/20 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Yes, that is = correct. Looks like you=92ve done very well in all of this for not being a = C/C++ programmer!

= =A0

David = Zoller

COLSA Corporation

Marshall Space Flight Cente= r

=A0<= /p>

From: ssireskin@gmail.= com [mailto:ss= ireskin@gmail.com]
Sent: Thursday, December 20, 2012 5:24 AM


To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]=
Cc: dtn-= users@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,
I am not a C/C++ programmer, could you please help me once again? Should = delete(b) be added at line 336 of TCPTunnel.cc before goto done?


--
Best regards,
Sergey Sireskin


2012/12/18 Zoller, David A. (MS= FC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Sergey,<= u>

You=92re= persistence got us here so you are welcome to make a patch if you would li= ke. I have a set of patches working their way through export approval now. = Once I get those thrown over the fence I don=92t mind making the additional= patch. I added options to override the socket buffer sizes that I only had= time to implement on the UDP side that I could look into adding to the TCP= side now that I=92ve got some experience with it.

Best regards,

DZ

=A0<= /p>

David Zoller

COLSA Corporation=

HOSC / C107= =A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: ssireskin@gmail.= com [mailto:ss= ireskin@gmail.com]
Sent: Tuesday, December 18, 2012 6:08 AM


To: Zoller, David A. (MSFC-EO60)[HOSC S= ERVICES CONTRACT]

<= b>Cc: dtn-users= @irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

<= /div>

=A0

Hi David,

Thanks for t= he hint. I have also noticed that memory usage of dtntunnel noticeably grow= s while running iperf with multiple flows.
Are you going to submit all the fixes to mercurial?

--
Best reg= ards,
Sergey Sireskin

2012/= 12/18 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

Looki= ng at the rest of the code, there are probably a couple of memory leaks. Th= e biggie is after the call to =93send_bundle(b_xmit<= /span>=94 on success b_xmit should be delete= d before setting it to NULL (near line 432). Similarly after the send_bundl= e for the =93error connecting=94 the APIBundle b should also be deleted bef= ore going to done (near line 342).

You=92re welcome,<= u>

DZ<= /span>

=A0<= /p>

David Zoller

COLSA Corporation=

Marsha= ll Space Flight Center

=A0<= /p>

From: ssireskin@gmail.com= [mailto:ssireskin= @gmail.com]
Sent: Tuesday, December 18, 2012 1:40 AM


To: Zoller, David A. (MSFC-EO60)[H= OSC SERVICES CONTRACT]
Cc: David Zoller; dtn-users@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

<= /div>

=A0

Hi David,

Now it works= like a charm, thank you very much!

2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SE= RVICES CONTRACT] <david.a.zoller@nasa.gov>

Sergey,

That was my bad. In the patch below I mov= ed the =93delete b_recv;=94 line above the =93if (recv_hdr->eof= _) {=94 block as a short cut to delet= e it before the possible exit. Move it back down below the if block and you= should be okay.

Sorry, that was one of th= ose possible loose ends I mentioned J,

DZ

=A0<= u>

David Zoller<= u>

COLSA Corporation=

HOSC / C107=A0

(Office: (256)=A0544-1= 820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: ssireskin@gmail.= com [mailto:ss= ireskin@gmail.com]
Sent: Monday, December 17, 2012 8:05 AM
To: Zoller, David = A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: David Zoller; dtn-users@irtf.org


Subject: Re: [dtn-users] A prob= lem with dtntunnel

=A0

David,

The problem is probably not with the reordering. Can it be, t= hat "if (recv_hdr->eof_)" is not the right condition to close = a connection? Or maybe it is not a sufficient condition? Which connection i= s recv_hdr about? Can it be a local TCP connection on the receiving side (d= tntunnel <-> nc)? If it is, then IMO this condition becomes true when= the receiving application doesn't send any data to the sending applica= tion, please see line 406 of the original code.

2012/12/17 ssireskin@gmail.com <ssireskin@gmail.com>

David,

I added this code and a log_debug inside the if-clause. Acco= rding to the log, this condition doesn't get true.
A have attached m= y logs and tcpdumps. Please, see dtntunnel_T.log - it's a receiving dtn= tunnel log.
There are 4 bundles after a eof bundle. This doesn't happen with the or= iginal dtntunnel.


--
Best regards,
Sergey Sireskin

2012/12/17 Zoller, David A. (MSFC-EO60)[H= OSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

=A0

Hi Sergey,

I don=92t have your setup and this has not been = tested but give this a try. In TCPTunnel::Connection::handle_bundle in the = while(1) loop add the test to see if the next_seqno_ is a match:<= /u>

=A0<= /p>

=A0=A0=A0 while (1) {

=A0=A0=A0=A0=A0=A0=A0 iter =3D reorder_table_.find(next_seq= no_);

=A0=A0=A0=A0=A0=A0=A0 if (ite= r =3D=3D reorder_table_.end()) {

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;

=A0=A0=A0=A0=A0=A0=A0 }

=A0

= =A0=A0=A0=A0=A0=A0=A0 // add this code:

=A0=A0=A0=A0=A0=A0=A0 // if seqno does not ma= tch then still a bundle missing

=A0=A0=A0=A0=A0=A0=A0 if (iter->first !=3D next_seqno_) {

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;

=A0=A0=A0 =A0=A0=A0=A0}

=A0

=A0=A0=A0=A0=A0= =A0=A0 bundle =3D iter->second;

=A0=A0=A0=A0=A0=A0=A0 log_debug("delivering %zu byte b= undle with seqno %d (from reorder table)",

=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 bundle->payload_.len(), ne= xt_seqno_);

=A0=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0reorder_table_.erase(iter);<= u>

=A0=A0=A0=A0=A0=A0=A0 next_seqno_++;

=A0=A0=A0

<= span style=3D"font-size:9.0pt;font-family:"Courier New"">=A0=A0= =A0=A0=A0=A0=A0=A0queue_.push_back(bundle);

=A0=A0=A0 }=A0=A0

=A0

=A0<= /p>

=A0

=A0<= /p>

David Zoller

COLSA Corporation=

HOSC / C107= =A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: dtn-users= -bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Monday, December 17, 2012 6:40 AM
To: David Zoller


Cc: dtn-users@irtf.org
= Subject: Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,



Thanks for your patch, it works! :) Connection is cl= eanly terminated now.

However, there still seems to be a problem. I run dtntunnel over a link= with high latency and about 30% packet loss. The receiving nc sometimes ge= ts FIN packet before it receives all the data. If I understand it right, on= a lossy link bundles may come out of order. In this case the EOF bundle ma= y come too early. I think that the reordering procedure in dtntunnel should= be reviewed. Just for information, pure TCP performs better that DTN over = this link.

2012/12/14 David Zoller &= lt;zollerd@gmail.com= >

Hi Sergey,<= /u>

I can't stay away from a problem that= is nagging at me so I made some time this morning. Going back to the origi= nal TCPTunnel.cc code, when a bundle is received that has the EOF indicatio= n, an empty bundle with the same EOF flag set needs to be returned to close= the loop (the second FIN). There may still be some loose ends or clean up = needed but try the code below at the end of the while loop.

=A0

I did not do a packet capture but I saw that if the receivin= g nc is not running when you start the sender nc then the receiving dtntunn= el logs an error on the failed connect and returns an EOF bundle which clos= es the sender nc. The EOF bundle could possibly be enhanced to include an e= rror message.

=A0

I agree that some applications could benefit from a "pr= etend to be a receiver" option with the default working as a bidirecti= onal end-to-end TCP connection.

=A0

Thanks for dragging me along kicking and screaming on this l= esson in DTN and TCP tunneling. I've only been using dtntunnel as a one= way UDP transmitter. I see a good fit for the TCP tunnel in an upcoming pr= oject and I'll try to make the pitch to use it. This is a pretty cool l= ittle app.

Best regards,

DZ

=A0

=A0=

=A0

=A0

=A0

484 =A0 =A0 =A0 =A0 =A0 =A0= delete b_recv;

485=A0

486 =A0 =A0 =A0 =A0 =A0 =A0 if (recv_hdr->eof_) {

487 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 log_info("bundle had eof bit set... closing connection");=

488=A0

489 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ( !sock_eof ) {<= /u>

490 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 sock_eof =3D true;

491 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 // send an empty bundle back since we did not initiate the close=

492 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 dtn::APIBundle* b =3D new dtn::APIBundle();

493 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 hdr.eof_ =3D 1;

494 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 hdr.seqno_ =3D ntohl(send_seqno++);=

495 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 memcpy(b->payload_.buf(sizeo= f(hdr)), &hdr, sizeof(hdr));

496 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 b->payload_.set_len= (sizeof(hdr));

497 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 int= err;

498 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ((err =3D tunnel->send_bundle(b, &des= t_eid_)) !=3D DTN_SUCCESS) {

499 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 log_err("error sending final socket closed packet bundle: = %s",

500 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 dtn_strerror(err));=

501 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 tcptun_->kill_connection(this);

=

502 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = exit(1);

503 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 }

<= p class=3D"MsoNormal">504 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 delete b;=

505 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 }

506 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sock_.close();

<= div>

507 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto done;

508 =A0 =A0 =A0 =A0 =A0 = =A0 }

509 =A0 =A0 =A0 =A0 }

51= 0 =A0 =A0 }

511=A0

512 =A0done:

=A0

<= /div>

=A0

=A0

<= /div>


--
Best regards= ,
Sergey Sireskin


--047d7b6d99a46921f604d148391e-- From ssireskin@gmail.com Thu Dec 20 09:52:18 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7F86F21F8946 for ; Thu, 20 Dec 2012 09:52:18 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.714 X-Spam-Level: X-Spam-Status: No, score=-2.714 tagged_above=-999 required=5 tests=[AWL=-0.357, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, SARE_LWSHORTT=1.24, WEIRD_PORT=0.001] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j-0wjFYVHEvZ for ; Thu, 20 Dec 2012 09:51:56 -0800 (PST) Received: from mail-vc0-f181.google.com (mail-vc0-f181.google.com [209.85.220.181]) by ietfa.amsl.com (Postfix) with ESMTP id 1F15621F8931 for ; Thu, 20 Dec 2012 09:51:55 -0800 (PST) Received: by mail-vc0-f181.google.com with SMTP id gb30so4008284vcb.26 for ; Thu, 20 Dec 2012 09:51:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=CVNiXKNeiqRVeI7HNBK26OqYeRnggyzG7tl7nqva1Xs=; b=waSn9PazjaIK2j4+HCjEzL7AriVJGDr5Z0Heh4BeT9BXEwmtAmBelsl+osm9NKItTj aMdA3KauIacoAcx1O98VwxWk6qxs79VmwHZ8aYdk2DiHzY+5tL8eSLbBCz/jRRa+ZWzN JuxCut5hqWMuwNcfXZI/CCiPFpG9huDBA+ZLf4RWU2Ifc57C+ifSSNRU9xgpn13gaXiA vkMRbprjN2XACpvKkkfK2D/r2R8XZ7Ujv//jfwjhskjqUXHpCGx2vx0XO79Bl9109A9L w3Ivhm6xD1mIskkU3GXXhjZqDN5Dk6L8ixVIdvrU/oikz0DKXQMJlJqHza21N3UB7/4k AOhQ== MIME-Version: 1.0 Received: by 10.58.198.135 with SMTP id jc7mr15827341vec.51.1356025908138; Thu, 20 Dec 2012 09:51:48 -0800 (PST) Received: by 10.58.219.65 with HTTP; Thu, 20 Dec 2012 09:51:47 -0800 (PST) In-Reply-To: References: Date: Thu, 20 Dec 2012 20:51:47 +0300 Message-ID: From: "ssireskin@gmail.com" To: Elwyn Davies Content-Type: multipart/alternative; boundary=047d7b5d8cd78f913604d14c6228 Cc: dtn-users@irtf.org Subject: Re: [dtn-users] Unable to compile DTN2 with NORM convergence layer X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Dec 2012 17:52:19 -0000 --047d7b5d8cd78f913604d14c6228 Content-Type: text/plain; charset=ISO-8859-1 Hi, Elwyn. It seems that you still not receiving my mail. This could be your mail server filtering my messages. So I am writing here. I have tried the patch, it applied cleanly, but I am still getting the same error during the build process as before. Have nice holidays! With kindest regards, Sergey > > Hi, Sergey. > > I sent you the message below a couple of weeks ago but I haven't seen > any response. Were you able to test it out? > > Best wishes for the festive season > Elwyn > > Hi, Sergey. > > Here is an advance copy of a patch for 2.9.0 that should fix your > problem. I haven't had chance to test it properly but it is basically a > clone of existing code for other convergence layers so it should work > OK. Let me know if it doesn't work please. > > Regards, > Elwyn > > On 07/12/12 10:33, ssireskin@gmail.com wrote: > > > Hi Elwyn, > > > > Thanks for your help! Please let me know, when this issue will be > > resolved, > > so that I update my DTN sources. I will use 2.8.0 until then. > > > > 2012/12/7 Elwyn Davies > > Hi. > > > > I can reproduce this problem. I suspect I know what is going > > on and > > will look into the problem later today. It may be a > > consequence of some > > changes that were made to allow persistent link names.. I > > *thought* it > > had been tested with all the convergwence layers, but clearly > > not. > > *>sigh<*. > > > > If you want to experiment in the short term, v2.8.0 compiles > > cleanly > > with NORM. > > > > Regards, > > Elwyn > > > > On Fri, 2012-12-07 at 09:59 +0300, ssireskin@gmail.com wrote: > > > Hello all, > > > > > > I have trouble compiling DTN2 with NORM convergence layer > > using > > > gcc-4.4.5 on a RHEL6-based Linux distro. > > > norm-svnsnap.tgz 09-Aug-2012 > > > protolib-svnsnap.tgz 03-Oct-2012 > > > Did anyone manage to build DTN2 with NORM-CL? What am I > > doing wrong? > > > > > > make -w -C applib > > > make[1]: Entering directory `/usr/src/dtn2/dtn-2.9.0/applib' > > > make[1]: Nothing to be done for `all'. > > > make[1]: Leaving directory `/usr/src/dtn2/dtn-2.9.0/applib' > > > make -w -C servlib > > > make[1]: Entering directory > > `/usr/src/dtn2/dtn-2.9.0/servlib' > > > g++ -g -fno-inline -fPIC -DPIC -MMD -MP -MT > > > "conv_layers/ConvergenceLayer.o > > conv_layers/ConvergenceLayer.E" > > > -DHAVE_CONFIG_H -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 > > -I.. -I.. > > > -I/opt/dtn2/include -I/opt/dtn2/include/oasys/ext > > -I../servlib > > > -I/opt/dtn2/include -Wall -W -Wcast-align -c > > > conv_layers/ConvergenceLayer.cc -o > > conv_layers/ConvergenceLayer.o > > > In file included from conv_layers/NORMConvergenceLayer.h:36, > > > from conv_layers/ConvergenceLayer.cc:29: > > > /opt/dtn2/include/normApi.h:616: warning: type qualifiers > > ignored on > > > function return type > > > conv_layers/ConvergenceLayer.cc: In static member function > > 'static > > > void dtn::ConvergenceLayer::init_clayers()': > > > conv_layers/ConvergenceLayer.cc:67: error: cannot allocate > > an object > > > of abstract type 'dtn::NORMConvergenceLayer' > > > conv_layers/NORMConvergenceLayer.h:245: note: because the > > following > > > virtual functions are pure within > > 'dtn::NORMConvergenceLayer': > > > conv_layers/ConvergenceLayer.h:224: note: virtual > > dtn::CLInfo* > > > dtn::ConvergenceLayer::new_link_params() > > > make[1]: *** [conv_layers/ConvergenceLayer.o] Error 1 > > > make[1]: Leaving directory `/usr/src/dtn2/dtn-2.9.0/servlib' > > > make: *** [servlib] Error 2 > > > > > > Best regards, > > > Sergey Sireskin > > > > > > > > _______________________________________________ > > > dtn-users mailing list > > > dtn-users@irtf.org > > > https://www.irtf.org/mailman/listinfo/dtn-users > > > > > > > > > > -- > > Kindest Regards > > > > Sergey Sireskin > > FGUP CNII EISU > > > > -- Best regards, Sergey Sireskin --047d7b5d8cd78f913604d14c6228 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi, Elwyn.

It seems that you still not receiving my mail. This could= be your mail server filtering my messages. So I am writing here.

I = have tried the patch, it applied cleanly, but I am still getting the same e= rror during the build process as before.

Have nice holidays!

With kindest regards,
Sergey

=


Hi, Sergey.

I sent = you the message below a couple of weeks ago but I haven't seen
any r= esponse. =A0Were you able to test it out?

Best wishes for the festiv= e season
Elwyn

Hi, Sergey.

Here is an advance copy of a patch for 2.9.= 0 that should fix your
problem. =A0I haven't had chance to test it p= roperly but it is basically a
clone of existing code for other convergen= ce layers so it should work
OK. =A0Let me know if it doesn't work please.

Regards,
Elwyn<= br>
On 07/12/12 10:33, ssireskin@gmail.com wrote:

> Hi Elwyn,
>
&g= t; Thanks for your help! Please let me know, when this issue will be
> resolved,
> so that I update my DTN sources. I will use 2.8.0 u= ntil then.
>
> 2012/12/7 Elwyn Davies <elwynd@folly.org.uk>
> = =A0 =A0 =A0 =A0 Hi.
> =A0 =A0 =A0 =A0
> =A0 =A0 =A0 =A0 I can reproduce this problem.= =A0I suspect I know what is going
> =A0 =A0 =A0 =A0 on and
> = =A0 =A0 =A0 =A0 will look into the problem later today. =A0It may be a
&= gt; =A0 =A0 =A0 =A0 consequence of some
> =A0 =A0 =A0 =A0 changes that were made to allow persistent link names.= . I
> =A0 =A0 =A0 =A0 *thought* it
> =A0 =A0 =A0 =A0 had been t= ested with all the convergwence layers, but clearly
> =A0 =A0 =A0 =A0= not.
> =A0 =A0 =A0 =A0 *>sigh<*.
> =A0 =A0 =A0 =A0
> =A0 =A0 =A0 =A0 If you want to experiment in = the short term, v2.8.0 compiles
> =A0 =A0 =A0 =A0 cleanly
> =A0= =A0 =A0 =A0 with NORM.
> =A0 =A0 =A0 =A0
> =A0 =A0 =A0 =A0 Re= gards,
> =A0 =A0 =A0 =A0 Elwyn
> =A0 =A0 =A0 =A0
> =A0 =A0 =A0 =A0 On Fri, 2012-12-07 at 09:59 +0300, ssireskin@gmail.com wrote:
>= ; =A0 =A0 =A0 =A0 > Hello all,
> =A0 =A0 =A0 =A0 >
> =A0 = =A0 =A0 =A0 > I have trouble compiling DTN2 with NORM convergence layer<= br> > =A0 =A0 =A0 =A0 using
> =A0 =A0 =A0 =A0 > gcc-4.4.5 on a RHEL= 6-based Linux distro.
> =A0 =A0 =A0 =A0 > norm-svnsnap.tgz 09-Aug-= 2012
> =A0 =A0 =A0 =A0 > protolib-svnsnap.tgz 03-Oct-2012
> = =A0 =A0 =A0 =A0 > Did anyone manage to build DTN2 with NORM-CL? What am = I
> =A0 =A0 =A0 =A0 doing wrong?
> =A0 =A0 =A0 =A0 >
> =A0 = =A0 =A0 =A0 > make -w -C applib
> =A0 =A0 =A0 =A0 > make[1]: En= tering directory `/usr/src/dtn2/dtn-2.9.0/applib'
> =A0 =A0 =A0 = =A0 > make[1]: Nothing to be done for `all'.
> =A0 =A0 =A0 =A0 > make[1]: Leaving directory `/usr/src/dtn2/dtn-2.9= .0/applib'
> =A0 =A0 =A0 =A0 > make -w -C servlib
> =A0 = =A0 =A0 =A0 > make[1]: Entering directory
> =A0 =A0 =A0 =A0 `/usr/= src/dtn2/dtn-2.9.0/servlib'
> =A0 =A0 =A0 =A0 > g++ -g -fno-inline =A0-fPIC -DPIC -MMD -MP -MT> =A0 =A0 =A0 =A0 > "conv_layers/ConvergenceLayer.o
> =A0= =A0 =A0 =A0 conv_layers/ConvergenceLayer.E"
> =A0 =A0 =A0 =A0 &= gt; -DHAVE_CONFIG_H -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=3D64
> =A0 =A0 =A0 =A0 -I.. -I..
> =A0 =A0 =A0 =A0 > -I/opt/dtn2/inc= lude -I/opt/dtn2/include/oasys/ext
> =A0 =A0 =A0 =A0 -I../servlib
= > =A0 =A0 =A0 =A0 > -I/opt/dtn2/include =A0 -Wall -W -Wcast-align =A0= -c
> =A0 =A0 =A0 =A0 > conv_layers/ConvergenceLayer.cc -o
> =A0 =A0 =A0 =A0 conv_layers/ConvergenceLayer.o
> =A0 =A0 =A0 =A0= > In file included from conv_layers/NORMConvergenceLayer.h:36,
> = =A0 =A0 =A0 =A0 > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0from conv_layers/Co= nvergenceLayer.cc:29:
> =A0 =A0 =A0 =A0 > /opt/dtn2/include/normAp= i.h:616: warning: type qualifiers
> =A0 =A0 =A0 =A0 ignored on
> =A0 =A0 =A0 =A0 > function retur= n type
> =A0 =A0 =A0 =A0 > conv_layers/ConvergenceLayer.cc: In sta= tic member function
> =A0 =A0 =A0 =A0 'static
> =A0 =A0 =A0= =A0 > void dtn::ConvergenceLayer::init_clayers()':
> =A0 =A0 =A0 =A0 > conv_layers/ConvergenceLayer.cc:67: error: cannot= allocate
> =A0 =A0 =A0 =A0 an object
> =A0 =A0 =A0 =A0 > of= abstract type 'dtn::NORMConvergenceLayer'
> =A0 =A0 =A0 =A0 = > conv_layers/NORMConvergenceLayer.h:245: note: =A0 because the
> =A0 =A0 =A0 =A0 following
> =A0 =A0 =A0 =A0 > virtual functio= ns are pure within
> =A0 =A0 =A0 =A0 'dtn::NORMConvergenceLayer&#= 39;:
> =A0 =A0 =A0 =A0 > conv_layers/ConvergenceLayer.h:224: note:= =A0 =A0 =A0 virtual
> =A0 =A0 =A0 =A0 dtn::CLInfo*
> =A0 =A0 =A0 =A0 > dtn::ConvergenceLayer::new_link_params()
> = =A0 =A0 =A0 =A0 > make[1]: *** [conv_layers/ConvergenceLayer.o] Error 1<= br>> =A0 =A0 =A0 =A0 > make[1]: Leaving directory `/usr/src/dtn2/dtn-= 2.9.0/servlib'
> =A0 =A0 =A0 =A0 > make: *** [servlib] Error 2
> =A0 =A0 =A0 = =A0 >
> =A0 =A0 =A0 =A0 > Best regards,
> =A0 =A0 =A0 =A0= > Sergey Sireskin
> =A0 =A0 =A0 =A0 >
> =A0 =A0 =A0 =A0 =
> =A0 =A0 =A0 =A0 > _____________________________________________= __
> =A0 =A0 =A0 =A0 > dtn-users mailing list
> =A0 =A0 =A0 =A0 &g= t; dtn-users@irtf.o= rg
> =A0 =A0 =A0 =A0 > https://www.irtf.org/mailman/listinf= o/dtn-users
> =A0 =A0 =A0 =A0
>
>
>
> --
> Kindes= t Regards
>
> Sergey Sireskin
> FGUP CNII EISU
> <= br>



--
Best= regards,
Sergey Sireskin

--047d7b5d8cd78f913604d14c6228-- From zollerd@gmail.com Fri Dec 14 09:32:03 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 09AE821F8A90 for ; Fri, 14 Dec 2012 09:32:03 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.998 X-Spam-Level: X-Spam-Status: No, score=-2.998 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5TnjgYSR8Udx for ; Fri, 14 Dec 2012 09:32:02 -0800 (PST) Received: from mail-da0-f54.google.com (mail-da0-f54.google.com [209.85.210.54]) by ietfa.amsl.com (Postfix) with ESMTP id 5E22B21F8A47 for ; Fri, 14 Dec 2012 09:32:02 -0800 (PST) Received: by mail-da0-f54.google.com with SMTP id n2so1448252dad.13 for ; Fri, 14 Dec 2012 09:32:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=YtWoq4mcAmFF1nsBAZjQoT4U9NIBAOD6XWRXRwoCNbI=; b=xex4S7bQldBEHnEgc+73JkLOjvsuFjax17gxcp4Hc9VliS4ASoUcstppFKUjzvUzy6 rpd49UgLKtDGlW+epGYuYWUn/Db80jGIPs9AuPAIwmdAYgixTzryi8h7Ugm3VpZocPoS jp29BK5axim3xCl/NIR5xaWCwqQc+pxAGW6nwB3fNH6WkxqKpoXkuESwmJI+AKz9SjjO RWY4L6/CdhY4y+Kz1V1nWfXnqdHW2Poae5ackdnawJUkhqW8TIY+Ph4YhW9m8SmSFOiT fX62wNZkFyufJA3RsJZ4J3vYc223r//uSZazlwPFhXEly3zRHeI67MlcJtUQGVAo885S VDmQ== MIME-Version: 1.0 Received: by 10.68.252.228 with SMTP id zv4mr17617835pbc.134.1355506322052; Fri, 14 Dec 2012 09:32:02 -0800 (PST) Received: by 10.68.234.137 with HTTP; Fri, 14 Dec 2012 09:32:01 -0800 (PST) Date: Fri, 14 Dec 2012 11:32:01 -0600 Message-ID: From: David Zoller To: ssireskin@gmail.com, dtn-users@irtf.org Content-Type: multipart/alternative; boundary=047d7b2e0e25d116ad04d0d3685b X-Mailman-Approved-At: Thu, 27 Dec 2012 08:05:59 -0800 Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Dec 2012 22:59:48 -0000 --047d7b2e0e25d116ad04d0d3685b Content-Type: text/plain; charset=ISO-8859-1 Hi Sergey, I can't stay away from a problem that is nagging at me so I made some time this morning. Going back to the original TCPTunnel.cc code, when a bundle is received that has the EOF indication, an empty bundle with the same EOF flag set needs to be returned to close the loop (the second FIN). There may still be some loose ends or clean up needed but try the code below at the end of the while loop. I did not do a packet capture but I saw that if the receiving nc is not running when you start the sender nc then the receiving dtntunnel logs an error on the failed connect and returns an EOF bundle which closes the sender nc. The EOF bundle could possibly be enhanced to include an error message. I agree that some applications could benefit from a "pretend to be a receiver" option with the default working as a bidirectional end-to-end TCP connection. Thanks for dragging me along kicking and screaming on this lesson in DTN and TCP tunneling. I've only been using dtntunnel as a one way UDP transmitter. I see a good fit for the TCP tunnel in an upcoming project and I'll try to make the pitch to use it. This is a pretty cool little app. Best regards, DZ 484 delete b_recv; 485 486 if (recv_hdr->eof_) { 487 log_info("bundle had eof bit set... closing connection"); 488 489 if ( !sock_eof ) { 490 sock_eof = true; 491 // send an empty bundle back since we did not initiate the close 492 dtn::APIBundle* b = new dtn::APIBundle(); 493 hdr.eof_ = 1; 494 hdr.seqno_ = ntohl(send_seqno++); 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, sizeof(hdr)); 496 b->payload_.set_len(sizeof(hdr)); 497 int err; 498 if ((err = tunnel->send_bundle(b, &dest_eid_)) != DTN_SUCCESS) { 499 log_err("error sending final socket closed packet bundle: %s", 500 dtn_strerror(err)); 501 tcptun_->kill_connection(this); 502 exit(1); 503 } 504 delete b; 505 } 506 sock_.close(); 507 goto done; 508 } 509 } 510 } 511 512 done: --047d7b2e0e25d116ad04d0d3685b Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi Sergey,
I can't stay away from a problem that is nagg= ing at me so I made some time this morning. Going back to the original TCPT= unnel.cc code, when a bundle is received that has the EOF indication, an em= pty bundle with the same EOF flag set needs to be returned to close the loo= p (the second FIN). There may still be some loose ends or clean up needed b= ut try the code below at the end of the while loop.

I did not do a packet capture but I saw that if the rec= eiving nc is not running when you start the sender nc then the receiving dt= ntunnel logs an error on the failed connect and returns an EOF bundle which= closes the sender nc. The EOF bundle could possibly be enhanced to include= an error message.

I agree that some applications could benefit from a &qu= ot;pretend to be a receiver" option with the default working as a bidi= rectional end-to-end TCP connection.

Thanks for dr= agging me along kicking and screaming on this lesson in DTN and TCP tunneli= ng. I've only been using dtntunnel as a one way UDP transmitter. I see = a good fit for the TCP tunnel in an upcoming project and I'll try to ma= ke the pitch to use it. This is a pretty cool little app.
Best regards,
DZ




484 =A0 =A0 =A0 =A0 =A0 =A0 = delete b_recv;
485=A0
486 =A0 =A0 =A0 =A0 =A0 =A0 if (r= ecv_hdr->eof_) {
487 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 log_info("bundle had eof bit = set... closing connection");
488=A0
489 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 if ( !sock_eof ) {
490 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 sock_eof =3D true;
491 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 // send an empty bundle back since we did not initi= ate the close
492 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 dtn::APIBundle* b =3D new = dtn::APIBundle();
493 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 hdr= .eof_ =3D 1;
494 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 hdr.seqn= o_ =3D ntohl(send_seqno++);
495 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, sizeof(hdr));
496 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 b->payload_.set_len(siz= eof(hdr));
497 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 int err;
498 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ((err =3D tunnel-&g= t;send_bundle(b, &dest_eid_)) !=3D DTN_SUCCESS) {
499 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 log_err("error sen= ding final socket closed packet bundle: %s",
500 =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 dtn_strerror(err));
501 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 tcptun_->kil= l_connection(this);
502 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 exit(1);
503 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 }
504 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 delete b;
505 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 }
506 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sock_.close();
507 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto done;
508 =A0 =A0 =A0 =A0 =A0 =A0 }
509 =A0 =A0 =A0 =A0 }
510 =A0 =A0 }
511=A0
512 =A0done:
--047d7b2e0e25d116ad04d0d3685b-- From vlyamtsev@mzeal.com Fri Dec 28 07:28:56 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B812C21F8CF3 for ; Fri, 28 Dec 2012 07:28:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.001 X-Spam-Level: X-Spam-Status: No, score=0.001 tagged_above=-999 required=5 tests=[BAYES_50=0.001] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hsu6vqRiwNxs for ; Fri, 28 Dec 2012 07:28:56 -0800 (PST) Received: from mzeal.com (mzeal.com [199.237.228.71]) by ietfa.amsl.com (Postfix) with ESMTP id 344E521F8CED for ; Fri, 28 Dec 2012 07:28:56 -0800 (PST) Received: (qmail 41335 invoked by uid 25671); 28 Dec 2012 15:28:54 -0000 Message-ID: <20121228152854.41334.qmail@mzeal.com> Content-Disposition: inline Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="US-ASCII"; format="flowed" MIME-Version: 1.0 X-Mailer: MIME::Lite 3.0104 (F2.72; T1.15; A1.47; B3.01; Q3.01) Date: Fri, 28 Dec 2012 15:28:54 +0000 From: "Victor Lyamtsev" To: dtn-users@irtf.org Subject: [dtn-users] dtn multicasting -? X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Dec 2012 15:28:56 -0000 Hello, Is there any completed standard/ implementation of bundle multicasting? P.S. I am not asking about something like NORM convergence layer with multicasting over ip, but the scenario with non-ip transport... Thank you, From william.d.ivancic@nasa.gov Fri Dec 28 10:33:11 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4661E21F8DFA for ; Fri, 28 Dec 2012 10:33:11 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.349 X-Spam-Level: X-Spam-Status: No, score=-6.349 tagged_above=-999 required=5 tests=[AWL=0.250, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WlfFolWrcv2T for ; Fri, 28 Dec 2012 10:33:10 -0800 (PST) Received: from ndmsnpf03.ndc.nasa.gov (ndmsnpf03.ndc.nasa.gov [198.117.0.123]) by ietfa.amsl.com (Postfix) with ESMTP id 3165321F87FD for ; Fri, 28 Dec 2012 10:33:10 -0800 (PST) Received: from ndjsppt03.ndc.nasa.gov (ndjsppt03.ndc.nasa.gov [198.117.1.102]) by ndmsnpf03.ndc.nasa.gov (Postfix) with ESMTP id E118C2D81F5; Fri, 28 Dec 2012 12:33:07 -0600 (CST) Received: from ndjshub06.ndc.nasa.gov (ndjshub06.ndc.nasa.gov [198.117.4.165]) by ndjsppt03.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBSIX77t001669; Fri, 28 Dec 2012 12:33:07 -0600 Received: from NDJSSCC07.ndc.nasa.gov ([198.117.4.178]) by ndjshub06.ndc.nasa.gov ([198.117.4.165]) with mapi; Fri, 28 Dec 2012 12:33:07 -0600 From: "Ivancic, William D. (GRC-RHN0)" To: Victor Lyamtsev , "dtn-users@irtf.org" Date: Fri, 28 Dec 2012 12:33:05 -0600 Thread-Topic: [dtn-users] dtn multicasting -? Thread-Index: Ac3lEA/ZCxfaNYeVS6Wz9FEDuQGcyQAGbai2 Message-ID: In-Reply-To: <20121228152854.41334.qmail@mzeal.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-Entourage/13.11.0.110726 acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-28_07:2012-12-28, 2012-12-28, 1970-01-01 signatures=0 Subject: Re: [dtn-users] dtn multicasting -? X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Dec 2012 18:33:11 -0000 I don't believe there is any formal "standard" for DTN multicasting so an implementation of a standard is problematic. I think there have been some experimentation. It is a naming thing as well as routing. You may checkout the following for guidance and ideas. Scott may have even implemented some of this for ION, but ION is generally geared toward scheduled contacts. That may not fit your needs. http://datatracker.ietf.org/doc/draft-burleigh-dtnrg-imc/?include_text=3D1 Will > From: Victor Lyamtsev > Date: Fri, 28 Dec 2012 09:28:54 -0600 > To: "dtn-users@irtf.org" > Subject: [dtn-users] dtn multicasting -? >=20 > Hello, > Is there any completed standard/ implementation of bundle multicasting? > P.S. I am not asking about something like NORM convergence layer with > multicasting over ip, but the scenario with non-ip transport... > Thank you, > _______________________________________________ > dtn-users mailing list > dtn-users@irtf.org > https://www.irtf.org/mailman/listinfo/dtn-users From william.d.ivancic@nasa.gov Fri Dec 28 10:35:50 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8E5F121F8DFE for ; Fri, 28 Dec 2012 10:35:50 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.385 X-Spam-Level: X-Spam-Status: No, score=-6.385 tagged_above=-999 required=5 tests=[AWL=0.214, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rCN4G4m1J0Al for ; Fri, 28 Dec 2012 10:35:50 -0800 (PST) Received: from ndmsnpf02.ndc.nasa.gov (ndmsnpf02.ndc.nasa.gov [198.117.0.122]) by ietfa.amsl.com (Postfix) with ESMTP id 0470E21F8DFD for ; Fri, 28 Dec 2012 10:35:50 -0800 (PST) Received: from ndjsppt03.ndc.nasa.gov (ndjsppt05.ndc.nasa.gov [198.117.1.104]) by ndmsnpf02.ndc.nasa.gov (Postfix) with ESMTP id 69C8FD004A; Fri, 28 Dec 2012 12:35:49 -0600 (CST) Received: from ndjshub02.ndc.nasa.gov (ndjshub02-pub.ndc.nasa.gov [198.117.1.161]) by ndjsppt05.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBSIZmrE023521; Fri, 28 Dec 2012 12:35:48 -0600 Received: from NDJSSCC07.ndc.nasa.gov ([198.117.4.178]) by ndjshub02.ndc.nasa.gov ([198.117.1.161]) with mapi; Fri, 28 Dec 2012 12:35:48 -0600 From: "Ivancic, William D. (GRC-RHN0)" To: Victor Lyamtsev , "dtn-users@irtf.org" Date: Fri, 28 Dec 2012 12:35:45 -0600 Thread-Topic: [dtn-users] dtn multicasting -? Thread-Index: Ac3lEA/ZCxfaNYeVS6Wz9FEDuQGcyQAGbai2AAAX2E8= Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-Entourage/13.11.0.110726 acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-28_07:2012-12-28, 2012-12-28, 1970-01-01 signatures=0 Subject: Re: [dtn-users] dtn multicasting -? X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Dec 2012 18:35:50 -0000 This may help also: http://tools.ietf.org/html/draft-pbasu-dtnrg-naming-00#section-2.2 Will ***************************** >> From: Victor Lyamtsev >> Date: Fri, 28 Dec 2012 09:28:54 -0600 >> To: "dtn-users@irtf.org" >> Subject: [dtn-users] dtn multicasting -? >>=20 >> Hello, >> Is there any completed standard/ implementation of bundle multicasting? >> P.S. I am not asking about something like NORM convergence layer with >> multicasting over ip, but the scenario with non-ip transport... >> Thank you, >> _______________________________________________ >> dtn-users mailing list >> dtn-users@irtf.org >> https://www.irtf.org/mailman/listinfo/dtn-users >=20 > _______________________________________________ > dtn-users mailing list > dtn-users@irtf.org > https://www.irtf.org/mailman/listinfo/dtn-users From robert.l.pitts@nasa.gov Fri Dec 28 10:41:31 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4928621F8DFC for ; Fri, 28 Dec 2012 10:41:31 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 69WGOEtwmi18 for ; Fri, 28 Dec 2012 10:41:30 -0800 (PST) Received: from ndmsnpf02.ndc.nasa.gov (ndmsnpf02.ndc.nasa.gov [198.117.0.122]) by ietfa.amsl.com (Postfix) with ESMTP id E8AB121F8BEA for ; Fri, 28 Dec 2012 10:41:29 -0800 (PST) Received: from ndmsppt05.ndc.nasa.gov (ndmsppt05.ndc.nasa.gov [198.117.0.104]) by ndmsnpf02.ndc.nasa.gov (Postfix) with ESMTP id 5BB90D0046; Fri, 28 Dec 2012 12:41:29 -0600 (CST) Received: from ndmshub06.ndc.nasa.gov (ndmshub06.ndc.nasa.gov [198.117.2.165]) by ndmsppt05.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBSIfTYa009636; Fri, 28 Dec 2012 12:41:29 -0600 Received: from NDMSSCC04.ndc.nasa.gov ([198.117.2.172]) by ndmshub06.ndc.nasa.gov ([198.117.2.165]) with mapi; Fri, 28 Dec 2012 12:41:28 -0600 From: "Pitts, Robert L. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "Ivancic, William D. (GRC-RHN0)" , Victor Lyamtsev , "dtn-users@irtf.org" Date: Fri, 28 Dec 2012 12:41:28 -0600 Thread-Topic: [dtn-users] dtn multicasting -? Thread-Index: Ac3lEA/ZCxfaNYeVS6Wz9FEDuQGcyQAGbai2AAAX2E8AAClI0A== Message-ID: <8A674285E0E0E344965DF27CCFEE30EC59B256E4DB@NDMSSCC04.ndc.nasa.gov> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-28_07:2012-12-28, 2012-12-28, 1970-01-01 signatures=0 Subject: Re: [dtn-users] dtn multicasting -? X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Dec 2012 18:41:31 -0000 I think you are right Will. We have talked about some implementations but = haven't had the time yet to go much further. It would be helpful to our pa= yload users. Lee -----Original Message----- From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Beh= alf Of Ivancic, William D. (GRC-RHN0) Sent: Friday, December 28, 2012 12:36 PM To: Victor Lyamtsev; dtn-users@irtf.org Subject: Re: [dtn-users] dtn multicasting -? This may help also: http://tools.ietf.org/html/draft-pbasu-dtnrg-naming-00#section-2.2 Will ***************************** >> From: Victor Lyamtsev >> Date: Fri, 28 Dec 2012 09:28:54 -0600 >> To: "dtn-users@irtf.org" >> Subject: [dtn-users] dtn multicasting -? >>=20 >> Hello, >> Is there any completed standard/ implementation of bundle multicasting? >> P.S. I am not asking about something like NORM convergence layer with=20 >> multicasting over ip, but the scenario with non-ip transport... >> Thank you, >> _______________________________________________ >> dtn-users mailing list >> dtn-users@irtf.org >> https://www.irtf.org/mailman/listinfo/dtn-users >=20 > _______________________________________________ > dtn-users mailing list > dtn-users@irtf.org > https://www.irtf.org/mailman/listinfo/dtn-users _______________________________________________ dtn-users mailing list dtn-users@irtf.org https://www.irtf.org/mailman/listinfo/dtn-users From elwynd@folly.org.uk Fri Dec 28 11:36:31 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 04F1C21F8E11 for ; Fri, 28 Dec 2012 11:36:31 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.289 X-Spam-Level: X-Spam-Status: No, score=-2.289 tagged_above=-999 required=5 tests=[AWL=0.310, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tcBaEbptpve8 for ; Fri, 28 Dec 2012 11:36:30 -0800 (PST) Received: from a.painless.aa.net.uk (a.painless.aa.net.uk [IPv6:2001:8b0:0:30::51bb:1e33]) by ietfa.amsl.com (Postfix) with ESMTP id 70DA221F8E10 for ; Fri, 28 Dec 2012 11:36:30 -0800 (PST) Received: from mightyatom.folly.org.uk ([81.187.254.250]) by a.painless.aa.net.uk with esmtp (Exim 4.77) (envelope-from ) id 1TofjF-0007N6-Kt; Fri, 28 Dec 2012 19:36:25 +0000 From: Elwyn Davies To: Victor Lyamtsev In-Reply-To: <20121228152854.41334.qmail@mzeal.com> References: <20121228152854.41334.qmail@mzeal.com> Content-Type: text/plain Organization: Folly Consulting Date: Fri, 28 Dec 2012 19:35:30 +0000 Message-Id: <1356723331.11916.16701.camel@mightyatom> Mime-Version: 1.0 X-Mailer: Evolution 2.26.3 Content-Transfer-Encoding: 7bit Cc: dtn-users@irtf.org Subject: Re: [dtn-users] dtn multicasting -? X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Dec 2012 19:36:31 -0000 Hi. The nearest thing to multicasting in DTN2 is epidemic routing which tries to send a received bundle to every other contact that the node makes until the bundle expires. At present the MULTINODE endpoint concept is effectively not implemented at all in DTN2 although the low levels have the concept defined. This would potentially give you a form of multicast group. The feature has been requested for ION (http://sourceforge.net/p/ion-dtn/feature-requests/3/) this year. I would also like to have it for certain ICN-like operations but implementing multinode capabilities requires a considerable amount of work in DTN2: - allowing nodes to have more than one local EID - new discovery/announcement processes for offering auxiliary EIDs as well as the 'master' one that identifies the administrative name. - managing 'auxiliary' logical links and routes that use the master physical link to reach EIDs containing the auxiliary EIDs - adapting exiting routing/forwarding protocols to deal with MULTINODE EIDs It might be possible to simplify the work on routing by treating the multipoint EID as a form of service on the 'master' EID rather than an independent EID in its own right. Thus sending to dtn://some.multicast.service/zzz which is marked as a MULTIPOINT EID is effectively equivalent to sending to dtn:/*/some.multicast.service/zzz. However, don't hold your breath for an imminent implementation in DTN2. Regards, Elwyn On Fri, 2012-12-28 at 15:28 +0000, Victor Lyamtsev wrote: > Hello, > Is there any completed standard/ implementation of bundle multicasting? > P.S. I am not asking about something like NORM convergence layer with multicasting over ip, but the scenario with non-ip transport... > Thank you, > _______________________________________________ > dtn-users mailing list > dtn-users@irtf.org > https://www.irtf.org/mailman/listinfo/dtn-users From scott.c.burleigh@jpl.nasa.gov Fri Dec 28 16:54:40 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9680021F8E09 for ; Fri, 28 Dec 2012 16:54:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.432 X-Spam-Level: X-Spam-Status: No, score=-6.432 tagged_above=-999 required=5 tests=[AWL=0.167, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sld8vX1Tk1st for ; Fri, 28 Dec 2012 16:54:40 -0800 (PST) Received: from mail.jpl.nasa.gov (smtp.jpl.nasa.gov [128.149.139.106]) by ietfa.amsl.com (Postfix) with ESMTP id 0CDD721F8E0B for ; Fri, 28 Dec 2012 16:54:39 -0800 (PST) Received: from mail.jpl.nasa.gov (ap-ehub-sp02.jpl.nasa.gov [128.149.137.149]) by smtp.jpl.nasa.gov (Sentrion-MTA-4.2.2/Sentrion-MTA-4.2.2) with ESMTP id qBT0sa8D031473 (using TLSv1/SSLv3 with cipher AES128-SHA (128 bits) verified NO); Fri, 28 Dec 2012 16:54:37 -0800 Received: from AP-EMBX-SP20.RES.AD.JPL ([169.254.8.182]) by ap-ehub-sp02.RES.AD.JPL ([fe80::dd85:7b07:1e36:7e3c%15]) with mapi id 14.02.0318.001; Fri, 28 Dec 2012 16:54:36 -0800 From: "Burleigh, Scott C (313B)" To: Elwyn Davies , Victor Lyamtsev Thread-Topic: [dtn-users] dtn multicasting -? Thread-Index: AQHN5RAP6ak5n9Ahk0aTv6v65PMVnJgvIPkA///QP1A= Date: Sat, 29 Dec 2012 00:54:35 +0000 Message-ID: References: <20121228152854.41334.qmail@mzeal.com> <1356723331.11916.16701.camel@mightyatom> In-Reply-To: <1356723331.11916.16701.camel@mightyatom> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [128.149.137.113] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Source-Sender: scott.c.burleigh@jpl.nasa.gov X-AUTH: Authorized Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] dtn multicasting -? X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Dec 2012 00:54:40 -0000 There is an implementation of multicast in ION that conforms to the "imc" I= nternet Draft that Will referenced. It's not necessarily limited to a sche= duled-contacts environment, but ION itself does not yet have any opportunis= tic forwarding capability; that may change later this year, but for now you= would be stuck with static routing or scheduled contacts. Also, IMC relie= s on establishment of a multicast forwarding tree, and currently in ION tha= t has to be done by hand, using an administration program. A spanning-tree= configuration protocol is needed, but it doesn't exist yet. Scott -----Original Message----- From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Beh= alf Of Elwyn Davies Sent: Friday, December 28, 2012 11:36 AM To: Victor Lyamtsev Cc: dtn-users@irtf.org Subject: Re: [dtn-users] dtn multicasting -? Hi. The nearest thing to multicasting in DTN2 is epidemic routing which tries t= o send a received bundle to every other contact that the node makes until t= he bundle expires. =20 At present the MULTINODE endpoint concept is effectively not implemented at= all in DTN2 although the low levels have the concept defined. This would = potentially give you a form of multicast group. The feature has been requested for ION (http://sourceforge.net/p/ion-dtn/feature-requests/3/) this year. I would also like to have it for certain ICN-like operations but implement= ing multinode capabilities requires a considerable amount of work in DTN2: - allowing nodes to have more than one local EID - new discovery/announcement processes for offering auxiliary EIDs as well = as the 'master' one that identifies the administrative name. - managing 'auxiliary' logical links and routes that use the master physica= l link to reach EIDs containing the auxiliary EIDs - adapting exiting routing/forwarding protocols to deal with MULTINODE EIDs It might be possible to simplify the work on routing by treating the multip= oint EID as a form of service on the 'master' EID rather than an independen= t EID in its own right. Thus sending to dtn://some.multicast.service/zzz w= hich is marked as a MULTIPOINT EID is effectively equivalent to sending to = dtn:/*/some.multicast.service/zzz. However, don't hold your breath for an imminent implementation in DTN2. Regards, Elwyn =20 On Fri, 2012-12-28 at 15:28 +0000, Victor Lyamtsev wrote: > Hello, > Is there any completed standard/ implementation of bundle multicasting? > P.S. I am not asking about something like NORM convergence layer with mul= ticasting over ip, but the scenario with non-ip transport...=20 > Thank you, > _______________________________________________ > dtn-users mailing list > dtn-users@irtf.org > https://www.irtf.org/mailman/listinfo/dtn-users _______________________________________________ dtn-users mailing list dtn-users@irtf.org https://www.irtf.org/mailman/listinfo/dtn-users From william.d.ivancic@nasa.gov Mon Dec 31 05:27:04 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 57FB321F8809 for ; Mon, 31 Dec 2012 05:27:04 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.411 X-Spam-Level: X-Spam-Status: No, score=-6.411 tagged_above=-999 required=5 tests=[AWL=0.188, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JtaKZS3307rE for ; Mon, 31 Dec 2012 05:27:03 -0800 (PST) Received: from ndmsnpf03.ndc.nasa.gov (ndmsnpf03.ndc.nasa.gov [198.117.0.123]) by ietfa.amsl.com (Postfix) with ESMTP id 90EAD21F87F9 for ; Mon, 31 Dec 2012 05:27:03 -0800 (PST) Received: from ndjsppt03.ndc.nasa.gov (ndjsppt05.ndc.nasa.gov [198.117.1.104]) by ndmsnpf03.ndc.nasa.gov (Postfix) with ESMTP id 8201A1812BE; Mon, 31 Dec 2012 07:27:02 -0600 (CST) Received: from ndjshub04.ndc.nasa.gov (ndjshub04-pub.ndc.nasa.gov [198.117.1.34]) by ndjsppt05.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBVDR1M6003573; Mon, 31 Dec 2012 07:27:02 -0600 Received: from NDJSSCC07.ndc.nasa.gov ([198.117.4.178]) by ndjshub04.ndc.nasa.gov ([10.202.202.163]) with mapi; Mon, 31 Dec 2012 07:27:01 -0600 From: "Ivancic, William D. (GRC-RHN0)" To: Victor Lyamtsev Date: Mon, 31 Dec 2012 07:26:59 -0600 Thread-Topic: [dtn-users] dtn multicasting -? Thread-Index: Ac3lMqWY3zmj4bzYQPGSweU9Dwvq9QCJ90jt Message-ID: In-Reply-To: <1356723331.11916.16701.camel@mightyatom> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-Entourage/13.11.0.110726 acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-31_02:2012-12-31, 2012-12-31, 1970-01-01 signatures=0 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] dtn multicasting -? X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2012 13:27:04 -0000 Victor, I agree with Elwyn that looking at multicast EID more like a service may help. Get a copy of John Day's book Patterns in Network Architecture - not an easy read. Chapter 9 goes into multicast, multi-homing and mobility and such and gives a good discussion on what thos= e concepts really mean. In the IP protocol we have multicast addresses, but they really are not addresses in the normal sense. Rather a multicast address represents a set of endpoints. Thus, a multicast EID represents a set of EIDs (multicast group members). I think if you look at IP multicast you can easily confused. The other thing to consider is what do you really want to accomplish with multicast? There may be better ways. Information Centric Networking has = a lot of multicast like features that may be all you need. --Will ****************************** > From: Elwyn Davies > Date: Fri, 28 Dec 2012 13:35:30 -0600 > To: Victor Lyamtsev > Cc: "dtn-users@irtf.org" > Subject: Re: [dtn-users] dtn multicasting -? >=20 > Hi. >=20 > The nearest thing to multicasting in DTN2 is epidemic routing which > tries to send a received bundle to every other contact that the node > makes until the bundle expires. >=20 > At present the MULTINODE endpoint concept is effectively not implemented > at all in DTN2 although the low levels have the concept defined. This > would potentially give you a form of multicast group. >=20 > The feature has been requested for ION > (http://sourceforge.net/p/ion-dtn/feature-requests/3/) this year. >=20 > I would also like to have it for certain ICN-like operations but > implementing multinode capabilities requires a considerable amount of > work in DTN2: > - allowing nodes to have more than one local EID > - new discovery/announcement processes for offering auxiliary EIDs as > well as the 'master' one that identifies the administrative name. > - managing 'auxiliary' logical links and routes that use the master > physical link to reach EIDs containing the auxiliary EIDs > - adapting exiting routing/forwarding protocols to deal with MULTINODE > EIDs >=20 > It might be possible to simplify the work on routing by treating the > multipoint EID as a form of service on the 'master' EID rather than an > independent EID in its own right. Thus sending to > dtn://some.multicast.service/zzz which is marked as a MULTIPOINT EID is > effectively equivalent to sending to dtn:/*/some.multicast.service/zzz. >=20 > However, don't hold your breath for an imminent implementation in DTN2. >=20 > Regards, > Elwyn > =20 > On Fri, 2012-12-28 at 15:28 +0000, Victor Lyamtsev wrote: >> Hello, >> Is there any completed standard/ implementation of bundle multicasting? >> P.S. I am not asking about something like NORM convergence layer with >> multicasting over ip, but the scenario with non-ip transport... >> Thank you, >> _______________________________________________ >> dtn-users mailing list >> dtn-users@irtf.org >> https://www.irtf.org/mailman/listinfo/dtn-users >=20 > _______________________________________________ > dtn-users mailing list > dtn-users@irtf.org > https://www.irtf.org/mailman/listinfo/dtn-users From vlyamtsev@mzeal.com Mon Dec 31 06:08:49 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5ABAD21F883B for ; Mon, 31 Dec 2012 06:08:49 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.299 X-Spam-Level: X-Spam-Status: No, score=-1.299 tagged_above=-999 required=5 tests=[AWL=1.300, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id erl3Fk2pc3p6 for ; Mon, 31 Dec 2012 06:08:48 -0800 (PST) Received: from mzeal.com (mzeal.com [199.237.228.71]) by ietfa.amsl.com (Postfix) with ESMTP id 809DB21F8839 for ; Mon, 31 Dec 2012 06:08:48 -0800 (PST) Received: (qmail 56482 invoked by uid 25671); 31 Dec 2012 14:08:45 -0000 Message-ID: <20121231140845.56481.qmail@mzeal.com> Content-Disposition: inline Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="US-ASCII"; format="flowed" MIME-Version: 1.0 X-Mailer: MIME::Lite 3.0104 (F2.72; T1.15; A1.47; B3.01; Q3.01) Date: Mon, 31 Dec 2012 14:08:45 +0000 From: "Victor Lyamtsev" To: william.d.ivancic@nasa.gov, vlyamtsev@mzeal.com Cc: dtn-users@irtf.org Subject: Re: [dtn-users] dtn multicasting -? X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2012 14:08:49 -0000 William, Elwyn: What i want to accomplish: reliable one-to-many message distribution in challenged MANETS, DTN is considered as one of the approaches... In IP the concept is pretty clear: the same packet must be replicated to the range of endpoints within class D address. The confusing part was that i didn't find ( or, missed?) any part of DTN standards that would define DTN multicasting as such, just many papers on the topic. Thank you very much for clarifications and happy New Year. -Victor ------- Original Message ------- On 12/31/2012 1:26 PM Ivancic, William D. (GRC-RHN0) wrote: Victor, I agree with Elwyn that looking at multicast EID more like a service may help. Get a copy of John Day's book Patterns in Network Architecture - not an easy read. Chapter 9 goes into multicast, multi-homing and mobility and such and gives a good discussion on what those concepts really mean. In the IP protocol we have multicast addresses, but they really are not addresses in the normal sense. Rather a multicast address represents a set of endpoints. Thus, a multicast EID represents a set of EIDs (multicast group members). I think if you look at IP multicast you can easily confused. The other thing to consider is what do you really want to accomplish with multicast? There may be better ways. Information Centric Networking has a lot of multicast like features that may be all you need. --Will ****************************** > From: Elwyn Davies > Date: Fri, 28 Dec 2012 13:35:30 -0600 > To: Victor Lyamtsev > Cc: "dtn-users@irtf.org" > Subject: Re: [dtn-users] dtn multicasting -? > > Hi. > > The nearest thing to multicasting in DTN2 is epidemic routing which > tries to send a received bundle to every other contact that the node > makes until the bundle expires. > > At present the MULTINODE endpoint concept is effectively not implemented > at all in DTN2 although the low levels have the concept defined. This > would potentially give you a form of multicast group. > > The feature has been requested for ION > (http://sourceforge.net/p/ion-dtn/feature-requests/3/) this year. > > I would also like to have it for certain ICN-like operations but > implementing multinode capabilities requires a considerable amount of > work in DTN2: > - allowing nodes to have more than one local EID > - new discovery/announcement processes for offering auxiliary EIDs as > well as the 'master' one that identifies the administrative name. > - managing 'auxiliary' logical links and routes that use the master > physical link to reach EIDs containing the auxiliary EIDs > - adapting exiting routing/forwarding protocols to deal with MULTINODE > EIDs > > It might be possible to simplify the work on routing by treating the > multipoint EID as a form of service on the 'master' EID rather than an > independent EID in its own right. Thus sending to > dtn://some.multicast.service/zzz which is marked as a MULTIPOINT EID is > effectively equivalent to sending to dtn:/*/some.multicast.service/zzz. > > However, don't hold your breath for an imminent implementation in DTN2. > > Regards, > Elwyn > > On Fri, 2012-12-28 at 15:28 +0000, Victor Lyamtsev wrote: >> Hello, >> Is there any completed standard/ implementation of bundle multicasting? >> P.S. I am not asking about something like NORM convergence layer with >> multicasting over ip, but the scenario with non-ip transport... >> Thank you, >> _______________________________________________ >> dtn-users mailing list >> dtn-users@irtf.org >> https://www.irtf.org/mailman/listinfo/dtn-users > > _______________________________________________ > dtn-users mailing list > dtn-users@irtf.org > https://www.irtf.org/mailman/listinfo/dtn-users From william.d.ivancic@nasa.gov Mon Dec 31 07:37:42 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7BD6B21F884A for ; Mon, 31 Dec 2012 07:37:42 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.432 X-Spam-Level: X-Spam-Status: No, score=-6.432 tagged_above=-999 required=5 tests=[AWL=0.167, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZaAcI0NnSjLL for ; Mon, 31 Dec 2012 07:37:41 -0800 (PST) Received: from ndmsnpf03.ndc.nasa.gov (ndmsnpf03.ndc.nasa.gov [198.117.0.123]) by ietfa.amsl.com (Postfix) with ESMTP id 5FA2A21F885E for ; Mon, 31 Dec 2012 07:37:41 -0800 (PST) Received: from ndjsppt02.ndc.nasa.gov (ndjsppt02.ndc.nasa.gov [198.117.1.101]) by ndmsnpf03.ndc.nasa.gov (Postfix) with ESMTP id 14FAA180912; Mon, 31 Dec 2012 09:37:40 -0600 (CST) Received: from ndjshub01.ndc.nasa.gov (ndjshub01-pub.ndc.nasa.gov [198.117.1.160]) by ndjsppt02.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBVFbdAH015771; Mon, 31 Dec 2012 09:37:39 -0600 Received: from NDJSSCC07.ndc.nasa.gov ([198.117.4.178]) by ndjshub01.ndc.nasa.gov ([198.117.1.160]) with mapi; Mon, 31 Dec 2012 09:37:39 -0600 From: "Ivancic, William D. (GRC-RHN0)" To: Victor Lyamtsev Date: Mon, 31 Dec 2012 09:37:37 -0600 Thread-Topic: [dtn-users] dtn multicasting -? Thread-Index: Ac3nYFmVR4Nmb1y/S829pm5rtMD7HQADGj0Y Message-ID: In-Reply-To: <20121231140845.56481.qmail@mzeal.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-Entourage/13.11.0.110726 acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-31_02:2012-12-31, 2012-12-31, 1970-01-01 signatures=0 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] dtn multicasting -? X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2012 15:37:42 -0000 A few things to think about. One-to-many. Home much is many? 5, 10, 50,000? If low, duplicate unicast my actually result in less system-wide resource utilization. Is your manet experiencing disconnection and disruption? Are you multicast groups pre-defined prior to deployment or is there dynami= c join and leave? Dynamic creation of multicast groups in a manet, is, to th= e best of my knowledge, not well defined. Dynamic creation of multicast groups in a DTN is definitely not defined. You might take a look at this paper for at least some EID naming concepts: Endpoint Naming for Space Delay / Disruption Tolerant Networking http://www.mitre.org/work/tech_papers/2010/09_5229/09_5229.pdf FYI, I may be able to create an EID such as: dtn://lat35.*lon-74.* And I may even figure out how to send it to that location. Now, how do the devices at that location know to listen? Lots of research and deployment issues that still need addressing. -- Will ****************************** William D. Ivancic Phone 216-433-3494 Fax 216-433-8705 Networking Lab 216-433-2620 Mobile 440-503-4892 http://roland.grc.nasa.gov/~ivancic > From: Victor Lyamtsev > Date: Mon, 31 Dec 2012 08:08:45 -0600 > To: "Ivancic, William D. (GRC-RHN0)" , > "vlyamtsev@mzeal.com" > Cc: "dtn-users@irtf.org" , "elwynd@folly.org.uk" > > Subject: RE: Re: [dtn-users] dtn multicasting -? >=20 > William, Elwyn: > What i want to accomplish: reliable one-to-many message distribution in > challenged MANETS, DTN is considered as one of the approaches... > In IP the concept is pretty clear: the same packet must be replicated to = the > range of endpoints within class D address. The confusing part was that i > didn't find ( or, missed?) any part of DTN standards that would define DT= N > multicasting as such, just many papers on the topic. > Thank you very much for clarifications and happy New Year. > -Victor >=20 > ------- Original Message ------- > On 12/31/2012 1:26 PM Ivancic, William D. (GRC-RHN0) wrote: > Victor, I agree with Elwyn that looking at multicast EID more like a > service may help. Get a copy of John Day's book Patterns in Network > Architecture - not an easy read. Chapter 9 goes into multicast, > multi-homing and mobility and such and gives a good discussion on what th= ose > concepts really mean. In the IP protocol we have multicast addresses, bu= t > they really are not addresses in the normal sense. Rather a multicast > address represents a set of endpoints. Thus, a multicast EID represents = a > set of EIDs (multicast group members). I think if you look at IP multicas= t > you can easily confused. >=20 >=20 > The other thing to consider is what do you really want to accomplish with > multicast? There may be better ways. Information Centric Networking ha= s a > lot of multicast like features that may be all you need. >=20 > --Will >=20 > ****************************** >=20 >=20 >=20 >> From: Elwyn Davies >> Date: Fri, 28 Dec 2012 13:35:30 -0600 >> To: Victor Lyamtsev >> Cc: "dtn-users@irtf.org" >> Subject: Re: [dtn-users] dtn multicasting -? >>=20 >> Hi. >>=20 >> The nearest thing to multicasting in DTN2 is epidemic routing which >> tries to send a received bundle to every other contact that the node >> makes until the bundle expires. >>=20 >> At present the MULTINODE endpoint concept is effectively not implemented >> at all in DTN2 although the low levels have the concept defined. This >> would potentially give you a form of multicast group. >>=20 >> The feature has been requested for ION >> (http://sourceforge.net/p/ion-dtn/feature-requests/3/) this year. >>=20 >> I would also like to have it for certain ICN-like operations but >> implementing multinode capabilities requires a considerable amount of >> work in DTN2: >> - allowing nodes to have more than one local EID >> - new discovery/announcement processes for offering auxiliary EIDs as >> well as the 'master' one that identifies the administrative name. >> - managing 'auxiliary' logical links and routes that use the master >> physical link to reach EIDs containing the auxiliary EIDs >> - adapting exiting routing/forwarding protocols to deal with MULTINODE >> EIDs >>=20 >> It might be possible to simplify the work on routing by treating the >> multipoint EID as a form of service on the 'master' EID rather than an >> independent EID in its own right. Thus sending to >> dtn://some.multicast.service/zzz which is marked as a MULTIPOINT EID is >> effectively equivalent to sending to dtn:/*/some.multicast.service/zzz. >>=20 >> However, don't hold your breath for an imminent implementation in DTN2. >>=20 >> Regards, >> Elwyn >> =20 >> On Fri, 2012-12-28 at 15:28 +0000, Victor Lyamtsev wrote: >>> Hello, >>> Is there any completed standard/ implementation of bundle multicasting? >>> P.S. I am not asking about something like NORM convergence layer with >>> multicasting over ip, but the scenario with non-ip transport... >>> Thank you, >>> _______________________________________________ >>> dtn-users mailing list >>> dtn-users@irtf.org >>> https://www.irtf.org/mailman/listinfo/dtn-users >>=20 >> _______________________________________________ >> dtn-users mailing list >> dtn-users@irtf.org >> https://www.irtf.org/mailman/listinfo/dtn-users >=20

Sergey,

You’re persistence go= t us here so you are welcome to make a patch if you would like. I have a se= t of patches working their way through export approval now. Once I get thos= e thrown over the fence I don’t mind making the additional patch. I a= dded options to override the socket buffer sizes that I only had time to im= plement on the UDP side that I could look into adding to the TCP side now t= hat I’ve got some experience with it.

Best regards,

DZ

 

David Zoller

COLSA Corporat= ion

HOSC / C10= 7 

(Off= ice: (256) 544-1820
*= EMail: david.a.zoller@n= asa.gov

=  

From: ssireskin@gmail.com [m= ailto:ssireskin@gmail.com]
Sent: Tuesday, December 18, 2012 6:08= AM
To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
<= b>Cc: dtn-users@irtf.org
Subject: Re: [dtn-users] A problem w= ith dtntunnel

 

Hi David,

Than= ks for the hint. I have also noticed that memory usage of dtntunnel noticea= bly grows while running iperf with multiple flows.
Are you going to subm= it all the fixes to mercurial?

--
Best regards,
Sergey Sires= kin

2012/12/18 Zoller, Davi= d A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

Loo= king at the rest of the code, there are probably a couple of memory leaks. = The biggie is after the call to “send_bundle(b_xmit” on success b_xmit should be deleted before setting it to NULL (n= ear line 432). Similarly after the send_bundle for the “error connect= ing” the APIBundle b should also be deleted before going to done (nea= r line 342).

You’re welcome,

DZ

&n= bsp;

David Zoller

COLSA Corporation

Marshall = Space Flight Center

 

From: ssireskin@gmail.com [mailto:ssireskin@gmail.com]
Sent: Tuesday, December = 18, 2012 1:40 AM


To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc:= David Zoller; dtn-= users@irtf.org
Subject: Re: [dtn-users] A problem with dtntun= nel

 

Hi Dav= id,

Now it works like a charm, thank you very much!

2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] &= lt;david.a.zol= ler@nasa.gov>

Sergey,

That was my bad. In the patch below I moved the = “delete b_recv;” line above the = “= if (recv_hdr->eof_) {” block as a short cut to d= elete it before the possible exit. Move it back down below the if block and= you should be okay.

Sorry, that was one= of those possible loose ends I mentioned J,

DZ

 

David Zoller

C= OLSA Corporation

HOSC / C107  =

(Office: (256) 5= 44-1820
*EMail: david.a.zoller@nasa.gov<= /span>

 

From= : ssireskin@gma= il.com [mailto:ssireskin@gmail.com]
Sent: Monday, December 17, 2012 8:05 A= M
To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
= Cc: David Zoller; dtn-users@irtf.org


Subj= ect: Re: [dtn-users] A problem with dtntunnel

 

David,

The problem is probab= ly not with the reordering. Can it be, that "if (recv_hdr->eof_)&qu= ot; is not the right condition to close a connection? Or maybe it is not a = sufficient condition? Which connection is recv_hdr about? Can it be a local= TCP connection on the receiving side (dtntunnel <-> nc)? If it is, t= hen IMO this condition becomes true when the receiving application doesn't = send any data to the sending application, please see line 406 of the origin= al code.

2012/12/17 ssireskin@gmail.com <ssireskin@gmail.com>

David,

I added this code and a log_debug inside the if-= clause. According to the log, this condition doesn't get true.
A have at= tached my logs and tcpdumps. Please, see dtntunnel_T.log - it's a receiving= dtntunnel log.
There are 4 bundles after a eof bundle. This doesn't hap= pen with the original dtntunnel.


--
Best regar= ds,
Sergey Sireskin

2012/12/17 Zoller, = David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

 

= Hi Sergey,

I don’t have you= r setup and this has not been tested but give this a try. In TCPTunnel::Con= nection::handle_bundle in the while(1) loop add the test to see if the next= _seqno_ is a match:

 

&n= bsp;   while (1) {

      = ;  iter =3D reorder_table_.find(next_seqno_);

  &= nbsp;     if (iter =3D=3D reorder_table_.end()) {

            br= eak;

        }

&nb= sp;

        // = add this code:

      = ;  // if seqno does not match then still a bundle missing<= /o:p>

        if (iter->first = !=3D next_seqno_) {

     =        break;

<= span style=3D'font-size:9.0pt;font-family:"Courier New";color:#00B050'>&nbs= p;       }

 

 = ;       bundle =3D iter->second;

 &nb= sp;            =     bundle->payload_.len(), next_seqno_);

&nbs= p;  

        = reorder_table_.erase(iter);

      =   next_seqno_++;

   

<= p class=3DMsoNormal style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:= auto'>  = ;      queue_.push_back(bundle);<= /o:p>

    }  

&n= bsp;

 

=  

 

David Zoller

COLSA Corporat= ion

HOSC / C107 

(Office: (256) 544-1820
*EMail: david.a.zoller@nasa.gov

 

From:<= span style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> dtn-users-bounces@= irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Monday, December 17, 2012 6:40 AM
To: David Zoller


Cc: dtn-users@irtf.org
Subject: Re: [dt= n-users] A problem with dtntunnel

 

Hi David,



Thanks for your pa= tch, it works! :) Connection is cleanly terminated now.

However, the= re still seems to be a problem. I run dtntunnel over a link with high laten= cy and about 30% packet loss. The receiving nc sometimes gets FIN packet be= fore it receives all the data. If I understand it right, on a lossy link bu= ndles may come out of order. In this case the EOF bundle may come too early= . I think that the reordering procedure in dtntunnel should be reviewed. Ju= st for information, pure TCP performs better that DTN over this link.<= /o:p>

2012/12/14 David Zoller <zollerd@gmail.com>= ;

Hi Sergey,

I ca= n't stay away from a problem that is nagging at me so I made some time this= morning. Going back to the original TCPTunnel.cc code, when a bundle is re= ceived that has the EOF indication, an empty bundle with the same EOF flag = set needs to be returned to close the loop (the second FIN). There may stil= l be some loose ends or clean up needed but try the code below at the end o= f the while loop.

 

I did not do a packet capture but I saw that if the receiving= nc is not running when you start the sender nc then the receiving dtntunne= l logs an error on the failed connect and returns an EOF bundle which close= s the sender nc. The EOF bundle could possibly be enhanced to include an er= ror message.

 

I agree that some applications could benefit from a "pretend = to be a receiver" option with the default working as a bidirectional e= nd-to-end TCP connection.

 =

Thanks for dragging me along kicking and screaming on= this lesson in DTN and TCP tunneling. I've only been using dtntunnel as a = one way UDP transmitter. I see a good fit for the TCP tunnel in an upcoming= project and I'll try to make the pitch to use it. This is a pretty cool li= ttle app.

Best regards,

DZ

 

=

 

 

 

 

484             dele= te b_recv;

485 

486             if (recv_hdr->eo= f_) {

487          = ;       log_info("bundle had eof bit set... closing con= nection");

488 

489                 = if ( !sock_eof ) {

490      =               sock_eof =3D true;

491             &nb= sp;       // send an empty bundle back since we did not init= iate the close

492       &nb= sp;             dtn::APIBundle* b =3D new dtn= ::APIBundle();

493       &nb= sp;             hdr.eof_ =3D 1;

494               &nb= sp;     hdr.seqno_ =3D ntohl(send_seqno++);

<= div>

495                 &nbs= p;   memcpy(b->payload_.buf(sizeof(hdr)), &hdr, sizeof(hdr));

496            = ;         b->payload_.set_len(sizeof(hdr));

497              = ;       int err;

498   =                   if ((err =3D= tunnel->send_bundle(b, &dest_eid_)) !=3D DTN_SUCCESS) {<= /p>

499               &= nbsp;         log_err("error sending final socket = closed packet bundle: %s",

500   =                      = ;         dtn_strerror(err));

=

501                   &= nbsp;     tcptun_->kill_connection(this);

=

502                 &nb= sp;       exit(1);

503  = ;                   }

504              = ;       delete b;

505  =               }

506                 sock_.= close();

507         &n= bsp;       goto done;

508 &n= bsp;           }

5= 09         }

510  =   }

511 

512  done:

 <= /o:p>

 

Hi Sergey= ,

Research indicates that = the TIME_WAIT state is a feature of TCP to prevent delayed packets for conn= ection from being delivered to a later connection between the same hosts. [= http://tools.ietf.org/html/draft-faber-time-wait-avoidance-00]

 

Based on that, the added “if (sock_eof)” bl= ock can be removed and the while condition can be changed to “while (= !sock_eof)” and you get the same result.

 

But, looking at the intent of the code and at the oasys::IPSocket, it app= ears that it is possible to establish a socket connection and then only clo= se the read side while keeping the write side open. I’ve never run ac= ross a socket used in that manner but that is what it looks like to me. So,= the TCPTunnel::Connection::run  while loop allows for the read side t= o close and continue transmitting bundle payloads on the write side until i= t gets a bundle with the EOF bit set. If there are no bundles coming back t= he other way as is the case here then the while loop never exits which keep= s the netcat connection open.

 

So, based on th= at, I think a better solution would be to implement an option that configur= es dtntunnel to close the connection when the read side terminates. (Someon= e in the know can weigh in) Your solution is probably sufficient for your t= est.

 

--

 

As to c= onnections not being removed from the table and sockets not being closed= 230;

 

I set up a test to kick off 100 netcats = on an 11MB file so that the connection would be open for several seconds:

= #!/bin/bash

ctr=3D0

while [ $ctr -lt 100 ]; do

    nc -w = 10 x.x.x.x 12345 < data11mg  &

    let ctr=3Dct= r+1

done

 

I also added some debug code to pri= nt out a connection accepted counter and closed counter as they are accepte= d and closed.

After 30 con= nections, the first close kicks in and then there is a mix of connects and = closes until finished.

All= of the connections were closed and I did not have any sockets left open.

 

On my first attempt I exceeded my payload quot= a and started seeing “error sending bundle: 141” messages so I = don’t think that is the issue you are running into.=

 

You might try putting in similar debug to see if you can deter= mine what is happening.

 

Hope this helps,=

DZ

 

 

David Zoller

<= span style=3D'font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F49= 7D'>COLSA Corporation

HOSC / C107 

(Office: (256) 544-1820=
*EMail: david.a.zoller@nasa.gov

 

From: ss= ireskin@gmail.com [mailto:ssireskin@gmail.com]
Sent: Wednesday, = December 12, 2012 2:07 PM
To: Zoller, David A. (MSFC-EO60)[HOSC S= ERVICES CONTRACT]
Cc: dtn-users@irtf.org
Subject: Re: [= dtn-users] A problem with dtntunnel

 

Hi David,

I meant exactly the same position before the end of the w= hile loop, just described it incorrect.

Please let me know if you fi= nd the solution for this problem.

Best regards,
Sergey=

2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC = SERVICES CONTRACT] <david.a.zoller@nasa.gov>

<= span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F= 497D'>Hi Sergey,

I was narrowing in on t= he same track... The FIN results in a returned length of zero on the read a= nd errno is also zero so you have to key on the return length which is alre= ady in the code.

 

You may have added a couple of lines of debug code and i= t should be inserted just before the end of the while(1) loop at line 491 i= n the latest version on Sourceforge.

&nb= sp;

I am seeing the sending socket stay = in the TIME_WAIT mode even if I add a call to sock_.shutdown(SHUT_RDWR) bef= ore closing the socket;

Getting close I = think.

 

The 0.0.0.0 was because I did not have anything at the other end o= f the tunnel and just stuck some easy numbers in there.

Thanks,
DZ

&nbs= p;

David Zoller

COLSA Corporation

Marshall Space Fl= ight Center

 

=

From: s= sireskin@gmail.com [mailto:ssireskin@gmail.com]
Sent: Wednesday, December = 12, 2012 11:06 AM


<= b>To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.or= g
Subject: Re: [dtn-users] A problem with dtntunnel

 

Hi David,

I = added the following code at the line 492 of TCPTunnel.cc and it seems that = it helped.

    if (sock_eof) {
   =     sock_.close();
       = goto done;
    }

The senders, both nc and iperf, = exit after having finished sending the data. There are
no more sockets i= n CLOSE_WAIT state. However I am completely unsure whether
my code is c= orrect and is in correct place.

Now I have encountered another probl= em. When iperf test is run with 100 parallel flows,
some dtntunnel conne= ctions are not removed from dtntunel's connection table, and the
related= sockets are not closed. I suppose this happens because my code addition is= far
from perfect.

And one more question, what does 0.0.0.0 mean = in -T tunnel specification? Is it just an
example host, or 0.0.0.0 has s= ome special meaning for dtntunnel?

2012/12/12 Zol= ler, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

<= span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F= 497D'>I have duplicated what you are reporting and have to retract my earli= er statement saying that this is the netcat design and not a problem J

The sender netcat attempts to close the soc= ket when it gets the [EOF] and not in response to the receiver closing its = end.

 

=  

netcat sender:

cat /etc/passwd | nc x.x.x.= x 12345

 

scenario 1 - netcat receiver with the keep listener option:

      &n= bsp;         nc -k -l 12345<= o:p>

·         Sender terminates and receiver stays alive and listening for a re= -connect

·      &= nbsp;  FIN is sent in both directions initiated by the s= ender

 

scenario 2 - dtntunnel receiver:

           &= nbsp;    dtntunnel -t -T 12345:0.0.0.0:54321 dtn://desteid/xxxx

·         Sender does not terminate

&midd= ot;   &= nbsp;     FIN is sent from sender to rece= iver but not the other way

&midd= ot;   &= nbsp;     Sender socket is in state FIN_W= AIT2

·      &nbs= p;  Receiver socket is in state CLOSE_WAIT

Kill the sender and repeat...=

·         Sender does not terminate

· &nb= sp;       FIN is sent from send= er to receiver but not the other way

· &nb= sp;       New Sender socket is = in state FIN_WAIT2 and the first one has timed out and died

·         Both Receiver sockets are in state CLOSE_WAIT

Kill the dtntunnel receiver…

·         2 FINs are sent from the receiver – 1 to each of the sockets (even = though the other ends have expired)

&nbs= p;

 

<= span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F= 497D'>I believe the issue would be in the oasys IPSocket or possibly the IP= Client. I’ll have a look at it unless there is a low-level socket exp= ert out there that wants to give it a go…

<= p class=3DMsoNormal style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:= auto'> 

Best regards,=

DZ

 

David Zoller

<= span style=3D'font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F49= 7D'>COLSA Corporation

Marshall Space Fligh= t Center

 

From: ssir= eskin@gmail.com [mailto:ssireskin@gmail.com]
Sent: Wednesday, December 1= 2, 2012 4:38 AM


To: Zoller, Dav= id A. (MSFC-EO60)[HOSC SERVICES CONTRACT]

Subject: Re: [dtn-users] A problem with dtntunnel

=

 

Hi David,

I have = investigated that issue a little. I have found, that no matter whether netc= at is run with or without the -w1 option, something
strange happens. Ac= cording to netstat -ntp on the sending node, dtntunnel remains in CLOSE_WAI= T state after nc finishes sending
data. I have run tcpdump on both send= er and receiver, and it showed that final FIN from the receiver doesn't rea= ch the sender.
This looked like a bug in dtntunnel to me. I decided to = re-check with iperf, and again, tcpdump showed the same problem.

Her= e are the last lines of the tcpdump. Remember that in my setup all outgoing= TCP packets with destination port 9999 are redirected
with the help of= IPTables to the local dtntunnel process, listening om port 19999. That is = why I run tcpdump on loopback interface.

Receiver (192.168.4.1): tcp= dump -nn -npi lo port 9999 or port 19999
15:05:02.496366 IP 192.168.4.1.= 34284 > 192.168.4.1.9999: Flags [P.], seq 123995:131097, ack 1, win 257,= options [nop,nop,TS val 11163723 ecr 11163718], length 7102
15:05:02.49= 6428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131097, win= 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0
15:05:03= .457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], seq 131097,= ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724], length 015:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ac= k 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length = 0
15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.]= , seq 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 1116468= 4], length 0
15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999:= Flags [.], ack 2, win 257, options [nop,nop,TS val 11165685 ecr 11165685],= length 0

Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or por= t 19999
15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Fla= gs [.], ack 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875]= , length 0
15:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: = Flags [.], seq 127448:130344, ack 1, win 46, options [nop,nop,TS val 111728= 87 ecr 11172887], length 2896
15:05:02.433783 IP 192.168.1.2.43360 > = 192.168.1.2.19999: Flags [P.], seq 130344:131096, ack 1, win 46, options [n= op,nop,TS val 11172887 ecr 11172887], length 752
15:05:02.444839 IP 192.= 168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131097, win 386, option= s [nop,nop,TS val 11172898 ecr 11172887], length 0
15:05:03.421915 IP 19= 2.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], seq 131096, ack 1, win = 46, options [nop,nop,TS val 11173875 ecr 11172898], length 0
15:05:03.46= 1742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131098, win= 386, options [nop,nop,TS val 11173915 ecr 11173875], length 0

I bel= ieve that the cause of why the sending netcat without -w1 option doesn't ex= it, is that it doesn't receive the final FIN from the dtntunnel.
Can thi= s issue be fixed with a small amount of work?

2= 012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.g= ov>

Hi Sergey,

I would not even call this a problem. netcat is designe= d to be extremely flexible and by default the sender keeps its connection o= pen as long as the other end does. On the other hand the receiver by defaul= t terminates after receipt of an EOF but there is a switch to keep listenin= g if that is the desired behavior.

&nbs= p;

With DTN, your sending nc may do its = thing while the destination node is not even on line and then an hour later= it becomes available and completes the transmission to the receiving nc. I= n this scenario, the sender would still be “hung up” possibly i= ndefinitely waiting for a terminating signal.

=  

I am not the DTN2 authHi o= rity, but, I don’t see a change to dtntunnel for this as it would ass= ume a specific usage that would probably break someone else’s usage (= like mine J).

&nbs= p;

Best regards,

DZ

 <= /p>

David Zoller

COLSA Corporat= ion

Marshall Space Flight Center

 

From:<= /b> ssireskin@gmail.com [mailto:ssireski= n@gmail.com]
Sent: Wednesday, December 05, 2012 2:56 AM
<= b>To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.or= g
Subject: Re: [dtn-users] A problem with dtntunnel

 

Hi David,

Thanks = for your help, nc -w1 did the trick. But shouldn't dtntunnel behavior be ch= anged? The receiving dtntunnel could signal the sending dtntunnel, that the= listening nc has disconnected, so that the sending dtntunnel close its con= nection to the sending nc. Or is this solely the problem of nc?<= /p>

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRAC= T] <david.a= .zoller@nasa.gov>

Hi Sergey,

On RHEL 5.7, I am running dtn-2.9.0 plus = modifications that should not impact the behavior of dtntunnel.=

I just ran your test without the iptables redirect = and I see the same behavior.

I believe t= hat the sender exits if you go directly from nc to nc because the receiver = exits when it gets an end of file.

You = can add a “-w 1” option to the sender nc so that it will timeou= t and exit after stdin is idle for 1 second.

<= span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F= 497D'>Best regards,

DZ=

 

David Zoller

COLSA Corporation

= HOSC / C107 

(Office: (256) 544-1820
*EMail: david= .a.zoller@nasa.gov

 

From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Tuesday, December 04, 2012 11:= 13 AM
To: dtn-users@irtf.org
Subject: [dtn-users] A problem with dtntu= nnel

 

Hell= o all!

I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am = having a problem when using netcat (nc) with dtntunnel.
On the sender no= de I run "cat /etc/passwd | nc receiver_ip". On the receiver node= I run "nc -l 9999". With the help
of Iptables port 9999 gets= redirected to the port 19999, which is listened by dtntunnel.

The f= ile /etc/passwd is successfully delivered to the receiver and is shown on t= he screen. After this the receiving nc exists.
No problem here. However= , nc on the sender node doesn't exit after it sends the file. It continues = to run forever. When I
run nc in the opposite direction, I get the same= problem - the sending nc doesn't exit.

My configuration on both nod= es is symmetric:
iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dpor= t 9999 -j DNAT --to $LOCAL_HOST:19999
dtntunnel -T $LOCAL_HOST:19999:$RE= MOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d
dtntunnel -L --local-eid $LO= CAL_NODE/dtntunnel/nc -d

dtnping works ok in both directions.
Please give me any advice.

With best regards,
Sergey Sireskin <= o:p>




-- Kindest Regards

Sergey Sireskin
FGUP CNII EISU




--
Kindest Regards<= br>
Sergey Sireskin
FGUP CNII EISU




--
Best regards,
Sergey Sireskin<= o:p>




--
Best regards,
Sergey= Sireskin

= --_000_04E3D99A62496240BCD6A576813E6E31E0BDECDC13NDMSSCC05ndcn_-- From ssireskin@gmail.com Thu Dec 13 01:23:05 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E5AD021F8561 for ; Thu, 13 Dec 2012 01:23:05 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.079 X-Spam-Level: X-Spam-Status: No, score=-3.079 tagged_above=-999 required=5 tests=[AWL=-0.083, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_LOW=-1, WEIRD_PORT=0.001] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RyNYCMioxmvg for ; Thu, 13 Dec 2012 01:23:03 -0800 (PST) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by ietfa.amsl.com (Postfix) with ESMTP id 2437821F855D for ; Thu, 13 Dec 2012 01:23:01 -0800 (PST) Received: by mail-vb0-f54.google.com with SMTP id l1so1934277vba.13 for ; Thu, 13 Dec 2012 01:23:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=rYEqlpeRTxWgGU2UhWajVIv4vIfxckdpJUcCThA2DXw=; b=EHaxyQCuRTDWKoC25zvVATN/PbPGiFpa/5PgOOLzKTmPWMPVxHtSvjEXvdl5EmsQZD K8K9lA9aVVhprO55xgll4apjYMizA5GfGCeaEK9ErW+KJOurQ7VzKLs8YsFa1/E35sMj /ADn9w9EHOQJI28hcgkfN8g/dzLmbLSh0bA/szgFZmvIBeB6C4GVGLIolDh/RhiuOQBx G2hfUKP95aFGZn/oxh2796+Yc3kS+/i90gnVqxvith0eivb5fPl6xFaQANksa3gOQ/a3 +gQjUBN4/rtCMW8J4zf5Sh2Zn3/LD/8sWrJgDN27RZ8HhGqVDvJ/XuiE+9jpdN/GoOi/ jXVQ== MIME-Version: 1.0 Received: by 10.59.13.197 with SMTP id fa5mr1934810ved.47.1355390580236; Thu, 13 Dec 2012 01:23:00 -0800 (PST) Received: by 10.58.219.65 with HTTP; Thu, 13 Dec 2012 01:23:00 -0800 (PST) In-Reply-To: <04E3D99A62496240BCD6A576813E6E31E0BDECDC13@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD834@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD996@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECDC13@NDMSSCC05.ndc.nasa.gov> Date: Thu, 13 Dec 2012 12:23:00 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/alternative; boundary=089e0118499c11200a04d0b8766e Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 09:23:06 -0000 --089e0118499c11200a04d0b8766e Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David, I'll start from the end. I have repeated the test with 100 parallel flows twice, and it passed flawlessly. I think that was my mistake, I could have counted the number of erased connections too early, before all of 100 connections have terminated. Now to the first part. I have replaced while(1) with while(!sock_eof) as you suggested. This leads to very strange results when sending a small file with nc. Note the last two SYN and RST packets on the receiver side. Sender: tcpdump -nn -npi lo port 9999 or port 19999 13:10:35.136283 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [S], seq 2925076366, win 5840, options [mss 1460,sackOK,TS val 90705589 ecr 0,nop,wscale 7], length 0 13:10:35.136346 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [S.], seq 2931196344, ack 2925076367, win 32768, options [mss 16396,sackOK,TS val 90705589 ecr 90705589,nop,wscale 7], length 0 13:10:35.136371 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [.], ack 2931196345, win 46, options [nop,nop,TS val 90705589 ecr 90705589], length = 0 13:10:35.136799 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [P.], seq 0:1, ack 1, win 46, options [nop,nop,TS val 90705590 ecr 90705589], length = 1 13:10:35.136819 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [.], ack 2, win 256, options [nop,nop,TS val 90705590 ecr 90705590], length 0 13:10:35.136892 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [F.], seq 1, ack 1, win 46, options [nop,nop,TS val 90705590 ecr 90705590], length 0 13:10:35.176755 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [.], ack 3, win 256, options [nop,nop,TS val 90705630 ecr 90705590], length 0 13:10:35.178602 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [F.], seq 1, ack 3, win 256, options [nop,nop,TS val 90705631 ecr 90705590], length 0 13:10:35.178623 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [.], ack 2, win 46, options [nop,nop,TS val 90705631 ecr 90705631], length 0 13:10:35.253829 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [S], seq 2934914533, win 5840, options [mss 1460,sackOK,TS val 90705707 ecr 0,nop,wscale 7], length 0 13:10:35.253860 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [S.], seq 2932249125, ack 2934914534, win 32768, options [mss 16396,sackOK,TS val 90705707 ecr 90705707,nop,wscale 7], length 0 13:10:35.253878 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [.], ack 2932249126, win 46, options [nop,nop,TS val 90705707 ecr 90705707], length = 0 13:10:36.380942 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [F.], seq 1, ack 1, win 256, options [nop,nop,TS val 90706834 ecr 90705707], length 0 13:10:36.381495 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [.], ack 2, win 46, options [nop,nop,TS val 90706835 ecr 90706834], length 0 13:10:36.398070 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [F.], seq 0, ack 2, win 46, options [nop,nop,TS val 90706851 ecr 90706834], length 0 13:10:36.398131 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [.], ack 2, win 256, options [nop,nop,TS val 90706851 ecr 90706851], length 0 Receiver: tcpdump -nn -npi lo port 9999 or port 19999 13:10:35.185211 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [S], seq 2927123945, win 32792, options [mss 16396,sackOK,TS val 90696412 ecr 0,nop,wscale 7], length 0 13:10:35.185266 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [S.], seq 2929621602, ack 2927123946, win 32768, options [mss 16396,sackOK,TS val 90696412 ecr 90696412,nop,wscale 7], length 0 13:10:35.185286 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [.], ack 1, win 257, options [nop,nop,TS val 90696412 ecr 90696412], length 0 13:10:35.206818 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [P.], seq 1:2, ack 1, win 257, options [nop,nop,TS val 90696434 ecr 90696412], length 1 13:10:35.206861 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [.], ack 2, win 256, options [nop,nop,TS val 90696434 ecr 90696434], length 0 13:10:35.222032 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [F.], seq 2, ack 1, win 257, options [nop,nop,TS val 90696449 ecr 90696434], length 0 13:10:35.222133 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [F.], seq 1, ack 3, win 256, options [nop,nop,TS val 90696449 ecr 90696449], length 0 13:10:35.222146 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [.], ack 2, win 257, options [nop,nop,TS val 90696449 ecr 90696449], length 0 13:10:36.325396 IP 192.168.4.1.44551 > 192.168.4.1.9999: Flags [S], seq 2945716889, win 32792, options [mss 16396,sackOK,TS val 90697552 ecr 0,nop,wscale 7], length 0 13:10:36.325447 IP 192.168.4.1.9999 > 192.168.4.1.44551: Flags [R.], seq 0, ack 2945716890, win 0, length 0 Another strange thing happend when I have run sending nc and forgot to run the receiving nc. Just try it yourself and see tcpdump. I have also conducted another test with the original dtntunnel code. I have run nc in the interactive mode, i.e. nc on the client side and nc -l on the server side. Then I terminated the connection on the client side by pressing CTRL+D. The server received a FIN packed from the client, replied with another FIN, and terminated. The client did not receive the FIN packet from the server and did not terminate. After that I have established a new connection and terminated it by pressing CTRL+D on the server side. This time server did send a FIN packet, the client got it, replied with its own FIN and terminated. The server did not receive the FIN from the client and continued running. This makes me think that dtntunnel, that have received the final FIN from a TCP application, doesn't transmit this FIN to its remote dtntunnel peer. Or the latter one doesn't retransmit this final FIN to the TCP application on its side. I'm not a TCP and sockets guru, so I cannot figure out how to translate this to the language of sockets, which I think is important to do in order to understand how to fix dtntunnel. I don't think that implementing an option that configures dtntunnel behavior in relation to closing connections is a good idea. I believe that there is a flaw in dtntunnel logic, and it could be fixed in a way that dtntunnel determines itself when to close a connection to a TCP application= . 2012/12/13 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < david.a.zoller@nasa.gov> > Hi Sergey,**** > > Research indicates that the TIME_WAIT state is a feature of TCP to preven= t > delayed packets for connection from being delivered to a later connection > between the same hosts. [ > http://tools.ietf.org/html/draft-faber-time-wait-avoidance-00]**** > > ** ** > > Based on that, the added =93if (sock_eof)=94 block can be removed and the > while condition can be changed to =93while (!sock_eof)=94 and you get the= same > result.**** > > ** ** > > But, looking at the intent of the code and at the oasys::IPSocket, it > appears that it is possible to establish a socket connection and then onl= y > close the read side while keeping the write side open. I=92ve never run > across a socket used in that manner but that is what it looks like to me. > So, the TCPTunnel::Connection::run while loop allows for the read side t= o > close and continue transmitting bundle payloads on the write side until i= t > gets a bundle with the EOF bit set. If there are no bundles coming back t= he > other way as is the case here then the while loop never exits which keeps > the netcat connection open.**** > > ** ** > > So, based on that, I think a better solution would be to implement an > option that configures dtntunnel to close the connection when the read si= de > terminates. (Someone in the know can weigh in) Your solution is probably > sufficient for your test.**** > > ** ** > > --**** > > ** ** > > As to connections not being removed from the table and sockets not being > closed=85**** > > ** ** > > I set up a test to kick off 100 netcats on an 11MB file so that the > connection would be open for several seconds:**** > > #!/bin/bash**** > > ctr=3D0**** > > while [ $ctr -lt 100 ]; do**** > > nc -w 10 x.x.x.x 12345 < data11mg &**** > > let ctr=3Dctr+1**** > > done**** > > ** ** > > I also added some debug code to print out a connection accepted counter > and closed counter as they are accepted and closed.**** > > After 30 connections, the first close kicks in and then there is a mix of > connects and closes until finished.**** > > All of the connections were closed and I did not have any sockets left > open.**** > > ** ** > > On my first attempt I exceeded my payload quota and started seeing =93err= or > sending bundle: 141=94 messages so I don=92t think that is the issue you = are > running into.**** > > ** ** > > You might try putting in similar debug to see if you can determine what i= s > happening.**** > > ** ** > > Hope this helps,**** > > DZ**** > > ** ** > > ** ** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > ** ** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 12, 2012 2:07 PM > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > ** ** > > Hi David, > > I meant exactly the same position before the end of the while loop, just > described it incorrect. > > Please let me know if you find the solution for this problem. > > Best regards, > Sergey**** > > 2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > I was narrowing in on the same track... The FIN results in a returned > length of zero on the read and errno is also zero so you have to key on t= he > return length which is already in the code.**** > > **** > > You may have added a couple of lines of debug code and it should be > inserted just before the end of the while(1) loop at line 491 in the late= st > version on Sourceforge.**** > > **** > > I am seeing the sending socket stay in the TIME_WAIT mode even if I add a > call to sock_.shutdown(SHUT_RDWR) before closing the socket;**** > > Getting close I think.**** > > **** > > The 0.0.0.0 was because I did not have anything at the other end of the > tunnel and just stuck some easy numbers in there.**** > > Thanks, > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 12, 2012 11:06 AM**** > > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > I added the following code at the line 492 of TCPTunnel.cc and it seems > that it helped. > > if (sock_eof) { > sock_.close(); > goto done; > } > > The senders, both nc and iperf, exit after having finished sending the > data. There are > no more sockets in CLOSE_WAIT state. However I am completely unsure > whether > my code is correct and is in correct place. > > Now I have encountered another problem. When iperf test is run with 100 > parallel flows, > some dtntunnel connections are not removed from dtntunel's connection > table, and the > related sockets are not closed. I suppose this happens because my code > addition is far > from perfect. > > And one more question, what does 0.0.0.0 mean in -T tunnel specification? > Is it just an > example host, or 0.0.0.0 has some special meaning for dtntunnel?**** > > 2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > I have duplicated what you are reporting and have to retract my earlier > statement saying that this is the netcat design and not a problem J**** > > The sender netcat attempts to close the socket when it gets the [EOF] and > not in response to the receiver closing its end.**** > > **** > > **** > > *netcat sender:***** > > cat /etc/passwd | nc x.x.x.x 12345**** > > **** > > *scenario 1 - netcat receiver with the keep listener option:***** > > nc -k -l 12345**** > > =B7 Sender terminates and receiver stays alive and listening for = a > re-connect**** > > =B7 FIN is sent in both directions initiated by the sender**** > > **** > > *scenario 2 - dtntunnel receiver:***** > > dtntunnel -t -T 12345:0.0.0.0:54321 dtn://desteid/xxxx***= * > > =B7 Sender does not terminate**** > > =B7 FIN is sent from sender to receiver but not the other way**** > > =B7 Sender socket is in state FIN_WAIT2**** > > =B7 Receiver socket is in state CLOSE_WAIT**** > > Kill the sender and repeat...**** > > =B7 Sender does not terminate**** > > =B7 FIN is sent from sender to receiver but not the other way**** > > =B7 New Sender socket is in state FIN_WAIT2 and the first one has > timed out and died**** > > =B7 Both Receiver sockets are in state CLOSE_WAIT**** > > Kill the dtntunnel receiver=85**** > > =B7 2 FINs are sent from the receiver =96 1 to each of the socket= s > (even though the other ends have expired)**** > > **** > > **** > > I believe the issue would be in the oasys IPSocket or possibly the > IPClient. I=92ll have a look at it unless there is a low-level socket exp= ert > out there that wants to give it a go=85**** > > **** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 12, 2012 4:38 AM**** > > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]**** > > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > I have investigated that issue a little. I have found, that no matter > whether netcat is run with or without the -w1 option, something > strange happens. According to netstat -ntp on the sending node, dtntunnel > remains in CLOSE_WAIT state after nc finishes sending > data. I have run tcpdump on both sender and receiver, and it showed that > final FIN from the receiver doesn't reach the sender. > This looked like a bug in dtntunnel to me. I decided to re-check with > iperf, and again, tcpdump showed the same problem. > > Here are the last lines of the tcpdump. Remember that in my setup all > outgoing TCP packets with destination port 9999 are redirected > with the help of IPTables to the local dtntunnel process, listening om > port 19999. That is why I run tcpdump on loopback interface. > > Receiver (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999 > 15:05:02.496366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq > 123995:131097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr > 11163718], length 7102 > 15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack > 131097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length = 0 > 15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], seq > 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724], > length 0 > 15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack > 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length = 0 > 15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], seq > 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684], > length 0 > 15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack 2= , > win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0 > > Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999 > 15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack > 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 0 > 15:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], seq > 127448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr > 11172887], length 2896 > 15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], seq > 130344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr > 11172887], length 752 > 15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack > 131097, win 386, options [nop,nop,TS val 11172898 ecr 11172887], length 0 > 15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], seq > 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], > length 0 > 15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack > 131098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], length 0 > > I believe that the cause of why the sending netcat without -w1 option > doesn't exit, is that it doesn't receive the final FIN from the dtntunnel= . > Can this issue be fixed with a small amount of work?**** > > 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > I would not even call this a problem. netcat is designed to be extremely > flexible and by default the sender keeps its connection open as long as t= he > other end does. On the other hand the receiver by default terminates afte= r > receipt of an EOF but there is a switch to keep listening if that is the > desired behavior. **** > > **** > > With DTN, your sending nc may do its thing while the destination node is > not even on line and then an hour later it becomes available and complete= s > the transmission to the receiving nc. In this scenario, the sender would > still be =93hung up=94 possibly indefinitely waiting for a terminating si= gnal. > **** > > **** > > I am not the DTN2 authHi ority, but, I don=92t see a change to dtntunnel = for > this as it would assume a specific usage that would probably break someon= e > else=92s usage (like mine J).**** > > **** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 05, 2012 2:56 AM > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > Thanks for your help, nc -w1 did the trick. But shouldn't dtntunnel > behavior be changed? The receiving dtntunnel could signal the sending > dtntunnel, that the listening nc has disconnected, so that the sending > dtntunnel close its connection to the sending nc. Or is this solely the > problem of nc?**** > > 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not > impact the behavior of dtntunnel.**** > > I just ran your test without the iptables redirect and I see the same > behavior.**** > > I believe that the sender exits if you go directly from nc to nc because > the receiver exits when it gets an end of file.**** > > You can add a =93-w 1=94 option to the sender nc so that it will timeout = and > exit after stdin is idle for 1 second.**** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *O= n > Behalf Of *ssireskin@gmail.com > *Sent:* Tuesday, December 04, 2012 11:13 AM > *To:* dtn-users@irtf.org > *Subject:* [dtn-users] A problem with dtntunnel**** > > **** > > Hello all! > > I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a > problem when using netcat (nc) with dtntunnel. > On the sender node I run "cat /etc/passwd | nc receiver_ip". On the > receiver node I run "nc -l 9999". With the help > of Iptables port 9999 gets redirected to the port 19999, which is listene= d > by dtntunnel. > > The file /etc/passwd is successfully delivered to the receiver and is > shown on the screen. After this the receiving nc exists. > No problem here. However, nc on the sender node doesn't exit after it > sends the file. It continues to run forever. When I > run nc in the opposite direction, I get the same problem - the sending nc > doesn't exit. > > My configuration on both nodes is symmetric: > iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --t= o > $LOCAL_HOST:19999 > dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/n= c > -d > dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d > > dtnping works ok in both directions. > > Please give me any advice. > > With best regards, > Sergey Sireskin **** > > > > > -- > Kindest Regards > > Sergey Sireskin > FGUP CNII EISU**** > > > > > -- > Kindest Regards > > Sergey Sireskin > FGUP CNII EISU**** > > > > > -- > Best regards, > Sergey Sireskin**** > > > > > -- > Best regards, > Sergey Sireskin > > **** > --=20 Best regards, Sergey Sireskin --089e0118499c11200a04d0b8766e Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David,

I'll start from the end. I have repeated the test with= 100 parallel flows twice, and it passed flawlessly. I think that was my mi= stake, I could have counted
the number of erased connections too early,= before all of 100 connections have terminated.

Now to the first part. I have replaced while(1) with while(!sock_eof) a= s you suggested. This leads to very strange results when sending a small fi= le with nc. Note the last two SYN and RST packets on the receiver side.

Sender: tcpdump -nn -npi lo port 9999 or port 19999
13:10:35.136283 = IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [S], seq 2925076366, win= 5840, options [mss 1460,sackOK,TS val 90705589 ecr 0,nop,wscale 7], length= 0
13:10:35.136346 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [S.], seq= 2931196344, ack 2925076367, win 32768, options [mss 16396,sackOK,TS val 90= 705589 ecr 90705589,nop,wscale 7], length 0
13:10:35.136371 IP 192.168.1= .2.32880 > 192.168.1.2.19999: Flags [.], ack 2931196345, win 46, options= [nop,nop,TS val 90705589 ecr 90705589], length 0
13:10:35.136799 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [P.], se= q 0:1, ack 1, win 46, options [nop,nop,TS val 90705590 ecr 90705589], lengt= h 1
13:10:35.136819 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [.= ], ack 2, win 256, options [nop,nop,TS val 90705590 ecr 90705590], length 0=
13:10:35.136892 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [F.], se= q 1, ack 1, win 46, options [nop,nop,TS val 90705590 ecr 90705590], length = 0
13:10:35.176755 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [.],= ack 3, win 256, options [nop,nop,TS val 90705630 ecr 90705590], length 0 13:10:35.178602 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [F.], seq= 1, ack 3, win 256, options [nop,nop,TS val 90705631 ecr 90705590], length = 0
13:10:35.178623 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [.]= , ack 2, win 46, options [nop,nop,TS val 90705631 ecr 90705631], length 0 13:10:35.253829 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [S], seq= 2934914533, win 5840, options [mss 1460,sackOK,TS val 90705707 ecr 0,nop,w= scale 7], length 0
13:10:35.253860 IP 192.168.4.1.9999 > 192.168.1.2.= 32881: Flags [S.], seq 2932249125, ack 2934914534, win 32768, options [mss = 16396,sackOK,TS val 90705707 ecr 90705707,nop,wscale 7], length 0
13:10:35.253878 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [.], ack= 2932249126, win 46, options [nop,nop,TS val 90705707 ecr 90705707], length= 0
13:10:36.380942 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [F.= ], seq 1, ack 1, win 256, options [nop,nop,TS val 90706834 ecr 90705707], l= ength 0
13:10:36.381495 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [.], ack= 2, win 46, options [nop,nop,TS val 90706835 ecr 90706834], length 0
13:= 10:36.398070 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [F.], seq 0= , ack 2, win 46, options [nop,nop,TS val 90706851 ecr 90706834], length 0 13:10:36.398131 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [.], ack = 2, win 256, options [nop,nop,TS val 90706851 ecr 90706851], length 0
Receiver: tcpdump -nn -npi lo port 9999 or port 19999
13:10:35.185211 I= P 192.168.4.1.44550 > 192.168.4.1.9999: Flags [S], seq 2927123945, win 3= 2792, options [mss 16396,sackOK,TS val 90696412 ecr 0,nop,wscale 7], length= 0
13:10:35.185266 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [S.], seq= 2929621602, ack 2927123946, win 32768, options [mss 16396,sackOK,TS val 90= 696412 ecr 90696412,nop,wscale 7], length 0
13:10:35.185286 IP 192.168.4= .1.44550 > 192.168.4.1.9999: Flags [.], ack 1, win 257, options [nop,nop= ,TS val 90696412 ecr 90696412], length 0
13:10:35.206818 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [P.], seq= 1:2, ack 1, win 257, options [nop,nop,TS val 90696434 ecr 90696412], lengt= h 1
13:10:35.206861 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [.= ], ack 2, win 256, options [nop,nop,TS val 90696434 ecr 90696434], length 0=
13:10:35.222032 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [F.], seq= 2, ack 1, win 257, options [nop,nop,TS val 90696449 ecr 90696434], length = 0
13:10:35.222133 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [F.]= , seq 1, ack 3, win 256, options [nop,nop,TS val 90696449 ecr 90696449], le= ngth 0
13:10:35.222146 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [.], ack = 2, win 257, options [nop,nop,TS val 90696449 ecr 90696449], length 0
13:= 10:36.325396 IP 192.168.4.1.44551 > 192.168.4.1.9999: Flags [S], seq 294= 5716889, win 32792, options [mss 16396,sackOK,TS val 90697552 ecr 0,nop,wsc= ale 7], length 0
13:10:36.325447 IP 192.168.4.1.9999 > 192.168.4.1.44551: Flags [R.], seq= 0, ack 2945716890, win 0, length 0

Another strange thing happend wh= en I have run sending nc and forgot to run the receiving nc. Just try it yo= urself and see tcpdump.

I have also conducted another test with the original dtntunnel code. I = have run nc in the interactive mode, i.e. nc <ip> <port> on the= client side and nc -l <port> on the server side. Then I terminated t= he connection on the client side by pressing CTRL+D. The server received a = FIN packed from the client, replied with another FIN, and terminated. The c= lient did not receive the FIN packet from the server and did not terminate.= After that I have established a new connection and terminated it by pressi= ng CTRL+D on the server side. This time server did send a FIN packet, the c= lient got it, replied with its own FIN and terminated. The server did not r= eceive the FIN from the client and continued running.

This makes me think that dtntunnel, that have received the final FIN fr= om a TCP application, doesn't transmit this FIN to its remote dtntunnel= peer. Or the latter one doesn't retransmit this final FIN to the TCP a= pplication on its side. I'm not a TCP and sockets guru, so I cannot fig= ure out how to translate this to the language of sockets, which I think is = important to do in order to understand how to fix dtntunnel.

I don't think that implementing an option that configures dtntunnel= behavior in relation to closing connections is a good idea. I believe that= there is a flaw in dtntunnel logic, and it could be fixed in a way that dt= ntunnel determines itself when to close a connection to a TCP application.<= br>
2012/12/13 Zoller, David A. (MSFC-EO60)[HOSC= SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

Research indicates that the TIME_WAIT state is a= feature of TCP to prevent delayed packets for connection from being delive= red to a later connection between the same hosts. [http://= tools.ietf.org/html/draft-faber-time-wait-avoidance-00]

=A0<= /p>

Based on that, the add= ed =93if (sock_eof)=94 block can be removed and the while condition can be = changed to =93while (!sock_eof)=94 and you get the same result.

=A0<= /p>

But, looking at the in= tent of the code and at the oasys::IPSocket, it appears that it is possible= to establish a socket connection and then only close the read side while k= eeping the write side open. I=92ve never run across a socket used in that m= anner but that is what it looks like to me. So, the TCPTunnel::Connection::= run =A0while loop allows for the read side to close and continue transmitti= ng bundle payloads on the write side until it gets a bundle with the EOF bi= t set. If there are no bundles coming back the other way as is the case her= e then the while loop never exits which keeps the netcat connection open.

=A0<= /p>

So, based on that, I t= hink a better solution would be to implement an option that configures dtnt= unnel to close the connection when the read side terminates. (Someone in th= e know can weigh in) Your solution is probably sufficient for your test.=

=A0<= /p>

--

=A0<= /p>

As to connections not = being removed from the table and sockets not being closed=85<= /span>

=A0<= /p>

I set up a test to kic= k off 100 netcats on an 11MB file so that the connection would be open for = several seconds:

ctr=3D0

while [ $ctr -= lt 100 ]; do

=A0=A0=A0 let ctr=3Dctr+1

done

=A0<= /p>

I also added some debu= g code to print out a connection accepted counter and closed counter as the= y are accepted and closed.

After 30 connections, the= first close kicks in and then there is a mix of connects and closes until = finished.

All of the connections we= re closed and I did not have any sockets left open.

=A0<= /p>

On my first attempt I = exceeded my payload quota and started seeing =93error sending bundle: 141= =94 messages so I don=92t think that is the issue you are running into.<= /u>

=A0<= /p>

You might try putting = in similar debug to see if you can determine what is happening.

=A0<= /p>

Hope this helps,

DZ

=A0

=A0<= /p>

David Zoller

COLSA Corporation

HOSC / C107=A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov

=A0<= /p>

From: ssireskin@gmail.= com [mailto:ss= ireskin@gmail.com]
Sent: Wednesday, December 12, 2012 2:07 PM


To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]=
Cc: dtn-= users@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,
I meant exactly the same position before the end of the while loop, just = described it incorrect.

Please let me know if you find the solution for this problem.

Be= st regards,
Sergey

2012/12/= 12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov&g= t;

Hi Sergey,

I was= narrowing in on the same track... The FIN results in a returned length of = zero on the read and errno is also zero so you have to key on the return le= ngth which is already in the code.

=A0<= /p>

You may have added a c= ouple of lines of debug code and it should be inserted just before the end = of the while(1) loop at line 491 in the latest version on Sourceforge.

=A0<= /p>

I am seeing the sendin= g socket stay in the TIME_WAIT mode even if I add a call to sock_.shutdown(= SHUT_RDWR) before closing the socket;

Getting close I think.

=A0=

The 0.0.0.0 was because I= did not have anything at the other end of the tunnel and just stuck some e= asy numbers in there.

Thanks,
DZ

=A0

David Zoller<= u>

COLSA Corporation=

Marshall Space Flight Cente= r

=A0

From: ssireskin@gmail.com<= /a> [mailto:ssires= kin@gmail.com]
Sent: Wednesday, December 12, 2012 11:06 AM

=


To: Zoller, David A. (MSFC-EO60= )[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

<= /div>

=A0

Hi David,

I added the = following code at the line 492 of TCPTunnel.cc and it seems that it helped.=

=A0=A0=A0 if (sock_eof) {
=A0=A0=A0 =A0=A0=A0 sock_.close();
=A0= =A0=A0 =A0=A0=A0 goto done;
=A0=A0=A0 }

The senders, both nc and = iperf, exit after having finished sending the data. There are
no more so= ckets in CLOSE_WAIT state. However I am completely unsure whether
my code is correct and is in correct place.

Now I have encountered a= nother problem. When iperf test is run with 100 parallel flows,
some dtn= tunnel connections are not removed from dtntunel's connection table, an= d the
related sockets are not closed. I suppose this happens because my code addi= tion is far
from perfect.

And one more question, what does 0.0.0.= 0 mean in -T tunnel specification? Is it just an
example host, or 0.0.0.= 0 has some special meaning for dtntunnel?

2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SE= RVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

I have duplicated what you are reporti= ng and have to retract my earlier statement saying that this is the netcat = design and not a problem J

The sender netcat attempt= s to close the socket when it gets the [EOF] and not in response to the rec= eiver closing its end.

=A0<= /p>

=A0

netcat sender:<= /b>

cat /etc/passwd | nc x.x.x.x 12345<= /u>

=A0<= /p>

scenario 1 - netcat= receiver with the keep listener option:

=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 nc -k -l 12345

=B7=A0=A0=A0= =A0=A0=A0=A0=A0 Sender terminates and rec= eiver stays alive and listening for a re-connect

=B7=A0=A0=A0=A0=A0=A0=A0=A0 FIN is sent i= n both directions initiated by the sender

=A0<= /p>

scenario 2 - dtntun= nel receiver:

=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 dtntunnel -t -T 12345:0.0.0.0:54321 dtn://desteid/xxxx

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Sender does n= ot terminate

=B7=A0=A0=A0=A0=A0=A0=A0=A0 FIN is sent f= rom sender to receiver but not the other way

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Sender socket= is in state FIN_WAIT2

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Receiver sock= et is in state CLOSE_WAIT

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Sender does not terminate

=B7=A0=A0=A0=A0=A0=A0=A0=A0 FIN is sent f= rom sender to receiver but not the other way

=B7=A0=A0=A0=A0=A0=A0=A0=A0 New Sender so= cket is in state FIN_WAIT2 and the first one has timed out and died<= u>

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Both Receiver= sockets are in state CLOSE_WAIT

=B7=A0=A0=A0=A0=A0=A0=A0=A0 2 FINs are sent from the receiver =96 1 to e= ach of the sockets (even though the other ends have expired)<= u>

=A0<= /p>

=A0

I believe the issue would= be in the oasys IPSocket or possibly the IPClient. I=92ll have a look at i= t unless there is a low-level socket expert out there that wants to give it= a go=85

=A0=

Best regards,

DZ

=A0

David Zoller<= u>

COLSA Corporation=

Marshall Space Flight Cente= r

=A0

From: ssireskin@gmail.com<= /a> [mailto:ssires= kin@gmail.com]
Sent: Wednesday, December 12, 2012 4:38 AM

<= div>


To: Zoller, David A. (MSFC-EO60)[HOSC= SERVICES CONTRACT]

Subject: Re: [dtn-users] A problem with dtntunnel

<= /div>

=A0

Hi David,

I have inves= tigated that issue a little. I have found, that no matter whether netcat is= run with or without the -w1 option, something
strange happens. According to netstat -ntp on the sending node, dtntunnel r= emains in CLOSE_WAIT state after nc finishes sending
data. I have run t= cpdump on both sender and receiver, and it showed that final FIN from the r= eceiver doesn't reach the sender.
This looked like a bug in dtntunnel to me. I decided to re-check with iperf= , and again, tcpdump showed the same problem.

Here are the last line= s of the tcpdump. Remember that in my setup all outgoing TCP packets with d= estination port 9999 are redirected
with the help of IPTables to the local dtntunnel process, listening om port= 19999. That is why I run tcpdump on loopback interface.

Receiver (1= 92.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.496366= IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 123995:131097,= ack 1, win 257, options [nop,nop,TS val 11163723 ecr 11163718], length 710= 2
15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack = 131097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0<= br>15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], = seq 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724],= length 0
15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack = 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length 0<= br>15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], = seq 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684]= , length 0
15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack = 2, win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0
Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack = 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 015:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], s= eq 127448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172= 887], length 2896
15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], se= q 130344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 111728= 87], length 752
15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.433= 60: Flags [.], ack 131097, win 386, options [nop,nop,TS val 11172898 ecr 11= 172887], length 0
15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], se= q 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], le= ngth 0
15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags= [.], ack 131098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], = length 0

I believe that the cause of why the sending netcat without -w1 option d= oesn't exit, is that it doesn't receive the final FIN from the dtnt= unnel.
Can this issue be fixed with a small amount of work?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SER= VICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

I would not even call this a problem. = netcat is designed to be extremely flexible and by default the sender keeps= its connection open as long as the other end does. On the other hand the r= eceiver by default terminates after receipt of an EOF but there is a switch= to keep listening if that is the desired behavior.

=A0<= /p>

With DTN, your sending= nc may do its thing while the destination node is not even on line and the= n an hour later it becomes available and completes the transmission to the = receiving nc. In this scenario, the sender would still be =93hung up=94 pos= sibly indefinitely waiting for a terminating signal. <= /p>

=A0<= /p>

I am not the DTN2 auth= Hi ority, but, I don=92t see a change to dtntunnel for this as it would ass= ume a specific usage that would probably break someone else=92s usage (like= mine J).

=A0=

Best regards,

DZ

=A0

David Zoller<= u>

COLSA Corporation=

Marshall Space Flight= Center

=A0

From: ssireskin@gmail.com [= mailto:ssireskin@g= mail.com]
Sent: Wednesday, December 05, 2012 2:56 AM
To: Zoller, Dav= id A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org
Subject:<= /b> Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,

Thanks for your help, nc = -w1 did the trick. But shouldn't dtntunnel behavior be changed? The rec= eiving dtntunnel could signal the sending dtntunnel, that the listening nc = has disconnected, so that the sending dtntunnel close its connection to the= sending nc. Or is this solely the problem of nc?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SER= VICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

On RHEL 5.7, I am running dtn-2.9.0 pl= us modifications that should not impact the behavior of dtntunnel.

I just ran your test with= out the iptables redirect and I see the same behavior.=

I believe that the sender= exits if you go directly from nc to nc because the receiver exits when it = gets an end of file.

You can add a =93-w 1=94 = option to the sender nc so that it will timeout and exit after stdin is idl= e for 1 second.

Best regards,

DZ=

=A0<= /p>

David Zoller

COLSA Corporation=

HOSC / C107= =A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: dtn-users-bounce= s@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Tuesday, December 04, 2012 11:13 AM
To: dtn-users@irtf.org
Su= bject: [dtn-users] A problem with dtntunnel

=A0

Hel= lo all!

I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am= having a problem when using netcat (nc) with dtntunnel.
On the sender n= ode I run "cat /etc/passwd | nc receiver_ip". On the receiver nod= e I run "nc -l 9999". With the help
of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel.

The file /etc/passwd is successfully delivered to the = receiver and is shown on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it s= ends the file. It continues to run forever. When I
run nc in the opposi= te direction, I get the same problem - the sending nc doesn't exit.

My configuration on both nodes is symmetric:
iptables -t nat -A OUTP= UT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to $LOCAL_HOST:19999
dt= ntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d=
dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d

dtnping works o= k in both directions.

Please give me any advice.

With best r= egards,
Sergey Sireskin




--
Kindest Regards

Sergey Sireskin
FGUP CNII EISU=




--
Kindest Regards

Sergey Sireskin
= FGUP CNII EISU




--
B= est regards,
Sergey Sireskin




--
= Best regards,
Sergey Sireskin




--
Best regards,
Sergey= Sireskin


--089e0118499c11200a04d0b8766e-- From ssireskin@gmail.com Thu Dec 13 05:39:55 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6524121F89E8 for ; Thu, 13 Dec 2012 05:39:55 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.372 X-Spam-Level: X-Spam-Status: No, score=-3.372 tagged_above=-999 required=5 tests=[AWL=0.226, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IK9+wfIWROrZ for ; Thu, 13 Dec 2012 05:39:55 -0800 (PST) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by ietfa.amsl.com (Postfix) with ESMTP id D422821F87F6 for ; Thu, 13 Dec 2012 05:39:54 -0800 (PST) Received: by mail-ie0-f182.google.com with SMTP id s9so3982541iec.13 for ; Thu, 13 Dec 2012 05:39:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=uL6GYgfK+FLG/475H7V1ym7e+QnYibyq8ohMZzdEUt4=; b=LN6nUkxf97UI2KSWrRSaoJbgaRiMuJkZld4ItMvJI/aWNmCE2/ZQC1SjZIU8Q38kIt C6oqhLlyvaPh3dLs81B2ism6yGwR7nbsGS4nYGaxzI82wPv+jWOgMUi8iWV8CG8sWkyx MbQARAByauiW5vgxf4pnFxWESPnZzhXiwyXnlLrasQxJS8yOTKDS81EoR9201pjLwY4p 9tr2PIImTh0IWdrmstLvctyRVy9kwLO6uUhbcQ+DNSWGk/nF2zOyGjjc/+N6TWCrWO7P IKuEFT0YnHEQyWx8e8YDyuGv6b9xH2uh1VtvgnrWWIzZO0kSUf5J+0aCUYuZxrLoLtM9 rZbQ== MIME-Version: 1.0 Received: by 10.50.1.200 with SMTP id 8mr1547264igo.51.1355405994405; Thu, 13 Dec 2012 05:39:54 -0800 (PST) Received: by 10.50.97.236 with HTTP; Thu, 13 Dec 2012 05:39:54 -0800 (PST) In-Reply-To: References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD834@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD996@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECDC13@NDMSSCC05.ndc.nasa.gov> Date: Thu, 13 Dec 2012 16:39:54 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/alternative; boundary=e89a8f502e0cd2aef404d0bc0cfb Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 13:39:55 -0000 --e89a8f502e0cd2aef404d0bc0cfb Content-Type: text/plain; charset=ISO-8859-1 Hi again, I have added sock_.shutdown(SHUT_RDWR) in two places inside the while(1) loop: 1) right before the sock_.close() call at line 486; 1) inside the if (sock_eof) {...} block of code just before the end of the while(1) loop. This seems to solve the problem with lost FINs. Now I'm trying to stress-test dtntunnel with multiple concurrent iperf transfers. With 10 flows dtntunnel works well, but when increasing the number of flows to 15 the already seen weird things happen. The _sending_ iperf starts receiving SYN packets from dtntunnel. I think this could happen because of a race condition. -- Best regards, Sergey Sireskin --e89a8f502e0cd2aef404d0bc0cfb Content-Type: text/html; charset=ISO-8859-1 Hi again,

I have added sock_.shutdown(SHUT_RDWR) in two places inside the while(1) loop:
1) right before the sock_.close() call at line 486;
1) inside the if (sock_eof) {...} block of code just before the end of the while(1) loop.

This seems to solve the problem with lost FINs.

Now I'm trying to stress-test dtntunnel with multiple concurrent iperf transfers. With
10 flows dtntunnel works well, but when increasing the number of flows to 15 the
already seen weird things happen. The _sending_ iperf starts receiving SYN packets
from dtntunnel. I think this could happen because of a race condition.

--
Best regards,
Sergey Sireskin


--e89a8f502e0cd2aef404d0bc0cfb-- From ssireskin@gmail.com Thu Dec 13 05:44:41 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3D7A221F8982 for ; Thu, 13 Dec 2012 05:44:41 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.516 X-Spam-Level: X-Spam-Status: No, score=-2.516 tagged_above=-999 required=5 tests=[AWL=-0.671, BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_BASE64_TEXT=1.753, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vn3Lv2065IOk for ; Thu, 13 Dec 2012 05:44:40 -0800 (PST) Received: from mail-ie0-f170.google.com (mail-ie0-f170.google.com [209.85.223.170]) by ietfa.amsl.com (Postfix) with ESMTP id 30AB221F887A for ; Thu, 13 Dec 2012 05:44:40 -0800 (PST) Received: by mail-ie0-f170.google.com with SMTP id k10so3940354iea.15 for ; Thu, 13 Dec 2012 05:44:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=p1KbIKZAjJzl9LvZEfReiKuKIftjFkU6h0Yh+3I9oUo=; b=gMzSOhw1+rMwdtxJSznAjlzE7Un/MkNZm8F7gLp+nXpEIVsxs+qtKdsACMtYMtPBvb 8LqmZrrmb1YSkJ0IQzBWrjcLhMZGDNP7TDwY3CQwNgvog6QKwIbJfQxlObWa3mdqaWBy qzFk3LSHWO5b1BabkpulbKJ/pQjbnqLhA7yVh8lQCh8D9YMPdsBHgdo1Mo+vMxZYtn/J 95iqsAUUlAHCKKNg2DoW1/X3J0fYZGYSqye9zA2amuFPA2DLJVYkAl6lPo8pyOv8E8ar XBOkhYR1kPizNrD9E8TMJi8Q3+0Y9uIXPiaOHNjNt96SC6F5F36DeZmGRr01YhlPVerc b80g== MIME-Version: 1.0 Received: by 10.50.213.69 with SMTP id nq5mr9437260igc.70.1355406279621; Thu, 13 Dec 2012 05:44:39 -0800 (PST) Received: by 10.50.97.236 with HTTP; Thu, 13 Dec 2012 05:44:39 -0800 (PST) In-Reply-To: References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD834@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD996@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECDC13@NDMSSCC05.ndc.nasa.gov> Date: Thu, 13 Dec 2012 16:44:39 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/alternative; boundary=14dae9340b51d2bd6d04d0bc1d6c Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 13:44:41 -0000 --14dae9340b51d2bd6d04d0bc1d6c Content-Type: text/plain; charset=ISO-8859-1 To clarify what changes I did... ... while(1) { ... if (recv_hdr->eof_) { log_info("bundle had eof bit set... closing connection"); *sock_.shutdown(SHUT_RDWR);* sock_.close(); } delete b_recv; } *if (sock_eof) { log_debug(">>>> sock_eof is true... closing connection"); sock_.shutdown(SHUT_RDWR); sock_.close(); goto done; }* } done: tcptun_->kill_connection(this); } --14dae9340b51d2bd6d04d0bc1d6c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: base64 VG8gY2xhcmlmeSB3aGF0IGNoYW5nZXMgSSBkaWQuLi48YnI+PGJyPi4uLjxicj6goKAgd2hpbGUo MSkgezxicj6goKCgoKCgIC4uLjxicj6goKCgoKCgoKCgoCBpZiAocmVjdl9oZHItJmd0O2VvZl8p IHs8YnI+oKCgoKCgoKCgoKCgoKCgIGxvZ19pbmZvKCZxdW90O2J1bmRsZSBoYWQgZW9mIGJpdCBz ZXQuLi4gY2xvc2luZyBjb25uZWN0aW9uJnF1b3Q7KTs8YnI+oKCgoKCgoKCgoKCgoKCgIDxiPnNv Y2tfLnNodXRkb3duKFNIVVRfUkRXUik7PC9iPjxicj4KoKCgoKCgoKCgoKCgoKCgIHNvY2tfLmNs b3NlKCk7PGJyPqCgoKCgoKCgoKCgIH08YnI+PGJyPqCgoKCgoKCgoKCgIGRlbGV0ZSBiX3JlY3Y7 PGJyPqCgoKCgoKAgfTxicj48YnI+oKCgoKCgoCA8Yj5pZiAoc29ja19lb2YpIHs8YnI+oKCgoKCg oKCgoKAgbG9nX2RlYnVnKCZxdW90OyZndDsmZ3Q7Jmd0OyZndDsgc29ja19lb2YgaXMgdHJ1ZS4u LiBjbG9zaW5nIGNvbm5lY3Rpb24mcXVvdDspOzxicj4KoKCgoKCgoKCgoKAgc29ja18uc2h1dGRv d24oU0hVVF9SRFdSKTs8YnI+oKCgoKCgoKCgoKAgc29ja18uY2xvc2UoKTs8YnI+oKCgoKCgoKCg oKAgZ290byBkb25lOzxicj6goKCgoKCgIH08L2I+PGJyPjxicj6goKAgfTxicj48YnI+oGRvbmU6 PGJyPqCgoCB0Y3B0dW5fLSZndDtraWxsX2Nvbm5lY3Rpb24odGhpcyk7PGJyPn08YnI+PGJyPgo= --14dae9340b51d2bd6d04d0bc1d6c-- From david.a.zoller@nasa.gov Thu Dec 13 05:46:41 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2609A21F8979 for ; Thu, 13 Dec 2012 05:46:41 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.197 X-Spam-Level: X-Spam-Status: No, score=-6.197 tagged_above=-999 required=5 tests=[AWL=-0.201, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_MED=-4, WEIRD_PORT=0.001] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 26t6QlJ378Vw for ; Thu, 13 Dec 2012 05:46:30 -0800 (PST) Received: from ndmsnpf01.ndc.nasa.gov (ndmsnpf01.ndc.nasa.gov [198.117.0.121]) by ietfa.amsl.com (Postfix) with ESMTP id 0E48021F887A for ; Thu, 13 Dec 2012 05:46:30 -0800 (PST) Received: from ndmsppt04.ndc.nasa.gov (ndmsppt04.ndc.nasa.gov [198.117.0.103]) by ndmsnpf01.ndc.nasa.gov (Postfix) with ESMTP id AABB8260424; Thu, 13 Dec 2012 07:46:24 -0600 (CST) Received: from ndmshub02.ndc.nasa.gov (ndmshub02-pub.ndc.nasa.gov [198.117.0.161]) by ndmsppt04.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBDDkONM018419; Thu, 13 Dec 2012 07:46:24 -0600 Received: from NDMSSCC05.ndc.nasa.gov ([198.117.2.174]) by ndmshub02.ndc.nasa.gov ([198.117.2.161]) with mapi; Thu, 13 Dec 2012 07:46:24 -0600 From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "ssireskin@gmail.com" Date: Thu, 13 Dec 2012 07:46:23 -0600 Thread-Topic: [dtn-users] A problem with dtntunnel Thread-Index: Ac3ZE3Ox0Spq9bamQjqW9TXgtzao9QAI6Gyw Message-ID: <04E3D99A62496240BCD6A576813E6E31E0BDECDCE2@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD834@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD996@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECDC13@NDMSSCC05.ndc.nasa.gov> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_04E3D99A62496240BCD6A576813E6E31E0BDECDCE2NDMSSCC05ndcn_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-13_05:2012-12-13, 2012-12-13, 1970-01-01 signatures=0 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 13:46:41 -0000 --_000_04E3D99A62496240BCD6A576813E6E31E0BDECDCE2NDMSSCC05ndcn_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Sergey, I see now the end to end effect you are wanting and I'll experiment some mo= re and get back with you but it may be a few days. I guess one question is: If both ends are not up in "near real time" would you want the sender to st= ay connected until the receiving end comes on line and can return the FIN i= n all circumstances? DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: ssireskin@gmail.com [mailto:ssireskin@gmail.com] Sent: Thursday, December 13, 2012 3:23 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, I'll start from the end. I have repeated the test with 100 parallel flows t= wice, and it passed flawlessly. I think that was my mistake, I could have c= ounted the number of erased connections too early, before all of 100 connections h= ave terminated. Now to the first part. I have replaced while(1) with while(!sock_eof) as yo= u suggested. This leads to very strange results when sending a small file w= ith nc. Note the last two SYN and RST packets on the receiver side. Sender: tcpdump -nn -npi lo port 9999 or port 19999 13:10:35.136283 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [S], seq 29= 25076366, win 5840, options [mss 1460,sackOK,TS val 90705589 ecr 0,nop,wsca= le 7], length 0 13:10:35.136346 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [S.], seq 29= 31196344, ack 2925076367, win 32768, options [mss 16396,sackOK,TS val 90705= 589 ecr 90705589,nop,wscale 7], length 0 13:10:35.136371 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [.], ack 29= 31196345, win 46, options [nop,nop,TS val 90705589 ecr 90705589], length 0 13:10:35.136799 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [P.], seq 0= :1, ack 1, win 46, options [nop,nop,TS val 90705590 ecr 90705589], length 1 13:10:35.136819 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [.], ack 2, = win 256, options [nop,nop,TS val 90705590 ecr 90705590], length 0 13:10:35.136892 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [F.], seq 1= , ack 1, win 46, options [nop,nop,TS val 90705590 ecr 90705590], length 0 13:10:35.176755 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [.], ack 3, = win 256, options [nop,nop,TS val 90705630 ecr 90705590], length 0 13:10:35.178602 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [F.], seq 1,= ack 3, win 256, options [nop,nop,TS val 90705631 ecr 90705590], length 0 13:10:35.178623 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [.], ack 2,= win 46, options [nop,nop,TS val 90705631 ecr 90705631], length 0 13:10:35.253829 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [S], seq 29= 34914533, win 5840, options [mss 1460,sackOK,TS val 90705707 ecr 0,nop,wsca= le 7], length 0 13:10:35.253860 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [S.], seq 29= 32249125, ack 2934914534, win 32768, options [mss 16396,sackOK,TS val 90705= 707 ecr 90705707,nop,wscale 7], length 0 13:10:35.253878 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [.], ack 29= 32249126, win 46, options [nop,nop,TS val 90705707 ecr 90705707], length 0 13:10:36.380942 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [F.], seq 1,= ack 1, win 256, options [nop,nop,TS val 90706834 ecr 90705707], length 0 13:10:36.381495 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [.], ack 2,= win 46, options [nop,nop,TS val 90706835 ecr 90706834], length 0 13:10:36.398070 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [F.], seq 0= , ack 2, win 46, options [nop,nop,TS val 90706851 ecr 90706834], length 0 13:10:36.398131 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [.], ack 2, = win 256, options [nop,nop,TS val 90706851 ecr 90706851], length 0 Receiver: tcpdump -nn -npi lo port 9999 or port 19999 13:10:35.185211 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [S], seq 292= 7123945, win 32792, options [mss 16396,sackOK,TS val 90696412 ecr 0,nop,wsc= ale 7], length 0 13:10:35.185266 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [S.], seq 29= 29621602, ack 2927123946, win 32768, options [mss 16396,sackOK,TS val 90696= 412 ecr 90696412,nop,wscale 7], length 0 13:10:35.185286 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [.], ack 1, = win 257, options [nop,nop,TS val 90696412 ecr 90696412], length 0 13:10:35.206818 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [P.], seq 1:= 2, ack 1, win 257, options [nop,nop,TS val 90696434 ecr 90696412], length 1 13:10:35.206861 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [.], ack 2, = win 256, options [nop,nop,TS val 90696434 ecr 90696434], length 0 13:10:35.222032 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [F.], seq 2,= ack 1, win 257, options [nop,nop,TS val 90696449 ecr 90696434], length 0 13:10:35.222133 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [F.], seq 1,= ack 3, win 256, options [nop,nop,TS val 90696449 ecr 90696449], length 0 13:10:35.222146 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [.], ack 2, = win 257, options [nop,nop,TS val 90696449 ecr 90696449], length 0 13:10:36.325396 IP 192.168.4.1.44551 > 192.168.4.1.9999: Flags [S], seq 294= 5716889, win 32792, options [mss 16396,sackOK,TS val 90697552 ecr 0,nop,wsc= ale 7], length 0 13:10:36.325447 IP 192.168.4.1.9999 > 192.168.4.1.44551: Flags [R.], seq 0,= ack 2945716890, win 0, length 0 Another strange thing happend when I have run sending nc and forgot to run = the receiving nc. Just try it yourself and see tcpdump. I have also conducted another test with the original dtntunnel code. I have= run nc in the interactive mode, i.e. nc on the client side and= nc -l on the server side. Then I terminated the connection on the c= lient side by pressing CTRL+D. The server received a FIN packed from the cl= ient, replied with another FIN, and terminated. The client did not receive = the FIN packet from the server and did not terminate. After that I have est= ablished a new connection and terminated it by pressing CTRL+D on the serve= r side. This time server did send a FIN packet, the client got it, replied = with its own FIN and terminated. The server did not receive the FIN from th= e client and continued running. This makes me think that dtntunnel, that have received the final FIN from a= TCP application, doesn't transmit this FIN to its remote dtntunnel peer. O= r the latter one doesn't retransmit this final FIN to the TCP application o= n its side. I'm not a TCP and sockets guru, so I cannot figure out how to t= ranslate this to the language of sockets, which I think is important to do = in order to understand how to fix dtntunnel. I don't think that implementing an option that configures dtntunnel behavio= r in relation to closing connections is a good idea. I believe that there i= s a flaw in dtntunnel logic, and it could be fixed in a way that dtntunnel = determines itself when to close a connection to a TCP application. 2012/12/13 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, Research indicates that the TIME_WAIT state is a feature of TCP to prevent = delayed packets for connection from being delivered to a later connection b= etween the same hosts. [http://tools.ietf.org/html/draft-faber-time-wait-av= oidance-00] Based on that, the added "if (sock_eof)" block can be removed and the while= condition can be changed to "while (!sock_eof)" and you get the same resul= t. But, looking at the intent of the code and at the oasys::IPSocket, it appea= rs that it is possible to establish a socket connection and then only close= the read side while keeping the write side open. I've never run across a s= ocket used in that manner but that is what it looks like to me. So, the TCP= Tunnel::Connection::run while loop allows for the read side to close and c= ontinue transmitting bundle payloads on the write side until it gets a bund= le with the EOF bit set. If there are no bundles coming back the other way = as is the case here then the while loop never exits which keeps the netcat = connection open. So, based on that, I think a better solution would be to implement an optio= n that configures dtntunnel to close the connection when the read side term= inates. (Someone in the know can weigh in) Your solution is probably suffic= ient for your test. -- As to connections not being removed from the table and sockets not being cl= osed... I set up a test to kick off 100 netcats on an 11MB file so that the connect= ion would be open for several seconds: #!/bin/bash ctr=3D0 while [ $ctr -lt 100 ]; do nc -w 10 x.x.x.x 12345 < data11mg & let ctr=3Dctr+1 done I also added some debug code to print out a connection accepted counter and= closed counter as they are accepted and closed. After 30 connections, the first close kicks in and then there is a mix of c= onnects and closes until finished. All of the connections were closed and I did not have any sockets left open= . On my first attempt I exceeded my payload quota and started seeing "error s= ending bundle: 141" messages so I don't think that is the issue you are run= ning into. You might try putting in similar debug to see if you can determine what is = happening. Hope this helps, DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Wednesday, December 12, 2012 2:07 PM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, I meant exactly the same position before the end of the while loop, just de= scribed it incorrect. Please let me know if you find the solution for this problem. Best regards, Sergey 2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I was narrowing in on the same track... The FIN results in a returned lengt= h of zero on the read and errno is also zero so you have to key on the retu= rn length which is already in the code. You may have added a couple of lines of debug code and it should be inserte= d just before the end of the while(1) loop at line 491 in the latest versio= n on Sourceforge. I am seeing the sending socket stay in the TIME_WAIT mode even if I add a c= all to sock_.shutdown(SHUT_RDWR) before closing the socket; Getting close I think. The 0.0.0.0 was because I did not have anything at the other end of the tun= nel and just stuck some easy numbers in there. Thanks, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Wednesday, December 12, 2012 11:06 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, I added the following code at the line 492 of TCPTunnel.cc and it seems tha= t it helped. if (sock_eof) { sock_.close(); goto done; } The senders, both nc and iperf, exit after having finished sending the data= . There are no more sockets in CLOSE_WAIT state. However I am completely unsure whether my code is correct and is in correct place. Now I have encountered another problem. When iperf test is run with 100 par= allel flows, some dtntunnel connections are not removed from dtntunel's connection table= , and the related sockets are not closed. I suppose this happens because my code addi= tion is far from perfect. And one more question, what does 0.0.0.0 mean in -T tunnel specification? I= s it just an example host, or 0.0.0.0 has some special meaning for dtntunnel? 2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I have duplicated what you are reporting and have to retract my earlier sta= tement saying that this is the netcat design and not a problem :) The sender netcat attempts to close the socket when it gets the [EOF] and n= ot in response to the receiver closing its end. netcat sender: cat /etc/passwd | nc x.x.x.x 12345 scenario 1 - netcat receiver with the keep listener option: nc -k -l 12345 * Sender terminates and receiver stays alive and listening for a re= -connect * FIN is sent in both directions initiated by the sender scenario 2 - dtntunnel receiver: dtntunnel -t -T 12345:0.0.0.0:54321 d= tn://desteid/xxxx * Sender does not terminate * FIN is sent from sender to receiver but not the other way * Sender socket is in state FIN_WAIT2 * Receiver socket is in state CLOSE_WAIT Kill the sender and repeat... * Sender does not terminate * FIN is sent from sender to receiver but not the other way * New Sender socket is in state FIN_WAIT2 and the first one has tim= ed out and died * Both Receiver sockets are in state CLOSE_WAIT Kill the dtntunnel receiver... * 2 FINs are sent from the receiver - 1 to each of the sockets (eve= n though the other ends have expired) I believe the issue would be in the oasys IPSocket or possibly the IPClient= . I'll have a look at it unless there is a low-level socket expert out ther= e that wants to give it a go... Best regards, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Wednesday, December 12, 2012 4:38 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Subject: Re: [dtn-users] A problem with dtntunnel Hi David, I have investigated that issue a little. I have found, that no matter wheth= er netcat is run with or without the -w1 option, something strange happens. According to netstat -ntp on the sending node, dtntunnel r= emains in CLOSE_WAIT state after nc finishes sending data. I have run tcpdump on both sender and receiver, and it showed that fi= nal FIN from the receiver doesn't reach the sender. This looked like a bug in dtntunnel to me. I decided to re-check with iperf= , and again, tcpdump showed the same problem. Here are the last lines of the tcpdump. Remember that in my setup all outgo= ing TCP packets with destination port 9999 are redirected with the help of IPTables to the local dtntunnel process, listening om port= 19999. That is why I run tcpdump on loopback interface. Receiver (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999 15:05:02.496366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 12= 3995:131097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr 11163718]= , length 7102 15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131= 097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0 15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], seq 13= 1097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724], lengt= h 0 15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack 131= 098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length 0 15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], seq 1,= ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684], leng= th 0 15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack 2, = win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0 Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999 15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 127= 449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 0 15:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], seq 12= 7448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887],= length 2896 15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], seq 1= 30344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887]= , length 752 15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131= 097, win 386, options [nop,nop,TS val 11172898 ecr 11172887], length 0 15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], seq 1= 31096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], lengt= h 0 15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack 131= 098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], length 0 I believe that the cause of why the sending netcat without -w1 option doesn= 't exit, is that it doesn't receive the final FIN from the dtntunnel. Can this issue be fixed with a small amount of work? 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I would not even call this a problem. netcat is designed to be extremely fl= exible and by default the sender keeps its connection open as long as the o= ther end does. On the other hand the receiver by default terminates after r= eceipt of an EOF but there is a switch to keep listening if that is the des= ired behavior. With DTN, your sending nc may do its thing while the destination node is no= t even on line and then an hour later it becomes available and completes th= e transmission to the receiving nc. In this scenario, the sender would stil= l be "hung up" possibly indefinitely waiting for a terminating signal. I am not the DTN2 authHi ority, but, I don't see a change to dtntunnel for = this as it would assume a specific usage that would probably break someone = else's usage (like mine :)). Best regards, DZ David Zoller COLSA Corporation Marshall Space Flight Center From: ssireskin@gmail.com [mailto:ssireskin@gma= il.com] Sent: Wednesday, December 05, 2012 2:56 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Thanks for your help, nc -w1 did the trick. But shouldn't dtntunnel behavio= r be changed? The receiving dtntunnel could signal the sending dtntunnel, t= hat the listening nc has disconnected, so that the sending dtntunnel close = its connection to the sending nc. Or is this solely the problem of nc? 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not impa= ct the behavior of dtntunnel. I just ran your test without the iptables redirect and I see the same behav= ior. I believe that the sender exits if you go directly from nc to nc because th= e receiver exits when it gets an end of file. You can add a "-w 1" option to the sender nc so that it will timeout and ex= it after stdin is idle for 1 second. Best regards, DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: dtn-users-bounces@irtf.org [mailto= :dtn-users-bounces@irtf.org] On Behalf O= f ssireskin@gmail.com Sent: Tuesday, December 04, 2012 11:13 AM To: dtn-users@irtf.org Subject: [dtn-users] A problem with dtntunnel Hello all! I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a probl= em when using netcat (nc) with dtntunnel. On the sender node I run "cat /etc/passwd | nc receiver_ip". On the receive= r node I run "nc -l 9999". With the help of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel. The file /etc/passwd is successfully delivered to the receiver and is shown= on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it sends= the file. It continues to run forever. When I run nc in the opposite direction, I get the same problem - the sending nc d= oesn't exit. My configuration on both nodes is symmetric: iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to = $LOCAL_HOST:19999 dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc = -d dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d dtnping works ok in both directions. Please give me any advice. With best regards, Sergey Sireskin -- Kindest Regards Sergey Sireskin FGUP CNII EISU -- Kindest Regards Sergey Sireskin FGUP CNII EISU -- Best regards, Sergey Sireskin -- Best regards, Sergey Sireskin -- Best regards, Sergey Sireskin --_000_04E3D99A62496240BCD6A576813E6E31E0BDECDCE2NDMSSCC05ndcn_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Sergey= ,

I see now the end to end= effect you are wanting and I’ll experiment some more and get back wi= th you but it may be a few days.

=  

I guess one = question is:

If both ends= are not up in “near real time” would you want the sender to st= ay connected until the receiving end comes on line and can return the FIN i= n all circumstances?

DZ

 

David Zoller

COLSA Corporation

HOSC / C107 

(Office:<= span style=3D'font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F= 497D'> (256) 544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

 <= /o:p>

From: ssireskin@gmail.com [mailto:ssires= kin@gmail.com]
Sent: Thursday, December 13, 2012 3:23 AM
T= o: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: d= tn-users@irtf.org
Subject: Re: [dtn-users] A problem with dtntunn= el

 

Hi David,

I'll start fro= m the end. I have repeated the test with 100 parallel flows twice, and it p= assed flawlessly. I think that was my mistake, I could have counted
the= number of erased connections too early, before all of 100 connections have= terminated.

Now to the first part. I have replaced while(1) with wh= ile(!sock_eof) as you suggested. This leads to very strange results when se= nding a small file with nc. Note the last two SYN and RST packets on the re= ceiver side.

Sender: tcpdump -nn -npi lo port 9999 or port 19999
= 13:10:35.136283 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [S], seq= 2925076366, win 5840, options [mss 1460,sackOK,TS val 90705589 ecr 0,nop,w= scale 7], length 0
13:10:35.136346 IP 192.168.4.1.9999 > 192.168.1.2.= 32880: Flags [S.], seq 2931196344, ack 2925076367, win 32768, options [mss = 16396,sackOK,TS val 90705589 ecr 90705589,nop,wscale 7], length 0
13:10:= 35.136371 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [.], ack 29311= 96345, win 46, options [nop,nop,TS val 90705589 ecr 90705589], length 0
= 13:10:35.136799 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [P.], se= q 0:1, ack 1, win 46, options [nop,nop,TS val 90705590 ecr 90705589], lengt= h 1
13:10:35.136819 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [.= ], ack 2, win 256, options [nop,nop,TS val 90705590 ecr 90705590], length 0=
13:10:35.136892 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [F.]= , seq 1, ack 1, win 46, options [nop,nop,TS val 90705590 ecr 90705590], len= gth 0
13:10:35.176755 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags = [.], ack 3, win 256, options [nop,nop,TS val 90705630 ecr 90705590], length= 0
13:10:35.178602 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [F.= ], seq 1, ack 3, win 256, options [nop,nop,TS val 90705631 ecr 90705590], l= ength 0
13:10:35.178623 IP 192.168.1.2.32880 > 192.168.1.2.19999: Fla= gs [.], ack 2, win 46, options [nop,nop,TS val 90705631 ecr 90705631], leng= th 0
13:10:35.253829 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags = [S], seq 2934914533, win 5840, options [mss 1460,sackOK,TS val 90705707 ecr= 0,nop,wscale 7], length 0
13:10:35.253860 IP 192.168.4.1.9999 > 192.= 168.1.2.32881: Flags [S.], seq 2932249125, ack 2934914534, win 32768, optio= ns [mss 16396,sackOK,TS val 90705707 ecr 90705707,nop,wscale 7], length 013:10:35.253878 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [.], a= ck 2932249126, win 46, options [nop,nop,TS val 90705707 ecr 90705707], leng= th 0
13:10:36.380942 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [= F.], seq 1, ack 1, win 256, options [nop,nop,TS val 90706834 ecr 90705707],= length 0
13:10:36.381495 IP 192.168.1.2.32881 > 192.168.1.2.19999: F= lags [.], ack 2, win 46, options [nop,nop,TS val 90706835 ecr 90706834], le= ngth 0
13:10:36.398070 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flag= s [F.], seq 0, ack 2, win 46, options [nop,nop,TS val 90706851 ecr 90706834= ], length 0
13:10:36.398131 IP 192.168.4.1.9999 > 192.168.1.2.32881: = Flags [.], ack 2, win 256, options [nop,nop,TS val 90706851 ecr 90706851], = length 0

Receiver: tcpdump -nn -npi lo port 9999 or port 19999
13= :10:35.185211 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [S], seq 29= 27123945, win 32792, options [mss 16396,sackOK,TS val 90696412 ecr 0,nop,ws= cale 7], length 0
13:10:35.185266 IP 192.168.4.1.9999 > 192.168.4.1.4= 4550: Flags [S.], seq 2929621602, ack 2927123946, win 32768, options [mss 1= 6396,sackOK,TS val 90696412 ecr 90696412,nop,wscale 7], length 0
13:10:3= 5.185286 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [.], ack 1, win = 257, options [nop,nop,TS val 90696412 ecr 90696412], length 0
13:10:35.2= 06818 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [P.], seq 1:2, ack = 1, win 257, options [nop,nop,TS val 90696434 ecr 90696412], length 1
13:= 10:35.206861 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [.], ack 2, = win 256, options [nop,nop,TS val 90696434 ecr 90696434], length 0
13:10:= 35.222032 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [F.], seq 2, ac= k 1, win 257, options [nop,nop,TS val 90696449 ecr 90696434], length 0
1= 3:10:35.222133 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [F.], seq = 1, ack 3, win 256, options [nop,nop,TS val 90696449 ecr 90696449], length 0=
13:10:35.222146 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [.], = ack 2, win 257, options [nop,nop,TS val 90696449 ecr 90696449], length 013:10:36.325396 IP 192.168.4.1.44551 > 192.168.4.1.9999: Flags [S], seq= 2945716889, win 32792, options [mss 16396,sackOK,TS val 90697552 ecr 0,nop= ,wscale 7], length 0
13:10:36.325447 IP 192.168.4.1.9999 > 192.168.4.= 1.44551: Flags [R.], seq 0, ack 2945716890, win 0, length 0

Another = strange thing happend when I have run sending nc and forgot to run the rece= iving nc. Just try it yourself and see tcpdump.

I have also conducte= d another test with the original dtntunnel code. I have run nc in the inter= active mode, i.e. nc <ip> <port> on the client side and nc -l &= lt;port> on the server side. Then I terminated the connection on the cli= ent side by pressing CTRL+D. The server received a FIN packed from the clie= nt, replied with another FIN, and terminated. The client did not receive th= e FIN packet from the server and did not terminate. After that I have estab= lished a new connection and terminated it by pressing CTRL+D on the server = side. This time server did send a FIN packet, the client got it, replied wi= th its own FIN and terminated. The server did not receive the FIN from the = client and continued running.

This makes me think that dtntunnel, th= at have received the final FIN from a TCP application, doesn't transmit thi= s FIN to its remote dtntunnel peer. Or the latter one doesn't retransmit th= is final FIN to the TCP application on its side. I'm not a TCP and sockets = guru, so I cannot figure out how to translate this to the language of socke= ts, which I think is important to do in order to understand how to fix dtnt= unnel.

I don't think that implementing an option that configures dtn= tunnel behavior in relation to closing connections is a good idea. I believ= e that there is a flaw in dtntunnel logic, and it could be fixed in a way t= hat dtntunnel determines itself when to close a connection to a TCP applica= tion.

2012/12/13 Zoller, David A. (= MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

Research = indicates that the TIME_WAIT state is a feature of TCP to prevent delayed p= ackets for connection from being delivered to a later connection between th= e same hosts. [http://tools.ietf.org/html/draft-faber-time= -wait-avoidance-00]

 

Based on that, the added “if (sock_eof)R= 21; block can be removed and the while condition can be changed to “w= hile (!sock_eof)” and you get the same result.

<= p class=3DMsoNormal style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:= auto'> 

But, looking at the = intent of the code and at the oasys::IPSocket, it appears that it is possib= le to establish a socket connection and then only close the read side while= keeping the write side open. I’ve never run across a socket used in = that manner but that is what it looks like to me. So, the TCPTunnel::Connec= tion::run  while loop allows for the read side to close and continue t= ransmitting bundle payloads on the write side until it gets a bundle with t= he EOF bit set. If there are no bundles coming back the other way as is the= case here then the while loop never exits which keeps the netcat connectio= n open.

 

So, based on that, I think a better solution would be to implemen= t an option that configures dtntunnel to close the connection when the read= side terminates. (Someone in the know can weigh in) Your solution is proba= bly sufficient for your test.

 

--

&nbs= p;

As to connections not being removed f= rom the table and sockets not being closed…

 

I set up a test to kick= off 100 netcats on an 11MB file so that the connection would be open for s= everal seconds:

#!/bin/= bash

ctr=3D0

while [ $ctr -lt 100 ]; do=

    nc -w 10 x= .x.x.x 12345 < data11mg  &

    let ctr=3Dctr+1

done

 <= /span>

I also added some debug code to print ou= t a connection accepted counter and closed counter as they are accepted and= closed.

After 30 connections, the first= close kicks in and then there is a mix of connects and closes until finish= ed.

All of the connections were closed a= nd I did not have any sockets left open.

 

On my first attempt I exceeded m= y payload quota and started seeing “error sending bundle: 141” = messages so I don’t think that is the issue you are running into.

 

Y= ou might try putting in similar debug to see if you can determine what is h= appening.

 

Hope this helps,

DZ

 

 

David Zoller

=

COLSA Corporation

HOSC / C107&= nbsp;

(Office: (256) 544-1820
*EMail: david.a.zoller@nasa.g= ov

 

From: ssireskin@gmail.com [mailto:ssireskin@gmail.com]
Sent: Wednesday, Decembe= r 12, 2012 2:07 PM


= To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.o= rg
Subject: Re: [dtn-users] A problem with dtntunnel

 

Hi David,

I= meant exactly the same position before the end of the while loop, just des= cribed it incorrect.

Please let me know if you find the solution for= this problem.

Best regards,
Sergey

2= 012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.g= ov>

Hi Sergey,

I was narrowing in on the same track... The FIN results= in a returned length of zero on the read and errno is also zero so you hav= e to key on the return length which is already in the code.

 

You may have= added a couple of lines of debug code and it should be inserted just befor= e the end of the while(1) loop at line 491 in the latest version on Sourcef= orge.

 

I am seeing the sending socket stay in the TIME_WAIT mode even if I= add a call to sock_.shutdown(SHUT_RDWR) before closing the socket;<= o:p>

Getting close I think.

 

The 0.0.0.0 was becaus= e I did not have anything at the other end of the tunnel and just stuck som= e easy numbers in there.

Thanks,
DZ

 

<= span style=3D'font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F49= 7D'>David Zoller

COLSA Corporation<= o:p>

Marshall Space Flight Center

 

From: ssireskin@gmail.com [mailto:<= a href=3D"mailto:ssireskin@gmail.com" target=3D"_blank">ssireskin@gmail.com= ]
Sent: Wednesday, December 12, 2012 11:06 AM


To: Zoller, David A. (MSFC-EO60)[HOSC SE= RVICES CONTRACT]
Cc: dtn-users@irtf.org
Subject: Re: [dtn-users] A pro= blem with dtntunnel

 <= /o:p>

Hi David,

I added the following code at the line 492 of TC= PTunnel.cc and it seems that it helped.

    if (sock= _eof) {
        sock_.close();
 &n= bsp;      goto done;
    }

The= senders, both nc and iperf, exit after having finished sending the data. T= here are
no more sockets in CLOSE_WAIT state. However I am completely un= sure whether
my code is correct and is in correct place.

Now I h= ave encountered another problem. When iperf test is run with 100 parallel f= lows,
some dtntunnel connections are not removed from dtntunel's connect= ion table, and the
related sockets are not closed. I suppose this happen= s because my code addition is far
from perfect.

And one more ques= tion, what does 0.0.0.0 mean in -T tunnel specification? Is it just an
e= xample host, or 0.0.0.0 has some special meaning for dtntunnel?<= /p>

2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRA= CT] <david.= a.zoller@nasa.gov>

Hi Sergey,<= /span>

I have duplicated what you are reporting= and have to retract my earlier statement saying that this is the netcat de= sign and not a problem J

The sender n= etcat attempts to close the socket when it gets the [EOF] and not in respon= se to the receiver closing its end.

&nbs= p;

 

<= b>netcat sender:

cat /etc/passwd | nc x.x.x.x 12345

=  

scenario 1 - netcat receiver w= ith the keep listener option:

&nbs= p;            &= nbsp;  nc -k -l 12345

&midd= ot;   &= nbsp;     Sender terminates and receiver = stays alive and listening for a re-connect

·&nb= sp;        FIN is sent in = both directions initiated by the sender

=  

scenario 2 - dtntunnel receive= r:

     &nb= sp;          dtntunnel -t -T 1= 2345:0.0.0.0:54321 d= tn://desteid/xxxx

·    &nbs= p;    Sender does not terminate

·         FIN is sent from sender to receiver but not the other way

·         Sender socket is in state FIN_WAIT2

· &= nbsp;       Receiver socket is = in state CLOSE_WAIT

Kil= l the sender and repeat...

&midd= ot;   &= nbsp;     Sender does not terminate

·         FIN is sent from sender to receiver but not the other way

·         New Sender socket is in state FIN_WAIT2 and the first one has t= imed out and died

·    &nbs= p;    Both Receiver sockets are in state CLOSE= _WAIT

Kill the dtntunne= l receiver…

·    &nbs= p;    2 FINs are sent from the receiver –= ; 1 to each of the sockets (even though the other ends have expired)=

 

&nbs= p;

I believe the issue would be in the o= asys IPSocket or possibly the IPClient. I’ll have a look at it unless= there is a low-level socket expert out there that wants to give it a go= 230;

 

Best regards,

DZ

 

David Zol= ler

COLSA Corporation

Marshall Space Flight Center

 

From: ssireskin@gmail.com [mailto:ssireskin@gmail.com]
= Sent: Wednesday, December 12, 2012 4:38 AM


To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]

Subject: Re: [dtn-users] A proble= m with dtntunnel

 

Hi David,

I have investigated that issue a little. I have fou= nd, that no matter whether netcat is run with or without the -w1 option, so= mething
strange happens. According to netstat -ntp on the sending node,= dtntunnel remains in CLOSE_WAIT state after nc finishes sending
data. = I have run tcpdump on both sender and receiver, and it showed that final FI= N from the receiver doesn't reach the sender.
This looked like a bug in= dtntunnel to me. I decided to re-check with iperf, and again, tcpdump show= ed the same problem.

Here are the last lines of the tcpdump. Remembe= r that in my setup all outgoing TCP packets with destination port 9999 are = redirected
with the help of IPTables to the local dtntunnel process, li= stening om port 19999. That is why I run tcpdump on loopback interface.
=
Receiver (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999
= 15:05:02.496366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq= 123995:131097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr 111637= 18], length 7102
15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34= 284: Flags [.], ack 131097, win 1154, options [nop,nop,TS val 11163724 ecr = 11163723], length 0
15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.= 1.9999: Flags [F.], seq 131097, ack 1, win 257, options [nop,nop,TS val 111= 64684 ecr 11163724], length 0
15:05:03.497482 IP 192.168.4.1.9999 > 1= 92.168.4.1.34284: Flags [.], ack 131098, win 1154, options [nop,nop,TS val = 11164725 ecr 11164684], length 0
15:05:04.457762 IP 192.168.4.1.9999 >= ; 192.168.4.1.34284: Flags [F.], seq 1, ack 131098, win 1154, options [nop,= nop,TS val 11165685 ecr 11164684], length 0
15:05:04.457856 IP 192.168.4= .1.34284 > 192.168.4.1.9999: Flags [.], ack 2, win 257, options [nop,nop= ,TS val 11165685 ecr 11165685], length 0

Sender (192.168.2.1): tcpdu= mp -nn -npi lo port 9999 or port 19999
15:05:02.433755 IP 192.168.4.1.9= 999 > 192.168.1.2.43360: Flags [.], ack 127449, win 386, options [nop,no= p,TS val 11172887 ecr 11172875], length 0
15:05:02.433776 IP 192.168.1.2= .43360 > 192.168.1.2.19999: Flags [.], seq 127448:130344, ack 1, win 46,= options [nop,nop,TS val 11172887 ecr 11172887], length 2896
15:05:02.43= 3783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], seq 130344:13= 1096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172887], length= 752
15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [= .], ack 131097, win 386, options [nop,nop,TS val 11172898 ecr 11172887], le= ngth 0
15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flag= s [F.], seq 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 111= 72898], length 0
15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43= 360: Flags [.], ack 131098, win 386, options [nop,nop,TS val 11173915 ecr 1= 1173875], length 0

I believe that the cause of why the sending netca= t without -w1 option doesn't exit, is that it doesn't receive the final FIN= from the dtntunnel.
Can this issue be fixed with a small amount of work= ?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SER= VICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

I would not even call this a= problem. netcat is designed to be extremely flexible and by default the se= nder keeps its connection open as long as the other end does. On the other = hand the receiver by default terminates after receipt of an EOF but there i= s a switch to keep listening if that is the desired behavior. <= /o:p>

 

With DTN, = your sending nc may do its thing while the destination node is not even on = line and then an hour later it becomes available and completes the transmis= sion to the receiving nc. In this scenario, the sender would still be ̶= 0;hung up” possibly indefinitely waiting for a terminating signal.

 

I am not the DTN2 authHi ority, but, I don’t see a change to dtntunn= el for this as it would assume a specific usage that would probably break s= omeone else’s usage (like mine J).

 

Best regards= ,

DZ

=  

David Zoller

<= p class=3DMsoNormal style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:= auto'>COLSA Corporation

Marsha= ll Space Flight Center

 

From: ssireskin@gmail.com [mailto:ssireskin@gmail.com]
Sent: Wednesday, Dece= mber 05, 2012 2:56 AM
To: Zoller, David A. (MSFC-EO60)[HOSC SERVI= CES CONTRACT]
Cc: dtn-users@irtf.org
Subject: Re: [dtn-users] A proble= m with dtntunnel

 

Hi David,

Thanks for your help, nc -w1 did the trick. But shouldn= 't dtntunnel behavior be changed? The receiving dtntunnel could signal the = sending dtntunnel, that the listening nc has disconnected, so that the send= ing dtntunnel close its connection to the sending nc. Or is this solely the= problem of nc?

2012/12/5 Zoller, David A. (MSFC-= EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

On RHEL 5.7,= I am running dtn-2.9.0 plus modifications that should not impact the behav= ior of dtntunnel.

I just ran your test w= ithout the iptables redirect and I see the same behavior.=

I believe that the sender exits if you go directly from n= c to nc because the receiver exits when it gets an end of file.=

You can add a “-w 1” option to the send= er nc so that it will timeout and exit after stdin is idle for 1 second.

Best regards,

= DZ

 

David Zoller

COLSA Corporation

HOSC / C107 

(<= /span>Office: (256) 544-1820*EMail: david.a.zoller@nasa.gov<= /p>

 

From: dtn-users-bounces@irtf.org [mailto:d= tn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Tu= esday, December 04, 2012 11:13 AM
To: dtn-users@irtf.org
Subject: [dtn= -users] A problem with dtntunnel

&nbs= p;

Hello all!

I am using dtn-2.9.0 on a RHEL 6 = based Linux distro and I am having a problem when using netcat (nc) with dt= ntunnel.
On the sender node I run "cat /etc/passwd | nc receiver_ip= ". On the receiver node I run "nc -l 9999". With the help of Iptables port 9999 gets redirected to the port 19999, which is listene= d by dtntunnel.

The file /etc/passwd is successfully delivered to th= e receiver and is shown on the screen. After this the receiving nc exists. =
No problem here. However, nc on the sender node doesn't exit after it s= ends the file. It continues to run forever. When I
run nc in the opposi= te direction, I get the same problem - the sending nc doesn't exit.

= My configuration on both nodes is symmetric:
iptables -t nat -A OUTPUT -= d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to $LOCAL_HOST:19999
dtntun= nel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d
= dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d

dtnping works o= k in both directions.

Please give me any advice.

With best r= egards,
Sergey Sireskin



--
Kindest Regards

Sergey Sireskin
FGUP= CNII EISU




--
Kindest Regards

Sergey Sireskin
FGUP CNII EISU




--
Best = regards,
Sergey Sireskin



--
Best regards,
Sergey Sireskin




--
Best regards,
Sergey Sireskin
=

= --_000_04E3D99A62496240BCD6A576813E6E31E0BDECDCE2NDMSSCC05ndcn_-- From ssireskin@gmail.com Thu Dec 13 06:09:10 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B0D3B21F89A5 for ; Thu, 13 Dec 2012 06:09:10 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.035 X-Spam-Level: X-Spam-Status: No, score=-3.035 tagged_above=-999 required=5 tests=[AWL=-0.039, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_LOW=-1, WEIRD_PORT=0.001] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lYrWEaZpadkZ for ; Thu, 13 Dec 2012 06:09:07 -0800 (PST) Received: from mail-ia0-f182.google.com (mail-ia0-f182.google.com [209.85.210.182]) by ietfa.amsl.com (Postfix) with ESMTP id 8C31921F8980 for ; Thu, 13 Dec 2012 06:09:07 -0800 (PST) Received: by mail-ia0-f182.google.com with SMTP id x2so2024095iad.13 for ; Thu, 13 Dec 2012 06:09:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=SBgXlCUO3wEB8Xwdunoa4oYhFki1irUb0k3DaLUg8R0=; b=aNB3sRb0E7okiGzEbKJR4QASshx+aN02hkQ0R4WWjg13TqNklrmPO2g+mJWTsyOfdV C7K77i+px/DesjQAYCg1VvRbnpjM0+QpK10nkxnxcV6wWWA9AMmmWRV7qE1gAvCjv3dC i5LTOWvYC2oRHHEZhADVbqtWT3XUx66qg/CqfYpUj/uHbB3vMuj50d5lS9DquF8IxzGS NYarriM4XuH2iEUkVgi6CdPBgdBsXsFn6/F97/QIlYtR5IGTZTBrqFMgIHIftZ2LhK4n 5uvuGcnAe7JRyWI/Me9xwG5pc1Md0yw7KekbnP4itMtMwW8mpJxoctKBMpEBk0Gg+Oul MSGg== MIME-Version: 1.0 Received: by 10.50.173.34 with SMTP id bh2mr1594779igc.70.1355407746888; Thu, 13 Dec 2012 06:09:06 -0800 (PST) Received: by 10.50.97.236 with HTTP; Thu, 13 Dec 2012 06:09:06 -0800 (PST) In-Reply-To: <04E3D99A62496240BCD6A576813E6E31E0BDECDCE2@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDBBA4FA@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDBBA685@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD834@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECD996@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECDC13@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDECDCE2@NDMSSCC05.ndc.nasa.gov> Date: Thu, 13 Dec 2012 17:09:06 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/alternative; boundary=e89a8f647b3b4775fa04d0bc7532 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 14:09:10 -0000 --e89a8f647b3b4775fa04d0bc7532 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David, I want dtntunnel to be as transparent to a TCP application as possible. Also I am not sure whether it will work or not, but it would be nice if in case of connection disruption, or even in case of no connection at the time when the sending application initiates the transfer, dtn could cache the data that it received from the application and send it to the receiving application when connection is restored. This will work only for limited class of applications that only send data and don't expect replies from the peer. But in case if an application on either sending or receiving side * explicitly* terminates the connection, closes socket, sends FIN (not the case of lost connectivity), dtntunnel must do the same to the application on the other side, so that it terminates correctly. This is exactly what happens in case of a direct connection between application client and server without dtntunnel. 2012/12/13 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < david.a.zoller@nasa.gov> > Hi Sergey,**** > > I see now the end to end effect you are wanting and I=92ll experiment som= e > more and get back with you but it may be a few days.**** > > ** ** > > I guess one question is: **** > > If both ends are not up in =93near real time=94 would you want the sender= to > stay connected until the receiving end comes on line and can return the F= IN > in all circumstances?**** > > DZ**** > > ** ** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > ** ** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Thursday, December 13, 2012 3:23 AM > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > ** ** > > Hi David, > > I'll start from the end. I have repeated the test with 100 parallel flows > twice, and it passed flawlessly. I think that was my mistake, I could hav= e > counted > the number of erased connections too early, before all of 100 connections > have terminated. > > Now to the first part. I have replaced while(1) with while(!sock_eof) as > you suggested. This leads to very strange results when sending a small fi= le > with nc. Note the last two SYN and RST packets on the receiver side. > > Sender: tcpdump -nn -npi lo port 9999 or port 19999 > 13:10:35.136283 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [S], seq > 2925076366, win 5840, options [mss 1460,sackOK,TS val 90705589 ecr > 0,nop,wscale 7], length 0 > 13:10:35.136346 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [S.], seq > 2931196344, ack 2925076367, win 32768, options [mss 16396,sackOK,TS val > 90705589 ecr 90705589,nop,wscale 7], length 0 > 13:10:35.136371 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [.], ack > 2931196345, win 46, options [nop,nop,TS val 90705589 ecr 90705589], lengt= h 0 > 13:10:35.136799 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [P.], seq > 0:1, ack 1, win 46, options [nop,nop,TS val 90705590 ecr 90705589], lengt= h 1 > 13:10:35.136819 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [.], ack 2= , > win 256, options [nop,nop,TS val 90705590 ecr 90705590], length 0 > 13:10:35.136892 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [F.], seq > 1, ack 1, win 46, options [nop,nop,TS val 90705590 ecr 90705590], length = 0 > 13:10:35.176755 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [.], ack 3= , > win 256, options [nop,nop,TS val 90705630 ecr 90705590], length 0 > 13:10:35.178602 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [F.], seq > 1, ack 3, win 256, options [nop,nop,TS val 90705631 ecr 90705590], length= 0 > 13:10:35.178623 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [.], ack > 2, win 46, options [nop,nop,TS val 90705631 ecr 90705631], length 0 > 13:10:35.253829 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [S], seq > 2934914533, win 5840, options [mss 1460,sackOK,TS val 90705707 ecr > 0,nop,wscale 7], length 0 > 13:10:35.253860 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [S.], seq > 2932249125, ack 2934914534, win 32768, options [mss 16396,sackOK,TS val > 90705707 ecr 90705707,nop,wscale 7], length 0 > 13:10:35.253878 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [.], ack > 2932249126, win 46, options [nop,nop,TS val 90705707 ecr 90705707], lengt= h 0 > 13:10:36.380942 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [F.], seq > 1, ack 1, win 256, options [nop,nop,TS val 90706834 ecr 90705707], length= 0 > 13:10:36.381495 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [.], ack > 2, win 46, options [nop,nop,TS val 90706835 ecr 90706834], length 0 > 13:10:36.398070 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [F.], seq > 0, ack 2, win 46, options [nop,nop,TS val 90706851 ecr 90706834], length = 0 > 13:10:36.398131 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [.], ack 2= , > win 256, options [nop,nop,TS val 90706851 ecr 90706851], length 0 > > Receiver: tcpdump -nn -npi lo port 9999 or port 19999 > 13:10:35.185211 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [S], seq > 2927123945, win 32792, options [mss 16396,sackOK,TS val 90696412 ecr > 0,nop,wscale 7], length 0 > 13:10:35.185266 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [S.], seq > 2929621602, ack 2927123946, win 32768, options [mss 16396,sackOK,TS val > 90696412 ecr 90696412,nop,wscale 7], length 0 > 13:10:35.185286 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [.], ack 1= , > win 257, options [nop,nop,TS val 90696412 ecr 90696412], length 0 > 13:10:35.206818 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [P.], seq > 1:2, ack 1, win 257, options [nop,nop,TS val 90696434 ecr 90696412], leng= th > 1 > 13:10:35.206861 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [.], ack 2= , > win 256, options [nop,nop,TS val 90696434 ecr 90696434], length 0 > 13:10:35.222032 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [F.], seq > 2, ack 1, win 257, options [nop,nop,TS val 90696449 ecr 90696434], length= 0 > 13:10:35.222133 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [F.], seq > 1, ack 3, win 256, options [nop,nop,TS val 90696449 ecr 90696449], length= 0 > 13:10:35.222146 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [.], ack 2= , > win 257, options [nop,nop,TS val 90696449 ecr 90696449], length 0 > 13:10:36.325396 IP 192.168.4.1.44551 > 192.168.4.1.9999: Flags [S], seq > 2945716889, win 32792, options [mss 16396,sackOK,TS val 90697552 ecr > 0,nop,wscale 7], length 0 > 13:10:36.325447 IP 192.168.4.1.9999 > 192.168.4.1.44551: Flags [R.], seq > 0, ack 2945716890, win 0, length 0 > > Another strange thing happend when I have run sending nc and forgot to ru= n > the receiving nc. Just try it yourself and see tcpdump. > > I have also conducted another test with the original dtntunnel code. I > have run nc in the interactive mode, i.e. nc on the client si= de > and nc -l on the server side. Then I terminated the connection on > the client side by pressing CTRL+D. The server received a FIN packed from > the client, replied with another FIN, and terminated. The client did not > receive the FIN packet from the server and did not terminate. After that = I > have established a new connection and terminated it by pressing CTRL+D on > the server side. This time server did send a FIN packet, the client got i= t, > replied with its own FIN and terminated. The server did not receive the F= IN > from the client and continued running. > > This makes me think that dtntunnel, that have received the final FIN from > a TCP application, doesn't transmit this FIN to its remote dtntunnel peer= . > Or the latter one doesn't retransmit this final FIN to the TCP applicatio= n > on its side. I'm not a TCP and sockets guru, so I cannot figure out how t= o > translate this to the language of sockets, which I think is important to = do > in order to understand how to fix dtntunnel. > > I don't think that implementing an option that configures dtntunnel > behavior in relation to closing connections is a good idea. I believe tha= t > there is a flaw in dtntunnel logic, and it could be fixed in a way that > dtntunnel determines itself when to close a connection to a TCP applicati= on. > **** > > 2012/12/13 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > Research indicates that the TIME_WAIT state is a feature of TCP to preven= t > delayed packets for connection from being delivered to a later connection > between the same hosts. [ > http://tools.ietf.org/html/draft-faber-time-wait-avoidance-00]**** > > **** > > Based on that, the added =93if (sock_eof)=94 block can be removed and the > while condition can be changed to =93while (!sock_eof)=94 and you get the= same > result.**** > > **** > > But, looking at the intent of the code and at the oasys::IPSocket, it > appears that it is possible to establish a socket connection and then onl= y > close the read side while keeping the write side open. I=92ve never run > across a socket used in that manner but that is what it looks like to me. > So, the TCPTunnel::Connection::run while loop allows for the read side t= o > close and continue transmitting bundle payloads on the write side until i= t > gets a bundle with the EOF bit set. If there are no bundles coming back t= he > other way as is the case here then the while loop never exits which keeps > the netcat connection open.**** > > **** > > So, based on that, I think a better solution would be to implement an > option that configures dtntunnel to close the connection when the read si= de > terminates. (Someone in the know can weigh in) Your solution is probably > sufficient for your test.**** > > **** > > --**** > > **** > > As to connections not being removed from the table and sockets not being > closed=85**** > > **** > > I set up a test to kick off 100 netcats on an 11MB file so that the > connection would be open for several seconds:**** > > #!/bin/bash**** > > ctr=3D0**** > > while [ $ctr -lt 100 ]; do**** > > nc -w 10 x.x.x.x 12345 < data11mg &**** > > let ctr=3Dctr+1**** > > done**** > > **** > > I also added some debug code to print out a connection accepted counter > and closed counter as they are accepted and closed.**** > > After 30 connections, the first close kicks in and then there is a mix of > connects and closes until finished.**** > > All of the connections were closed and I did not have any sockets left > open.**** > > **** > > On my first attempt I exceeded my payload quota and started seeing =93err= or > sending bundle: 141=94 messages so I don=92t think that is the issue you = are > running into.**** > > **** > > You might try putting in similar debug to see if you can determine what i= s > happening.**** > > **** > > Hope this helps,**** > > DZ**** > > **** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 12, 2012 2:07 PM**** > > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > I meant exactly the same position before the end of the while loop, just > described it incorrect. > > Please let me know if you find the solution for this problem. > > Best regards, > Sergey**** > > 2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > I was narrowing in on the same track... The FIN results in a returned > length of zero on the read and errno is also zero so you have to key on t= he > return length which is already in the code.**** > > **** > > You may have added a couple of lines of debug code and it should be > inserted just before the end of the while(1) loop at line 491 in the late= st > version on Sourceforge.**** > > **** > > I am seeing the sending socket stay in the TIME_WAIT mode even if I add a > call to sock_.shutdown(SHUT_RDWR) before closing the socket;**** > > Getting close I think.**** > > **** > > The 0.0.0.0 was because I did not have anything at the other end of the > tunnel and just stuck some easy numbers in there.**** > > Thanks, > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 12, 2012 11:06 AM**** > > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > I added the following code at the line 492 of TCPTunnel.cc and it seems > that it helped. > > if (sock_eof) { > sock_.close(); > goto done; > } > > The senders, both nc and iperf, exit after having finished sending the > data. There are > no more sockets in CLOSE_WAIT state. However I am completely unsure > whether > my code is correct and is in correct place. > > Now I have encountered another problem. When iperf test is run with 100 > parallel flows, > some dtntunnel connections are not removed from dtntunel's connection > table, and the > related sockets are not closed. I suppose this happens because my code > addition is far > from perfect. > > And one more question, what does 0.0.0.0 mean in -T tunnel specification? > Is it just an > example host, or 0.0.0.0 has some special meaning for dtntunnel?**** > > 2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > I have duplicated what you are reporting and have to retract my earlier > statement saying that this is the netcat design and not a problem J**** > > The sender netcat attempts to close the socket when it gets the [EOF] and > not in response to the receiver closing its end.**** > > **** > > **** > > *netcat sender:***** > > cat /etc/passwd | nc x.x.x.x 12345**** > > **** > > *scenario 1 - netcat receiver with the keep listener option:***** > > nc -k -l 12345**** > > =B7 Sender terminates and receiver stays alive and listening for = a > re-connect**** > > =B7 FIN is sent in both directions initiated by the sender**** > > **** > > *scenario 2 - dtntunnel receiver:***** > > dtntunnel -t -T 12345:0.0.0.0:54321 dtn://desteid/xxxx***= * > > =B7 Sender does not terminate**** > > =B7 FIN is sent from sender to receiver but not the other way**** > > =B7 Sender socket is in state FIN_WAIT2**** > > =B7 Receiver socket is in state CLOSE_WAIT**** > > Kill the sender and repeat...**** > > =B7 Sender does not terminate**** > > =B7 FIN is sent from sender to receiver but not the other way**** > > =B7 New Sender socket is in state FIN_WAIT2 and the first one has > timed out and died**** > > =B7 Both Receiver sockets are in state CLOSE_WAIT**** > > Kill the dtntunnel receiver=85**** > > =B7 2 FINs are sent from the receiver =96 1 to each of the socket= s > (even though the other ends have expired)**** > > **** > > **** > > I believe the issue would be in the oasys IPSocket or possibly the > IPClient. I=92ll have a look at it unless there is a low-level socket exp= ert > out there that wants to give it a go=85**** > > **** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 12, 2012 4:38 AM**** > > > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]**** > > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > I have investigated that issue a little. I have found, that no matter > whether netcat is run with or without the -w1 option, something > strange happens. According to netstat -ntp on the sending node, dtntunnel > remains in CLOSE_WAIT state after nc finishes sending > data. I have run tcpdump on both sender and receiver, and it showed that > final FIN from the receiver doesn't reach the sender. > This looked like a bug in dtntunnel to me. I decided to re-check with > iperf, and again, tcpdump showed the same problem. > > Here are the last lines of the tcpdump. Remember that in my setup all > outgoing TCP packets with destination port 9999 are redirected > with the help of IPTables to the local dtntunnel process, listening om > port 19999. That is why I run tcpdump on loopback interface. > > Receiver (192.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999 > 15:05:02.496366 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq > 123995:131097, ack 1, win 257, options [nop,nop,TS val 11163723 ecr > 11163718], length 7102 > 15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack > 131097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length = 0 > 15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], seq > 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724], > length 0 > 15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack > 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length = 0 > 15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], seq > 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684], > length 0 > 15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack 2= , > win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0 > > Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999 > 15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack > 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 0 > 15:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], seq > 127448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr > 11172887], length 2896 > 15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], seq > 130344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr > 11172887], length 752 > 15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack > 131097, win 386, options [nop,nop,TS val 11172898 ecr 11172887], length 0 > 15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], seq > 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], > length 0 > 15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack > 131098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], length 0 > > I believe that the cause of why the sending netcat without -w1 option > doesn't exit, is that it doesn't receive the final FIN from the dtntunnel= . > Can this issue be fixed with a small amount of work?**** > > 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > I would not even call this a problem. netcat is designed to be extremely > flexible and by default the sender keeps its connection open as long as t= he > other end does. On the other hand the receiver by default terminates afte= r > receipt of an EOF but there is a switch to keep listening if that is the > desired behavior. **** > > **** > > With DTN, your sending nc may do its thing while the destination node is > not even on line and then an hour later it becomes available and complete= s > the transmission to the receiving nc. In this scenario, the sender would > still be =93hung up=94 possibly indefinitely waiting for a terminating si= gnal. > **** > > **** > > I am not the DTN2 authHi ority, but, I don=92t see a change to dtntunnel = for > this as it would assume a specific usage that would probably break someon= e > else=92s usage (like mine J).**** > > **** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > Marshall Space Flight Center**** > > **** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Wednesday, December 05, 2012 2:56 AM > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David, > > Thanks for your help, nc -w1 did the trick. But shouldn't dtntunnel > behavior be changed? The receiving dtntunnel could signal the sending > dtntunnel, that the listening nc has disconnected, so that the sending > dtntunnel close its connection to the sending nc. Or is this solely the > problem of nc?**** > > 2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > Hi Sergey,**** > > On RHEL 5.7, I am running dtn-2.9.0 plus modifications that should not > impact the behavior of dtntunnel.**** > > I just ran your test without the iptables redirect and I see the same > behavior.**** > > I believe that the sender exits if you go directly from nc to nc because > the receiver exits when it gets an end of file.**** > > You can add a =93-w 1=94 option to the sender nc so that it will timeout = and > exit after stdin is idle for 1 second.**** > > Best regards,**** > > DZ**** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *O= n > Behalf Of *ssireskin@gmail.com > *Sent:* Tuesday, December 04, 2012 11:13 AM > *To:* dtn-users@irtf.org > *Subject:* [dtn-users] A problem with dtntunnel**** > > **** > > Hello all! > > I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am having a > problem when using netcat (nc) with dtntunnel. > On the sender node I run "cat /etc/passwd | nc receiver_ip". On the > receiver node I run "nc -l 9999". With the help > of Iptables port 9999 gets redirected to the port 19999, which is listene= d > by dtntunnel. > > The file /etc/passwd is successfully delivered to the receiver and is > shown on the screen. After this the receiving nc exists. > No problem here. However, nc on the sender node doesn't exit after it > sends the file. It continues to run forever. When I > run nc in the opposite direction, I get the same problem - the sending nc > doesn't exit. > > My configuration on both nodes is symmetric: > iptables -t nat -A OUTPUT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --t= o > $LOCAL_HOST:19999 > dtntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/n= c > -d > dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d > > dtnping works ok in both directions. > > Please give me any advice. > > With best regards, > Sergey Sireskin **** > > > > > -- > Kindest Regards > > Sergey Sireskin > FGUP CNII EISU**** > > > > > -- > Kindest Regards > > Sergey Sireskin > FGUP CNII EISU**** > > > > > -- > Best regards, > Sergey Sireskin**** > > > > > -- > Best regards, > Sergey Sireskin**** > > > > > -- > Best regards, > Sergey Sireskin > > **** > --=20 Best regards, Sergey Sireskin --e89a8f647b3b4775fa04d0bc7532 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David,

I want dtntunnel to be as transparent to a TCP application= as possible.

Also I am not sure whether it will work or not, but it= would be nice if in case of connection disruption, or even in case of no c= onnection at the time when the sending application initiates the transfer, = dtn could cache the data that it received from the application and send it = to the receiving application when connection is restored. This will work on= ly for limited class of applications that only send data and don't expe= ct replies from the peer.

But in case if an application on either sending or receiving side ex= plicitly terminates the connection, closes socket, sends FIN (not the c= ase of lost connectivity), dtntunnel must do the same to the application on= the other side, so that it terminates correctly. This is exactly what happ= ens in case of a direct connection between application client and server wi= thout dtntunnel.


2012/12/13 Zoller, David A. (MSFC-EO60)[= HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">

Hi Sergey,

I see now the end to end effect you are wanting = and I=92ll experiment some more and get back with you but it may be a few d= ays.

=A0<= /p>

I guess one question i= s:

If both ends are not up i= n =93near real time=94 would you want the sender to stay connected until th= e receiving end comes on line and can return the FIN in all circumstances?<= u>

DZ

=A0

David Zoller

COLSA Corporation<= /u>

HOSC / C107=A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov

=A0<= /p>

From: ssireskin@gmail.= com [mailto:ss= ireskin@gmail.com]
Sent: Thursday, December 13, 2012 3:23 AM


To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]=
Cc: dtn-= users@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,
I'll start from the end. I have repeated the test with 100 parallel f= lows twice, and it passed flawlessly. I think that was my mistake, I could = have counted
the number of erased connections too early, before all of 100 connections h= ave terminated.

Now to the first part. I have replaced while(1) with= while(!sock_eof) as you suggested. This leads to very strange results when= sending a small file with nc. Note the last two SYN and RST packets on the= receiver side.

Sender: tcpdump -nn -npi lo port 9999 or port 19999
13:10:35.136283 = IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [S], seq 2925076366, win= 5840, options [mss 1460,sackOK,TS val 90705589 ecr 0,nop,wscale 7], length= 0
13:10:35.136346 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [S.], seq= 2931196344, ack 2925076367, win 32768, options [mss 16396,sackOK,TS val 90= 705589 ecr 90705589,nop,wscale 7], length 0
13:10:35.136371 IP 192.168.1= .2.32880 > 192.168.1.2.19999: Flags [.], ack 2931196345, win 46, options= [nop,nop,TS val 90705589 ecr 90705589], length 0
13:10:35.136799 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [P.], se= q 0:1, ack 1, win 46, options [nop,nop,TS val 90705590 ecr 90705589], lengt= h 1
13:10:35.136819 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [.= ], ack 2, win 256, options [nop,nop,TS val 90705590 ecr 90705590], length 0=
13:10:35.136892 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [F.], se= q 1, ack 1, win 46, options [nop,nop,TS val 90705590 ecr 90705590], length = 0
13:10:35.176755 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [.],= ack 3, win 256, options [nop,nop,TS val 90705630 ecr 90705590], length 0 13:10:35.178602 IP 192.168.4.1.9999 > 192.168.1.2.32880: Flags [F.], seq= 1, ack 3, win 256, options [nop,nop,TS val 90705631 ecr 90705590], length = 0
13:10:35.178623 IP 192.168.1.2.32880 > 192.168.1.2.19999: Flags [.]= , ack 2, win 46, options [nop,nop,TS val 90705631 ecr 90705631], length 0 13:10:35.253829 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [S], seq= 2934914533, win 5840, options [mss 1460,sackOK,TS val 90705707 ecr 0,nop,w= scale 7], length 0
13:10:35.253860 IP 192.168.4.1.9999 > 192.168.1.2.= 32881: Flags [S.], seq 2932249125, ack 2934914534, win 32768, options [mss = 16396,sackOK,TS val 90705707 ecr 90705707,nop,wscale 7], length 0
13:10:35.253878 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [.], ack= 2932249126, win 46, options [nop,nop,TS val 90705707 ecr 90705707], length= 0
13:10:36.380942 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [F.= ], seq 1, ack 1, win 256, options [nop,nop,TS val 90706834 ecr 90705707], l= ength 0
13:10:36.381495 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [.], ack= 2, win 46, options [nop,nop,TS val 90706835 ecr 90706834], length 0
13:= 10:36.398070 IP 192.168.1.2.32881 > 192.168.1.2.19999: Flags [F.], seq 0= , ack 2, win 46, options [nop,nop,TS val 90706851 ecr 90706834], length 0 13:10:36.398131 IP 192.168.4.1.9999 > 192.168.1.2.32881: Flags [.], ack = 2, win 256, options [nop,nop,TS val 90706851 ecr 90706851], length 0
Receiver: tcpdump -nn -npi lo port 9999 or port 19999
13:10:35.185211 I= P 192.168.4.1.44550 > 192.168.4.1.9999: Flags [S], seq 2927123945, win 3= 2792, options [mss 16396,sackOK,TS val 90696412 ecr 0,nop,wscale 7], length= 0
13:10:35.185266 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [S.], seq= 2929621602, ack 2927123946, win 32768, options [mss 16396,sackOK,TS val 90= 696412 ecr 90696412,nop,wscale 7], length 0
13:10:35.185286 IP 192.168.4= .1.44550 > 192.168.4.1.9999: Flags [.], ack 1, win 257, options [nop,nop= ,TS val 90696412 ecr 90696412], length 0
13:10:35.206818 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [P.], seq= 1:2, ack 1, win 257, options [nop,nop,TS val 90696434 ecr 90696412], lengt= h 1
13:10:35.206861 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [.= ], ack 2, win 256, options [nop,nop,TS val 90696434 ecr 90696434], length 0=
13:10:35.222032 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [F.], seq= 2, ack 1, win 257, options [nop,nop,TS val 90696449 ecr 90696434], length = 0
13:10:35.222133 IP 192.168.4.1.9999 > 192.168.4.1.44550: Flags [F.]= , seq 1, ack 3, win 256, options [nop,nop,TS val 90696449 ecr 90696449], le= ngth 0
13:10:35.222146 IP 192.168.4.1.44550 > 192.168.4.1.9999: Flags [.], ack = 2, win 257, options [nop,nop,TS val 90696449 ecr 90696449], length 0
13:= 10:36.325396 IP 192.168.4.1.44551 > 192.168.4.1.9999: Flags [S], seq 294= 5716889, win 32792, options [mss 16396,sackOK,TS val 90697552 ecr 0,nop,wsc= ale 7], length 0
13:10:36.325447 IP 192.168.4.1.9999 > 192.168.4.1.44551: Flags [R.], seq= 0, ack 2945716890, win 0, length 0

Another strange thing happend wh= en I have run sending nc and forgot to run the receiving nc. Just try it yo= urself and see tcpdump.

I have also conducted another test with the original dtntunnel code. I = have run nc in the interactive mode, i.e. nc <ip> <port> on the= client side and nc -l <port> on the server side. Then I terminated t= he connection on the client side by pressing CTRL+D. The server received a = FIN packed from the client, replied with another FIN, and terminated. The c= lient did not receive the FIN packet from the server and did not terminate.= After that I have established a new connection and terminated it by pressi= ng CTRL+D on the server side. This time server did send a FIN packet, the c= lient got it, replied with its own FIN and terminated. The server did not r= eceive the FIN from the client and continued running.

This makes me think that dtntunnel, that have received the final FIN fr= om a TCP application, doesn't transmit this FIN to its remote dtntunnel= peer. Or the latter one doesn't retransmit this final FIN to the TCP a= pplication on its side. I'm not a TCP and sockets guru, so I cannot fig= ure out how to translate this to the language of sockets, which I think is = important to do in order to understand how to fix dtntunnel.

I don't think that implementing an option that configures dtntunnel= behavior in relation to closing connections is a good idea. I believe that= there is a flaw in dtntunnel logic, and it could be fixed in a way that dt= ntunnel determines itself when to close a connection to a TCP application.<= u>

2012/12/13 Zoller, David A. (MSFC-EO60)[HOSC SE= RVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

Research indicates that the TIME_WAIT = state is a feature of TCP to prevent delayed packets for connection from be= ing delivered to a later connection between the same hosts. [http://tools.ietf.org/html/draft-faber-time-wait-avoidance-00]

=A0<= /p>

Based on that, the add= ed =93if (sock_eof)=94 block can be removed and the while condition can be = changed to =93while (!sock_eof)=94 and you get the same result.

=A0<= /p>

But, looking at the in= tent of the code and at the oasys::IPSocket, it appears that it is possible= to establish a socket connection and then only close the read side while k= eeping the write side open. I=92ve never run across a socket used in that m= anner but that is what it looks like to me. So, the TCPTunnel::Connection::= run =A0while loop allows for the read side to close and continue transmitti= ng bundle payloads on the write side until it gets a bundle with the EOF bi= t set. If there are no bundles coming back the other way as is the case her= e then the while loop never exits which keeps the netcat connection open.

=A0<= /p>

So, based on that, I t= hink a better solution would be to implement an option that configures dtnt= unnel to close the connection when the read side terminates. (Someone in th= e know can weigh in) Your solution is probably sufficient for your test.

=A0<= /p>

--

=A0<= /p>

As to connections not = being removed from the table and sockets not being closed=85<= u>

=A0<= /p>

I set up a test to kic= k off 100 netcats on an 11MB file so that the connection would be open for = several seconds:

ctr=3D0

while [ $ctr -= lt 100 ]; do

=A0=A0=A0 let ctr=3Dctr+1

done

=A0<= /p>

I also added some debu= g code to print out a connection accepted counter and closed counter as the= y are accepted and closed.

After 30 connections, the= first close kicks in and then there is a mix of connects and closes until = finished.

All of the connections we= re closed and I did not have any sockets left open.

=A0<= /p>

On my first attempt I = exceeded my payload quota and started seeing =93error sending bundle: 141= =94 messages so I don=92t think that is the issue you are running into.

=A0<= /p>

You might try putting = in similar debug to see if you can determine what is happening.

=A0<= /p>

Hope this helps,

DZ<= /u>

=A0<= u>

=A0<= /p>

David Zoller

COLSA Corporation=

HOSC / C107= =A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: ssireskin@gmail.= com [mailto:ss= ireskin@gmail.com]
Sent: Wednesday, December 12, 2012 2:07 PM

<= div>


To: Zoller, David A. (MSFC-EO60)= [HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

<= /div>

=A0

Hi David,

I meant exac= tly the same position before the end of the while loop, just described it i= ncorrect.

Please let me know if you find the solution for this problem.

Be= st regards,
Sergey

2012/12/= 12 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov&g= t;

Hi Sergey,

I was= narrowing in on the same track... The FIN results in a returned length of = zero on the read and errno is also zero so you have to key on the return le= ngth which is already in the code.

=A0<= /p>

You may have added a c= ouple of lines of debug code and it should be inserted just before the end = of the while(1) loop at line 491 in the latest version on Sourceforge.

=A0<= /p>

I am seeing the sendin= g socket stay in the TIME_WAIT mode even if I add a call to sock_.shutdown(= SHUT_RDWR) before closing the socket;

Getting close I think.

=A0=

The 0.0.0.0 was because I= did not have anything at the other end of the tunnel and just stuck some e= asy numbers in there.

Thanks,
DZ

=A0

David Zoller<= u>

COLSA Corporation=

Marshall Space Flight Cente= r

=A0

From: ssireskin@gmail.com<= /a> [mailto:ssires= kin@gmail.com]
Sent: Wednesday, December 12, 2012 11:06 AM

=


To: Zoller, David A. (MSFC-EO60= )[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

<= /div>

=A0

Hi David,

I added the = following code at the line 492 of TCPTunnel.cc and it seems that it helped.=

=A0=A0=A0 if (sock_eof) {
=A0=A0=A0 =A0=A0=A0 sock_.close();
=A0= =A0=A0 =A0=A0=A0 goto done;
=A0=A0=A0 }

The senders, both nc and = iperf, exit after having finished sending the data. There are
no more so= ckets in CLOSE_WAIT state. However I am completely unsure whether
my code is correct and is in correct place.

Now I have encountered a= nother problem. When iperf test is run with 100 parallel flows,
some dtn= tunnel connections are not removed from dtntunel's connection table, an= d the
related sockets are not closed. I suppose this happens because my code addi= tion is far
from perfect.

And one more question, what does 0.0.0.= 0 mean in -T tunnel specification? Is it just an
example host, or 0.0.0.= 0 has some special meaning for dtntunnel?

2012/12/12 Zoller, David A. (MSFC-EO60)[HOSC SE= RVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

I have duplicated what you are reporti= ng and have to retract my earlier statement saying that this is the netcat = design and not a problem J

The sender netcat attempt= s to close the socket when it gets the [EOF] and not in response to the rec= eiver closing its end.

=A0<= /p>

=A0

netcat sender:<= /b>

cat /etc/passwd | nc x.x.x.x 12345<= /u>

=A0<= /p>

scenario 1 - netcat= receiver with the keep listener option:

=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 nc -k -l 12345

=B7=A0=A0=A0= =A0=A0=A0=A0=A0 Sender terminates and rec= eiver stays alive and listening for a re-connect

=B7=A0=A0=A0=A0=A0=A0=A0=A0 FIN is sent i= n both directions initiated by the sender

=A0<= /p>

scenario 2 - dtntun= nel receiver:

=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 dtntunnel -t -T 12345:0.0.0.0:54321 dtn://desteid/xxxx

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Sender does n= ot terminate

=B7=A0=A0=A0=A0=A0=A0=A0=A0 FIN is sent f= rom sender to receiver but not the other way

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Sender socket= is in state FIN_WAIT2

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Receiver sock= et is in state CLOSE_WAIT

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Sender does not terminate

=B7=A0=A0=A0=A0=A0=A0=A0=A0 FIN is sent f= rom sender to receiver but not the other way

=B7=A0=A0=A0=A0=A0=A0=A0=A0 New Sender so= cket is in state FIN_WAIT2 and the first one has timed out and died<= u>

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Both Receiver= sockets are in state CLOSE_WAIT

=B7=A0=A0=A0=A0=A0=A0=A0=A0 2 FINs are sent from the receiver =96 1 to e= ach of the sockets (even though the other ends have expired)<= u>

=A0<= /p>

=A0

I believe the issue would= be in the oasys IPSocket or possibly the IPClient. I=92ll have a look at i= t unless there is a low-level socket expert out there that wants to give it= a go=85

=A0=

Best regards,

DZ

=A0

David Zoller<= u>

COLSA Corporation=

Marshall Space Flight Cente= r

=A0

From: ssireskin@gmail.com<= /a> [mailto:ssires= kin@gmail.com]
Sent: Wednesday, December 12, 2012 4:38 AM

<= div>


To: Zoller, David A. (MSFC-EO60)[HOSC= SERVICES CONTRACT]

Subject: Re: [dtn-users] A problem with dtntunnel

<= /div>

=A0

Hi David,

I have inves= tigated that issue a little. I have found, that no matter whether netcat is= run with or without the -w1 option, something
strange happens. According to netstat -ntp on the sending node, dtntunnel r= emains in CLOSE_WAIT state after nc finishes sending
data. I have run t= cpdump on both sender and receiver, and it showed that final FIN from the r= eceiver doesn't reach the sender.
This looked like a bug in dtntunnel to me. I decided to re-check with iperf= , and again, tcpdump showed the same problem.

Here are the last line= s of the tcpdump. Remember that in my setup all outgoing TCP packets with d= estination port 9999 are redirected
with the help of IPTables to the local dtntunnel process, listening om port= 19999. That is why I run tcpdump on loopback interface.

Receiver (1= 92.168.4.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.496366= IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [P.], seq 123995:131097,= ack 1, win 257, options [nop,nop,TS val 11163723 ecr 11163718], length 710= 2
15:05:02.496428 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack = 131097, win 1154, options [nop,nop,TS val 11163724 ecr 11163723], length 0<= br>15:05:03.457421 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [F.], = seq 131097, ack 1, win 257, options [nop,nop,TS val 11164684 ecr 11163724],= length 0
15:05:03.497482 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [.], ack = 131098, win 1154, options [nop,nop,TS val 11164725 ecr 11164684], length 0<= br>15:05:04.457762 IP 192.168.4.1.9999 > 192.168.4.1.34284: Flags [F.], = seq 1, ack 131098, win 1154, options [nop,nop,TS val 11165685 ecr 11164684]= , length 0
15:05:04.457856 IP 192.168.4.1.34284 > 192.168.4.1.9999: Flags [.], ack = 2, win 257, options [nop,nop,TS val 11165685 ecr 11165685], length 0
Sender (192.168.2.1): tcpdump -nn -npi lo port 9999 or port 19999
15:05:02.433755 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags [.], ack = 127449, win 386, options [nop,nop,TS val 11172887 ecr 11172875], length 015:05:02.433776 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [.], s= eq 127448:130344, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 11172= 887], length 2896
15:05:02.433783 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [P.], se= q 130344:131096, ack 1, win 46, options [nop,nop,TS val 11172887 ecr 111728= 87], length 752
15:05:02.444839 IP 192.168.4.1.9999 > 192.168.1.2.433= 60: Flags [.], ack 131097, win 386, options [nop,nop,TS val 11172898 ecr 11= 172887], length 0
15:05:03.421915 IP 192.168.1.2.43360 > 192.168.1.2.19999: Flags [F.], se= q 131096, ack 1, win 46, options [nop,nop,TS val 11173875 ecr 11172898], le= ngth 0
15:05:03.461742 IP 192.168.4.1.9999 > 192.168.1.2.43360: Flags= [.], ack 131098, win 386, options [nop,nop,TS val 11173915 ecr 11173875], = length 0

I believe that the cause of why the sending netcat without -w1 option d= oesn't exit, is that it doesn't receive the final FIN from the dtnt= unnel.
Can this issue be fixed with a small amount of work?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SER= VICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

I would not even call this a problem. = netcat is designed to be extremely flexible and by default the sender keeps= its connection open as long as the other end does. On the other hand the r= eceiver by default terminates after receipt of an EOF but there is a switch= to keep listening if that is the desired behavior.

=A0<= /p>

With DTN, your sending= nc may do its thing while the destination node is not even on line and the= n an hour later it becomes available and completes the transmission to the = receiving nc. In this scenario, the sender would still be =93hung up=94 pos= sibly indefinitely waiting for a terminating signal. <= /p>

=A0<= /p>

I am not the DTN2 auth= Hi ority, but, I don=92t see a change to dtntunnel for this as it would ass= ume a specific usage that would probably break someone else=92s usage (like= mine J).

=A0=

Best regards,

DZ

=A0

David Zoller<= u>

COLSA Corporation=

Marshall Space Flight= Center

=A0

From: ssireskin@gmail.com [= mailto:ssireskin@g= mail.com]
Sent: Wednesday, December 05, 2012 2:56 AM
To: Zoller, Dav= id A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: dtn-users@irtf.org
Subject:<= /b> Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,

Thanks for your help, nc = -w1 did the trick. But shouldn't dtntunnel behavior be changed? The rec= eiving dtntunnel could signal the sending dtntunnel, that the listening nc = has disconnected, so that the sending dtntunnel close its connection to the= sending nc. Or is this solely the problem of nc?

2012/12/5 Zoller, David A. (MSFC-EO60)[HOSC SER= VICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,

On RHEL 5.7, I am running dtn-2.9.0 pl= us modifications that should not impact the behavior of dtntunnel.

I just ran your test with= out the iptables redirect and I see the same behavior.=

I believe that the sender= exits if you go directly from nc to nc because the receiver exits when it = gets an end of file.

You can add a =93-w 1=94 = option to the sender nc so that it will timeout and exit after stdin is idl= e for 1 second.

Best regards,

DZ=

=A0<= /p>

David Zoller

COLSA Corporation=

HOSC / C107= =A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: dtn-users-bounce= s@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Tuesday, December 04, 2012 11:13 AM
To: dtn-users@irtf.org
Su= bject: [dtn-users] A problem with dtntunnel

=A0

Hel= lo all!

I am using dtn-2.9.0 on a RHEL 6 based Linux distro and I am= having a problem when using netcat (nc) with dtntunnel.
On the sender n= ode I run "cat /etc/passwd | nc receiver_ip". On the receiver nod= e I run "nc -l 9999". With the help
of Iptables port 9999 gets redirected to the port 19999, which is listened = by dtntunnel.

The file /etc/passwd is successfully delivered to the = receiver and is shown on the screen. After this the receiving nc exists. No problem here. However, nc on the sender node doesn't exit after it s= ends the file. It continues to run forever. When I
run nc in the opposi= te direction, I get the same problem - the sending nc doesn't exit.

My configuration on both nodes is symmetric:
iptables -t nat -A OUTP= UT -d $REMOTE_HOST -p tcp --dport 9999 -j DNAT --to $LOCAL_HOST:19999
dt= ntunnel -T $LOCAL_HOST:19999:$REMOTE_HOST:9999 $REMOTE_NODE/dtntunnel/nc -d=
dtntunnel -L --local-eid $LOCAL_NODE/dtntunnel/nc -d

dtnping works o= k in both directions.

Please give me any advice.

With best r= egards,
Sergey Sireskin




--
Kindest Regards

Sergey Sireskin
FGUP CNII EISU=




--
Kindest Regards

Sergey Sireskin
= FGUP CNII EISU




--
B= est regards,
Sergey Sireskin




--
= Best regards,
Sergey Sireskin

<= /div>




--
Best regards,
Sergey Sireskin




--
Best= regards,
Sergey Sireskin


--e89a8f647b3b4775fa04d0bc7532-- From ssireskin@gmail.com Thu Dec 13 09:14:59 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9882C21F8634 for ; Thu, 13 Dec 2012 09:14:59 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 1.912 X-Spam-Level: * X-Spam-Status: No, score=1.912 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FROM_EXCESS_BASE64=1.456, HTML_MESSAGE=0.001, J_CHICKENPOX_52=0.6, J_CHICKENPOX_61=0.6, J_CHICKENPOX_62=0.6, J_CHICKENPOX_63=0.6, J_CHICKENPOX_72=0.6, J_CHICKENPOX_92=0.6, MIME_8BIT_HEADER=0.3, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_ENC_UTF8=0.152, WEIRD_PORT=0.001] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Za+jOddB-FCl for ; Thu, 13 Dec 2012 09:14:48 -0800 (PST) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by ietfa.amsl.com (Postfix) with ESMTP id 9363D21F860A for ; Thu, 13 Dec 2012 09:14:47 -0800 (PST) Received: by mail-lb0-f182.google.com with SMTP id go10so1998438lbb.13 for ; Thu, 13 Dec 2012 09:14:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:to:from:cc:subject:date:mime-version:content-type; bh=7JdLN08It21f1WjYrZrG8WFf3QuRZzZ70Xp04zacPts=; b=uwBxw7CjLWPEJhpN6972Sk9SisNlPx9kIwDrocVOc3UBaBis6z54xwnyAs+1cyHC/R aKZjlR2GfD4DRns/Uuz4g3aQeg7+ln53Ei0BSFM9nUis6Kep5PVeNbHhuHFWnmjRMOUp ybmWtZZdtq2Tjf/z+UZu5pEALZmlEgW1PrjqcJpXUqcEX5/Kx/BjYZVWlSHs8xYTiRoZ Btro8Hm0oQBSPZ4vqHEidsUyQkQRoINdBPTS1NoZKLicE+qSn7y4+0lu0QA7kw74DPkg DtIqutGHKjga2oCHhRYlc3nNtcyU/Z+X4+7Ko/CxGtGoSZfrHTkS/7NE0UnOTAM3DyH4 1PVw== Received: by 10.112.11.105 with SMTP id p9mr1132229lbb.129.1355418886192; Thu, 13 Dec 2012 09:14:46 -0800 (PST) Received: from [192.168.55.18] ([176.195.103.92]) by mx.google.com with ESMTPS id k7sm983818lbf.4.2012.12.13.09.14.43 (version=SSLv3 cipher=OTHER); Thu, 13 Dec 2012 09:14:44 -0800 (PST) Message-ID: <50ca0d04.4717700a.7136.2ee8@mx.google.com> To: "=?utf-8?B?RGF2aWQgQS4gKE1TRkMtRU82MClbSE9TQyBTRVJWSUNFUyBDT05UUkFDVF0gWm9sbGVy?=" , "=?utf-8?B?0KHQtdGA0LPQtdC5INCh0YvRgNC10YHQutC40L0=?=" From: "=?utf-8?B?c3NpcmVza2luQGdtYWlsLmNvbQ==?=" Date: Thu, 13 Dec 2012 21:14:49 +0400 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_0_1355418889033" Cc: =?utf-8?B?ZHRuLXVzZXJzQGlydGYub3Jn?= Subject: [dtn-users] =?utf-8?q?=D0=9D=D0=90=3A__A_problem_with_dtntunnel?= X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 17:14:59 -0000 ------=_Part_0_1355418889033 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 Content-Disposition: inline SGkgRGF2aWQsCgpJIHRoaW5rLCB0aGF0IGl0IHNob3VsZCBiZSBwb3NzaWJsZSB0byBzcGVjaWZ5 IHRoZSBiZWhhdmlvciBvZiBkdG50dW5uZWwgaW4gY2FzZSBpZiBib3RoIGVuZHMgYXJlIG5vdCB1 cCBpbiAibmVhciByZWFsIHRpbWUiIHdpdGggYSBjb21tYW5kIGxpbmUgb3B0aW9uLiBUaGVyZSBz aG91bGQgYmUgMiBvcHRpb25zOgoxKSB3YWl0IGZvciB0aGUgcmVjZWl2ZXIgdG8gY29tZSBvbmxp bmU7CjIpIHByZXRlbmQgdG8gYmUgYSByZWNlaXZlciwgZ2V0IGFsbCBkYXRhLCBzZW5kIGZpbiB0 byB0aGUgc2VuZGVyLiBXaGVuIHRoZSByZWFsIHJlY2VpdmVyIGNvbWVzIG9uIGxpbmUsIHNlbmQg ZGF0YS4KVGhlIHNlY29uZCBvcHRpb24gd291bGQgYmUgdXNlZnVsbCBmb3IgdGhlIGFwcHMgdGhh dCBvbmx5IHNlbmQgZGF0YSBvbmUgd2F5LCBub3QgcmVxdWVzdC1yZXBseS4KCgpCZXN0IHJlZ2Fy ZHMsClNlcmdleQoKLS0tLS0gUmVwbHkgbWVzc2FnZSAtLS0tLQrQntGCOiAiWm9sbGVyLCBEYXZp ZCBBLiAoTVNGQy1FTzYwKVtIT1NDIFNFUlZJQ0VTIENPTlRSQUNUXSIgPGRhdmlkLmEuem9sbGVy QG5hc2EuZ292PgrQmtC+0LzRgzogInNzaXJlc2tpbkBnbWFpbC5jb20iIDxzc2lyZXNraW5AZ21h aWwuY29tPgrQmtC+0L/QuNGPOiAiZHRuLXVzZXJzQGlydGYub3JnIiA8ZHRuLXVzZXJzQGlydGYu b3JnPgrQotC10LzQsDogW2R0bi11c2Vyc10gQSBwcm9ibGVtIHdpdGggZHRudHVubmVsCtCU0LDR gtCwOiDRh9GCLCDQtNC10LogMTMsIDIwMTIgMTc6NDYKSGkgU2VyZ2V5LEkgc2VlIG5vdyB0aGUg ZW5kIHRvIGVuZCBlZmZlY3QgeW91IGFyZSB3YW50aW5nIGFuZCBJ4oCZbGwgZXhwZXJpbWVudCBz b21lIG1vcmUgYW5kIGdldCBiYWNrIHdpdGggeW91IGJ1dCBpdCBtYXkgYmUgYSBmZXcgZGF5cy4g SSBndWVzcyBvbmUgcXVlc3Rpb24gaXM6IElmIGJvdGggZW5kcyBhcmUgbm90IHVwIGluIOKAnG5l YXIgcmVhbCB0aW1l4oCdIHdvdWxkIHlvdSB3YW50IHRoZSBzZW5kZXIgdG8gc3RheSBjb25uZWN0 ZWQgdW50aWwgdGhlIHJlY2VpdmluZyBlbmQgY29tZXMgb24gbGluZSBhbmQgY2FuIHJldHVybiB0 aGUgRklOIGluIGFsbCBjaXJjdW1zdGFuY2VzP0RaIERhdmlkIFpvbGxlckNPTFNBIENvcnBvcmF0 aW9uSE9TQyAvIEMxMDcgIChPZmZpY2U6ICgyNTYpIDU0NC0xODIwCipFTWFpbDogZGF2aWQuYS56 b2xsZXJAbmFzYS5nb3YgRnJvbTogc3NpcmVza2luQGdtYWlsLmNvbSBbbWFpbHRvOnNzaXJlc2tp bkBnbWFpbC5jb21dIApTZW50OiBUaHVyc2RheSwgRGVjZW1iZXIgMTMsIDIwMTIgMzoyMyBBTQpU bzogWm9sbGVyLCBEYXZpZCBBLiAoTVNGQy1FTzYwKVtIT1NDIFNFUlZJQ0VTIENPTlRSQUNUXQpD YzogZHRuLXVzZXJzQGlydGYub3JnClN1YmplY3Q6IFJlOiBbZHRuLXVzZXJzXSBBIHByb2JsZW0g d2l0aCBkdG50dW5uZWwgSGkgRGF2aWQsCgpJJ2xsIHN0YXJ0IGZyb20gdGhlIGVuZC4gSSBoYXZl IHJlcGVhdGVkIHRoZSB0ZXN0IHdpdGggMTAwIHBhcmFsbGVsIGZsb3dzIHR3aWNlLCBhbmQgaXQg cGFzc2VkIGZsYXdsZXNzbHkuIEkgdGhpbmsgdGhhdCB3YXMgbXkgbWlzdGFrZSwgSSBjb3VsZCBo YXZlIGNvdW50ZWQgCnRoZSBudW1iZXIgb2YgZXJhc2VkIGNvbm5lY3Rpb25zIHRvbyBlYXJseSwg YmVmb3JlIGFsbCBvZiAxMDAgY29ubmVjdGlvbnMgaGF2ZSB0ZXJtaW5hdGVkLgoKTm93IHRvIHRo ZSBmaXJzdCBwYXJ0LiBJIGhhdmUgcmVwbGFjZWQgd2hpbGUoMSkgd2l0aCB3aGlsZSghc29ja19l b2YpIGFzIHlvdSBzdWdnZXN0ZWQuIFRoaXMgbGVhZHMgdG8gdmVyeSBzdHJhbmdlIHJlc3VsdHMg d2hlbiBzZW5kaW5nIGEgc21hbGwgZmlsZSB3aXRoIG5jLiBOb3RlIHRoZSBsYXN0IHR3byBTWU4g YW5kIFJTVCBwYWNrZXRzIG9uIHRoZSByZWNlaXZlciBzaWRlLgoKU2VuZGVyOiB0Y3BkdW1wIC1u biAtbnBpIGxvIHBvcnQgOTk5OSBvciBwb3J0IDE5OTk5CjEzOjEwOjM1LjEzNjI4MyBJUCAxOTIu MTY4LjEuMi4zMjg4MCA+IDE5Mi4xNjguMS4yLjE5OTk5OiBGbGFncyBbU10sIHNlcSAyOTI1MDc2 MzY2LCB3aW4gNTg0MCwgb3B0aW9ucyBbbXNzIDE0NjAsc2Fja09LLFRTIHZhbCA5MDcwNTU4OSBl Y3IgMCxub3Asd3NjYWxlIDddLCBsZW5ndGggMAoxMzoxMDozNS4xMzYzNDYgSVAgMTkyLjE2OC40 LjEuOTk5OSA+IDE5Mi4xNjguMS4yLjMyODgwOiBGbGFncyBbUy5dLCBzZXEgMjkzMTE5NjM0NCwg YWNrIDI5MjUwNzYzNjcsIHdpbiAzMjc2OCwgb3B0aW9ucyBbbXNzIDE2Mzk2LHNhY2tPSyxUUyB2 YWwgOTA3MDU1ODkgZWNyIDkwNzA1NTg5LG5vcCx3c2NhbGUgN10sIGxlbmd0aCAwCjEzOjEwOjM1 LjEzNjM3MSBJUCAxOTIuMTY4LjEuMi4zMjg4MCA+IDE5Mi4xNjguMS4yLjE5OTk5OiBGbGFncyBb Ll0sIGFjayAyOTMxMTk2MzQ1LCB3aW4gNDYsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDkwNzA1 NTg5IGVjciA5MDcwNTU4OV0sIGxlbmd0aCAwCjEzOjEwOjM1LjEzNjc5OSBJUCAxOTIuMTY4LjEu Mi4zMjg4MCA+IDE5Mi4xNjguMS4yLjE5OTk5OiBGbGFncyBbUC5dLCBzZXEgMDoxLCBhY2sgMSwg d2luIDQ2LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCA5MDcwNTU5MCBlY3IgOTA3MDU1ODldLCBs ZW5ndGggMQoxMzoxMDozNS4xMzY4MTkgSVAgMTkyLjE2OC40LjEuOTk5OSA+IDE5Mi4xNjguMS4y LjMyODgwOiBGbGFncyBbLl0sIGFjayAyLCB3aW4gMjU2LCBvcHRpb25zIFtub3Asbm9wLFRTIHZh bCA5MDcwNTU5MCBlY3IgOTA3MDU1OTBdLCBsZW5ndGggMAoxMzoxMDozNS4xMzY4OTIgSVAgMTky LjE2OC4xLjIuMzI4ODAgPiAxOTIuMTY4LjEuMi4xOTk5OTogRmxhZ3MgW0YuXSwgc2VxIDEsIGFj ayAxLCB3aW4gNDYsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDkwNzA1NTkwIGVjciA5MDcwNTU5 MF0sIGxlbmd0aCAwCjEzOjEwOjM1LjE3Njc1NSBJUCAxOTIuMTY4LjQuMS45OTk5ID4gMTkyLjE2 OC4xLjIuMzI4ODA6IEZsYWdzIFsuXSwgYWNrIDMsIHdpbiAyNTYsIG9wdGlvbnMgW25vcCxub3As VFMgdmFsIDkwNzA1NjMwIGVjciA5MDcwNTU5MF0sIGxlbmd0aCAwCjEzOjEwOjM1LjE3ODYwMiBJ UCAxOTIuMTY4LjQuMS45OTk5ID4gMTkyLjE2OC4xLjIuMzI4ODA6IEZsYWdzIFtGLl0sIHNlcSAx LCBhY2sgMywgd2luIDI1Niwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgOTA3MDU2MzEgZWNyIDkw NzA1NTkwXSwgbGVuZ3RoIDAKMTM6MTA6MzUuMTc4NjIzIElQIDE5Mi4xNjguMS4yLjMyODgwID4g MTkyLjE2OC4xLjIuMTk5OTk6IEZsYWdzIFsuXSwgYWNrIDIsIHdpbiA0Niwgb3B0aW9ucyBbbm9w LG5vcCxUUyB2YWwgOTA3MDU2MzEgZWNyIDkwNzA1NjMxXSwgbGVuZ3RoIDAKMTM6MTA6MzUuMjUz ODI5IElQIDE5Mi4xNjguMS4yLjMyODgxID4gMTkyLjE2OC4xLjIuMTk5OTk6IEZsYWdzIFtTXSwg c2VxIDI5MzQ5MTQ1MzMsIHdpbiA1ODQwLCBvcHRpb25zIFttc3MgMTQ2MCxzYWNrT0ssVFMgdmFs IDkwNzA1NzA3IGVjciAwLG5vcCx3c2NhbGUgN10sIGxlbmd0aCAwCjEzOjEwOjM1LjI1Mzg2MCBJ UCAxOTIuMTY4LjQuMS45OTk5ID4gMTkyLjE2OC4xLjIuMzI4ODE6IEZsYWdzIFtTLl0sIHNlcSAy OTMyMjQ5MTI1LCBhY2sgMjkzNDkxNDUzNCwgd2luIDMyNzY4LCBvcHRpb25zIFttc3MgMTYzOTYs c2Fja09LLFRTIHZhbCA5MDcwNTcwNyBlY3IgOTA3MDU3MDcsbm9wLHdzY2FsZSA3XSwgbGVuZ3Ro IDAKMTM6MTA6MzUuMjUzODc4IElQIDE5Mi4xNjguMS4yLjMyODgxID4gMTkyLjE2OC4xLjIuMTk5 OTk6IEZsYWdzIFsuXSwgYWNrIDI5MzIyNDkxMjYsIHdpbiA0Niwgb3B0aW9ucyBbbm9wLG5vcCxU UyB2YWwgOTA3MDU3MDcgZWNyIDkwNzA1NzA3XSwgbGVuZ3RoIDAKMTM6MTA6MzYuMzgwOTQyIElQ IDE5Mi4xNjguNC4xLjk5OTkgPiAxOTIuMTY4LjEuMi4zMjg4MTogRmxhZ3MgW0YuXSwgc2VxIDEs IGFjayAxLCB3aW4gMjU2LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCA5MDcwNjgzNCBlY3IgOTA3 MDU3MDddLCBsZW5ndGggMAoxMzoxMDozNi4zODE0OTUgSVAgMTkyLjE2OC4xLjIuMzI4ODEgPiAx OTIuMTY4LjEuMi4xOTk5OTogRmxhZ3MgWy5dLCBhY2sgMiwgd2luIDQ2LCBvcHRpb25zIFtub3As bm9wLFRTIHZhbCA5MDcwNjgzNSBlY3IgOTA3MDY4MzRdLCBsZW5ndGggMAoxMzoxMDozNi4zOTgw NzAgSVAgMTkyLjE2OC4xLjIuMzI4ODEgPiAxOTIuMTY4LjEuMi4xOTk5OTogRmxhZ3MgW0YuXSwg c2VxIDAsIGFjayAyLCB3aW4gNDYsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDkwNzA2ODUxIGVj ciA5MDcwNjgzNF0sIGxlbmd0aCAwCjEzOjEwOjM2LjM5ODEzMSBJUCAxOTIuMTY4LjQuMS45OTk5 ID4gMTkyLjE2OC4xLjIuMzI4ODE6IEZsYWdzIFsuXSwgYWNrIDIsIHdpbiAyNTYsIG9wdGlvbnMg W25vcCxub3AsVFMgdmFsIDkwNzA2ODUxIGVjciA5MDcwNjg1MV0sIGxlbmd0aCAwCgpSZWNlaXZl cjogdGNwZHVtcCAtbm4gLW5waSBsbyBwb3J0IDk5OTkgb3IgcG9ydCAxOTk5OQoxMzoxMDozNS4x ODUyMTEgSVAgMTkyLjE2OC40LjEuNDQ1NTAgPiAxOTIuMTY4LjQuMS45OTk5OiBGbGFncyBbU10s IHNlcSAyOTI3MTIzOTQ1LCB3aW4gMzI3OTIsIG9wdGlvbnMgW21zcyAxNjM5NixzYWNrT0ssVFMg dmFsIDkwNjk2NDEyIGVjciAwLG5vcCx3c2NhbGUgN10sIGxlbmd0aCAwCjEzOjEwOjM1LjE4NTI2 NiBJUCAxOTIuMTY4LjQuMS45OTk5ID4gMTkyLjE2OC40LjEuNDQ1NTA6IEZsYWdzIFtTLl0sIHNl cSAyOTI5NjIxNjAyLCBhY2sgMjkyNzEyMzk0Niwgd2luIDMyNzY4LCBvcHRpb25zIFttc3MgMTYz OTYsc2Fja09LLFRTIHZhbCA5MDY5NjQxMiBlY3IgOTA2OTY0MTIsbm9wLHdzY2FsZSA3XSwgbGVu Z3RoIDAKMTM6MTA6MzUuMTg1Mjg2IElQIDE5Mi4xNjguNC4xLjQ0NTUwID4gMTkyLjE2OC40LjEu OTk5OTogRmxhZ3MgWy5dLCBhY2sgMSwgd2luIDI1Nywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwg OTA2OTY0MTIgZWNyIDkwNjk2NDEyXSwgbGVuZ3RoIDAKMTM6MTA6MzUuMjA2ODE4IElQIDE5Mi4x NjguNC4xLjQ0NTUwID4gMTkyLjE2OC40LjEuOTk5OTogRmxhZ3MgW1AuXSwgc2VxIDE6MiwgYWNr IDEsIHdpbiAyNTcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDkwNjk2NDM0IGVjciA5MDY5NjQx Ml0sIGxlbmd0aCAxCjEzOjEwOjM1LjIwNjg2MSBJUCAxOTIuMTY4LjQuMS45OTk5ID4gMTkyLjE2 OC40LjEuNDQ1NTA6IEZsYWdzIFsuXSwgYWNrIDIsIHdpbiAyNTYsIG9wdGlvbnMgW25vcCxub3As VFMgdmFsIDkwNjk2NDM0IGVjciA5MDY5NjQzNF0sIGxlbmd0aCAwCjEzOjEwOjM1LjIyMjAzMiBJ UCAxOTIuMTY4LjQuMS40NDU1MCA+IDE5Mi4xNjguNC4xLjk5OTk6IEZsYWdzIFtGLl0sIHNlcSAy LCBhY2sgMSwgd2luIDI1Nywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgOTA2OTY0NDkgZWNyIDkw Njk2NDM0XSwgbGVuZ3RoIDAKMTM6MTA6MzUuMjIyMTMzIElQIDE5Mi4xNjguNC4xLjk5OTkgPiAx OTIuMTY4LjQuMS40NDU1MDogRmxhZ3MgW0YuXSwgc2VxIDEsIGFjayAzLCB3aW4gMjU2LCBvcHRp b25zIFtub3Asbm9wLFRTIHZhbCA5MDY5NjQ0OSBlY3IgOTA2OTY0NDldLCBsZW5ndGggMAoxMzox MDozNS4yMjIxNDYgSVAgMTkyLjE2OC40LjEuNDQ1NTAgPiAxOTIuMTY4LjQuMS45OTk5OiBGbGFn cyBbLl0sIGFjayAyLCB3aW4gMjU3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCA5MDY5NjQ0OSBl Y3IgOTA2OTY0NDldLCBsZW5ndGggMAoxMzoxMDozNi4zMjUzOTYgSVAgMTkyLjE2OC40LjEuNDQ1 NTEgPiAxOTIuMTY4LjQuMS45OTk5OiBGbGFncyBbU10sIHNlcSAyOTQ1NzE2ODg5LCB3aW4gMzI3 OTIsIG9wdGlvbnMgW21zcyAxNjM5NixzYWNrT0ssVFMgdmFsIDkwNjk3NTUyIGVjciAwLG5vcCx3 c2NhbGUgN10sIGxlbmd0aCAwCjEzOjEwOjM2LjMyNTQ0NyBJUCAxOTIuMTY4LjQuMS45OTk5ID4g MTkyLjE2OC40LjEuNDQ1NTE6IEZsYWdzIFtSLl0sIHNlcSAwLCBhY2sgMjk0NTcxNjg5MCwgd2lu IDAsIGxlbmd0aCAwCgpBbm90aGVyIHN0cmFuZ2UgdGhpbmcgaGFwcGVuZCB3aGVuIEkgaGF2ZSBy dW4gc2VuZGluZyBuYyBhbmQgZm9yZ290IHRvIHJ1biB0aGUgcmVjZWl2aW5nIG5jLiBKdXN0IHRy eSBpdCB5b3Vyc2VsZiBhbmQgc2VlIHRjcGR1bXAuCgpJIGhhdmUgYWxzbyBjb25kdWN0ZWQgYW5v dGhlciB0ZXN0IHdpdGggdGhlIG9yaWdpbmFsIGR0bnR1bm5lbCBjb2RlLiBJIGhhdmUgcnVuIG5j IGluIHRoZSBpbnRlcmFjdGl2ZSBtb2RlLCBpLmUuIG5jIDxpcD4gPHBvcnQ+IG9uIHRoZSBjbGll bnQgc2lkZSBhbmQgbmMgLWwgPHBvcnQ+IG9uIHRoZSBzZXJ2ZXIgc2lkZS4gVGhlbiBJIHRlcm1p bmF0ZWQgdGhlIGNvbm5lY3Rpb24gb24gdGhlIGNsaWVudCBzaWRlIGJ5IHByZXNzaW5nIENUUkwr RC4gVGhlIHNlcnZlciByZWNlaXZlZCBhIEZJTiBwYWNrZWQgZnJvbSB0aGUgY2xpZW50LCByZXBs aWVkIHdpdGggYW5vdGhlciBGSU4sIGFuZCB0ZXJtaW5hdGVkLiBUaGUgY2xpZW50IGRpZCBub3Qg cmVjZWl2ZSB0aGUgRklOIHBhY2tldCBmcm9tIHRoZSBzZXJ2ZXIgYW5kIGRpZCBub3QgdGVybWlu YXRlLiBBZnRlciB0aGF0IEkgaGF2ZSBlc3RhYmxpc2hlZCBhIG5ldyBjb25uZWN0aW9uIGFuZCB0 ZXJtaW5hdGVkIGl0IGJ5IHByZXNzaW5nIENUUkwrRCBvbiB0aGUgc2VydmVyIHNpZGUuIFRoaXMg dGltZSBzZXJ2ZXIgZGlkIHNlbmQgYSBGSU4gcGFja2V0LCB0aGUgY2xpZW50IGdvdCBpdCwgcmVw bGllZCB3aXRoIGl0cyBvd24gRklOIGFuZCB0ZXJtaW5hdGVkLiBUaGUgc2VydmVyIGRpZCBub3Qg cmVjZWl2ZSB0aGUgRklOIGZyb20gdGhlIGNsaWVudCBhbmQgY29udGludWVkIHJ1bm5pbmcuCgpU aGlzIG1ha2VzIG1lIHRoaW5rIHRoYXQgZHRudHVubmVsLCB0aGF0IGhhdmUgcmVjZWl2ZWQgdGhl IGZpbmFsIEZJTiBmcm9tIGEgVENQIGFwcGxpY2F0aW9uLCBkb2Vzbid0IHRyYW5zbWl0IHRoaXMg RklOIHRvIGl0cyByZW1vdGUgZHRudHVubmVsIHBlZXIuIE9yIHRoZSBsYXR0ZXIgb25lIGRvZXNu J3QgcmV0cmFuc21pdCB0aGlzIGZpbmFsIEZJTiB0byB0aGUgVENQIGFwcGxpY2F0aW9uIG9uIGl0 cyBzaWRlLiBJJ20gbm90IGEgVENQIGFuZCBzb2NrZXRzIGd1cnUsIHNvIEkgY2Fubm90IGZpZ3Vy ZSBvdXQgaG93IHRvIHRyYW5zbGF0ZSB0aGlzIHRvIHRoZSBsYW5ndWFnZSBvZiBzb2NrZXRzLCB3 aGljaCBJIHRoaW5rIGlzIGltcG9ydGFudCB0byBkbyBpbiBvcmRlciB0byB1bmRlcnN0YW5kIGhv dyB0byBmaXggZHRudHVubmVsLgoKSSBkb24ndCB0aGluayB0aGF0IGltcGxlbWVudGluZyBhbiBv cHRpb24gdGhhdCBjb25maWd1cmVzIGR0bnR1bm5lbCBiZWhhdmlvciBpbiByZWxhdGlvbiB0byBj bG9zaW5nIGNvbm5lY3Rpb25zIGlzIGEgZ29vZCBpZGVhLiBJIGJlbGlldmUgdGhhdCB0aGVyZSBp cyBhIGZsYXcgaW4gZHRudHVubmVsIGxvZ2ljLCBhbmQgaXQgY291bGQgYmUgZml4ZWQgaW4gYSB3 YXkgdGhhdCBkdG50dW5uZWwgZGV0ZXJtaW5lcyBpdHNlbGYgd2hlbiB0byBjbG9zZSBhIGNvbm5l Y3Rpb24gdG8gYSBUQ1AgYXBwbGljYXRpb24uMjAxMi8xMi8xMyBab2xsZXIsIERhdmlkIEEuIChN U0ZDLUVPNjApW0hPU0MgU0VSVklDRVMgQ09OVFJBQ1RdIDxkYXZpZC5hLnpvbGxlckBuYXNhLmdv dj5IaSBTZXJnZXksUmVzZWFyY2ggaW5kaWNhdGVzIHRoYXQgdGhlIFRJTUVfV0FJVCBzdGF0ZSBp cyBhIGZlYXR1cmUgb2YgVENQIHRvIHByZXZlbnQgZGVsYXllZCBwYWNrZXRzIGZvciBjb25uZWN0 aW9uIGZyb20gYmVpbmcgZGVsaXZlcmVkIHRvIGEgbGF0ZXIgY29ubmVjdGlvbiBiZXR3ZWVuIHRo ZSBzYW1lIGhvc3RzLiBbaHR0cDovL3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtZmFiZXItdGlt ZS13YWl0LWF2b2lkYW5jZS0wMF0gQmFzZWQgb24gdGhhdCwgdGhlIGFkZGVkIOKAnGlmIChzb2Nr X2VvZinigJ0gYmxvY2sgY2FuIGJlIHJlbW92ZWQgYW5kIHRoZSB3aGlsZSBjb25kaXRpb24gY2Fu IGJlIGNoYW5nZWQgdG8g4oCcd2hpbGUgKCFzb2NrX2VvZinigJ0gYW5kIHlvdSBnZXQgdGhlIHNh bWUgcmVzdWx0LiBCdXQsIGxvb2tpbmcgYXQgdGhlIGludGVudCBvZiB0aGUgY29kZSBhbmQgYXQg dGhlIG9hc3lzOjpJUFNvY2tldCwgaXQgYXBwZWFycyB0aGF0IGl0IGlzIHBvc3NpYmxlIHRvIGVz dGFibGlzaCBhIHNvY2tldCBjb25uZWN0aW9uIGFuZCB0aGVuIG9ubHkgY2xvc2UgdGhlIHJlYWQg c2lkZSB3aGlsZSBrZWVwaW5nIHRoZSB3cml0ZSBzaWRlIG9wZW4uIEnigJl2ZSBuZXZlciBydW4g YWNyb3NzIGEgc29ja2V0IHVzZWQgaW4gdGhhdCBtYW5uZXIgYnV0IHRoYXQgaXMgd2hhdCBpdCBs b29rcyBsaWtlIHRvIG1lLiBTbywgdGhlIFRDUFR1bm5lbDo6Q29ubmVjdGlvbjo6cnVuICB3aGls ZSBsb29wIGFsbG93cyBmb3IgdGhlIHJlYWQgc2lkZSB0byBjbG9zZSBhbmQgY29udGludWUgdHJh bnNtaXR0aW5nIGJ1bmRsZSBwYXlsb2FkcyBvbiB0aGUgd3JpdGUgc2lkZSB1bnRpbCBpdCBnZXRz IGEgYnVuZGxlIHdpdGggdGhlIEVPRiBiaXQgc2V0LiBJZiB0aGVyZSBhcmUgbm8gYnVuZGxlcyBj b21pbmcgYmFjayB0aGUgb3RoZXIgd2F5IGFzIGlzIHRoZSBjYXNlIGhlcmUgdGhlbiB0aGUgd2hp bGUgbG9vcCBuZXZlciBleGl0cyB3aGljaCBrZWVwcyB0aGUgbmV0Y2F0IGNvbm5lY3Rpb24gb3Bl bi4gU28sIGJhc2VkIG9uIHRoYXQsIEkgdGhpbmsgYSBiZXR0ZXIgc29sdXRpb24gd291bGQgYmUg dG8gaW1wbGVtZW50IGFuIG9wdGlvbiB0aGF0IGNvbmZpZ3VyZXMgZHRudHVubmVsIHRvIGNsb3Nl IHRoZSBjb25uZWN0aW9uIHdoZW4gdGhlIHJlYWQgc2lkZSB0ZXJtaW5hdGVzLiAoU29tZW9uZSBp biB0aGUga25vdyBjYW4gd2VpZ2ggaW4pIFlvdXIgc29sdXRpb24gaXMgcHJvYmFibHkgc3VmZmlj aWVudCBmb3IgeW91ciB0ZXN0LiAtLSBBcyB0byBjb25uZWN0aW9ucyBub3QgYmVpbmcgcmVtb3Zl ZCBmcm9tIHRoZSB0YWJsZSBhbmQgc29ja2V0cyBub3QgYmVpbmcgY2xvc2Vk4oCmIEkgc2V0IHVw IGEgdGVzdCB0byBraWNrIG9mZiAxMDAgbmV0Y2F0cyBvbiBhbiAxMU1CIGZpbGUgc28gdGhhdCB0 aGUgY29ubmVjdGlvbiB3b3VsZCBiZSBvcGVuIGZvciBzZXZlcmFsIHNlY29uZHM6IyEvYmluL2Jh c2hjdHI9MHdoaWxlIFsgJGN0ciAtbHQgMTAwIF07IGRvICAgIG5jIC13IDEwIHgueC54LnggMTIz NDUgPCBkYXRhMTFtZyAgJiAgICBsZXQgY3RyPWN0cisxZG9uZSBJIGFsc28gYWRkZWQgc29tZSBk ZWJ1ZyBjb2RlIHRvIHByaW50IG91dCBhIGNvbm5lY3Rpb24gYWNjZXB0ZWQgY291bnRlciBhbmQg Y2xvc2VkIGNvdW50ZXIgYXMgdGhleSBhcmUgYWNjZXB0ZWQgYW5kIGNsb3NlZC5BZnRlciAzMCBj b25uZWN0aW9ucywgdGhlIGZpcnN0IGNsb3NlIGtpY2tzIGluIGFuZCB0aGVuIHRoZXJlIGlzIGEg bWl4IG9mIGNvbm5lY3RzIGFuZCBjbG9zZXMgdW50aWwgZmluaXNoZWQuQWxsIG9mIHRoZSBjb25u ZWN0aW9ucyB3ZXJlIGNsb3NlZCBhbmQgSSBkaWQgbm90IGhhdmUgYW55IHNvY2tldHMgbGVmdCBv cGVuLiBPbiBteSBmaXJzdCBhdHRlbXB0IEkgZXhjZWVkZWQgbXkgcGF5bG9hZCBxdW90YSBhbmQg c3RhcnRlZCBzZWVpbmcg4oCcZXJyb3Igc2VuZGluZyBidW5kbGU6IDE0MeKAnSBtZXNzYWdlcyBz byBJIGRvbuKAmXQgdGhpbmsgdGhhdCBpcyB0aGUgaXNzdWUgeW91IGFyZSBydW5uaW5nIGludG8u IFlvdSBtaWdodCB0cnkgcHV0dGluZyBpbiBzaW1pbGFyIGRlYnVnIHRvIHNlZSBpZiB5b3UgY2Fu IGRldGVybWluZSB3aGF0IGlzIGhhcHBlbmluZy4gSG9wZSB0aGlzIGhlbHBzLERaICBEYXZpZCBa b2xsZXJDT0xTQSBDb3Jwb3JhdGlvbkhPU0MgLyBDMTA3ICAoT2ZmaWNlOiAoMjU2KSA1NDQtMTgy MAoqRU1haWw6IGRhdmlkLmEuem9sbGVyQG5hc2EuZ292IEZyb206IHNzaXJlc2tpbkBnbWFpbC5j b20gW21haWx0bzpzc2lyZXNraW5AZ21haWwuY29tXSAKU2VudDogV2VkbmVzZGF5LCBEZWNlbWJl ciAxMiwgMjAxMiAyOjA3IFBNClRvOiBab2xsZXIsIERhdmlkIEEuIChNU0ZDLUVPNjApW0hPU0Mg U0VSVklDRVMgQ09OVFJBQ1RdCkNjOiBkdG4tdXNlcnNAaXJ0Zi5vcmcKU3ViamVjdDogUmU6IFtk dG4tdXNlcnNdIEEgcHJvYmxlbSB3aXRoIGR0bnR1bm5lbCBIaSBEYXZpZCwKCkkgbWVhbnQgZXhh Y3RseSB0aGUgc2FtZSBwb3NpdGlvbiBiZWZvcmUgdGhlIGVuZCBvZiB0aGUgd2hpbGUgbG9vcCwg anVzdCBkZXNjcmliZWQgaXQgaW5jb3JyZWN0LgoKUGxlYXNlIGxldCBtZSBrbm93IGlmIHlvdSBm aW5kIHRoZSBzb2x1dGlvbiBmb3IgdGhpcyBwcm9ibGVtLgoKQmVzdCByZWdhcmRzLApTZXJnZXky MDEyLzEyLzEyIFpvbGxlciwgRGF2aWQgQS4gKE1TRkMtRU82MClbSE9TQyBTRVJWSUNFUyBDT05U UkFDVF0gPGRhdmlkLmEuem9sbGVyQG5hc2EuZ292PkhpIFNlcmdleSxJIHdhcyBuYXJyb3dpbmcg aW4gb24gdGhlIHNhbWUgdHJhY2suLi4gVGhlIEZJTiByZXN1bHRzIGluIGEgcmV0dXJuZWQgbGVu Z3RoIG9mIHplcm8gb24gdGhlIHJlYWQgYW5kIGVycm5vIGlzIGFsc28gemVybyBzbyB5b3UgaGF2 ZSB0byBrZXkgb24gdGhlIHJldHVybiBsZW5ndGggd2hpY2ggaXMgYWxyZWFkeSBpbiB0aGUgY29k ZS4gWW91IG1heSBoYXZlIGFkZGVkIGEgY291cGxlIG9mIGxpbmVzIG9mIGRlYnVnIGNvZGUgYW5k IGl0IHNob3VsZCBiZSBpbnNlcnRlZCBqdXN0IGJlZm9yZSB0aGUgZW5kIG9mIHRoZSB3aGlsZSgx KSBsb29wIGF0IGxpbmUgNDkxIGluIHRoZSBsYXRlc3QgdmVyc2lvbiBvbiBTb3VyY2Vmb3JnZS4g SSBhbSBzZWVpbmcgdGhlIHNlbmRpbmcgc29ja2V0IHN0YXkgaW4gdGhlIFRJTUVfV0FJVCBtb2Rl IGV2ZW4gaWYgSSBhZGQgYSBjYWxsIHRvIHNvY2tfLnNodXRkb3duKFNIVVRfUkRXUikgYmVmb3Jl IGNsb3NpbmcgdGhlIHNvY2tldDtHZXR0aW5nIGNsb3NlIEkgdGhpbmsuIFRoZSAwLjAuMC4wIHdh cyBiZWNhdXNlIEkgZGlkIG5vdCBoYXZlIGFueXRoaW5nIGF0IHRoZSBvdGhlciBlbmQgb2YgdGhl IHR1bm5lbCBhbmQganVzdCBzdHVjayBzb21lIGVhc3kgbnVtYmVycyBpbiB0aGVyZS5UaGFua3Ms CkRaIERhdmlkIFpvbGxlckNPTFNBIENvcnBvcmF0aW9uTWFyc2hhbGwgU3BhY2UgRmxpZ2h0IENl bnRlciBGcm9tOiBzc2lyZXNraW5AZ21haWwuY29tIFttYWlsdG86c3NpcmVza2luQGdtYWlsLmNv bV0gClNlbnQ6IFdlZG5lc2RheSwgRGVjZW1iZXIgMTIsIDIwMTIgMTE6MDYgQU0KVG86IFpvbGxl ciwgRGF2aWQgQS4gKE1TRkMtRU82MClbSE9TQyBTRVJWSUNFUyBDT05UUkFDVF0KQ2M6IGR0bi11 c2Vyc0BpcnRmLm9yZwpTdWJqZWN0OiBSZTogW2R0bi11c2Vyc10gQSBwcm9ibGVtIHdpdGggZHRu dHVubmVsIEhpIERhdmlkLAoKSSBhZGRlZCB0aGUgZm9sbG93aW5nIGNvZGUgYXQgdGhlIGxpbmUg NDkyIG9mIFRDUFR1bm5lbC5jYyBhbmQgaXQgc2VlbXMgdGhhdCBpdCBoZWxwZWQuIAoKaWYgKHNv Y2tfZW9mKSB7CnNvY2tfLmNsb3NlKCk7CmdvdG8gZG9uZTsKfQoKVGhlIHNlbmRlcnMsIGJvdGgg bmMgYW5kIGlwZXJmLCBleGl0IGFmdGVyIGhhdmluZyBmaW5pc2hlZCBzZW5kaW5nIHRoZSBkYXRh LiBUaGVyZSBhcmUKbm8gbW9yZSBzb2NrZXRzIGluIENMT1NFX1dBSVQgc3RhdGUuIEhvd2V2ZXIg SSBhbSBjb21wbGV0ZWx5IHVuc3VyZSB3aGV0aGVyIApteSBjb2RlIGlzIGNvcnJlY3QgYW5kIGlz IGluIGNvcnJlY3QgcGxhY2UuCgpOb3cgSSBoYXZlIGVuY291bnRlcmVkIGFub3RoZXIgcHJvYmxl bS4gV2hlbiBpcGVyZiB0ZXN0IGlzIHJ1biB3aXRoIDEwMCBwYXJhbGxlbCBmbG93cywKc29tZSBk dG50dW5uZWwgY29ubmVjdGlvbnMgYXJlIG5vdCByZW1vdmVkIGZyb20gZHRudHVuZWwncyBjb25u ZWN0aW9uIHRhYmxlLCBhbmQgdGhlCnJlbGF0ZWQgc29ja2V0cyBhcmUgbm90IGNsb3NlZC4gSSBz dXBwb3NlIHRoaXMgaGFwcGVucyBiZWNhdXNlIG15IGNvZGUgYWRkaXRpb24gaXMgZmFyCmZyb20g cGVyZmVjdC4KCkFuZCBvbmUgbW9yZSBxdWVzdGlvbiwgd2hhdCBkb2VzIDAuMC4wLjAgbWVhbiBp biAtVCB0dW5uZWwgc3BlY2lmaWNhdGlvbj8gSXMgaXQganVzdCBhbgpleGFtcGxlIGhvc3QsIG9y IDAuMC4wLjAgaGFzIHNvbWUgc3BlY2lhbCBtZWFuaW5nIGZvciBkdG50dW5uZWw/MjAxMi8xMi8x MiBab2xsZXIsIERhdmlkIEEuIChNU0ZDLUVPNjApW0hPU0MgU0VSVklDRVMgQ09OVFJBQ1RdIDxk YXZpZC5hLnpvbGxlckBuYXNhLmdvdj5IaSBTZXJnZXksSSBoYXZlIGR1cGxpY2F0ZWQgd2hhdCB5 b3UgYXJlIHJlcG9ydGluZyBhbmQgaGF2ZSB0byByZXRyYWN0IG15IGVhcmxpZXIgc3RhdGVtZW50 IHNheWluZyB0aGF0IHRoaXMgaXMgdGhlIG5ldGNhdCBkZXNpZ24gYW5kIG5vdCBhIHByb2JsZW0g SlRoZSBzZW5kZXIgbmV0Y2F0IGF0dGVtcHRzIHRvIGNsb3NlIHRoZSBzb2NrZXQgd2hlbiBpdCBn ZXRzIHRoZSBbRU9GXSBhbmQgbm90IGluIHJlc3BvbnNlIHRvIHRoZSByZWNlaXZlciBjbG9zaW5n IGl0cyBlbmQuICBuZXRjYXQgc2VuZGVyOmNhdCAvZXRjL3Bhc3N3ZCB8IG5jIHgueC54LnggMTIz NDUgc2NlbmFyaW8gMSAtIG5ldGNhdCByZWNlaXZlciB3aXRoIHRoZSBrZWVwIGxpc3RlbmVyIG9w dGlvbjogICAgICAgICAgICAgICAgbmMgLWsgLWwgMTIzNDUmbWlkZG90OyAgICAgICAgIFNlbmRl ciB0ZXJtaW5hdGVzIGFuZCByZWNlaXZlciBzdGF5cyBhbGl2ZSBhbmQgbGlzdGVuaW5nIGZvciBh IHJlLWNvbm5lY3QmbWlkZG90OyAgICAgICAgIEZJTiBpcyBzZW50IGluIGJvdGggZGlyZWN0aW9u cyBpbml0aWF0ZWQgYnkgdGhlIHNlbmRlciBzY2VuYXJpbyAyIC0gZHRudHVubmVsIHJlY2VpdmVy OiAgICAgICAgICAgICAgICBkdG50dW5uZWwgLXQgLVQgMTIzNDU6MC4wLjAuMDo1NDMyMSBkdG46 Ly9kZXN0ZWlkL3h4eHgmbWlkZG90OyAgICAgICAgIFNlbmRlciBkb2VzIG5vdCB0ZXJtaW5hdGUm bWlkZG90OyAgICAgICAgIEZJTiBpcyBzZW50IGZyb20gc2VuZGVyIHRvIHJlY2VpdmVyIGJ1dCBu b3QgdGhlIG90aGVyIHdheSZtaWRkb3Q7ICAgICAgICAgU2VuZGVyIHNvY2tldCBpcyBpbiBzdGF0 ZSBGSU5fV0FJVDImbWlkZG90OyAgICAgICAgIFJlY2VpdmVyIHNvY2tldCBpcyBpbiBzdGF0ZSBD TE9TRV9XQUlUS2lsbCB0aGUgc2VuZGVyIGFuZCByZXBlYXQuLi4mbWlkZG90OyAgICAgICAgIFNl bmRlciBkb2VzIG5vdCB0ZXJtaW5hdGUmbWlkZG90OyAgICAgICAgIEZJTiBpcyBzZW50IGZyb20g c2VuZGVyIHRvIHJlY2VpdmVyIGJ1dCBub3QgdGhlIG90aGVyIHdheSZtaWRkb3Q7ICAgICAgICAg TmV3IFNlbmRlciBzb2NrZXQgaXMgaW4gc3RhdGUgRklOX1dBSVQyIGFuZCB0aGUgZmlyc3Qgb25l IGhhcyB0aW1lZCBvdXQgYW5kIGRpZWQmbWlkZG90OyAgICAgICAgIEJvdGggUmVjZWl2ZXIgc29j a2V0cyBhcmUgaW4gc3RhdGUgQ0xPU0VfV0FJVEtpbGwgdGhlIGR0bnR1bm5lbCByZWNlaXZlcuKA piZtaWRkb3Q7ICAgICAgICAgMiBGSU5zIGFyZSBzZW50IGZyb20gdGhlIHJlY2VpdmVyIOKAkyAx IHRvIGVhY2ggb2YgdGhlIHNvY2tldHMgKGV2ZW4gdGhvdWdoIHRoZSBvdGhlciBlbmRzIGhhdmUg ZXhwaXJlZCkgIEkgYmVsaWV2ZSB0aGUgaXNzdWUgd291bGQgYmUgaW4gdGhlIG9hc3lzIElQU29j a2V0IG9yIHBvc3NpYmx5IHRoZSBJUENsaWVudC4gSeKAmWxsIGhhdmUgYSBsb29rIGF0IGl0IHVu bGVzcyB0aGVyZSBpcyBhIGxvdy1sZXZlbCBzb2NrZXQgZXhwZXJ0IG91dCB0aGVyZSB0aGF0IHdh bnRzIHRvIGdpdmUgaXQgYSBnb+KApiBCZXN0IHJlZ2FyZHMsRFogRGF2aWQgWm9sbGVyQ09MU0Eg Q29ycG9yYXRpb25NYXJzaGFsbCBTcGFjZSBGbGlnaHQgQ2VudGVyIEZyb206IHNzaXJlc2tpbkBn bWFpbC5jb20gW21haWx0bzpzc2lyZXNraW5AZ21haWwuY29tXSAKU2VudDogV2VkbmVzZGF5LCBE ZWNlbWJlciAxMiwgMjAxMiA0OjM4IEFNClRvOiBab2xsZXIsIERhdmlkIEEuIChNU0ZDLUVPNjAp W0hPU0MgU0VSVklDRVMgQ09OVFJBQ1RdU3ViamVjdDogUmU6IFtkdG4tdXNlcnNdIEEgcHJvYmxl bSB3aXRoIGR0bnR1bm5lbCBIaSBEYXZpZCwKCkkgaGF2ZSBpbnZlc3RpZ2F0ZWQgdGhhdCBpc3N1 ZSBhIGxpdHRsZS4gSSBoYXZlIGZvdW5kLCB0aGF0IG5vIG1hdHRlciB3aGV0aGVyIG5ldGNhdCBp cyBydW4gd2l0aCBvciB3aXRob3V0IHRoZSAtdzEgb3B0aW9uLCBzb21ldGhpbmcgCnN0cmFuZ2Ug aGFwcGVucy4gQWNjb3JkaW5nIHRvIG5ldHN0YXQgLW50cCBvbiB0aGUgc2VuZGluZyBub2RlLCBk dG50dW5uZWwgcmVtYWlucyBpbiBDTE9TRV9XQUlUIHN0YXRlIGFmdGVyIG5jIGZpbmlzaGVzIHNl bmRpbmcgCmRhdGEuIEkgaGF2ZSBydW4gdGNwZHVtcCBvbiBib3RoIHNlbmRlciBhbmQgcmVjZWl2 ZXIsIGFuZCBpdCBzaG93ZWQgdGhhdCBmaW5hbCBGSU4gZnJvbSB0aGUgcmVjZWl2ZXIgZG9lc24n dCByZWFjaCB0aGUgc2VuZGVyLiAKVGhpcyBsb29rZWQgbGlrZSBhIGJ1ZyBpbiBkdG50dW5uZWwg dG8gbWUuIEkgZGVjaWRlZCB0byByZS1jaGVjayB3aXRoIGlwZXJmLCBhbmQgYWdhaW4sIHRjcGR1 bXAgc2hvd2VkIHRoZSBzYW1lIHByb2JsZW0uCgpIZXJlIGFyZSB0aGUgbGFzdCBsaW5lcyBvZiB0 aGUgdGNwZHVtcC4gUmVtZW1iZXIgdGhhdCBpbiBteSBzZXR1cCBhbGwgb3V0Z29pbmcgVENQIHBh Y2tldHMgd2l0aCBkZXN0aW5hdGlvbiBwb3J0IDk5OTkgYXJlIHJlZGlyZWN0ZWQgCndpdGggdGhl IGhlbHAgb2YgSVBUYWJsZXMgdG8gdGhlIGxvY2FsIGR0bnR1bm5lbCBwcm9jZXNzLCBsaXN0ZW5p bmcgb20gcG9ydCAxOTk5OS4gVGhhdCBpcyB3aHkgSSBydW4gdGNwZHVtcCBvbiBsb29wYmFjayBp bnRlcmZhY2UuCgpSZWNlaXZlciAoMTkyLjE2OC40LjEpOiB0Y3BkdW1wIC1ubiAtbnBpIGxvIHBv cnQgOTk5OSBvciBwb3J0IDE5OTk5CjE1OjA1OjAyLjQ5NjM2NiBJUCAxOTIuMTY4LjQuMS4zNDI4 NCA+IDE5Mi4xNjguNC4xLjk5OTk6IEZsYWdzIFtQLl0sIHNlcSAxMjM5OTU6MTMxMDk3LCBhY2sg MSwgd2luIDI1Nywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExNjM3MjMgZWNyIDExMTYzNzE4 XSwgbGVuZ3RoIDcxMDIKMTU6MDU6MDIuNDk2NDI4IElQIDE5Mi4xNjguNC4xLjk5OTkgPiAxOTIu MTY4LjQuMS4zNDI4NDogRmxhZ3MgWy5dLCBhY2sgMTMxMDk3LCB3aW4gMTE1NCwgb3B0aW9ucyBb bm9wLG5vcCxUUyB2YWwgMTExNjM3MjQgZWNyIDExMTYzNzIzXSwgbGVuZ3RoIDAKMTU6MDU6MDMu NDU3NDIxIElQIDE5Mi4xNjguNC4xLjM0Mjg0ID4gMTkyLjE2OC40LjEuOTk5OTogRmxhZ3MgW0Yu XSwgc2VxIDEzMTA5NywgYWNrIDEsIHdpbiAyNTcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDEx MTY0Njg0IGVjciAxMTE2MzcyNF0sIGxlbmd0aCAwCjE1OjA1OjAzLjQ5NzQ4MiBJUCAxOTIuMTY4 LjQuMS45OTk5ID4gMTkyLjE2OC40LjEuMzQyODQ6IEZsYWdzIFsuXSwgYWNrIDEzMTA5OCwgd2lu IDExNTQsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTY0NzI1IGVjciAxMTE2NDY4NF0sIGxl bmd0aCAwCjE1OjA1OjA0LjQ1Nzc2MiBJUCAxOTIuMTY4LjQuMS45OTk5ID4gMTkyLjE2OC40LjEu MzQyODQ6IEZsYWdzIFtGLl0sIHNlcSAxLCBhY2sgMTMxMDk4LCB3aW4gMTE1NCwgb3B0aW9ucyBb bm9wLG5vcCxUUyB2YWwgMTExNjU2ODUgZWNyIDExMTY0Njg0XSwgbGVuZ3RoIDAKMTU6MDU6MDQu NDU3ODU2IElQIDE5Mi4xNjguNC4xLjM0Mjg0ID4gMTkyLjE2OC40LjEuOTk5OTogRmxhZ3MgWy5d LCBhY2sgMiwgd2luIDI1Nywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExNjU2ODUgZWNyIDEx MTY1Njg1XSwgbGVuZ3RoIDAKClNlbmRlciAoMTkyLjE2OC4yLjEpOiB0Y3BkdW1wIC1ubiAtbnBp IGxvIHBvcnQgOTk5OSBvciBwb3J0IDE5OTk5IAoxNTowNTowMi40MzM3NTUgSVAgMTkyLjE2OC40 LjEuOTk5OSA+IDE5Mi4xNjguMS4yLjQzMzYwOiBGbGFncyBbLl0sIGFjayAxMjc0NDksIHdpbiAz ODYsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTcyODg3IGVjciAxMTE3Mjg3NV0sIGxlbmd0 aCAwCjE1OjA1OjAyLjQzMzc3NiBJUCAxOTIuMTY4LjEuMi40MzM2MCA+IDE5Mi4xNjguMS4yLjE5 OTk5OiBGbGFncyBbLl0sIHNlcSAxMjc0NDg6MTMwMzQ0LCBhY2sgMSwgd2luIDQ2LCBvcHRpb25z IFtub3Asbm9wLFRTIHZhbCAxMTE3Mjg4NyBlY3IgMTExNzI4ODddLCBsZW5ndGggMjg5NgoxNTow NTowMi40MzM3ODMgSVAgMTkyLjE2OC4xLjIuNDMzNjAgPiAxOTIuMTY4LjEuMi4xOTk5OTogRmxh Z3MgW1AuXSwgc2VxIDEzMDM0NDoxMzEwOTYsIGFjayAxLCB3aW4gNDYsIG9wdGlvbnMgW25vcCxu b3AsVFMgdmFsIDExMTcyODg3IGVjciAxMTE3Mjg4N10sIGxlbmd0aCA3NTIKMTU6MDU6MDIuNDQ0 ODM5IElQIDE5Mi4xNjguNC4xLjk5OTkgPiAxOTIuMTY4LjEuMi40MzM2MDogRmxhZ3MgWy5dLCBh Y2sgMTMxMDk3LCB3aW4gMzg2LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTE3Mjg5OCBlY3Ig MTExNzI4ODddLCBsZW5ndGggMAoxNTowNTowMy40MjE5MTUgSVAgMTkyLjE2OC4xLjIuNDMzNjAg PiAxOTIuMTY4LjEuMi4xOTk5OTogRmxhZ3MgW0YuXSwgc2VxIDEzMTA5NiwgYWNrIDEsIHdpbiA0 Niwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExNzM4NzUgZWNyIDExMTcyODk4XSwgbGVuZ3Ro IDAKMTU6MDU6MDMuNDYxNzQyIElQIDE5Mi4xNjguNC4xLjk5OTkgPiAxOTIuMTY4LjEuMi40MzM2 MDogRmxhZ3MgWy5dLCBhY2sgMTMxMDk4LCB3aW4gMzg2LCBvcHRpb25zIFtub3Asbm9wLFRTIHZh bCAxMTE3MzkxNSBlY3IgMTExNzM4NzVdLCBsZW5ndGggMAoKSSBiZWxpZXZlIHRoYXQgdGhlIGNh dXNlIG9mIHdoeSB0aGUgc2VuZGluZyBuZXRjYXQgd2l0aG91dCAtdzEgb3B0aW9uIGRvZXNuJ3Qg ZXhpdCwgaXMgdGhhdCBpdCBkb2Vzbid0IHJlY2VpdmUgdGhlIGZpbmFsIEZJTiBmcm9tIHRoZSBk dG50dW5uZWwuCkNhbiB0aGlzIGlzc3VlIGJlIGZpeGVkIHdpdGggYSBzbWFsbCBhbW91bnQgb2Yg d29yaz8yMDEyLzEyLzUgWm9sbGVyLCBEYXZpZCBBLiAoTVNGQy1FTzYwKVtIT1NDIFNFUlZJQ0VT IENPTlRSQUNUXSA8ZGF2aWQuYS56b2xsZXJAbmFzYS5nb3Y+SGkgU2VyZ2V5LEkgd291bGQgbm90 IGV2ZW4gY2FsbCB0aGlzIGEgcHJvYmxlbS4gbmV0Y2F0IGlzIGRlc2lnbmVkIHRvIGJlIGV4dHJl bWVseSBmbGV4aWJsZSBhbmQgYnkgZGVmYXVsdCB0aGUgc2VuZGVyIGtlZXBzIGl0cyBjb25uZWN0 aW9uIG9wZW4gYXMgbG9uZyBhcyB0aGUgb3RoZXIgZW5kIGRvZXMuIE9uIHRoZSBvdGhlciBoYW5k IHRoZSByZWNlaXZlciBieSBkZWZhdWx0IHRlcm1pbmF0ZXMgYWZ0ZXIgcmVjZWlwdCBvZiBhbiBF T0YgYnV0IHRoZXJlIGlzIGEgc3dpdGNoIHRvIGtlZXAgbGlzdGVuaW5nIGlmIHRoYXQgaXMgdGhl IGRlc2lyZWQgYmVoYXZpb3IuICBXaXRoIERUTiwgeW91ciBzZW5kaW5nIG5jIG1heSBkbyBpdHMg dGhpbmcgd2hpbGUgdGhlIGRlc3RpbmF0aW9uIG5vZGUgaXMgbm90IGV2ZW4gb24gbGluZSBhbmQg dGhlbiBhbiBob3VyIGxhdGVyIGl0IGJlY29tZXMgYXZhaWxhYmxlIGFuZCBjb21wbGV0ZXMgdGhl IHRyYW5zbWlzc2lvbiB0byB0aGUgcmVjZWl2aW5nIG5jLiBJbiB0aGlzIHNjZW5hcmlvLCB0aGUg c2VuZGVyIHdvdWxkIHN0aWxsIGJlIOKAnGh1bmcgdXDigJ0gcG9zc2libHkgaW5kZWZpbml0ZWx5 IHdhaXRpbmcgZm9yIGEgdGVybWluYXRpbmcgc2lnbmFsLiAgSSBhbSBub3QgdGhlIERUTjIgYXV0 aEhpIG9yaXR5LCBidXQsIEkgZG9u4oCZdCBzZWUgYSBjaGFuZ2UgdG8gZHRudHVubmVsIGZvciB0 aGlzIGFzIGl0IHdvdWxkIGFzc3VtZSBhIHNwZWNpZmljIHVzYWdlIHRoYXQgd291bGQgcHJvYmFi bHkgYnJlYWsgc29tZW9uZSBlbHNl4oCZcyB1c2FnZSAobGlrZSBtaW5lIEopLiBCZXN0IHJlZ2Fy ZHMsRFogRGF2aWQgWm9sbGVyQ09MU0EgQ29ycG9yYXRpb25NYXJzaGFsbCBTcGFjZSBGbGlnaHQg Q2VudGVyIEZyb206IHNzaXJlc2tpbkBnbWFpbC5jb20gW21haWx0bzpzc2lyZXNraW5AZ21haWwu Y29tXSAKU2VudDogV2VkbmVzZGF5LCBEZWNlbWJlciAwNSwgMjAxMiAyOjU2IEFNClRvOiBab2xs ZXIsIERhdmlkIEEuIChNU0ZDLUVPNjApW0hPU0MgU0VSVklDRVMgQ09OVFJBQ1RdCkNjOiBkdG4t dXNlcnNAaXJ0Zi5vcmcKU3ViamVjdDogUmU6IFtkdG4tdXNlcnNdIEEgcHJvYmxlbSB3aXRoIGR0 bnR1bm5lbCBIaSBEYXZpZCwKClRoYW5rcyBmb3IgeW91ciBoZWxwLCBuYyAtdzEgZGlkIHRoZSB0 cmljay4gQnV0IHNob3VsZG4ndCBkdG50dW5uZWwgYmVoYXZpb3IgYmUgY2hhbmdlZD8gVGhlIHJl Y2VpdmluZyBkdG50dW5uZWwgY291bGQgc2lnbmFsIHRoZSBzZW5kaW5nIGR0bnR1bm5lbCwgdGhh dCB0aGUgbGlzdGVuaW5nIG5jIGhhcyBkaXNjb25uZWN0ZWQsIHNvIHRoYXQgdGhlIHNlbmRpbmcg ZHRudHVubmVsIGNsb3NlIGl0cyBjb25uZWN0aW9uIHRvIHRoZSBzZW5kaW5nIG5jLiBPciBpcyB0 aGlzIHNvbGVseSB0aGUgcHJvYmxlbSBvZiBuYz8yMDEyLzEyLzUgWm9sbGVyLCBEYXZpZCBBLiAo TVNGQy1FTzYwKVtIT1NDIFNFUlZJQ0VTIENPTlRSQUNUXSA8ZGF2aWQuYS56b2xsZXJAbmFzYS5n b3Y+SGkgU2VyZ2V5LE9uIFJIRUwgNS43LCBJIGFtIHJ1bm5pbmcgZHRuLTIuOS4wIHBsdXMgbW9k aWZpY2F0aW9ucyB0aGF0IHNob3VsZCBub3QgaW1wYWN0IHRoZSBiZWhhdmlvciBvZiBkdG50dW5u ZWwuSSBqdXN0IHJhbiB5b3VyIHRlc3Qgd2l0aG91dCB0aGUgaXB0YWJsZXMgcmVkaXJlY3QgYW5k IEkgc2VlIHRoZSBzYW1lIGJlaGF2aW9yLkkgYmVsaWV2ZSB0aGF0IHRoZSBzZW5kZXIgZXhpdHMg aWYgeW91IGdvIGRpcmVjdGx5IGZyb20gbmMgdG8gbmMgYmVjYXVzZSB0aGUgcmVjZWl2ZXIgZXhp dHMgd2hlbiBpdCBnZXRzIGFuIGVuZCBvZiBmaWxlLllvdSBjYW4gYWRkIGEg4oCcLXcgMeKAnSBv cHRpb24gdG8gdGhlIHNlbmRlciBuYyBzbyB0aGF0IGl0IHdpbGwgdGltZW91dCBhbmQgZXhpdCBh ZnRlciBzdGRpbiBpcyBpZGxlIGZvciAxIHNlY29uZC5CZXN0IHJlZ2FyZHMsRFogRGF2aWQgWm9s bGVyQ09MU0EgQ29ycG9yYXRpb25IT1NDIC8gQzEwNyAgKE9mZmljZTogKDI1NikgNTQ0LTE4MjAK KkVNYWlsOiBkYXZpZC5hLnpvbGxlckBuYXNhLmdvdiBGcm9tOiBkdG4tdXNlcnMtYm91bmNlc0Bp cnRmLm9yZyBbbWFpbHRvOmR0bi11c2Vycy1ib3VuY2VzQGlydGYub3JnXSBPbiBCZWhhbGYgT2Yg c3NpcmVza2luQGdtYWlsLmNvbQpTZW50OiBUdWVzZGF5LCBEZWNlbWJlciAwNCwgMjAxMiAxMTox MyBBTQpUbzogZHRuLXVzZXJzQGlydGYub3JnClN1YmplY3Q6IFtkdG4tdXNlcnNdIEEgcHJvYmxl bSB3aXRoIGR0bnR1bm5lbCBIZWxsbyBhbGwhCgpJIGFtIHVzaW5nIGR0bi0yLjkuMCBvbiBhIFJI RUwgNiBiYXNlZCBMaW51eCBkaXN0cm8gYW5kIEkgYW0gaGF2aW5nIGEgcHJvYmxlbSB3aGVuIHVz aW5nIG5ldGNhdCAobmMpIHdpdGggZHRudHVubmVsLgpPbiB0aGUgc2VuZGVyIG5vZGUgSSBydW4g ImNhdCAvZXRjL3Bhc3N3ZCB8IG5jIHJlY2VpdmVyX2lwIi4gT24gdGhlIHJlY2VpdmVyIG5vZGUg SSBydW4gIm5jIC1sIDk5OTkiLiBXaXRoIHRoZSBoZWxwIApvZiBJcHRhYmxlcyBwb3J0IDk5OTkg Z2V0cyByZWRpcmVjdGVkIHRvIHRoZSBwb3J0IDE5OTk5LCB3aGljaCBpcyBsaXN0ZW5lZCBieSBk dG50dW5uZWwuCgpUaGUgZmlsZSAvZXRjL3Bhc3N3ZCBpcyBzdWNjZXNzZnVsbHkgZGVsaXZlcmVk IHRvIHRoZSByZWNlaXZlciBhbmQgaXMgc2hvd24gb24gdGhlIHNjcmVlbi4gQWZ0ZXIgdGhpcyB0 aGUgcmVjZWl2aW5nIG5jIGV4aXN0cy4gCk5vIHByb2JsZW0gaGVyZS4gSG93ZXZlciwgbmMgb24g dGhlIHNlbmRlciBub2RlIGRvZXNuJ3QgZXhpdCBhZnRlciBpdCBzZW5kcyB0aGUgZmlsZS4gSXQg Y29udGludWVzIHRvIHJ1biBmb3JldmVyLiBXaGVuIEkgCnJ1biBuYyBpbiB0aGUgb3Bwb3NpdGUg ZGlyZWN0aW9uLCBJIGdldCB0aGUgc2FtZSBwcm9ibGVtIC0gdGhlIHNlbmRpbmcgbmMgZG9lc24n dCBleGl0LgoKTXkgY29uZmlndXJhdGlvbiBvbiBib3RoIG5vZGVzIGlzIHN5bW1ldHJpYzoKaXB0 YWJsZXMgLXQgbmF0IC1BIE9VVFBVVCAtZCAkUkVNT1RFX0hPU1QgLXAgdGNwIC0tZHBvcnQgOTk5 OSAtaiBETkFUIC0tdG8gJExPQ0FMX0hPU1Q6MTk5OTkKZHRudHVubmVsIC1UICRMT0NBTF9IT1NU OjE5OTk5OiRSRU1PVEVfSE9TVDo5OTk5ICRSRU1PVEVfTk9ERS9kdG50dW5uZWwvbmMgLWQKZHRu dHVubmVsIC1MIC0tbG9jYWwtZWlkICRMT0NBTF9OT0RFL2R0bnR1bm5lbC9uYyAtZAoKZHRucGlu ZyB3b3JrcyBvayBpbiBib3RoIGRpcmVjdGlvbnMuCgpQbGVhc2UgZ2l2ZSBtZSBhbnkgYWR2aWNl LiAKCldpdGggYmVzdCByZWdhcmRzLApTZXJnZXkgU2lyZXNraW4gCgotLSAKS2luZGVzdCBSZWdh cmRzCgpTZXJnZXkgU2lyZXNraW4KRkdVUCBDTklJIEVJU1UKCi0tIApLaW5kZXN0IFJlZ2FyZHMK ClNlcmdleSBTaXJlc2tpbgpGR1VQIENOSUkgRUlTVQoKLS0gCkJlc3QgcmVnYXJkcywKU2VyZ2V5 IFNpcmVza2luCgotLSAKQmVzdCByZWdhcmRzLApTZXJnZXkgU2lyZXNraW4KCi0tIApCZXN0IHJl Z2FyZHMsClNlcmdleSBTaXJlc2tpbg== ------=_Part_0_1355418889033 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 Content-Disposition: inline PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+PGhlYWQ+PE1FVEEgSFRUUC1FUVVJVj0iQ29udGVudC1UeXBlIiBDT05U RU5UPSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXMtYXNjaWkiPjxtZXRhIG5hbWU9R2VuZXJhdG9yIGNv bnRlbnQ9Ik1pY3Jvc29mdCBXb3JkIDE0IChmaWx0ZXJlZCBtZWRpdW0pIj48c3R5bGU+PCEtLQov KiBGb250IERlZmluaXRpb25zICovCkBmb250LWZhY2UKCXtmb250LWZhbWlseTpXaW5nZGluZ3M7 CglwYW5vc2UtMTo1IDAgMCAwIDAgMCAwIDAgMCAwO30KQGZvbnQtZmFjZQoJe2ZvbnQtZmFtaWx5 OldpbmdkaW5nczsKCXBhbm9zZS0xOjUgMCAwIDAgMCAwIDAgMCAwIDA7fQpAZm9udC1mYWNlCgl7 Zm9udC1mYW1pbHk6Q2FsaWJyaTsKCXBhbm9zZS0xOjIgMTUgNSAyIDIgMiA0IDMgMiA0O30KQGZv bnQtZmFjZQoJe2ZvbnQtZmFtaWx5OlRhaG9tYTsKCXBhbm9zZS0xOjIgMTEgNiA0IDMgNSA0IDQg MiA0O30KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8KcC5Nc29Ob3JtYWwsIGxpLk1zb05vcm1hbCwg ZGl2Lk1zb05vcm1hbAoJe21hcmdpbjowaW47CgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7Cglmb250 LXNpemU6MTIuMHB0OwoJZm9udC1mYW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsInNlcmlmIjt9CmE6 bGluaywgc3Bhbi5Nc29IeXBlcmxpbmsKCXttc28tc3R5bGUtcHJpb3JpdHk6OTk7Cgljb2xvcjpi bHVlOwoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9CmE6dmlzaXRlZCwgc3Bhbi5Nc29IeXBl cmxpbmtGb2xsb3dlZAoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsKCWNvbG9yOnB1cnBsZTsKCXRl eHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQpwCgl7bXNvLXN0eWxlLXByaW9yaXR5Ojk5OwoJbXNv LW1hcmdpbi10b3AtYWx0OmF1dG87CgltYXJnaW4tcmlnaHQ6MGluOwoJbXNvLW1hcmdpbi1ib3R0 b20tYWx0OmF1dG87CgltYXJnaW4tbGVmdDowaW47Cglmb250LXNpemU6MTIuMHB0OwoJZm9udC1m YW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsInNlcmlmIjt9CnAuTXNvQWNldGF0ZSwgbGkuTXNvQWNl dGF0ZSwgZGl2Lk1zb0FjZXRhdGUKCXttc28tc3R5bGUtcHJpb3JpdHk6OTk7Cgltc28tc3R5bGUt bGluazoiQmFsbG9vbiBUZXh0IENoYXIiOwoJbWFyZ2luOjBpbjsKCW1hcmdpbi1ib3R0b206LjAw MDFwdDsKCWZvbnQtc2l6ZTo4LjBwdDsKCWZvbnQtZmFtaWx5OiJUYWhvbWEiLCJzYW5zLXNlcmlm Ijt9CnNwYW4uQmFsbG9vblRleHRDaGFyCgl7bXNvLXN0eWxlLW5hbWU6IkJhbGxvb24gVGV4dCBD aGFyIjsKCW1zby1zdHlsZS1wcmlvcml0eTo5OTsKCW1zby1zdHlsZS1saW5rOiJCYWxsb29uIFRl eHQiOwoJZm9udC1mYW1pbHk6IlRhaG9tYSIsInNhbnMtc2VyaWYiO30Kc3Bhbi5FbWFpbFN0eWxl MjAKCXttc28tc3R5bGUtdHlwZTpwZXJzb25hbC1yZXBseTsKCWZvbnQtZmFtaWx5OiJDYWxpYnJp Iiwic2Fucy1zZXJpZiI7Cgljb2xvcjojMUY0OTdEO30KLk1zb0NocERlZmF1bHQKCXttc28tc3R5 bGUtdHlwZTpleHBvcnQtb25seTsKCWZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7 fQpAcGFnZSBXb3JkU2VjdGlvbjEKCXtzaXplOjguNWluIDExLjBpbjsKCW1hcmdpbjoxLjBpbiAx LjBpbiAxLjBpbiAxLjBpbjt9CmRpdi5Xb3JkU2VjdGlvbjEKCXtwYWdlOldvcmRTZWN0aW9uMTt9 Ci0tPjwvc3R5bGU+PCEtLVtpZiBndGUgbXNvIDldPjx4bWw+CjxvOnNoYXBlZGVmYXVsdHMgdjpl eHQ9ImVkaXQiIHNwaWRtYXg9IjEwMjYiIC8+CjwveG1sPjwhW2VuZGlmXS0tPjwhLS1baWYgZ3Rl IG1zbyA5XT48eG1sPgo8bzpzaGFwZWxheW91dCB2OmV4dD0iZWRpdCI+CjxvOmlkbWFwIHY6ZXh0 PSJlZGl0IiBkYXRhPSIxIiAvPgo8L286c2hhcGVsYXlvdXQ+PC94bWw+PCFbZW5kaWZdLS0+PC9o ZWFkPjxib2R5IGxhbmc9RU4tVVMgbGluaz1ibHVlIHZsaW5rPXB1cnBsZT48c3BhbiBzdHlsZT0i Zm9udC1mYW1pbHk6IEFyaWFsOyI+SGkgRGF2aWQsPGJyPjxicj5JIHRoaW5rLCB0aGF0IGl0IHNo b3VsZCBiZSBwb3NzaWJsZSB0byBzcGVjaWZ5IHRoZSBiZWhhdmlvciBvZiBkdG50dW5uZWwgaW4g Y2FzZSBpZiBib3RoIGVuZHMgYXJlIG5vdCB1cCBpbiAmcXVvdDtuZWFyIHJlYWwgdGltZSZxdW90 OyB3aXRoIGEgY29tbWFuZCBsaW5lIG9wdGlvbi4gVGhlcmUgc2hvdWxkIGJlIDIgb3B0aW9uczo8 YnI+MSkgd2FpdCBmb3IgdGhlIHJlY2VpdmVyIHRvIGNvbWUgb25saW5lOzxicj4yKSBwcmV0ZW5k IHRvIGJlIGEgcmVjZWl2ZXIsIGdldCBhbGwgZGF0YSwgc2VuZCBmaW4gdG8gdGhlIHNlbmRlci4g V2hlbiB0aGUgcmVhbCByZWNlaXZlciBjb21lcyBvbiBsaW5lLCBzZW5kIGRhdGEuPGJyPlRoZSBz ZWNvbmQgb3B0aW9uIHdvdWxkIGJlIHVzZWZ1bGwgZm9yIHRoZSBhcHBzIHRoYXQgb25seSBzZW5k IGRhdGEgb25lIHdheSwgbm90IHJlcXVlc3QtcmVwbHkuPGJyPjxicj48YnI+QmVzdCByZWdhcmRz LDxicj5TZXJnZXk8YnI+PGJyPjxkaXYgaWQ9Imh0Y19oZWFkZXIiIHN0eWxlPSIiPi0tLS0tIFJl cGx5IG1lc3NhZ2UgLS0tLS08YnI+0J7RgjogJnF1b3Q7Wm9sbGVyLCBEYXZpZCBBLiAoTVNGQy1F TzYwKVtIT1NDIFNFUlZJQ0VTIENPTlRSQUNUXSZxdW90OyAmbHQ7ZGF2aWQuYS56b2xsZXJAbmFz YS5nb3YmZ3Q7PGJyPtCa0L7QvNGDOiAmcXVvdDtzc2lyZXNraW5AZ21haWwuY29tJnF1b3Q7ICZs dDtzc2lyZXNraW5AZ21haWwuY29tJmd0Ozxicj7QmtC+0L/QuNGPOiAmcXVvdDtkdG4tdXNlcnNA aXJ0Zi5vcmcmcXVvdDsgJmx0O2R0bi11c2Vyc0BpcnRmLm9yZyZndDs8YnI+0KLQtdC80LA6IFtk dG4tdXNlcnNdIEEgcHJvYmxlbSB3aXRoIGR0bnR1bm5lbDxicj7QlNCw0YLQsDog0YfRgiwg0LTQ tdC6IDEzLCAyMDEyIDE3OjQ2PGJyPjxicj48L2Rpdj48L3NwYW4+PGJyPjxkaXYgY2xhc3M9V29y ZFNlY3Rpb24xPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBw dDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkhpIFNl cmdleSw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIHN0eWxl PSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29s b3I6IzFGNDk3RCc+SSBzZWUgbm93IHRoZSBlbmQgdG8gZW5kIGVmZmVjdCB5b3UgYXJlIHdhbnRp bmcgYW5kIEkmIzgyMTc7bGwgZXhwZXJpbWVudCBzb21lIG1vcmUgYW5kIGdldCBiYWNrIHdpdGgg eW91IGJ1dCBpdCBtYXkgYmUgYSBmZXcgZGF5cy48bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xh c3M9TXNvTm9ybWFsPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJD YWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+PG86cD4mbmJzcDs8L286cD48L3Nw YW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtm b250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkkgZ3Vlc3Mg b25lIHF1ZXN0aW9uIGlzOiA8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFs PjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fu cy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+SWYgYm90aCBlbmRzIGFyZSBub3QgdXAgaW4gJiM4MjIw O25lYXIgcmVhbCB0aW1lJiM4MjIxOyB3b3VsZCB5b3Ugd2FudCB0aGUgc2VuZGVyIHRvIHN0YXkg Y29ubmVjdGVkIHVudGlsIHRoZSByZWNlaXZpbmcgZW5kIGNvbWVzIG9uIGxpbmUgYW5kIGNhbiBy ZXR1cm4gdGhlIEZJTiBpbiBhbGwgY2lyY3Vtc3RhbmNlcz88bzpwPjwvbzpwPjwvc3Bhbj48L3A+ PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFt aWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+RFo8bzpwPjwvbzpwPjwv c3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0 O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+PG86cD4m bmJzcDs8L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBzdHlsZT0nZm9u dC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiQXJpYWwiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0 OTdEJz5EYXZpZCBab2xsZXI8L3NwYW4+PHNwYW4gc3R5bGU9J2NvbG9yOiMxRjQ5N0QnPjxvOnA+ PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWw+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6 ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkFyaWFsIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+ Q09MU0EgQ29ycG9yYXRpb248L3NwYW4+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9u dC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz48bzpwPjwvbzpw Pjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTAu MHB0O2ZvbnQtZmFtaWx5OiJBcmlhbCIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkhPU0Mg LyBDMTA3Jm5ic3A7IDwvc3Bhbj48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZh bWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPjxvOnA+PC9vOnA+PC9z cGFuPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWw+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7 Zm9udC1mYW1pbHk6V2luZ2RpbmdzO2NvbG9yOiM5OTAwMDAnPig8L3NwYW4+PHNwYW4gc3R5bGU9 J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkFyaWFsIiwic2Fucy1zZXJpZiI7Y29sb3I6 IzFGNDk3RCc+T2ZmaWNlOjwvc3Bhbj48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250 LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPiA8L3NwYW4+PHNw YW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkFyaWFsIiwic2Fucy1zZXJp ZiI7Y29sb3I6IzFGNDk3RCc+KDI1NikmbmJzcDs1NDQtMTgyMDwvc3Bhbj48c3BhbiBzdHlsZT0n Zm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiQXJpYWwiLCJzYW5zLXNlcmlmIjtjb2xvcjpu YXZ5Jz48YnI+PC9zcGFuPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5 OldpbmdkaW5ncztjb2xvcjojMzM2NkZGJz4qPC9zcGFuPjxzcGFuIHN0eWxlPSdmb250LXNpemU6 MTAuMHB0O2ZvbnQtZmFtaWx5OiJBcmlhbCIsInNhbnMtc2VyaWYiO2NvbG9yOmJsYWNrJz5FTWFp bDogPC9zcGFuPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJBcmlh bCIsInNhbnMtc2VyaWYiO2NvbG9yOm5hdnknPjxhIGhyZWY9Im1haWx0bzpkYXZpZC5hLnpvbGxl ckBuYXNhLmdvdiI+ZGF2aWQuYS56b2xsZXJAbmFzYS5nb3Y8L2E+PC9zcGFuPjxzcGFuIHN0eWxl PSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29s b3I6IzFGNDk3RCc+PG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3Bh biBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2Vy aWYiO2NvbG9yOiMxRjQ5N0QnPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1N c29Ob3JtYWw+PGI+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IlRh aG9tYSIsInNhbnMtc2VyaWYiJz5Gcm9tOjwvc3Bhbj48L2I+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6 ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IlRhaG9tYSIsInNhbnMtc2VyaWYiJz4gc3NpcmVza2luQGdt YWlsLmNvbSBbbWFpbHRvOnNzaXJlc2tpbkBnbWFpbC5jb21dIDxicj48Yj5TZW50OjwvYj4gVGh1 cnNkYXksIERlY2VtYmVyIDEzLCAyMDEyIDM6MjMgQU08YnI+PGI+VG86PC9iPiBab2xsZXIsIERh dmlkIEEuIChNU0ZDLUVPNjApW0hPU0MgU0VSVklDRVMgQ09OVFJBQ1RdPGJyPjxiPkNjOjwvYj4g ZHRuLXVzZXJzQGlydGYub3JnPGJyPjxiPlN1YmplY3Q6PC9iPiBSZTogW2R0bi11c2Vyc10gQSBw cm9ibGVtIHdpdGggZHRudHVubmVsPG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05v cm1hbD48bzpwPiZuYnNwOzwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21hcmdp bi1ib3R0b206MTIuMHB0Jz5IaSBEYXZpZCw8YnI+PGJyPkknbGwgc3RhcnQgZnJvbSB0aGUgZW5k LiBJIGhhdmUgcmVwZWF0ZWQgdGhlIHRlc3Qgd2l0aCAxMDAgcGFyYWxsZWwgZmxvd3MgdHdpY2Us IGFuZCBpdCBwYXNzZWQgZmxhd2xlc3NseS4gSSB0aGluayB0aGF0IHdhcyBteSBtaXN0YWtlLCBJ IGNvdWxkIGhhdmUgY291bnRlZCA8YnI+dGhlIG51bWJlciBvZiBlcmFzZWQgY29ubmVjdGlvbnMg dG9vIGVhcmx5LCBiZWZvcmUgYWxsIG9mIDEwMCBjb25uZWN0aW9ucyBoYXZlIHRlcm1pbmF0ZWQu PGJyPjxicj5Ob3cgdG8gdGhlIGZpcnN0IHBhcnQuIEkgaGF2ZSByZXBsYWNlZCB3aGlsZSgxKSB3 aXRoIHdoaWxlKCFzb2NrX2VvZikgYXMgeW91IHN1Z2dlc3RlZC4gVGhpcyBsZWFkcyB0byB2ZXJ5 IHN0cmFuZ2UgcmVzdWx0cyB3aGVuIHNlbmRpbmcgYSBzbWFsbCBmaWxlIHdpdGggbmMuIE5vdGUg dGhlIGxhc3QgdHdvIFNZTiBhbmQgUlNUIHBhY2tldHMgb24gdGhlIHJlY2VpdmVyIHNpZGUuPGJy Pjxicj5TZW5kZXI6IHRjcGR1bXAgLW5uIC1ucGkgbG8gcG9ydCA5OTk5IG9yIHBvcnQgMTk5OTk8 YnI+MTM6MTA6MzUuMTM2MjgzIElQIDE5Mi4xNjguMS4yLjMyODgwICZndDsgMTkyLjE2OC4xLjIu MTk5OTk6IEZsYWdzIFtTXSwgc2VxIDI5MjUwNzYzNjYsIHdpbiA1ODQwLCBvcHRpb25zIFttc3Mg MTQ2MCxzYWNrT0ssVFMgdmFsIDkwNzA1NTg5IGVjciAwLG5vcCx3c2NhbGUgN10sIGxlbmd0aCAw PGJyPjEzOjEwOjM1LjEzNjM0NiBJUCAxOTIuMTY4LjQuMS45OTk5ICZndDsgMTkyLjE2OC4xLjIu MzI4ODA6IEZsYWdzIFtTLl0sIHNlcSAyOTMxMTk2MzQ0LCBhY2sgMjkyNTA3NjM2Nywgd2luIDMy NzY4LCBvcHRpb25zIFttc3MgMTYzOTYsc2Fja09LLFRTIHZhbCA5MDcwNTU4OSBlY3IgOTA3MDU1 ODksbm9wLHdzY2FsZSA3XSwgbGVuZ3RoIDA8YnI+MTM6MTA6MzUuMTM2MzcxIElQIDE5Mi4xNjgu MS4yLjMyODgwICZndDsgMTkyLjE2OC4xLjIuMTk5OTk6IEZsYWdzIFsuXSwgYWNrIDI5MzExOTYz NDUsIHdpbiA0Niwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgOTA3MDU1ODkgZWNyIDkwNzA1NTg5 XSwgbGVuZ3RoIDA8YnI+MTM6MTA6MzUuMTM2Nzk5IElQIDE5Mi4xNjguMS4yLjMyODgwICZndDsg MTkyLjE2OC4xLjIuMTk5OTk6IEZsYWdzIFtQLl0sIHNlcSAwOjEsIGFjayAxLCB3aW4gNDYsIG9w dGlvbnMgW25vcCxub3AsVFMgdmFsIDkwNzA1NTkwIGVjciA5MDcwNTU4OV0sIGxlbmd0aCAxPGJy PjEzOjEwOjM1LjEzNjgxOSBJUCAxOTIuMTY4LjQuMS45OTk5ICZndDsgMTkyLjE2OC4xLjIuMzI4 ODA6IEZsYWdzIFsuXSwgYWNrIDIsIHdpbiAyNTYsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDkw NzA1NTkwIGVjciA5MDcwNTU5MF0sIGxlbmd0aCAwPGJyPjEzOjEwOjM1LjEzNjg5MiBJUCAxOTIu MTY4LjEuMi4zMjg4MCAmZ3Q7IDE5Mi4xNjguMS4yLjE5OTk5OiBGbGFncyBbRi5dLCBzZXEgMSwg YWNrIDEsIHdpbiA0Niwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgOTA3MDU1OTAgZWNyIDkwNzA1 NTkwXSwgbGVuZ3RoIDA8YnI+MTM6MTA6MzUuMTc2NzU1IElQIDE5Mi4xNjguNC4xLjk5OTkgJmd0 OyAxOTIuMTY4LjEuMi4zMjg4MDogRmxhZ3MgWy5dLCBhY2sgMywgd2luIDI1Niwgb3B0aW9ucyBb bm9wLG5vcCxUUyB2YWwgOTA3MDU2MzAgZWNyIDkwNzA1NTkwXSwgbGVuZ3RoIDA8YnI+MTM6MTA6 MzUuMTc4NjAyIElQIDE5Mi4xNjguNC4xLjk5OTkgJmd0OyAxOTIuMTY4LjEuMi4zMjg4MDogRmxh Z3MgW0YuXSwgc2VxIDEsIGFjayAzLCB3aW4gMjU2LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCA5 MDcwNTYzMSBlY3IgOTA3MDU1OTBdLCBsZW5ndGggMDxicj4xMzoxMDozNS4xNzg2MjMgSVAgMTky LjE2OC4xLjIuMzI4ODAgJmd0OyAxOTIuMTY4LjEuMi4xOTk5OTogRmxhZ3MgWy5dLCBhY2sgMiwg d2luIDQ2LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCA5MDcwNTYzMSBlY3IgOTA3MDU2MzFdLCBs ZW5ndGggMDxicj4xMzoxMDozNS4yNTM4MjkgSVAgMTkyLjE2OC4xLjIuMzI4ODEgJmd0OyAxOTIu MTY4LjEuMi4xOTk5OTogRmxhZ3MgW1NdLCBzZXEgMjkzNDkxNDUzMywgd2luIDU4NDAsIG9wdGlv bnMgW21zcyAxNDYwLHNhY2tPSyxUUyB2YWwgOTA3MDU3MDcgZWNyIDAsbm9wLHdzY2FsZSA3XSwg bGVuZ3RoIDA8YnI+MTM6MTA6MzUuMjUzODYwIElQIDE5Mi4xNjguNC4xLjk5OTkgJmd0OyAxOTIu MTY4LjEuMi4zMjg4MTogRmxhZ3MgW1MuXSwgc2VxIDI5MzIyNDkxMjUsIGFjayAyOTM0OTE0NTM0 LCB3aW4gMzI3NjgsIG9wdGlvbnMgW21zcyAxNjM5NixzYWNrT0ssVFMgdmFsIDkwNzA1NzA3IGVj ciA5MDcwNTcwNyxub3Asd3NjYWxlIDddLCBsZW5ndGggMDxicj4xMzoxMDozNS4yNTM4NzggSVAg MTkyLjE2OC4xLjIuMzI4ODEgJmd0OyAxOTIuMTY4LjEuMi4xOTk5OTogRmxhZ3MgWy5dLCBhY2sg MjkzMjI0OTEyNiwgd2luIDQ2LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCA5MDcwNTcwNyBlY3Ig OTA3MDU3MDddLCBsZW5ndGggMDxicj4xMzoxMDozNi4zODA5NDIgSVAgMTkyLjE2OC40LjEuOTk5 OSAmZ3Q7IDE5Mi4xNjguMS4yLjMyODgxOiBGbGFncyBbRi5dLCBzZXEgMSwgYWNrIDEsIHdpbiAy NTYsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDkwNzA2ODM0IGVjciA5MDcwNTcwN10sIGxlbmd0 aCAwPGJyPjEzOjEwOjM2LjM4MTQ5NSBJUCAxOTIuMTY4LjEuMi4zMjg4MSAmZ3Q7IDE5Mi4xNjgu MS4yLjE5OTk5OiBGbGFncyBbLl0sIGFjayAyLCB3aW4gNDYsIG9wdGlvbnMgW25vcCxub3AsVFMg dmFsIDkwNzA2ODM1IGVjciA5MDcwNjgzNF0sIGxlbmd0aCAwPGJyPjEzOjEwOjM2LjM5ODA3MCBJ UCAxOTIuMTY4LjEuMi4zMjg4MSAmZ3Q7IDE5Mi4xNjguMS4yLjE5OTk5OiBGbGFncyBbRi5dLCBz ZXEgMCwgYWNrIDIsIHdpbiA0Niwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgOTA3MDY4NTEgZWNy IDkwNzA2ODM0XSwgbGVuZ3RoIDA8YnI+MTM6MTA6MzYuMzk4MTMxIElQIDE5Mi4xNjguNC4xLjk5 OTkgJmd0OyAxOTIuMTY4LjEuMi4zMjg4MTogRmxhZ3MgWy5dLCBhY2sgMiwgd2luIDI1Niwgb3B0 aW9ucyBbbm9wLG5vcCxUUyB2YWwgOTA3MDY4NTEgZWNyIDkwNzA2ODUxXSwgbGVuZ3RoIDA8YnI+ PGJyPlJlY2VpdmVyOiB0Y3BkdW1wIC1ubiAtbnBpIGxvIHBvcnQgOTk5OSBvciBwb3J0IDE5OTk5 PGJyPjEzOjEwOjM1LjE4NTIxMSBJUCAxOTIuMTY4LjQuMS40NDU1MCAmZ3Q7IDE5Mi4xNjguNC4x Ljk5OTk6IEZsYWdzIFtTXSwgc2VxIDI5MjcxMjM5NDUsIHdpbiAzMjc5Miwgb3B0aW9ucyBbbXNz IDE2Mzk2LHNhY2tPSyxUUyB2YWwgOTA2OTY0MTIgZWNyIDAsbm9wLHdzY2FsZSA3XSwgbGVuZ3Ro IDA8YnI+MTM6MTA6MzUuMTg1MjY2IElQIDE5Mi4xNjguNC4xLjk5OTkgJmd0OyAxOTIuMTY4LjQu MS40NDU1MDogRmxhZ3MgW1MuXSwgc2VxIDI5Mjk2MjE2MDIsIGFjayAyOTI3MTIzOTQ2LCB3aW4g MzI3NjgsIG9wdGlvbnMgW21zcyAxNjM5NixzYWNrT0ssVFMgdmFsIDkwNjk2NDEyIGVjciA5MDY5 NjQxMixub3Asd3NjYWxlIDddLCBsZW5ndGggMDxicj4xMzoxMDozNS4xODUyODYgSVAgMTkyLjE2 OC40LjEuNDQ1NTAgJmd0OyAxOTIuMTY4LjQuMS45OTk5OiBGbGFncyBbLl0sIGFjayAxLCB3aW4g MjU3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCA5MDY5NjQxMiBlY3IgOTA2OTY0MTJdLCBsZW5n dGggMDxicj4xMzoxMDozNS4yMDY4MTggSVAgMTkyLjE2OC40LjEuNDQ1NTAgJmd0OyAxOTIuMTY4 LjQuMS45OTk5OiBGbGFncyBbUC5dLCBzZXEgMToyLCBhY2sgMSwgd2luIDI1Nywgb3B0aW9ucyBb bm9wLG5vcCxUUyB2YWwgOTA2OTY0MzQgZWNyIDkwNjk2NDEyXSwgbGVuZ3RoIDE8YnI+MTM6MTA6 MzUuMjA2ODYxIElQIDE5Mi4xNjguNC4xLjk5OTkgJmd0OyAxOTIuMTY4LjQuMS40NDU1MDogRmxh Z3MgWy5dLCBhY2sgMiwgd2luIDI1Niwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgOTA2OTY0MzQg ZWNyIDkwNjk2NDM0XSwgbGVuZ3RoIDA8YnI+MTM6MTA6MzUuMjIyMDMyIElQIDE5Mi4xNjguNC4x LjQ0NTUwICZndDsgMTkyLjE2OC40LjEuOTk5OTogRmxhZ3MgW0YuXSwgc2VxIDIsIGFjayAxLCB3 aW4gMjU3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCA5MDY5NjQ0OSBlY3IgOTA2OTY0MzRdLCBs ZW5ndGggMDxicj4xMzoxMDozNS4yMjIxMzMgSVAgMTkyLjE2OC40LjEuOTk5OSAmZ3Q7IDE5Mi4x NjguNC4xLjQ0NTUwOiBGbGFncyBbRi5dLCBzZXEgMSwgYWNrIDMsIHdpbiAyNTYsIG9wdGlvbnMg W25vcCxub3AsVFMgdmFsIDkwNjk2NDQ5IGVjciA5MDY5NjQ0OV0sIGxlbmd0aCAwPGJyPjEzOjEw OjM1LjIyMjE0NiBJUCAxOTIuMTY4LjQuMS40NDU1MCAmZ3Q7IDE5Mi4xNjguNC4xLjk5OTk6IEZs YWdzIFsuXSwgYWNrIDIsIHdpbiAyNTcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDkwNjk2NDQ5 IGVjciA5MDY5NjQ0OV0sIGxlbmd0aCAwPGJyPjEzOjEwOjM2LjMyNTM5NiBJUCAxOTIuMTY4LjQu MS40NDU1MSAmZ3Q7IDE5Mi4xNjguNC4xLjk5OTk6IEZsYWdzIFtTXSwgc2VxIDI5NDU3MTY4ODks IHdpbiAzMjc5Miwgb3B0aW9ucyBbbXNzIDE2Mzk2LHNhY2tPSyxUUyB2YWwgOTA2OTc1NTIgZWNy IDAsbm9wLHdzY2FsZSA3XSwgbGVuZ3RoIDA8YnI+MTM6MTA6MzYuMzI1NDQ3IElQIDE5Mi4xNjgu NC4xLjk5OTkgJmd0OyAxOTIuMTY4LjQuMS40NDU1MTogRmxhZ3MgW1IuXSwgc2VxIDAsIGFjayAy OTQ1NzE2ODkwLCB3aW4gMCwgbGVuZ3RoIDA8YnI+PGJyPkFub3RoZXIgc3RyYW5nZSB0aGluZyBo YXBwZW5kIHdoZW4gSSBoYXZlIHJ1biBzZW5kaW5nIG5jIGFuZCBmb3Jnb3QgdG8gcnVuIHRoZSBy ZWNlaXZpbmcgbmMuIEp1c3QgdHJ5IGl0IHlvdXJzZWxmIGFuZCBzZWUgdGNwZHVtcC48YnI+PGJy PkkgaGF2ZSBhbHNvIGNvbmR1Y3RlZCBhbm90aGVyIHRlc3Qgd2l0aCB0aGUgb3JpZ2luYWwgZHRu dHVubmVsIGNvZGUuIEkgaGF2ZSBydW4gbmMgaW4gdGhlIGludGVyYWN0aXZlIG1vZGUsIGkuZS4g bmMgJmx0O2lwJmd0OyAmbHQ7cG9ydCZndDsgb24gdGhlIGNsaWVudCBzaWRlIGFuZCBuYyAtbCAm bHQ7cG9ydCZndDsgb24gdGhlIHNlcnZlciBzaWRlLiBUaGVuIEkgdGVybWluYXRlZCB0aGUgY29u bmVjdGlvbiBvbiB0aGUgY2xpZW50IHNpZGUgYnkgcHJlc3NpbmcgQ1RSTCtELiBUaGUgc2VydmVy IHJlY2VpdmVkIGEgRklOIHBhY2tlZCBmcm9tIHRoZSBjbGllbnQsIHJlcGxpZWQgd2l0aCBhbm90 aGVyIEZJTiwgYW5kIHRlcm1pbmF0ZWQuIFRoZSBjbGllbnQgZGlkIG5vdCByZWNlaXZlIHRoZSBG SU4gcGFja2V0IGZyb20gdGhlIHNlcnZlciBhbmQgZGlkIG5vdCB0ZXJtaW5hdGUuIEFmdGVyIHRo YXQgSSBoYXZlIGVzdGFibGlzaGVkIGEgbmV3IGNvbm5lY3Rpb24gYW5kIHRlcm1pbmF0ZWQgaXQg YnkgcHJlc3NpbmcgQ1RSTCtEIG9uIHRoZSBzZXJ2ZXIgc2lkZS4gVGhpcyB0aW1lIHNlcnZlciBk aWQgc2VuZCBhIEZJTiBwYWNrZXQsIHRoZSBjbGllbnQgZ290IGl0LCByZXBsaWVkIHdpdGggaXRz IG93biBGSU4gYW5kIHRlcm1pbmF0ZWQuIFRoZSBzZXJ2ZXIgZGlkIG5vdCByZWNlaXZlIHRoZSBG SU4gZnJvbSB0aGUgY2xpZW50IGFuZCBjb250aW51ZWQgcnVubmluZy48YnI+PGJyPlRoaXMgbWFr ZXMgbWUgdGhpbmsgdGhhdCBkdG50dW5uZWwsIHRoYXQgaGF2ZSByZWNlaXZlZCB0aGUgZmluYWwg RklOIGZyb20gYSBUQ1AgYXBwbGljYXRpb24sIGRvZXNuJ3QgdHJhbnNtaXQgdGhpcyBGSU4gdG8g aXRzIHJlbW90ZSBkdG50dW5uZWwgcGVlci4gT3IgdGhlIGxhdHRlciBvbmUgZG9lc24ndCByZXRy YW5zbWl0IHRoaXMgZmluYWwgRklOIHRvIHRoZSBUQ1AgYXBwbGljYXRpb24gb24gaXRzIHNpZGUu IEknbSBub3QgYSBUQ1AgYW5kIHNvY2tldHMgZ3VydSwgc28gSSBjYW5ub3QgZmlndXJlIG91dCBo b3cgdG8gdHJhbnNsYXRlIHRoaXMgdG8gdGhlIGxhbmd1YWdlIG9mIHNvY2tldHMsIHdoaWNoIEkg dGhpbmsgaXMgaW1wb3J0YW50IHRvIGRvIGluIG9yZGVyIHRvIHVuZGVyc3RhbmQgaG93IHRvIGZp eCBkdG50dW5uZWwuPGJyPjxicj5JIGRvbid0IHRoaW5rIHRoYXQgaW1wbGVtZW50aW5nIGFuIG9w dGlvbiB0aGF0IGNvbmZpZ3VyZXMgZHRudHVubmVsIGJlaGF2aW9yIGluIHJlbGF0aW9uIHRvIGNs b3NpbmcgY29ubmVjdGlvbnMgaXMgYSBnb29kIGlkZWEuIEkgYmVsaWV2ZSB0aGF0IHRoZXJlIGlz IGEgZmxhdyBpbiBkdG50dW5uZWwgbG9naWMsIGFuZCBpdCBjb3VsZCBiZSBmaXhlZCBpbiBhIHdh eSB0aGF0IGR0bnR1bm5lbCBkZXRlcm1pbmVzIGl0c2VsZiB3aGVuIHRvIGNsb3NlIGEgY29ubmVj dGlvbiB0byBhIFRDUCBhcHBsaWNhdGlvbi48bzpwPjwvbzpwPjwvcD48ZGl2PjxwIGNsYXNzPU1z b05vcm1hbD4yMDEyLzEyLzEzIFpvbGxlciwgRGF2aWQgQS4gKE1TRkMtRU82MClbSE9TQyBTRVJW SUNFUyBDT05UUkFDVF0gJmx0OzxhIGhyZWY9Im1haWx0bzpkYXZpZC5hLnpvbGxlckBuYXNhLmdv diIgdGFyZ2V0PSJfYmxhbmsiPmRhdmlkLmEuem9sbGVyQG5hc2EuZ292PC9hPiZndDs8bzpwPjwv bzpwPjwvcD48ZGl2PjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRv cC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQt c2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0 OTdEJz5IaSBTZXJnZXksPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBz dHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8n PjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fu cy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+UmVzZWFyY2ggaW5kaWNhdGVzIHRoYXQgdGhlIFRJTUVf V0FJVCBzdGF0ZSBpcyBhIGZlYXR1cmUgb2YgVENQIHRvIHByZXZlbnQgZGVsYXllZCBwYWNrZXRz IGZvciBjb25uZWN0aW9uIGZyb20gYmVpbmcgZGVsaXZlcmVkIHRvIGEgbGF0ZXIgY29ubmVjdGlv biBiZXR3ZWVuIHRoZSBzYW1lIGhvc3RzLiBbPGEgaHJlZj0iaHR0cDovL3Rvb2xzLmlldGYub3Jn L2h0bWwvZHJhZnQtZmFiZXItdGltZS13YWl0LWF2b2lkYW5jZS0wMCIgdGFyZ2V0PSJfYmxhbmsi Pmh0dHA6Ly90b29scy5pZXRmLm9yZy9odG1sL2RyYWZ0LWZhYmVyLXRpbWUtd2FpdC1hdm9pZGFu Y2UtMDA8L2E+XTwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9 J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3Bh biBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2Vy aWYiO2NvbG9yOiMxRjQ5N0QnPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1N c29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9t LWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2Fs aWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkJhc2VkIG9uIHRoYXQsIHRoZSBhZGRl ZCAmIzgyMjA7aWYgKHNvY2tfZW9mKSYjODIyMTsgYmxvY2sgY2FuIGJlIHJlbW92ZWQgYW5kIHRo ZSB3aGlsZSBjb25kaXRpb24gY2FuIGJlIGNoYW5nZWQgdG8gJiM4MjIwO3doaWxlICghc29ja19l b2YpJiM4MjIxOyBhbmQgeW91IGdldCB0aGUgc2FtZSByZXN1bHQuPC9zcGFuPjxvOnA+PC9vOnA+ PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNv LW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2Zv bnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+Jm5ic3A7PC9z cGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10 b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250 LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFG NDk3RCc+QnV0LCBsb29raW5nIGF0IHRoZSBpbnRlbnQgb2YgdGhlIGNvZGUgYW5kIGF0IHRoZSBv YXN5czo6SVBTb2NrZXQsIGl0IGFwcGVhcnMgdGhhdCBpdCBpcyBwb3NzaWJsZSB0byBlc3RhYmxp c2ggYSBzb2NrZXQgY29ubmVjdGlvbiBhbmQgdGhlbiBvbmx5IGNsb3NlIHRoZSByZWFkIHNpZGUg d2hpbGUga2VlcGluZyB0aGUgd3JpdGUgc2lkZSBvcGVuLiBJJiM4MjE3O3ZlIG5ldmVyIHJ1biBh Y3Jvc3MgYSBzb2NrZXQgdXNlZCBpbiB0aGF0IG1hbm5lciBidXQgdGhhdCBpcyB3aGF0IGl0IGxv b2tzIGxpa2UgdG8gbWUuIFNvLCB0aGUgVENQVHVubmVsOjpDb25uZWN0aW9uOjpydW4gJm5ic3A7 d2hpbGUgbG9vcCBhbGxvd3MgZm9yIHRoZSByZWFkIHNpZGUgdG8gY2xvc2UgYW5kIGNvbnRpbnVl IHRyYW5zbWl0dGluZyBidW5kbGUgcGF5bG9hZHMgb24gdGhlIHdyaXRlIHNpZGUgdW50aWwgaXQg Z2V0cyBhIGJ1bmRsZSB3aXRoIHRoZSBFT0YgYml0IHNldC4gSWYgdGhlcmUgYXJlIG5vIGJ1bmRs ZXMgY29taW5nIGJhY2sgdGhlIG90aGVyIHdheSBhcyBpcyB0aGUgY2FzZSBoZXJlIHRoZW4gdGhl IHdoaWxlIGxvb3AgbmV2ZXIgZXhpdHMgd2hpY2gga2VlcHMgdGhlIG5ldGNhdCBjb25uZWN0aW9u IG9wZW4uPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNv LW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0 eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7 Y29sb3I6IzFGNDk3RCc+Jm5ic3A7PC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05v cm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0 OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJp Iiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+U28sIGJhc2VkIG9uIHRoYXQsIEkgdGhpbmsg YSBiZXR0ZXIgc29sdXRpb24gd291bGQgYmUgdG8gaW1wbGVtZW50IGFuIG9wdGlvbiB0aGF0IGNv bmZpZ3VyZXMgZHRudHVubmVsIHRvIGNsb3NlIHRoZSBjb25uZWN0aW9uIHdoZW4gdGhlIHJlYWQg c2lkZSB0ZXJtaW5hdGVzLiAoU29tZW9uZSBpbiB0aGUga25vdyBjYW4gd2VpZ2ggaW4pIFlvdXIg c29sdXRpb24gaXMgcHJvYmFibHkgc3VmZmljaWVudCBmb3IgeW91ciB0ZXN0Ljwvc3Bhbj48bzpw PjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDph dXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEx LjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPiZu YnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1t YXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHls ZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2Nv bG9yOiMxRjQ5N0QnPi0tPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBz dHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8n PjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fu cy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+Jm5ic3A7PC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNs YXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1i b3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5 OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+QXMgdG8gY29ubmVjdGlvbnMg bm90IGJlaW5nIHJlbW92ZWQgZnJvbSB0aGUgdGFibGUgYW5kIHNvY2tldHMgbm90IGJlaW5nIGNs b3NlZCYjODIzMDs8L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxl PSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNw YW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNl cmlmIjtjb2xvcjojMUY0OTdEJz4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9 TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv bS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNh bGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5JIHNldCB1cCBhIHRlc3QgdG8ga2lj ayBvZmYgMTAwIG5ldGNhdHMgb24gYW4gMTFNQiBmaWxlIHNvIHRoYXQgdGhlIGNvbm5lY3Rpb24g d291bGQgYmUgb3BlbiBmb3Igc2V2ZXJhbCBzZWNvbmRzOjwvc3Bhbj48bzpwPjwvbzpwPjwvcD48 cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJn aW4tYm90dG9tLWFsdDphdXRvO3RleHQtaW5kZW50Oi41aW4nPjxzcGFuIHN0eWxlPSdmb250LXNp emU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3 RCc+IyEvYmluL2Jhc2g8L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0 eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzt0 ZXh0LWluZGVudDouNWluJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWls eToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPmN0cj0wPC9zcGFuPjxvOnA+ PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1 dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87dGV4dC1pbmRlbnQ6LjVpbic+PHNwYW4gc3R5 bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtj b2xvcjojMUY0OTdEJz53aGlsZSBbICRjdHIgLWx0IDEwMCBdOyBkbzwvc3Bhbj48bzpwPjwvbzpw PjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21z by1tYXJnaW4tYm90dG9tLWFsdDphdXRvO3RleHQtaW5kZW50Oi41aW4nPjxzcGFuIHN0eWxlPSdm b250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6 IzFGNDk3RCc+Jm5ic3A7Jm5ic3A7Jm5ic3A7IG5jIC13IDEwIHgueC54LnggMTIzNDUgJmx0OyBk YXRhMTFtZyZuYnNwOyAmYW1wOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3Jt YWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDph dXRvO3RleHQtaW5kZW50Oi41aW4nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQt ZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+Jm5ic3A7Jm5ic3A7 Jm5ic3A7IGxldCBjdHI9Y3RyKzE8L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9y bWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6 YXV0bzt0ZXh0LWluZGVudDouNWluJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250 LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPmRvbmU8L3NwYW4+ PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1h bHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6 ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdE Jz4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdt c28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4g c3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlm Ijtjb2xvcjojMUY0OTdEJz5JIGFsc28gYWRkZWQgc29tZSBkZWJ1ZyBjb2RlIHRvIHByaW50IG91 dCBhIGNvbm5lY3Rpb24gYWNjZXB0ZWQgY291bnRlciBhbmQgY2xvc2VkIGNvdW50ZXIgYXMgdGhl eSBhcmUgYWNjZXB0ZWQgYW5kIGNsb3NlZC48L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9 TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv bS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNh bGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5BZnRlciAzMCBjb25uZWN0aW9ucywg dGhlIGZpcnN0IGNsb3NlIGtpY2tzIGluIGFuZCB0aGVuIHRoZXJlIGlzIGEgbWl4IG9mIGNvbm5l Y3RzIGFuZCBjbG9zZXMgdW50aWwgZmluaXNoZWQuPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNs YXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1i b3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5 OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+QWxsIG9mIHRoZSBjb25uZWN0 aW9ucyB3ZXJlIGNsb3NlZCBhbmQgSSBkaWQgbm90IGhhdmUgYW55IHNvY2tldHMgbGVmdCBvcGVu Ljwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJn aW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0n Zm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9y OiMxRjQ5N0QnPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwg c3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRv Jz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNh bnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPk9uIG15IGZpcnN0IGF0dGVtcHQgSSBleGNlZWRlZCBt eSBwYXlsb2FkIHF1b3RhIGFuZCBzdGFydGVkIHNlZWluZyAmIzgyMjA7ZXJyb3Igc2VuZGluZyBi dW5kbGU6IDE0MSYjODIyMTsgbWVzc2FnZXMgc28gSSBkb24mIzgyMTc7dCB0aGluayB0aGF0IGlz IHRoZSBpc3N1ZSB5b3UgYXJlIHJ1bm5pbmcgaW50by48L3NwYW4+PG86cD48L286cD48L3A+PHAg Y2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2lu LWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1p bHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz4mbmJzcDs8L3NwYW4+PG86 cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6 YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZTox MS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5Z b3UgbWlnaHQgdHJ5IHB1dHRpbmcgaW4gc2ltaWxhciBkZWJ1ZyB0byBzZWUgaWYgeW91IGNhbiBk ZXRlcm1pbmUgd2hhdCBpcyBoYXBwZW5pbmcuPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNz PU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0 b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJD YWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+Jm5ic3A7PC9zcGFuPjxvOnA+PC9v OnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87 bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0 O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+SG9wZSB0 aGlzIGhlbHBzLDwvc3Bhbj48bzpwPjwvbzpwPjwvcD48ZGl2PjxwIGNsYXNzPU1zb05vcm1hbCBz dHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8n PjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fu cy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+RFo8L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9 TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv bS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNh bGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz4mbmJzcDs8L3NwYW4+PG86cD48L286 cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bztt c28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7 Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz4mbmJzcDs8 L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2lu LXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2Zv bnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkFyaWFsIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFG NDk3RCc+RGF2aWQgWm9sbGVyPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1h bCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1 dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJBcmlhbCIsInNh bnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkNPTFNBIENvcnBvcmF0aW9uPC9zcGFuPjxvOnA+PC9v OnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87 bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTAuMHB0 O2ZvbnQtZmFtaWx5OiJBcmlhbCIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkhPU0MgLyBD MTA3Jm5ic3A7IDwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9 J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3Bh biBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTpXaW5nZGluZ3M7Y29sb3I6Izk5 MDAwMCc+KDwvc3Bhbj48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToi QXJpYWwiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5PZmZpY2U6PC9zcGFuPjxzcGFuIHN0 eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7 Y29sb3I6IzFGNDk3RCc+IDwvc3Bhbj48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250 LWZhbWlseToiQXJpYWwiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz4oMjU2KSZuYnNwOzU0 NC0xODIwPC9zcGFuPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJB cmlhbCIsInNhbnMtc2VyaWYiO2NvbG9yOm5hdnknPjxicj48L3NwYW4+PHNwYW4gc3R5bGU9J2Zv bnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6V2luZ2RpbmdzO2NvbG9yOiMzMzY2RkYnPio8L3Nw YW4+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkFyaWFsIiwic2Fu cy1zZXJpZiInPkVNYWlsOiA8c3BhbiBzdHlsZT0nY29sb3I6bmF2eSc+PGEgaHJlZj0ibWFpbHRv OmRhdmlkLmEuem9sbGVyQG5hc2EuZ292IiB0YXJnZXQ9Il9ibGFuayI+ZGF2aWQuYS56b2xsZXJA bmFzYS5nb3Y8L2E+PC9zcGFuPjwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3Jt YWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDph dXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIs InNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD48 L2Rpdj48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21z by1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48Yj48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEwLjBw dDtmb250LWZhbWlseToiVGFob21hIiwic2Fucy1zZXJpZiInPkZyb206PC9zcGFuPjwvYj48c3Bh biBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiVGFob21hIiwic2Fucy1zZXJp ZiInPiA8YSBocmVmPSJtYWlsdG86c3NpcmVza2luQGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsi PnNzaXJlc2tpbkBnbWFpbC5jb208L2E+IFttYWlsdG86PGEgaHJlZj0ibWFpbHRvOnNzaXJlc2tp bkBnbWFpbC5jb20iIHRhcmdldD0iX2JsYW5rIj5zc2lyZXNraW5AZ21haWwuY29tPC9hPl0gPGJy PjxiPlNlbnQ6PC9iPiBXZWRuZXNkYXksIERlY2VtYmVyIDEyLCAyMDEyIDI6MDcgUE08L3NwYW4+ PG86cD48L286cD48L3A+PGRpdj48ZGl2PjxwIGNsYXNzPU1zb05vcm1hbD48YnI+PGI+VG86PC9i PiBab2xsZXIsIERhdmlkIEEuIChNU0ZDLUVPNjApW0hPU0MgU0VSVklDRVMgQ09OVFJBQ1RdPGJy PjxiPkNjOjwvYj4gPGEgaHJlZj0ibWFpbHRvOmR0bi11c2Vyc0BpcnRmLm9yZyIgdGFyZ2V0PSJf YmxhbmsiPmR0bi11c2Vyc0BpcnRmLm9yZzwvYT48YnI+PGI+U3ViamVjdDo8L2I+IFJlOiBbZHRu LXVzZXJzXSBBIHByb2JsZW0gd2l0aCBkdG50dW5uZWw8bzpwPjwvbzpwPjwvcD48L2Rpdj48L2Rp dj48ZGl2PjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6 YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+Jm5ic3A7PG86cD48L286cD48L3A+PHAg Y2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttYXJnaW4tYm90 dG9tOjEyLjBwdCc+SGkgRGF2aWQsPGJyPjxicj5JIG1lYW50IGV4YWN0bHkgdGhlIHNhbWUgcG9z aXRpb24gYmVmb3JlIHRoZSBlbmQgb2YgdGhlIHdoaWxlIGxvb3AsIGp1c3QgZGVzY3JpYmVkIGl0 IGluY29ycmVjdC48YnI+PGJyPlBsZWFzZSBsZXQgbWUga25vdyBpZiB5b3UgZmluZCB0aGUgc29s dXRpb24gZm9yIHRoaXMgcHJvYmxlbS48YnI+PGJyPkJlc3QgcmVnYXJkcyw8YnI+U2VyZ2V5PG86 cD48L286cD48L3A+PGRpdj48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9w LWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz4yMDEyLzEyLzEyIFpvbGxlciwg RGF2aWQgQS4gKE1TRkMtRU82MClbSE9TQyBTRVJWSUNFUyBDT05UUkFDVF0gJmx0OzxhIGhyZWY9 Im1haWx0bzpkYXZpZC5hLnpvbGxlckBuYXNhLmdvdiIgdGFyZ2V0PSJfYmxhbmsiPmRhdmlkLmEu em9sbGVyQG5hc2EuZ292PC9hPiZndDs8bzpwPjwvbzpwPjwvcD48ZGl2PjxkaXY+PHAgY2xhc3M9 TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv bS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNh bGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5IaSBTZXJnZXksPC9zcGFuPjxvOnA+ PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1 dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEu MHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+SSB3 YXMgbmFycm93aW5nIGluIG9uIHRoZSBzYW1lIHRyYWNrLi4uIFRoZSBGSU4gcmVzdWx0cyBpbiBh IHJldHVybmVkIGxlbmd0aCBvZiB6ZXJvIG9uIHRoZSByZWFkIGFuZCBlcnJubyBpcyBhbHNvIHpl cm8gc28geW91IGhhdmUgdG8ga2V5IG9uIHRoZSByZXR1cm4gbGVuZ3RoIHdoaWNoIGlzIGFscmVh ZHkgaW4gdGhlIGNvZGUuPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBz dHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8n PjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fu cy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+Jm5ic3A7PC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNs YXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1i b3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5 OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+WW91IG1heSBoYXZlIGFkZGVk IGEgY291cGxlIG9mIGxpbmVzIG9mIGRlYnVnIGNvZGUgYW5kIGl0IHNob3VsZCBiZSBpbnNlcnRl ZCBqdXN0IGJlZm9yZSB0aGUgZW5kIG9mIHRoZSB3aGlsZSgxKSBsb29wIGF0IGxpbmUgNDkxIGlu IHRoZSBsYXRlc3QgdmVyc2lvbiBvbiBTb3VyY2Vmb3JnZS48L3NwYW4+PG86cD48L286cD48L3A+ PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFy Z2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1m YW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz4mbmJzcDs8L3NwYW4+ PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1h bHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6 ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdE Jz5JIGFtIHNlZWluZyB0aGUgc2VuZGluZyBzb2NrZXQgc3RheSBpbiB0aGUgVElNRV9XQUlUIG1v ZGUgZXZlbiBpZiBJIGFkZCBhIGNhbGwgdG8gc29ja18uc2h1dGRvd24oU0hVVF9SRFdSKSBiZWZv cmUgY2xvc2luZyB0aGUgc29ja2V0Ozwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29O b3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFs dDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJy aSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkdldHRpbmcgY2xvc2UgSSB0aGluay48L3Nw YW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRv cC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQt c2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0 OTdEJz4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxl PSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNw YW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNl cmlmIjtjb2xvcjojMUY0OTdEJz5UaGUgMC4wLjAuMCB3YXMgYmVjYXVzZSBJIGRpZCBub3QgaGF2 ZSBhbnl0aGluZyBhdCB0aGUgb3RoZXIgZW5kIG9mIHRoZSB0dW5uZWwgYW5kIGp1c3Qgc3R1Y2sg c29tZSBlYXN5IG51bWJlcnMgaW4gdGhlcmUuPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNz PU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0 b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJD YWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+VGhhbmtzLDxicj5EWjwvc3Bhbj48 bzpwPjwvbzpwPjwvcD48ZGl2PjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10 b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250 LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFG NDk3RCc+Jm5ic3A7PC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHls ZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxz cGFuIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJBcmlhbCIsInNhbnMtc2Vy aWYiO2NvbG9yOiMxRjQ5N0QnPkRhdmlkIFpvbGxlcjwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBj bGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4t Ym90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWls eToiQXJpYWwiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5DT0xTQSBDb3Jwb3JhdGlvbjwv c3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4t dG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9u dC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiQXJpYWwiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0 OTdEJz5NYXJzaGFsbCBTcGFjZSBGbGlnaHQgQ2VudGVyPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxw IGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdp bi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFt aWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+Jm5ic3A7PC9zcGFuPjxv OnA+PC9vOnA+PC9wPjwvZGl2PjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10 b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxiPjxzcGFuIHN0eWxlPSdm b250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJUYWhvbWEiLCJzYW5zLXNlcmlmIic+RnJvbTo8 L3NwYW4+PC9iPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJUYWhv bWEiLCJzYW5zLXNlcmlmIic+IDxhIGhyZWY9Im1haWx0bzpzc2lyZXNraW5AZ21haWwuY29tIiB0 YXJnZXQ9Il9ibGFuayI+c3NpcmVza2luQGdtYWlsLmNvbTwvYT4gW21haWx0bzo8YSBocmVmPSJt YWlsdG86c3NpcmVza2luQGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsiPnNzaXJlc2tpbkBnbWFp bC5jb208L2E+XSA8YnI+PGI+U2VudDo8L2I+IFdlZG5lc2RheSwgRGVjZW1iZXIgMTIsIDIwMTIg MTE6MDYgQU08L3NwYW4+PG86cD48L286cD48L3A+PGRpdj48ZGl2PjxwIGNsYXNzPU1zb05vcm1h bCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1 dG8nPjxicj48Yj5Ubzo8L2I+IFpvbGxlciwgRGF2aWQgQS4gKE1TRkMtRU82MClbSE9TQyBTRVJW SUNFUyBDT05UUkFDVF08YnI+PGI+Q2M6PC9iPiA8YSBocmVmPSJtYWlsdG86ZHRuLXVzZXJzQGly dGYub3JnIiB0YXJnZXQ9Il9ibGFuayI+ZHRuLXVzZXJzQGlydGYub3JnPC9hPjxicj48Yj5TdWJq ZWN0OjwvYj4gUmU6IFtkdG4tdXNlcnNdIEEgcHJvYmxlbSB3aXRoIGR0bnR1bm5lbDxvOnA+PC9v OnA+PC9wPjwvZGl2PjwvZGl2PjxkaXY+PGRpdj48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21z by1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz4mbmJzcDs8 bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFs dDphdXRvO21hcmdpbi1ib3R0b206MTIuMHB0Jz5IaSBEYXZpZCw8YnI+PGJyPkkgYWRkZWQgdGhl IGZvbGxvd2luZyBjb2RlIGF0IHRoZSBsaW5lIDQ5MiBvZiBUQ1BUdW5uZWwuY2MgYW5kIGl0IHNl ZW1zIHRoYXQgaXQgaGVscGVkLiA8YnI+PGJyPiZuYnNwOyZuYnNwOyZuYnNwOyBpZiAoc29ja19l b2YpIHs8YnI+Jm5ic3A7Jm5ic3A7Jm5ic3A7ICZuYnNwOyZuYnNwOyZuYnNwOyBzb2NrXy5jbG9z ZSgpOzxicj4mbmJzcDsmbmJzcDsmbmJzcDsgJm5ic3A7Jm5ic3A7Jm5ic3A7IGdvdG8gZG9uZTs8 YnI+Jm5ic3A7Jm5ic3A7Jm5ic3A7IH08YnI+PGJyPlRoZSBzZW5kZXJzLCBib3RoIG5jIGFuZCBp cGVyZiwgZXhpdCBhZnRlciBoYXZpbmcgZmluaXNoZWQgc2VuZGluZyB0aGUgZGF0YS4gVGhlcmUg YXJlPGJyPm5vIG1vcmUgc29ja2V0cyBpbiBDTE9TRV9XQUlUIHN0YXRlLiBIb3dldmVyIEkgYW0g Y29tcGxldGVseSB1bnN1cmUgd2hldGhlciA8YnI+bXkgY29kZSBpcyBjb3JyZWN0IGFuZCBpcyBp biBjb3JyZWN0IHBsYWNlLjxicj48YnI+Tm93IEkgaGF2ZSBlbmNvdW50ZXJlZCBhbm90aGVyIHBy b2JsZW0uIFdoZW4gaXBlcmYgdGVzdCBpcyBydW4gd2l0aCAxMDAgcGFyYWxsZWwgZmxvd3MsPGJy PnNvbWUgZHRudHVubmVsIGNvbm5lY3Rpb25zIGFyZSBub3QgcmVtb3ZlZCBmcm9tIGR0bnR1bmVs J3MgY29ubmVjdGlvbiB0YWJsZSwgYW5kIHRoZTxicj5yZWxhdGVkIHNvY2tldHMgYXJlIG5vdCBj bG9zZWQuIEkgc3VwcG9zZSB0aGlzIGhhcHBlbnMgYmVjYXVzZSBteSBjb2RlIGFkZGl0aW9uIGlz IGZhcjxicj5mcm9tIHBlcmZlY3QuPGJyPjxicj5BbmQgb25lIG1vcmUgcXVlc3Rpb24sIHdoYXQg ZG9lcyAwLjAuMC4wIG1lYW4gaW4gLVQgdHVubmVsIHNwZWNpZmljYXRpb24/IElzIGl0IGp1c3Qg YW48YnI+ZXhhbXBsZSBob3N0LCBvciAwLjAuMC4wIGhhcyBzb21lIHNwZWNpYWwgbWVhbmluZyBm b3IgZHRudHVubmVsPzxvOnA+PC9vOnA+PC9wPjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxl PSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+MjAx Mi8xMi8xMiBab2xsZXIsIERhdmlkIEEuIChNU0ZDLUVPNjApW0hPU0MgU0VSVklDRVMgQ09OVFJB Q1RdICZsdDs8YSBocmVmPSJtYWlsdG86ZGF2aWQuYS56b2xsZXJAbmFzYS5nb3YiIHRhcmdldD0i X2JsYW5rIj5kYXZpZC5hLnpvbGxlckBuYXNhLmdvdjwvYT4mZ3Q7PG86cD48L286cD48L3A+PGRp dj48ZGl2PjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87 bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0 O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+SGkgU2Vy Z2V5LDwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1t YXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHls ZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2Nv bG9yOiMxRjQ5N0QnPkkgaGF2ZSBkdXBsaWNhdGVkIHdoYXQgeW91IGFyZSByZXBvcnRpbmcgYW5k IGhhdmUgdG8gcmV0cmFjdCBteSBlYXJsaWVyIHN0YXRlbWVudCBzYXlpbmcgdGhhdCB0aGlzIGlz IHRoZSBuZXRjYXQgZGVzaWduIGFuZCBub3QgYSBwcm9ibGVtIDwvc3Bhbj48c3BhbiBzdHlsZT0n Zm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTpXaW5nZGluZ3M7Y29sb3I6IzFGNDk3RCc+Sjwv c3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4t dG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9u dC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMx RjQ5N0QnPlRoZSBzZW5kZXIgbmV0Y2F0IGF0dGVtcHRzIHRvIGNsb3NlIHRoZSBzb2NrZXQgd2hl biBpdCBnZXRzIHRoZSBbRU9GXSBhbmQgbm90IGluIHJlc3BvbnNlIHRvIHRoZSByZWNlaXZlciBj bG9zaW5nIGl0cyBlbmQuPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBz dHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8n PjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fu cy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+Jm5ic3A7PC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNs YXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1i b3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5 OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+Jm5ic3A7PC9zcGFuPjxvOnA+ PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1 dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxiPjxzcGFuIHN0eWxlPSdmb250LXNpemU6 MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+ bmV0Y2F0IHNlbmRlcjo8L3NwYW4+PC9iPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1h bCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1 dG87dGV4dC1pbmRlbnQ6LjVpbic+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1m YW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5jYXQgL2V0Yy9wYXNz d2QgfCBuYyB4LngueC54IDEyMzQ1PC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05v cm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0 OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJp Iiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+Jm5ic3A7PC9zcGFuPjxvOnA+PC9vOnA+PC9w PjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1h cmdpbi1ib3R0b20tYWx0OmF1dG8nPjxiPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2Zv bnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+c2NlbmFyaW8g MSAtIG5ldGNhdCByZWNlaXZlciB3aXRoIHRoZSBrZWVwIGxpc3RlbmVyIG9wdGlvbjo8L3NwYW4+ PC9iPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10 b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250 LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFG NDk3RCc+Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IG5jIC1rIC1sIDEyMzQ1PC9z cGFuPjxvOnA+PC9vOnA+PC9wPjxwIHN0eWxlPSdtYXJnaW4tbGVmdDouNzVpbic+PHNwYW4gc3R5 bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6U3ltYm9sO2NvbG9yOiMxRjQ5N0QnPiZt aWRkb3Q7PC9zcGFuPjxzcGFuIHN0eWxlPSdmb250LXNpemU6Ny4wcHQ7Y29sb3I6IzFGNDk3RCc+ Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IDwvc3Bhbj48 c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMt c2VyaWYiO2NvbG9yOiMxRjQ5N0QnPlNlbmRlciB0ZXJtaW5hdGVzIGFuZCByZWNlaXZlciBzdGF5 cyBhbGl2ZSBhbmQgbGlzdGVuaW5nIGZvciBhIHJlLWNvbm5lY3Q8L3NwYW4+PG86cD48L286cD48 L3A+PHAgc3R5bGU9J21hcmdpbi1sZWZ0Oi43NWluJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEx LjBwdDtmb250LWZhbWlseTpTeW1ib2w7Y29sb3I6IzFGNDk3RCc+Jm1pZGRvdDs8L3NwYW4+PHNw YW4gc3R5bGU9J2ZvbnQtc2l6ZTo3LjBwdDtjb2xvcjojMUY0OTdEJz4mbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsgPC9zcGFuPjxzcGFuIHN0eWxlPSdmb250 LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFG NDk3RCc+RklOIGlzIHNlbnQgaW4gYm90aCBkaXJlY3Rpb25zIGluaXRpYXRlZCBieSB0aGUgc2Vu ZGVyPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1h cmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxl PSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29s b3I6IzFGNDk3RCc+Jm5ic3A7PC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1h bCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1 dG8nPjxiPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJp Iiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+c2NlbmFyaW8gMiAtIGR0bnR1bm5lbCByZWNl aXZlcjo8L3NwYW4+PC9iPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0n bXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFu IHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJp ZiI7Y29sb3I6IzFGNDk3RCc+Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IGR0bnR1 bm5lbCAtdCAtVCAxMjM0NTo8YSBocmVmPSJodHRwOi8vMC4wLjAuMDo1NDMyMSIgdGFyZ2V0PSJf YmxhbmsiPjAuMC4wLjA6NTQzMjE8L2E+IGR0bjovL2Rlc3RlaWQveHh4eDwvc3Bhbj48bzpwPjwv bzpwPjwvcD48cCBzdHlsZT0nbWFyZ2luLWxlZnQ6Ljc1aW4nPjxzcGFuIHN0eWxlPSdmb250LXNp emU6MTEuMHB0O2ZvbnQtZmFtaWx5OlN5bWJvbDtjb2xvcjojMUY0OTdEJz4mbWlkZG90Ozwvc3Bh bj48c3BhbiBzdHlsZT0nZm9udC1zaXplOjcuMHB0O2NvbG9yOiMxRjQ5N0QnPiZuYnNwOyZuYnNw OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyA8L3NwYW4+PHNwYW4gc3R5bGU9 J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xv cjojMUY0OTdEJz5TZW5kZXIgZG9lcyBub3QgdGVybWluYXRlPC9zcGFuPjxvOnA+PC9vOnA+PC9w PjxwIHN0eWxlPSdtYXJnaW4tbGVmdDouNzVpbic+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4w cHQ7Zm9udC1mYW1pbHk6U3ltYm9sO2NvbG9yOiMxRjQ5N0QnPiZtaWRkb3Q7PC9zcGFuPjxzcGFu IHN0eWxlPSdmb250LXNpemU6Ny4wcHQ7Y29sb3I6IzFGNDk3RCc+Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IDwvc3Bhbj48c3BhbiBzdHlsZT0nZm9udC1z aXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5 N0QnPkZJTiBpcyBzZW50IGZyb20gc2VuZGVyIHRvIHJlY2VpdmVyIGJ1dCBub3QgdGhlIG90aGVy IHdheTwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBzdHlsZT0nbWFyZ2luLWxlZnQ6Ljc1aW4nPjxz cGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OlN5bWJvbDtjb2xvcjojMUY0 OTdEJz4mbWlkZG90Ozwvc3Bhbj48c3BhbiBzdHlsZT0nZm9udC1zaXplOjcuMHB0O2NvbG9yOiMx RjQ5N0QnPiZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyA8 L3NwYW4+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmki LCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5TZW5kZXIgc29ja2V0IGlzIGluIHN0YXRlIEZJ Tl9XQUlUMjwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBzdHlsZT0nbWFyZ2luLWxlZnQ6Ljc1aW4n PjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OlN5bWJvbDtjb2xvcjoj MUY0OTdEJz4mbWlkZG90Ozwvc3Bhbj48c3BhbiBzdHlsZT0nZm9udC1zaXplOjcuMHB0O2NvbG9y OiMxRjQ5N0QnPiZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw OyA8L3NwYW4+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGli cmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5SZWNlaXZlciBzb2NrZXQgaXMgaW4gc3Rh dGUgQ0xPU0VfV0FJVDwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5 bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvO3Rl eHQtaW5kZW50Oi41aW4nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5 OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+S2lsbCB0aGUgc2VuZGVyIGFu ZCByZXBlYXQuLi48L3NwYW4+PG86cD48L286cD48L3A+PHAgc3R5bGU9J21hcmdpbi1sZWZ0Oi43 NWluJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTpTeW1ib2w7Y29s b3I6IzFGNDk3RCc+Jm1pZGRvdDs8L3NwYW4+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZTo3LjBwdDtj b2xvcjojMUY0OTdEJz4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsgPC9zcGFuPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJD YWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+U2VuZGVyIGRvZXMgbm90IHRlcm1p bmF0ZTwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBzdHlsZT0nbWFyZ2luLWxlZnQ6Ljc1aW4nPjxz cGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OlN5bWJvbDtjb2xvcjojMUY0 OTdEJz4mbWlkZG90Ozwvc3Bhbj48c3BhbiBzdHlsZT0nZm9udC1zaXplOjcuMHB0O2NvbG9yOiMx RjQ5N0QnPiZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyA8 L3NwYW4+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmki LCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5GSU4gaXMgc2VudCBmcm9tIHNlbmRlciB0byBy ZWNlaXZlciBidXQgbm90IHRoZSBvdGhlciB3YXk8L3NwYW4+PG86cD48L286cD48L3A+PHAgc3R5 bGU9J21hcmdpbi1sZWZ0Oi43NWluJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250 LWZhbWlseTpTeW1ib2w7Y29sb3I6IzFGNDk3RCc+Jm1pZGRvdDs8L3NwYW4+PHNwYW4gc3R5bGU9 J2ZvbnQtc2l6ZTo3LjBwdDtjb2xvcjojMUY0OTdEJz4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsgPC9zcGFuPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEu MHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+TmV3 IFNlbmRlciBzb2NrZXQgaXMgaW4gc3RhdGUgRklOX1dBSVQyIGFuZCB0aGUgZmlyc3Qgb25lIGhh cyB0aW1lZCBvdXQgYW5kIGRpZWQ8L3NwYW4+PG86cD48L286cD48L3A+PHAgc3R5bGU9J21hcmdp bi1sZWZ0Oi43NWluJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTpT eW1ib2w7Y29sb3I6IzFGNDk3RCc+Jm1pZGRvdDs8L3NwYW4+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6 ZTo3LjBwdDtjb2xvcjojMUY0OTdEJz4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsgPC9zcGFuPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQt ZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+Qm90aCBSZWNlaXZl ciBzb2NrZXRzIGFyZSBpbiBzdGF0ZSBDTE9TRV9XQUlUPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxw IGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdp bi1ib3R0b20tYWx0OmF1dG87dGV4dC1pbmRlbnQ6LjVpbic+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6 ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdE Jz5LaWxsIHRoZSBkdG50dW5uZWwgcmVjZWl2ZXImIzgyMzA7PC9zcGFuPjxvOnA+PC9vOnA+PC9w PjxwIHN0eWxlPSdtYXJnaW4tbGVmdDouNzVpbic+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4w cHQ7Zm9udC1mYW1pbHk6U3ltYm9sO2NvbG9yOiMxRjQ5N0QnPiZtaWRkb3Q7PC9zcGFuPjxzcGFu IHN0eWxlPSdmb250LXNpemU6Ny4wcHQ7Y29sb3I6IzFGNDk3RCc+Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IDwvc3Bhbj48c3BhbiBzdHlsZT0nZm9udC1z aXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5 N0QnPjIgRklOcyBhcmUgc2VudCBmcm9tIHRoZSByZWNlaXZlciAmIzgyMTE7IDEgdG8gZWFjaCBv ZiB0aGUgc29ja2V0cyAoZXZlbiB0aG91Z2ggdGhlIG90aGVyIGVuZHMgaGF2ZSBleHBpcmVkKTwv c3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4t dG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9u dC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMx RjQ5N0QnPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5 bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48 c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMt c2VyaWYiO2NvbG9yOiMxRjQ5N0QnPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFz cz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90 dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToi Q2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkkgYmVsaWV2ZSB0aGUgaXNzdWUg d291bGQgYmUgaW4gdGhlIG9hc3lzIElQU29ja2V0IG9yIHBvc3NpYmx5IHRoZSBJUENsaWVudC4g SSYjODIxNztsbCBoYXZlIGEgbG9vayBhdCBpdCB1bmxlc3MgdGhlcmUgaXMgYSBsb3ctbGV2ZWwg c29ja2V0IGV4cGVydCBvdXQgdGhlcmUgdGhhdCB3YW50cyB0byBnaXZlIGl0IGEgZ28mIzgyMzA7 PC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28t bWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5 bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtj b2xvcjojMUY0OTdEJz4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9y bWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6 YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmki LCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5CZXN0IHJlZ2FyZHMsPC9zcGFuPjxvOnA+PC9v OnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87 bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0 O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+RFo8L3Nw YW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRv cC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQt c2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0 OTdEJz4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxl PSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNw YW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkFyaWFsIiwic2Fucy1zZXJp ZiI7Y29sb3I6IzFGNDk3RCc+RGF2aWQgWm9sbGVyPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNs YXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1i b3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5 OiJBcmlhbCIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkNPTFNBIENvcnBvcmF0aW9uPC9z cGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10 b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250 LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJBcmlhbCIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5 N0QnPk1hcnNoYWxsIFNwYWNlIEZsaWdodCBDZW50ZXI8L3NwYW4+PG86cD48L286cD48L3A+PHAg Y2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2lu LWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1p bHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz4mbmJzcDs8L3NwYW4+PG86 cD48L286cD48L3A+PC9kaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRv cC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PGI+PHNwYW4gc3R5bGU9J2Zv bnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IlRhaG9tYSIsInNhbnMtc2VyaWYiJz5Gcm9tOjwv c3Bhbj48L2I+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IlRhaG9t YSIsInNhbnMtc2VyaWYiJz4gPGEgaHJlZj0ibWFpbHRvOnNzaXJlc2tpbkBnbWFpbC5jb20iIHRh cmdldD0iX2JsYW5rIj5zc2lyZXNraW5AZ21haWwuY29tPC9hPiBbbWFpbHRvOjxhIGhyZWY9Im1h aWx0bzpzc2lyZXNraW5AZ21haWwuY29tIiB0YXJnZXQ9Il9ibGFuayI+c3NpcmVza2luQGdtYWls LmNvbTwvYT5dIDxicj48Yj5TZW50OjwvYj4gV2VkbmVzZGF5LCBEZWNlbWJlciAxMiwgMjAxMiA0 OjM4IEFNPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxl PSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PGJy PjxiPlRvOjwvYj4gWm9sbGVyLCBEYXZpZCBBLiAoTVNGQy1FTzYwKVtIT1NDIFNFUlZJQ0VTIENP TlRSQUNUXTxvOnA+PC9vOnA+PC9wPjwvZGl2PjxkaXY+PGRpdj48cCBjbGFzcz1Nc29Ob3JtYWwg c3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRv Jz48Yj5TdWJqZWN0OjwvYj4gUmU6IFtkdG4tdXNlcnNdIEEgcHJvYmxlbSB3aXRoIGR0bnR1bm5l bDxvOnA+PC9vOnA+PC9wPjwvZGl2PjwvZGl2PjxkaXY+PGRpdj48cCBjbGFzcz1Nc29Ob3JtYWwg c3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRv Jz4mbmJzcDs8bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJn aW4tdG9wLWFsdDphdXRvO21hcmdpbi1ib3R0b206MTIuMHB0Jz5IaSBEYXZpZCw8YnI+PGJyPkkg aGF2ZSBpbnZlc3RpZ2F0ZWQgdGhhdCBpc3N1ZSBhIGxpdHRsZS4gSSBoYXZlIGZvdW5kLCB0aGF0 IG5vIG1hdHRlciB3aGV0aGVyIG5ldGNhdCBpcyBydW4gd2l0aCBvciB3aXRob3V0IHRoZSAtdzEg b3B0aW9uLCBzb21ldGhpbmcgPGJyPnN0cmFuZ2UgaGFwcGVucy4gQWNjb3JkaW5nIHRvIG5ldHN0 YXQgLW50cCBvbiB0aGUgc2VuZGluZyBub2RlLCBkdG50dW5uZWwgcmVtYWlucyBpbiBDTE9TRV9X QUlUIHN0YXRlIGFmdGVyIG5jIGZpbmlzaGVzIHNlbmRpbmcgPGJyPmRhdGEuIEkgaGF2ZSBydW4g dGNwZHVtcCBvbiBib3RoIHNlbmRlciBhbmQgcmVjZWl2ZXIsIGFuZCBpdCBzaG93ZWQgdGhhdCBm aW5hbCBGSU4gZnJvbSB0aGUgcmVjZWl2ZXIgZG9lc24ndCByZWFjaCB0aGUgc2VuZGVyLiA8YnI+ VGhpcyBsb29rZWQgbGlrZSBhIGJ1ZyBpbiBkdG50dW5uZWwgdG8gbWUuIEkgZGVjaWRlZCB0byBy ZS1jaGVjayB3aXRoIGlwZXJmLCBhbmQgYWdhaW4sIHRjcGR1bXAgc2hvd2VkIHRoZSBzYW1lIHBy b2JsZW0uPGJyPjxicj5IZXJlIGFyZSB0aGUgbGFzdCBsaW5lcyBvZiB0aGUgdGNwZHVtcC4gUmVt ZW1iZXIgdGhhdCBpbiBteSBzZXR1cCBhbGwgb3V0Z29pbmcgVENQIHBhY2tldHMgd2l0aCBkZXN0 aW5hdGlvbiBwb3J0IDk5OTkgYXJlIHJlZGlyZWN0ZWQgPGJyPndpdGggdGhlIGhlbHAgb2YgSVBU YWJsZXMgdG8gdGhlIGxvY2FsIGR0bnR1bm5lbCBwcm9jZXNzLCBsaXN0ZW5pbmcgb20gcG9ydCAx OTk5OS4gVGhhdCBpcyB3aHkgSSBydW4gdGNwZHVtcCBvbiBsb29wYmFjayBpbnRlcmZhY2UuPGJy Pjxicj5SZWNlaXZlciAoMTkyLjE2OC40LjEpOiB0Y3BkdW1wIC1ubiAtbnBpIGxvIHBvcnQgOTk5 OSBvciBwb3J0IDE5OTk5PGJyPjE1OjA1OjAyLjQ5NjM2NiBJUCAxOTIuMTY4LjQuMS4zNDI4NCAm Z3Q7IDE5Mi4xNjguNC4xLjk5OTk6IEZsYWdzIFtQLl0sIHNlcSAxMjM5OTU6MTMxMDk3LCBhY2sg MSwgd2luIDI1Nywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExNjM3MjMgZWNyIDExMTYzNzE4 XSwgbGVuZ3RoIDcxMDI8YnI+MTU6MDU6MDIuNDk2NDI4IElQIDE5Mi4xNjguNC4xLjk5OTkgJmd0 OyAxOTIuMTY4LjQuMS4zNDI4NDogRmxhZ3MgWy5dLCBhY2sgMTMxMDk3LCB3aW4gMTE1NCwgb3B0 aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExNjM3MjQgZWNyIDExMTYzNzIzXSwgbGVuZ3RoIDA8YnI+ MTU6MDU6MDMuNDU3NDIxIElQIDE5Mi4xNjguNC4xLjM0Mjg0ICZndDsgMTkyLjE2OC40LjEuOTk5 OTogRmxhZ3MgW0YuXSwgc2VxIDEzMTA5NywgYWNrIDEsIHdpbiAyNTcsIG9wdGlvbnMgW25vcCxu b3AsVFMgdmFsIDExMTY0Njg0IGVjciAxMTE2MzcyNF0sIGxlbmd0aCAwPGJyPjE1OjA1OjAzLjQ5 NzQ4MiBJUCAxOTIuMTY4LjQuMS45OTk5ICZndDsgMTkyLjE2OC40LjEuMzQyODQ6IEZsYWdzIFsu XSwgYWNrIDEzMTA5OCwgd2luIDExNTQsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTY0NzI1 IGVjciAxMTE2NDY4NF0sIGxlbmd0aCAwPGJyPjE1OjA1OjA0LjQ1Nzc2MiBJUCAxOTIuMTY4LjQu MS45OTk5ICZndDsgMTkyLjE2OC40LjEuMzQyODQ6IEZsYWdzIFtGLl0sIHNlcSAxLCBhY2sgMTMx MDk4LCB3aW4gMTE1NCwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExNjU2ODUgZWNyIDExMTY0 Njg0XSwgbGVuZ3RoIDA8YnI+MTU6MDU6MDQuNDU3ODU2IElQIDE5Mi4xNjguNC4xLjM0Mjg0ICZn dDsgMTkyLjE2OC40LjEuOTk5OTogRmxhZ3MgWy5dLCBhY2sgMiwgd2luIDI1Nywgb3B0aW9ucyBb bm9wLG5vcCxUUyB2YWwgMTExNjU2ODUgZWNyIDExMTY1Njg1XSwgbGVuZ3RoIDA8YnI+PGJyPlNl bmRlciAoMTkyLjE2OC4yLjEpOiB0Y3BkdW1wIC1ubiAtbnBpIGxvIHBvcnQgOTk5OSBvciBwb3J0 IDE5OTk5IDxicj4xNTowNTowMi40MzM3NTUgSVAgMTkyLjE2OC40LjEuOTk5OSAmZ3Q7IDE5Mi4x NjguMS4yLjQzMzYwOiBGbGFncyBbLl0sIGFjayAxMjc0NDksIHdpbiAzODYsIG9wdGlvbnMgW25v cCxub3AsVFMgdmFsIDExMTcyODg3IGVjciAxMTE3Mjg3NV0sIGxlbmd0aCAwPGJyPjE1OjA1OjAy LjQzMzc3NiBJUCAxOTIuMTY4LjEuMi40MzM2MCAmZ3Q7IDE5Mi4xNjguMS4yLjE5OTk5OiBGbGFn cyBbLl0sIHNlcSAxMjc0NDg6MTMwMzQ0LCBhY2sgMSwgd2luIDQ2LCBvcHRpb25zIFtub3Asbm9w LFRTIHZhbCAxMTE3Mjg4NyBlY3IgMTExNzI4ODddLCBsZW5ndGggMjg5Njxicj4xNTowNTowMi40 MzM3ODMgSVAgMTkyLjE2OC4xLjIuNDMzNjAgJmd0OyAxOTIuMTY4LjEuMi4xOTk5OTogRmxhZ3Mg W1AuXSwgc2VxIDEzMDM0NDoxMzEwOTYsIGFjayAxLCB3aW4gNDYsIG9wdGlvbnMgW25vcCxub3As VFMgdmFsIDExMTcyODg3IGVjciAxMTE3Mjg4N10sIGxlbmd0aCA3NTI8YnI+MTU6MDU6MDIuNDQ0 ODM5IElQIDE5Mi4xNjguNC4xLjk5OTkgJmd0OyAxOTIuMTY4LjEuMi40MzM2MDogRmxhZ3MgWy5d LCBhY2sgMTMxMDk3LCB3aW4gMzg2LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTE3Mjg5OCBl Y3IgMTExNzI4ODddLCBsZW5ndGggMDxicj4xNTowNTowMy40MjE5MTUgSVAgMTkyLjE2OC4xLjIu NDMzNjAgJmd0OyAxOTIuMTY4LjEuMi4xOTk5OTogRmxhZ3MgW0YuXSwgc2VxIDEzMTA5NiwgYWNr IDEsIHdpbiA0Niwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExNzM4NzUgZWNyIDExMTcyODk4 XSwgbGVuZ3RoIDA8YnI+MTU6MDU6MDMuNDYxNzQyIElQIDE5Mi4xNjguNC4xLjk5OTkgJmd0OyAx OTIuMTY4LjEuMi40MzM2MDogRmxhZ3MgWy5dLCBhY2sgMTMxMDk4LCB3aW4gMzg2LCBvcHRpb25z IFtub3Asbm9wLFRTIHZhbCAxMTE3MzkxNSBlY3IgMTExNzM4NzVdLCBsZW5ndGggMDxicj48YnI+ SSBiZWxpZXZlIHRoYXQgdGhlIGNhdXNlIG9mIHdoeSB0aGUgc2VuZGluZyBuZXRjYXQgd2l0aG91 dCAtdzEgb3B0aW9uIGRvZXNuJ3QgZXhpdCwgaXMgdGhhdCBpdCBkb2Vzbid0IHJlY2VpdmUgdGhl IGZpbmFsIEZJTiBmcm9tIHRoZSBkdG50dW5uZWwuPGJyPkNhbiB0aGlzIGlzc3VlIGJlIGZpeGVk IHdpdGggYSBzbWFsbCBhbW91bnQgb2Ygd29yaz88bzpwPjwvbzpwPjwvcD48ZGl2PjxwIGNsYXNz PU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0 b20tYWx0OmF1dG8nPjIwMTIvMTIvNSBab2xsZXIsIERhdmlkIEEuIChNU0ZDLUVPNjApW0hPU0Mg U0VSVklDRVMgQ09OVFJBQ1RdICZsdDs8YSBocmVmPSJtYWlsdG86ZGF2aWQuYS56b2xsZXJAbmFz YS5nb3YiIHRhcmdldD0iX2JsYW5rIj5kYXZpZC5hLnpvbGxlckBuYXNhLmdvdjwvYT4mZ3Q7PG86 cD48L286cD48L3A+PGRpdj48ZGl2PjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdp bi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdm b250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6 IzFGNDk3RCc+SGkgU2VyZ2V5LDwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3Jt YWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDph dXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIs InNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkkgd291bGQgbm90IGV2ZW4gY2FsbCB0aGlzIGEg cHJvYmxlbS4gbmV0Y2F0IGlzIGRlc2lnbmVkIHRvIGJlIGV4dHJlbWVseSBmbGV4aWJsZSBhbmQg YnkgZGVmYXVsdCB0aGUgc2VuZGVyIGtlZXBzIGl0cyBjb25uZWN0aW9uIG9wZW4gYXMgbG9uZyBh cyB0aGUgb3RoZXIgZW5kIGRvZXMuIE9uIHRoZSBvdGhlciBoYW5kIHRoZSByZWNlaXZlciBieSBk ZWZhdWx0IHRlcm1pbmF0ZXMgYWZ0ZXIgcmVjZWlwdCBvZiBhbiBFT0YgYnV0IHRoZXJlIGlzIGEg c3dpdGNoIHRvIGtlZXAgbGlzdGVuaW5nIGlmIHRoYXQgaXMgdGhlIGRlc2lyZWQgYmVoYXZpb3Iu IDwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJn aW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0n Zm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9y OiMxRjQ5N0QnPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwg c3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRv Jz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNh bnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPldpdGggRFROLCB5b3VyIHNlbmRpbmcgbmMgbWF5IGRv IGl0cyB0aGluZyB3aGlsZSB0aGUgZGVzdGluYXRpb24gbm9kZSBpcyBub3QgZXZlbiBvbiBsaW5l IGFuZCB0aGVuIGFuIGhvdXIgbGF0ZXIgaXQgYmVjb21lcyBhdmFpbGFibGUgYW5kIGNvbXBsZXRl cyB0aGUgdHJhbnNtaXNzaW9uIHRvIHRoZSByZWNlaXZpbmcgbmMuIEluIHRoaXMgc2NlbmFyaW8s IHRoZSBzZW5kZXIgd291bGQgc3RpbGwgYmUgJiM4MjIwO2h1bmcgdXAmIzgyMjE7IHBvc3NpYmx5 IGluZGVmaW5pdGVseSB3YWl0aW5nIGZvciBhIHRlcm1pbmF0aW5nIHNpZ25hbC4gPC9zcGFuPjxv OnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0 OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6 MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+ Jm5ic3A7PC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNv LW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0 eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7 Y29sb3I6IzFGNDk3RCc+SSBhbSBub3QgdGhlIERUTjIgYXV0aEhpIG9yaXR5LCBidXQsIEkgZG9u JiM4MjE3O3Qgc2VlIGEgY2hhbmdlIHRvIGR0bnR1bm5lbCBmb3IgdGhpcyBhcyBpdCB3b3VsZCBh c3N1bWUgYSBzcGVjaWZpYyB1c2FnZSB0aGF0IHdvdWxkIHByb2JhYmx5IGJyZWFrIHNvbWVvbmUg ZWxzZSYjODIxNztzIHVzYWdlIChsaWtlIG1pbmUgPC9zcGFuPjxzcGFuIHN0eWxlPSdmb250LXNp emU6MTEuMHB0O2ZvbnQtZmFtaWx5OldpbmdkaW5ncztjb2xvcjojMUY0OTdEJz5KPC9zcGFuPjxz cGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1z ZXJpZiI7Y29sb3I6IzFGNDk3RCc+KS48L3NwYW4+PG86cD48L286cD48L3A+PGRpdj48cCBjbGFz cz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90 dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToi Q2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPiZuYnNwOzwvc3Bhbj48bzpwPjwv bzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRv O21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBw dDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkJlc3Qg cmVnYXJkcyw8L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdt c28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4g c3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlm Ijtjb2xvcjojMUY0OTdEJz5EWjwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3Jt YWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDph dXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIs InNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD48 cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJn aW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZh bWlseToiQXJpYWwiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5EYXZpZCBab2xsZXI8L3Nw YW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRv cC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQt c2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkFyaWFsIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3 RCc+Q09MU0EgQ29ycG9yYXRpb248L3NwYW4+PG86cD48L286cD48L3A+PC9kaXY+PHAgY2xhc3M9 TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv bS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkFy aWFsIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+TWFyc2hhbGwgU3BhY2UgRmxpZ2h0IENl bnRlcjwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1t YXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHls ZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2Nv bG9yOiMxRjQ5N0QnPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3Jt YWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDph dXRvJz48Yj48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiVGFob21h Iiwic2Fucy1zZXJpZiInPkZyb206PC9zcGFuPjwvYj48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEw LjBwdDtmb250LWZhbWlseToiVGFob21hIiwic2Fucy1zZXJpZiInPiA8YSBocmVmPSJtYWlsdG86 c3NpcmVza2luQGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsiPnNzaXJlc2tpbkBnbWFpbC5jb208 L2E+IFttYWlsdG86PGEgaHJlZj0ibWFpbHRvOnNzaXJlc2tpbkBnbWFpbC5jb20iIHRhcmdldD0i X2JsYW5rIj5zc2lyZXNraW5AZ21haWwuY29tPC9hPl0gPGJyPjxiPlNlbnQ6PC9iPiBXZWRuZXNk YXksIERlY2VtYmVyIDA1LCAyMDEyIDI6NTYgQU08YnI+PGI+VG86PC9iPiBab2xsZXIsIERhdmlk IEEuIChNU0ZDLUVPNjApW0hPU0MgU0VSVklDRVMgQ09OVFJBQ1RdPGJyPjxiPkNjOjwvYj4gPGEg aHJlZj0ibWFpbHRvOmR0bi11c2Vyc0BpcnRmLm9yZyIgdGFyZ2V0PSJfYmxhbmsiPmR0bi11c2Vy c0BpcnRmLm9yZzwvYT48YnI+PGI+U3ViamVjdDo8L2I+IFJlOiBbZHRuLXVzZXJzXSBBIHByb2Js ZW0gd2l0aCBkdG50dW5uZWw8L3NwYW4+PG86cD48L286cD48L3A+PGRpdj48ZGl2PjxwIGNsYXNz PU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0 b20tYWx0OmF1dG8nPiZuYnNwOzxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHls ZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bWFyZ2luLWJvdHRvbToxMi4wcHQnPkhpIERhdmlk LDxicj48YnI+VGhhbmtzIGZvciB5b3VyIGhlbHAsIG5jIC13MSBkaWQgdGhlIHRyaWNrLiBCdXQg c2hvdWxkbid0IGR0bnR1bm5lbCBiZWhhdmlvciBiZSBjaGFuZ2VkPyBUaGUgcmVjZWl2aW5nIGR0 bnR1bm5lbCBjb3VsZCBzaWduYWwgdGhlIHNlbmRpbmcgZHRudHVubmVsLCB0aGF0IHRoZSBsaXN0 ZW5pbmcgbmMgaGFzIGRpc2Nvbm5lY3RlZCwgc28gdGhhdCB0aGUgc2VuZGluZyBkdG50dW5uZWwg Y2xvc2UgaXRzIGNvbm5lY3Rpb24gdG8gdGhlIHNlbmRpbmcgbmMuIE9yIGlzIHRoaXMgc29sZWx5 IHRoZSBwcm9ibGVtIG9mIG5jPzxvOnA+PC9vOnA+PC9wPjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFs IHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0 byc+MjAxMi8xMi81IFpvbGxlciwgRGF2aWQgQS4gKE1TRkMtRU82MClbSE9TQyBTRVJWSUNFUyBD T05UUkFDVF0gJmx0OzxhIGhyZWY9Im1haWx0bzpkYXZpZC5hLnpvbGxlckBuYXNhLmdvdiIgdGFy Z2V0PSJfYmxhbmsiPmRhdmlkLmEuem9sbGVyQG5hc2EuZ292PC9hPiZndDs8bzpwPjwvbzpwPjwv cD48ZGl2PjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6 YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZTox MS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5I aSBTZXJnZXksPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0n bXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFu IHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJp ZiI7Y29sb3I6IzFGNDk3RCc+T24gUkhFTCA1LjcsIEkgYW0gcnVubmluZyBkdG4tMi45LjAgcGx1 cyBtb2RpZmljYXRpb25zIHRoYXQgc2hvdWxkIG5vdCBpbXBhY3QgdGhlIGJlaGF2aW9yIG9mIGR0 bnR1bm5lbC48L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdt c28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4g c3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlm Ijtjb2xvcjojMUY0OTdEJz5JIGp1c3QgcmFuIHlvdXIgdGVzdCB3aXRob3V0IHRoZSBpcHRhYmxl cyByZWRpcmVjdCBhbmQgSSBzZWUgdGhlIHNhbWUgYmVoYXZpb3IuPC9zcGFuPjxvOnA+PC9vOnA+ PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNv LW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2Zv bnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+SSBiZWxpZXZl IHRoYXQgdGhlIHNlbmRlciBleGl0cyBpZiB5b3UgZ28gZGlyZWN0bHkgZnJvbSBuYyB0byBuYyBi ZWNhdXNlIHRoZSByZWNlaXZlciBleGl0cyB3aGVuIGl0IGdldHMgYW4gZW5kIG9mIGZpbGUuPC9z cGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10 b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250 LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFG NDk3RCc+WW91IGNhbiBhZGQgYSAmIzgyMjA7LXcgMSYjODIyMTsgb3B0aW9uIHRvIHRoZSBzZW5k ZXIgbmMgc28gdGhhdCBpdCB3aWxsIHRpbWVvdXQgYW5kIGV4aXQgYWZ0ZXIgc3RkaW4gaXMgaWRs ZSBmb3IgMSBzZWNvbmQuPC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBz dHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8n PjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fu cy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+QmVzdCByZWdhcmRzLDwvc3Bhbj48bzpwPjwvbzpwPjwv cD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1t YXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250 LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkRaPC9zcGFuPjxv OnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0 OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6 MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+ Jm5ic3A7PC9zcGFuPjxvOnA+PC9vOnA+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNv LW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0 eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJBcmlhbCIsInNhbnMtc2VyaWYiO2Nv bG9yOiMxRjQ5N0QnPkRhdmlkIFpvbGxlcjwvc3Bhbj48bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1N c29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9t LWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiQXJp YWwiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5DT0xTQSBDb3Jwb3JhdGlvbjwvc3Bhbj48 bzpwPjwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFs dDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXpl OjEwLjBwdDtmb250LWZhbWlseToiQXJpYWwiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz5I T1NDIC8gQzEwNyZuYnNwOyA8L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFs IHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0 byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6V2luZ2RpbmdzO2Nv bG9yOiM5OTAwMDAnPig8L3NwYW4+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1m YW1pbHk6IkFyaWFsIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+T2ZmaWNlOjwvc3Bhbj48 c3BhbiBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMt c2VyaWYiO2NvbG9yOiMxRjQ5N0QnPiA8L3NwYW4+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4w cHQ7Zm9udC1mYW1pbHk6IkFyaWFsIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+KDI1Nikm bmJzcDs1NDQtMTgyMDwvc3Bhbj48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZh bWlseToiQXJpYWwiLCJzYW5zLXNlcmlmIjtjb2xvcjpuYXZ5Jz48YnI+PC9zcGFuPjxzcGFuIHN0 eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OldpbmdkaW5ncztjb2xvcjojMzM2NkZG Jz4qPC9zcGFuPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJBcmlh bCIsInNhbnMtc2VyaWYiJz5FTWFpbDogPHNwYW4gc3R5bGU9J2NvbG9yOm5hdnknPjxhIGhyZWY9 Im1haWx0bzpkYXZpZC5hLnpvbGxlckBuYXNhLmdvdiIgdGFyZ2V0PSJfYmxhbmsiPmRhdmlkLmEu em9sbGVyQG5hc2EuZ292PC9hPjwvc3Bhbj48L3NwYW4+PG86cD48L286cD48L3A+PHAgY2xhc3M9 TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv bS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6IkNh bGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz4mbmJzcDs8L3NwYW4+PG86cD48L286 cD48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bztt c28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+PGI+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4w cHQ7Zm9udC1mYW1pbHk6IlRhaG9tYSIsInNhbnMtc2VyaWYiJz5Gcm9tOjwvc3Bhbj48L2I+PHNw YW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IlRhaG9tYSIsInNhbnMtc2Vy aWYiJz4gPGEgaHJlZj0ibWFpbHRvOmR0bi11c2Vycy1ib3VuY2VzQGlydGYub3JnIiB0YXJnZXQ9 Il9ibGFuayI+ZHRuLXVzZXJzLWJvdW5jZXNAaXJ0Zi5vcmc8L2E+IFttYWlsdG86PGEgaHJlZj0i bWFpbHRvOmR0bi11c2Vycy1ib3VuY2VzQGlydGYub3JnIiB0YXJnZXQ9Il9ibGFuayI+ZHRuLXVz ZXJzLWJvdW5jZXNAaXJ0Zi5vcmc8L2E+XSA8Yj5PbiBCZWhhbGYgT2YgPC9iPjxhIGhyZWY9Im1h aWx0bzpzc2lyZXNraW5AZ21haWwuY29tIiB0YXJnZXQ9Il9ibGFuayI+c3NpcmVza2luQGdtYWls LmNvbTwvYT48YnI+PGI+U2VudDo8L2I+IFR1ZXNkYXksIERlY2VtYmVyIDA0LCAyMDEyIDExOjEz IEFNPGJyPjxiPlRvOjwvYj4gPGEgaHJlZj0ibWFpbHRvOmR0bi11c2Vyc0BpcnRmLm9yZyIgdGFy Z2V0PSJfYmxhbmsiPmR0bi11c2Vyc0BpcnRmLm9yZzwvYT48YnI+PGI+U3ViamVjdDo8L2I+IFtk dG4tdXNlcnNdIEEgcHJvYmxlbSB3aXRoIGR0bnR1bm5lbDwvc3Bhbj48bzpwPjwvbzpwPjwvcD48 ZGl2PjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0 bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byc+Jm5ic3A7PG86cD48L286cD48L3A+PHAgY2xh c3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJv dHRvbS1hbHQ6YXV0byc+SGVsbG8gYWxsITxicj48YnI+SSBhbSB1c2luZyBkdG4tMi45LjAgb24g YSBSSEVMIDYgYmFzZWQgTGludXggZGlzdHJvIGFuZCBJIGFtIGhhdmluZyBhIHByb2JsZW0gd2hl biB1c2luZyBuZXRjYXQgKG5jKSB3aXRoIGR0bnR1bm5lbC48YnI+T24gdGhlIHNlbmRlciBub2Rl IEkgcnVuICZxdW90O2NhdCAvZXRjL3Bhc3N3ZCB8IG5jIHJlY2VpdmVyX2lwJnF1b3Q7LiBPbiB0 aGUgcmVjZWl2ZXIgbm9kZSBJIHJ1biAmcXVvdDtuYyAtbCA5OTk5JnF1b3Q7LiBXaXRoIHRoZSBo ZWxwIDxicj5vZiBJcHRhYmxlcyBwb3J0IDk5OTkgZ2V0cyByZWRpcmVjdGVkIHRvIHRoZSBwb3J0 IDE5OTk5LCB3aGljaCBpcyBsaXN0ZW5lZCBieSBkdG50dW5uZWwuPGJyPjxicj5UaGUgZmlsZSAv ZXRjL3Bhc3N3ZCBpcyBzdWNjZXNzZnVsbHkgZGVsaXZlcmVkIHRvIHRoZSByZWNlaXZlciBhbmQg aXMgc2hvd24gb24gdGhlIHNjcmVlbi4gQWZ0ZXIgdGhpcyB0aGUgcmVjZWl2aW5nIG5jIGV4aXN0 cy4gPGJyPk5vIHByb2JsZW0gaGVyZS4gSG93ZXZlciwgbmMgb24gdGhlIHNlbmRlciBub2RlIGRv ZXNuJ3QgZXhpdCBhZnRlciBpdCBzZW5kcyB0aGUgZmlsZS4gSXQgY29udGludWVzIHRvIHJ1biBm b3JldmVyLiBXaGVuIEkgPGJyPnJ1biBuYyBpbiB0aGUgb3Bwb3NpdGUgZGlyZWN0aW9uLCBJIGdl dCB0aGUgc2FtZSBwcm9ibGVtIC0gdGhlIHNlbmRpbmcgbmMgZG9lc24ndCBleGl0Ljxicj48YnI+ TXkgY29uZmlndXJhdGlvbiBvbiBib3RoIG5vZGVzIGlzIHN5bW1ldHJpYzo8YnI+aXB0YWJsZXMg LXQgbmF0IC1BIE9VVFBVVCAtZCAkUkVNT1RFX0hPU1QgLXAgdGNwIC0tZHBvcnQgOTk5OSAtaiBE TkFUIC0tdG8gJExPQ0FMX0hPU1Q6MTk5OTk8YnI+ZHRudHVubmVsIC1UICRMT0NBTF9IT1NUOjE5 OTk5OiRSRU1PVEVfSE9TVDo5OTk5ICRSRU1PVEVfTk9ERS9kdG50dW5uZWwvbmMgLWQ8YnI+ZHRu dHVubmVsIC1MIC0tbG9jYWwtZWlkICRMT0NBTF9OT0RFL2R0bnR1bm5lbC9uYyAtZDxicj48YnI+ ZHRucGluZyB3b3JrcyBvayBpbiBib3RoIGRpcmVjdGlvbnMuPGJyPjxicj5QbGVhc2UgZ2l2ZSBt ZSBhbnkgYWR2aWNlLiA8YnI+PGJyPldpdGggYmVzdCByZWdhcmRzLDxicj5TZXJnZXkgU2lyZXNr aW4gPG86cD48L286cD48L3A+PC9kaXY+PC9kaXY+PC9kaXY+PC9kaXY+PC9kaXY+PHAgY2xhc3M9 TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttYXJnaW4tYm90dG9tOjEy LjBwdCc+PGJyPjxiciBjbGVhcj1hbGw+PGJyPi0tIDxicj5LaW5kZXN0IFJlZ2FyZHM8YnI+PGJy PlNlcmdleSBTaXJlc2tpbjxicj5GR1VQIENOSUkgRUlTVTxvOnA+PC9vOnA+PC9wPjwvZGl2Pjwv ZGl2PjwvZGl2PjwvZGl2PjwvZGl2PjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdp bi10b3AtYWx0OmF1dG87bWFyZ2luLWJvdHRvbToxMi4wcHQnPjxicj48YnIgY2xlYXI9YWxsPjxi cj4tLSA8YnI+S2luZGVzdCBSZWdhcmRzPGJyPjxicj5TZXJnZXkgU2lyZXNraW48YnI+RkdVUCBD TklJIEVJU1U8bzpwPjwvbzpwPjwvcD48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48cCBj bGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21hcmdpbi1ib3R0 b206MTIuMHB0Jz48YnI+PGJyIGNsZWFyPWFsbD48YnI+LS0gPGJyPkJlc3QgcmVnYXJkcyw8YnI+ U2VyZ2V5IFNpcmVza2luPG86cD48L286cD48L3A+PC9kaXY+PC9kaXY+PC9kaXY+PC9kaXY+PC9k aXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttYXJn aW4tYm90dG9tOjEyLjBwdCc+PGJyPjxiciBjbGVhcj1hbGw+PGJyPi0tIDxicj5CZXN0IHJlZ2Fy ZHMsPGJyPlNlcmdleSBTaXJlc2tpbjxvOnA+PC9vOnA+PC9wPjwvZGl2PjwvZGl2PjwvZGl2Pjwv ZGl2PjwvZGl2PjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbWFyZ2luLWJvdHRvbToxMi4wcHQn Pjxicj48YnIgY2xlYXI9YWxsPjxicj4tLSA8YnI+QmVzdCByZWdhcmRzLDxicj5TZXJnZXkgU2ly ZXNraW48YnI+PGJyPjxvOnA+PC9vOnA+PC9wPjwvZGl2PjwvYm9keT48L2h0bWw+Cg== ------=_Part_0_1355418889033-- From ssireskin@gmail.com Mon Dec 17 04:39:57 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 88DE721F8A61 for ; Mon, 17 Dec 2012 04:39:57 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.033 X-Spam-Level: X-Spam-Status: No, score=-3.033 tagged_above=-999 required=5 tests=[AWL=-0.035, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VQ8Q7xXBsUOR for ; Mon, 17 Dec 2012 04:39:56 -0800 (PST) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by ietfa.amsl.com (Postfix) with ESMTP id A006521F89E2 for ; Mon, 17 Dec 2012 04:39:56 -0800 (PST) Received: by mail-vc0-f182.google.com with SMTP id fy27so4850393vcb.13 for ; Mon, 17 Dec 2012 04:39:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=P2A7LO1xPYVc3yWbJPcB04NNxLCtp8LzPuzzsep9YQ4=; b=rnjJ0DhXnTMPMXgASH+Jgpo2QuOsfuvrzgOmxWbhSuu1f4S+vfVt+fjWvDOfdYUjk1 /QEzn9BkFGo1KER7E2Stlp/7HPNX6xy0q+CalY9po7FlJnYPqHJ2Wo9zBKIZeqHVPV0E UheQo6YF1Yn22LgnWPwdZnLVjDir1Hc7ZwCB8j/ZbWt+2Sw8YZHiULXN3W3VUdrPChjS JiGqaUpYU2ROLMaRpx4aTg7VDunJV6a7plUKlgRIX4Zz9pn/vaz6mB73JucFx3crdbGn z8lzj14B6n8AnOkt8WEEyb6NKpbKorZvvbxqF9iHR3DQID4PoW7LhOptKFFC1Qpo7Kt7 m1Ww== MIME-Version: 1.0 Received: by 10.52.18.147 with SMTP id w19mr19951634vdd.94.1355747995908; Mon, 17 Dec 2012 04:39:55 -0800 (PST) Received: by 10.58.219.65 with HTTP; Mon, 17 Dec 2012 04:39:54 -0800 (PST) In-Reply-To: References: Date: Mon, 17 Dec 2012 15:39:54 +0300 Message-ID: From: "ssireskin@gmail.com" To: David Zoller Content-Type: multipart/alternative; boundary=bcaec50406eab37a9504d10bad70 Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 12:39:57 -0000 --bcaec50406eab37a9504d10bad70 Content-Type: text/plain; charset=ISO-8859-1 Hi David, Thanks for your patch, it works! :) Connection is cleanly terminated now. However, there still seems to be a problem. I run dtntunnel over a link with high latency and about 30% packet loss. The receiving nc sometimes gets FIN packet before it receives all the data. If I understand it right, on a lossy link bundles may come out of order. In this case the EOF bundle may come too early. I think that the reordering procedure in dtntunnel should be reviewed. Just for information, pure TCP performs better that DTN over this link. 2012/12/14 David Zoller > Hi Sergey, > I can't stay away from a problem that is nagging at me so I made some time > this morning. Going back to the original TCPTunnel.cc code, when a bundle > is received that has the EOF indication, an empty bundle with the same EOF > flag set needs to be returned to close the loop (the second FIN). There may > still be some loose ends or clean up needed but try the code below at the > end of the while loop. > > I did not do a packet capture but I saw that if the receiving nc is not > running when you start the sender nc then the receiving dtntunnel logs an > error on the failed connect and returns an EOF bundle which closes the > sender nc. The EOF bundle could possibly be enhanced to include an error > message. > > I agree that some applications could benefit from a "pretend to be a > receiver" option with the default working as a bidirectional end-to-end TCP > connection. > > Thanks for dragging me along kicking and screaming on this lesson in DTN > and TCP tunneling. I've only been using dtntunnel as a one way UDP > transmitter. I see a good fit for the TCP tunnel in an upcoming project and > I'll try to make the pitch to use it. This is a pretty cool little app. > Best regards, > DZ > > > > > > 484 delete b_recv; > 485 > 486 if (recv_hdr->eof_) { > 487 log_info("bundle had eof bit set... closing > connection"); > 488 > 489 if ( !sock_eof ) { > 490 sock_eof = true; > 491 // send an empty bundle back since we did not > initiate the close > 492 dtn::APIBundle* b = new dtn::APIBundle(); > 493 hdr.eof_ = 1; > 494 hdr.seqno_ = ntohl(send_seqno++); > 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, > sizeof(hdr)); > 496 b->payload_.set_len(sizeof(hdr)); > 497 int err; > 498 if ((err = tunnel->send_bundle(b, &dest_eid_)) != > DTN_SUCCESS) { > 499 log_err("error sending final socket closed > packet bundle: %s", > 500 dtn_strerror(err)); > 501 tcptun_->kill_connection(this); > 502 exit(1); > 503 } > 504 delete b; > 505 } > 506 sock_.close(); > 507 goto done; > 508 } > 509 } > 510 } > 511 > 512 done: > > -- Best regards, Sergey Sireskin --bcaec50406eab37a9504d10bad70 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi David,

Thanks for your patch, it works! :) Connection is cleanly = terminated now.

However, there still seems to be a problem. I run dt= ntunnel over a link with high latency and about 30% packet loss. The receiv= ing nc sometimes gets FIN packet before it receives all the data. If I und= erstand it right, on a lossy link bundles may come out of order. In this ca= se the EOF bundle may come too early. I think that the reordering procedure= in dtntunnel should be reviewed. Just for information, pure TCP performs b= etter that DTN over this link.

2012/12/14 David Zoller &l= t;zollerd@gmail.com<= /a>>
Hi Sergey,
I can't stay away from a problem that is nagg= ing at me so I made some time this morning. Going back to the original TCPT= unnel.cc code, when a bundle is received that has the EOF indication, an em= pty bundle with the same EOF flag set needs to be returned to close the loo= p (the second FIN). There may still be some loose ends or clean up needed b= ut try the code below at the end of the while loop.

I did not do a packet capture but I saw that if the rec= eiving nc is not running when you start the sender nc then the receiving dt= ntunnel logs an error on the failed connect and returns an EOF bundle which= closes the sender nc. The EOF bundle could possibly be enhanced to include= an error message.

I agree that some applications could benefit from a &qu= ot;pretend to be a receiver" option with the default working as a bidi= rectional end-to-end TCP connection.

Thanks for dr= agging me along kicking and screaming on this lesson in DTN and TCP tunneli= ng. I've only been using dtntunnel as a one way UDP transmitter. I see = a good fit for the TCP tunnel in an upcoming project and I'll try to ma= ke the pitch to use it. This is a pretty cool little app.
Best regards,
DZ




484 =A0 =A0 =A0 =A0 =A0 =A0 = delete b_recv;
485=A0
486 =A0 =A0 =A0 =A0 =A0 =A0 if (r= ecv_hdr->eof_) {
487 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 log_info("bundle had eof bit = set... closing connection");
488=A0
489 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 if ( !sock_eof ) {
490 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 sock_eof =3D true;
491 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 // send an empty bundle back since we did not initi= ate the close
492 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 dtn::APIBundle* b =3D new = dtn::APIBundle();
493 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 hdr= .eof_ =3D 1;
494 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 hdr.seqn= o_ =3D ntohl(send_seqno++);
495 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, sizeof(hdr));
496 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 b->payload_.set_len(siz= eof(hdr));
497 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 int err;
498 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ((err =3D tunnel-&g= t;send_bundle(b, &dest_eid_)) !=3D DTN_SUCCESS) {
499 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 log_err("error sen= ding final socket closed packet bundle: %s",
500 =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 dtn_strerror(err));
501 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 tcptun_->kil= l_connection(this);
502 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 exit(1);
503 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 }
504 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 delete b;
505 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 }
506 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sock_.close();
507 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto done;
508 =A0 =A0 =A0 =A0 =A0 =A0 }
509 =A0 =A0 =A0 =A0 }
510 =A0 =A0 }
511=A0
512 =A0done:



--
Best regards,
Sergey= Sireskin


--bcaec50406eab37a9504d10bad70-- From ssireskin@gmail.com Mon Dec 17 05:09:09 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 494C621F8AA6 for ; Mon, 17 Dec 2012 05:09:09 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.331 X-Spam-Level: X-Spam-Status: No, score=-3.331 tagged_above=-999 required=5 tests=[AWL=0.267, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JPd6+WdJhl+Z for ; Mon, 17 Dec 2012 05:09:08 -0800 (PST) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by ietfa.amsl.com (Postfix) with ESMTP id B623321F8505 for ; Mon, 17 Dec 2012 05:09:08 -0800 (PST) Received: by mail-vb0-f54.google.com with SMTP id l1so7041413vba.13 for ; Mon, 17 Dec 2012 05:09:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=q7AgBgm6QLcVJig3e/Z8q/dKiVmzfYPQBd9poBia2T4=; b=PX+VFyIEDVEEKxLJcbW00907Imd0A3KGhlTe/wb1AWFwPO3PEHVg5TlGiLGk57trIi 2OvdV5uMlWEayn6b0VSjwg8/17diT0HXFp8SPD07n44GVUgtrXOwaag0YyVUcQWAy98K DZsuTa184WGVfG2EPVCwv+IHwE2Mw/kdbhf7M0QOk57DwoJoj7VTz7Rh6csB6p49rSRA iR0zBgna7n3+eihpKYOln6JWRVfdj4xdJLtWUfWtHng6ZEKDonerlAyZ4drSs6JDRnv7 YfKAs/NeJRvZULkNQM1YVpC4E2EPVTEHpMfAPW5GZJ8AMOqAT63EhQjgmDZtujmL6TXF C3gA== MIME-Version: 1.0 Received: by 10.220.156.197 with SMTP id y5mr22602051vcw.17.1355749747938; Mon, 17 Dec 2012 05:09:07 -0800 (PST) Received: by 10.58.219.65 with HTTP; Mon, 17 Dec 2012 05:09:07 -0800 (PST) In-Reply-To: References: Date: Mon, 17 Dec 2012 16:09:07 +0300 Message-ID: From: "ssireskin@gmail.com" To: David Zoller Content-Type: multipart/alternative; boundary=f46d042f91102155cb04d10c1644 Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 13:09:09 -0000 --f46d042f91102155cb04d10c1644 Content-Type: text/plain; charset=ISO-8859-1 A series of tests showed that the original dtntunnel doesn't have the problem with a too early FIN. -- Best regards, Sergey Sireskin --f46d042f91102155cb04d10c1644 Content-Type: text/html; charset=ISO-8859-1 A series of tests showed that the original dtntunnel doesn't have the problem with a too early FIN.

--
Best regards,
Sergey Sireskin

--f46d042f91102155cb04d10c1644-- From david.a.zoller@nasa.gov Mon Dec 17 05:15:24 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4FC8C21F8AB8 for ; Mon, 17 Dec 2012 05:15:24 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.169 X-Spam-Level: X-Spam-Status: No, score=-6.169 tagged_above=-999 required=5 tests=[AWL=-0.171, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aisu6KiF4LZX for ; Mon, 17 Dec 2012 05:15:20 -0800 (PST) Received: from ndmsnpf01.ndc.nasa.gov (ndmsnpf01.ndc.nasa.gov [198.117.0.121]) by ietfa.amsl.com (Postfix) with ESMTP id CD3D821F8AB0 for ; Mon, 17 Dec 2012 05:15:19 -0800 (PST) Received: from ndmsppt02.ndc.nasa.gov (ndmsppt02.ndc.nasa.gov [198.117.0.101]) by ndmsnpf01.ndc.nasa.gov (Postfix) with ESMTP id 35CEC260286; Mon, 17 Dec 2012 07:15:19 -0600 (CST) Received: from ndmshub03.ndc.nasa.gov (ndmshub03-pub.ndc.nasa.gov [198.117.0.42]) by ndmsppt02.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBHDFIs9015040; Mon, 17 Dec 2012 07:15:19 -0600 Received: from NDMSSCC05.ndc.nasa.gov ([198.117.2.174]) by ndmshub03.ndc.nasa.gov ([198.117.2.162]) with mapi; Mon, 17 Dec 2012 07:15:19 -0600 From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "ssireskin@gmail.com" , David Zoller Date: Mon, 17 Dec 2012 07:15:18 -0600 Thread-Topic: [dtn-users] A problem with dtntunnel Thread-Index: Ac3cU6G/PYEyrcZETpS14MucqZ7OQAAAcm/g Message-ID: <04E3D99A62496240BCD6A576813E6E31E0BDF5C59C@NDMSSCC05.ndc.nasa.gov> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_04E3D99A62496240BCD6A576813E6E31E0BDF5C59CNDMSSCC05ndcn_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-17_02:2012-12-17, 2012-12-17, 1970-01-01 signatures=0 Cc: "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 13:15:24 -0000 --_000_04E3D99A62496240BCD6A576813E6E31E0BDF5C59CNDMSSCC05ndcn_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Sergey, I don't have your setup and this has not been tested but give this a try. I= n TCPTunnel::Connection::handle_bundle in the while(1) loop add the test to= see if the next_seqno_ is a match: while (1) { iter =3D reorder_table_.find(next_seqno_); if (iter =3D=3D reorder_table_.end()) { break; } // add this code: // if seqno does not match then still a bundle missing if (iter->first !=3D next_seqno_) { break; } bundle =3D iter->second; log_debug("delivering %zu byte bundle with seqno %d (from reorder t= able)", bundle->payload_.len(), next_seqno_); reorder_table_.erase(iter); next_seqno_++; queue_.push_back(bundle); } David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Beh= alf Of ssireskin@gmail.com Sent: Monday, December 17, 2012 6:40 AM To: David Zoller Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Thanks for your patch, it works! :) Connection is cleanly terminated now. However, there still seems to be a problem. I run dtntunnel over a link wit= h high latency and about 30% packet loss. The receiving nc sometimes gets F= IN packet before it receives all the data. If I understand it right, on a l= ossy link bundles may come out of order. In this case the EOF bundle may co= me too early. I think that the reordering procedure in dtntunnel should be = reviewed. Just for information, pure TCP performs better that DTN over this= link. 2012/12/14 David Zoller > Hi Sergey, I can't stay away from a problem that is nagging at me so I made some time = this morning. Going back to the original TCPTunnel.cc code, when a bundle i= s received that has the EOF indication, an empty bundle with the same EOF f= lag set needs to be returned to close the loop (the second FIN). There may = still be some loose ends or clean up needed but try the code below at the e= nd of the while loop. I did not do a packet capture but I saw that if the receiving nc is not run= ning when you start the sender nc then the receiving dtntunnel logs an erro= r on the failed connect and returns an EOF bundle which closes the sender n= c. The EOF bundle could possibly be enhanced to include an error message. I agree that some applications could benefit from a "pretend to be a receiv= er" option with the default working as a bidirectional end-to-end TCP conne= ction. Thanks for dragging me along kicking and screaming on this lesson in DTN an= d TCP tunneling. I've only been using dtntunnel as a one way UDP transmitte= r. I see a good fit for the TCP tunnel in an upcoming project and I'll try = to make the pitch to use it. This is a pretty cool little app. Best regards, DZ 484 delete b_recv; 485 486 if (recv_hdr->eof_) { 487 log_info("bundle had eof bit set... closing connection"= ); 488 489 if ( !sock_eof ) { 490 sock_eof =3D true; 491 // send an empty bundle back since we did not initi= ate the close 492 dtn::APIBundle* b =3D new dtn::APIBundle(); 493 hdr.eof_ =3D 1; 494 hdr.seqno_ =3D ntohl(send_seqno++); 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, sizeof(h= dr)); 496 b->payload_.set_len(sizeof(hdr)); 497 int err; 498 if ((err =3D tunnel->send_bundle(b, &dest_eid_)) != =3D DTN_SUCCESS) { 499 log_err("error sending final socket closed pack= et bundle: %s", 500 dtn_strerror(err)); 501 tcptun_->kill_connection(this); 502 exit(1); 503 } 504 delete b; 505 } 506 sock_.close(); 507 goto done; 508 } 509 } 510 } 511 512 done: -- Best regards, Sergey Sireskin --_000_04E3D99A62496240BCD6A576813E6E31E0BDF5C59CNDMSSCC05ndcn_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Sergey= ,

I don’t have your = setup and this has not been tested but give this a try. In TCPTunnel::Conne= ction::handle_bundle in the while(1) loop add the test to see if the next_s= eqno_ is a match:

&nb= sp;

    while (1) {

        iter =3D reorder_table_.fin= d(next_seqno_);

     &nbs= p;  if (iter =3D=3D reorder_table_.end()) {

&= nbsp;           break;

        }

 

  = ;      // add this code:

        // if seqno does n= ot match then still a bundle missing

        if (iter->first !=3D next_se= qno_) {

    &nbs= p;       break;

        }

 

      &n= bsp; bundle =3D iter->second;

=    = ;     log_debug("delivering %zu byte bundle with s= eqno %d (from reorder table)",

  &n= bsp;            = ;   bundle->payload_.len(), next_seqno_);

   

    &= nbsp;   reorder_table_.erase(iter);

        next_seqno_++;

   

   &nb= sp;    queue_.push_back(bundle);

<= p class=3DMsoNormal> 

&nb= sp;

 

 

David Zoller=

COLSA Corporation

HOSC / C107 

(Office: = (256) 544-1820
*EMail: david.a.zoller@nasa.gov

 

<= p class=3DMsoNormal>From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounce= s@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Monday,= December 17, 2012 6:40 AM
To: David Zoller
Cc: dtn-use= rs@irtf.org
Subject: Re: [dtn-users] A problem with dtntunnel

 

Hi David,

Thanks for your patch= , it works! :) Connection is cleanly terminated now.

However, there = still seems to be a problem. I run dtntunnel over a link with high latency = and about 30% packet loss. The receiving nc sometimes gets FIN packet befor= e it receives all the data. If I understand it right, on a lossy link bundl= es may come out of order. In this case the EOF bundle may come too early. I= think that the reordering procedure in dtntunnel should be reviewed. Just = for information, pure TCP performs better that DTN over this link.

2012/12/14 David Zoller <zollerd@gmail.com>

Hi Sergey,

I can't stay away from a problem that is nagging at me so I ma= de some time this morning. Going back to the original TCPTunnel.cc code, wh= en a bundle is received that has the EOF indication, an empty bundle with t= he same EOF flag set needs to be returned to close the loop (the second FIN= ). There may still be some loose ends or clean up needed but try the code b= elow at the end of the while loop.

 

I did not do a p= acket capture but I saw that if the receiving nc is not running when you st= art the sender nc then the receiving dtntunnel logs an error on the failed = connect and returns an EOF bundle which closes the sender nc. The EOF bundl= e could possibly be enhanced to include an error message.

 

I agree that some applications could benefit from a "pretend to= be a receiver" option with the default working as a bidirectional end= -to-end TCP connection.

=  

Thanks for dragging me alon= g kicking and screaming on this lesson in DTN and TCP tunneling. I've only = been using dtntunnel as a one way UDP transmitter. I see a good fit for the= TCP tunnel in an upcoming project and I'll try to make the pitch to use it= . This is a pretty cool little app.

Best regards,

DZ<= /o:p>

 

 

 

 

 

484             delete b_recv;

485 

=

486             if = (recv_hdr->eof_) {

487 &nb= sp;               log_info("bundle = had eof bit set... closing connection");

488 

48= 9                 if ( !sock_eof ) = {

490       &n= bsp;             sock_eof =3D true;

491          =           // send an empty bundle back since we d= id not initiate the close

492=                     dtn:= :APIBundle* b =3D new dtn::APIBundle();

493                 &n= bsp;   hdr.eof_ =3D 1;

4= 94                     hd= r.seqno_ =3D ntohl(send_seqno++);

495                   &nb= sp; memcpy(b->payload_.buf(sizeof(hdr)), &hdr, sizeof(hdr));

496         &nbs= p;           b->payload_.set_len(sizeof(hdr));<= o:p>

497       &nbs= p;             int err;

<= div>

498             &nbs= p;       if ((err =3D tunnel->send_bundle(b, &dest_ei= d_)) !=3D DTN_SUCCESS) {

499 =                      = ;   log_err("error sending final socket closed packet bundle: %s&= quot;,

500     &nbs= p;                     &n= bsp;     dtn_strerror(err));

501                  = ;       tcptun_->kill_connection(this);

502             =             exit(1);

503               =       }

504 &n= bsp;                   delete = b;

505       &= nbsp;         }

506                 sock_.cl= ose();

507     &nbs= p;           goto done;

<= p class=3DMsoNormal>508             }

509         }

510     }<= /p>

511 

512  done:

 




--
Best regards,
Sergey S= ireskin

= --_000_04E3D99A62496240BCD6A576813E6E31E0BDF5C59CNDMSSCC05ndcn_-- From ssireskin@gmail.com Mon Dec 17 05:43:28 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 12F3D21F86D9 for ; Mon, 17 Dec 2012 05:43:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.049 X-Spam-Level: X-Spam-Status: No, score=-3.049 tagged_above=-999 required=5 tests=[AWL=-0.051, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 60x9o8kaBqJy for ; Mon, 17 Dec 2012 05:43:23 -0800 (PST) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by ietfa.amsl.com (Postfix) with ESMTP id CE74021F84CD for ; Mon, 17 Dec 2012 05:43:22 -0800 (PST) Received: by mail-vb0-f54.google.com with SMTP id l1so7091494vba.13 for ; Mon, 17 Dec 2012 05:43:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=gMX3Ud3T+1UQ1ecZ5z2hLHJ151aBQA3JXcUzNeYssjo=; b=xb4U852BPKmfRzwFiOQ325Kr9IPMY9mp9IO+JZCS8J4WIunJ1EGmJ0u6Tx/3utIYMU qC/Ycr5sZZaLc8zaNrFaFZTo5L5AtPbpc6ffBDuy/lW6QmIiFLTsqw3WkPTHK1QGzWNE +W6RLmkS5lHGrJpZlHLDDy8gyGdFgIiJJbRWe9XwJJAaQv8zzXOuVuQWDZWkEZyuHhEl msPiFTcXH4LM33qrQD/szvwFFEWigzdDjgvDc50Lq8aCloWr83HHSoDhPiucAOOAIRCh vndPdpQxxMNFNB1U4xV6vWMkq3p7fZ3wbfm39bYqMM0Rc9aP0Tx8ekkHPyzFJ63fxTY1 U8YA== MIME-Version: 1.0 Received: by 10.58.198.135 with SMTP id jc7mr23440487vec.51.1355751802344; Mon, 17 Dec 2012 05:43:22 -0800 (PST) Received: by 10.58.219.65 with HTTP; Mon, 17 Dec 2012 05:43:22 -0800 (PST) In-Reply-To: <04E3D99A62496240BCD6A576813E6E31E0BDF5C59C@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDF5C59C@NDMSSCC05.ndc.nasa.gov> Date: Mon, 17 Dec 2012 16:43:22 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/mixed; boundary=047d7b5d8cd795139a04d10c9019 Cc: David Zoller , "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 13:43:28 -0000 --047d7b5d8cd795139a04d10c9019 Content-Type: multipart/alternative; boundary=047d7b5d8cd795139604d10c9017 --047d7b5d8cd795139604d10c9017 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable David, I added this code and a log_debug inside the if-clause. According to the log, this condition doesn't get true. A have attached my logs and tcpdumps. Please, see dtntunnel_T.log - it's a receiving dtntunnel log. There are 4 bundles after a eof bundle. This doesn't happen with the original dtntunnel. --=20 Best regards, Sergey Sireskin 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < david.a.zoller@nasa.gov> > Hi Sergey,**** > > I don=92t have your setup and this has not been tested but give this a tr= y. > In TCPTunnel::Connection::handle_bundle in the while(1) loop add the test > to see if the next_seqno_ is a match:**** > > ** ** > > while (1) {**** > > iter =3D reorder_table_.find(next_seqno_);**** > > if (iter =3D=3D reorder_table_.end()) {**** > > break;**** > > }**** > > ** ** > > // add this code:**** > > // if seqno does not match then still a bundle missing**** > > if (iter->first !=3D next_seqno_) {**** > > break;**** > > }**** > > ** ** > > bundle =3D iter->second;**** > > log_debug("delivering %zu byte bundle with seqno %d (from reorder > table)",**** > > bundle->payload_.len(), next_seqno_);**** > > **** > > reorder_table_.erase(iter);**** > > next_seqno_++;**** > > **** > > queue_.push_back(bundle);**** > > } **** > > ** ** > > ** ** > > ** ** > > ** ** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > ** ** > > *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *O= n > Behalf Of *ssireskin@gmail.com > *Sent:* Monday, December 17, 2012 6:40 AM > *To:* David Zoller > > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > ** ** > > Hi David, > > > Thanks for your patch, it works! :) Connection is cleanly terminated now. > > However, there still seems to be a problem. I run dtntunnel over a link > with high latency and about 30% packet loss. The receiving nc sometimes > gets FIN packet before it receives all the data. If I understand it right= , > on a lossy link bundles may come out of order. In this case the EOF bundl= e > may come too early. I think that the reordering procedure in dtntunnel > should be reviewed. Just for information, pure TCP performs better that D= TN > over this link.**** > > 2012/12/14 David Zoller **** > > Hi Sergey,**** > > I can't stay away from a problem that is nagging at me so I made some tim= e > this morning. Going back to the original TCPTunnel.cc code, when a bundle > is received that has the EOF indication, an empty bundle with the same EO= F > flag set needs to be returned to close the loop (the second FIN). There m= ay > still be some loose ends or clean up needed but try the code below at the > end of the while loop.**** > > ** ** > > I did not do a packet capture but I saw that if the receiving nc is not > running when you start the sender nc then the receiving dtntunnel logs an > error on the failed connect and returns an EOF bundle which closes the > sender nc. The EOF bundle could possibly be enhanced to include an error > message.**** > > ** ** > > I agree that some applications could benefit from a "pretend to be a > receiver" option with the default working as a bidirectional end-to-end T= CP > connection.**** > > ** ** > > Thanks for dragging me along kicking and screaming on this lesson in DTN > and TCP tunneling. I've only been using dtntunnel as a one way UDP > transmitter. I see a good fit for the TCP tunnel in an upcoming project a= nd > I'll try to make the pitch to use it. This is a pretty cool little app.**= * > * > > Best regards,**** > > DZ**** > > ** ** > > ** ** > > ** ** > > ** ** > > ** ** > > 484 delete b_recv;**** > > 485 **** > > 486 if (recv_hdr->eof_) {**** > > 487 log_info("bundle had eof bit set... closing > connection");**** > > 488 **** > > 489 if ( !sock_eof ) {**** > > 490 sock_eof =3D true;**** > > 491 // send an empty bundle back since we did not > initiate the close**** > > 492 dtn::APIBundle* b =3D new dtn::APIBundle();**** > > 493 hdr.eof_ =3D 1;**** > > 494 hdr.seqno_ =3D ntohl(send_seqno++);**** > > 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, > sizeof(hdr));**** > > 496 b->payload_.set_len(sizeof(hdr));**** > > 497 int err;**** > > 498 if ((err =3D tunnel->send_bundle(b, &dest_eid_)) = !=3D > DTN_SUCCESS) {**** > > 499 log_err("error sending final socket closed > packet bundle: %s",**** > > 500 dtn_strerror(err));**** > > 501 tcptun_->kill_connection(this);**** > > 502 exit(1);**** > > 503 }**** > > 504 delete b;**** > > 505 }**** > > 506 sock_.close();**** > > 507 goto done;**** > > 508 }**** > > 509 }**** > > 510 }**** > > 511 **** > > 512 done:**** > > ** ** > > > > > -- > Best regards, > Sergey Sireskin > > **** > --047d7b5d8cd795139604d10c9017 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable David,

I added this code and a log_debug inside the if-clause. Acco= rding to the log, this condition doesn't get true.
A have attached m= y logs and tcpdumps. Please, see dtntunnel_T.log - it's a receiving dtn= tunnel log.
There are 4 bundles after a eof bundle. This doesn't happen with the or= iginal dtntunnel.

--
Best regards,
Sergey Sireskin

2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC S= ERVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,=

I don=92t have your setup= and this has not been tested but give this a try. In TCPTunnel::Connection= ::handle_bundle in the while(1) loop add the test to see if the next_seqno_= is a match:

=A0<= /p>

=A0=A0=A0 while (1) {

=A0=A0=A0=A0=A0=A0=A0 iter =3D reorder_table_.find(next_seq= no_);

=A0=A0=A0=A0=A0=A0=A0 if (ite= r =3D=3D reorder_table_.end()) {

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;

=A0=A0=A0=A0=A0=A0=A0 }

=A0

= =A0=A0=A0=A0=A0=A0=A0 // add this code:

=A0=A0=A0=A0=A0=A0=A0 // if seqno does not ma= tch then still a bundle missing

=A0=A0=A0=A0=A0=A0=A0 if (iter->first !=3D next_seqno_) {<= /u>

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;

=A0=A0=A0 =A0=A0=A0=A0}

=A0

=A0=A0=A0=A0=A0= =A0=A0 bundle =3D iter->second;

=A0=A0=A0=A0=A0=A0=A0 log_debug("delivering %zu byte b= undle with seqno %d (from reorder table)",

=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 bundle->payload_.len(), ne= xt_seqno_);

=A0=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0reorder_table_.erase(iter);<= u>

=A0=A0=A0=A0=A0=A0=A0 next_seqno_++;

=A0=A0=A0

<= span style=3D"font-size:9.0pt;font-family:"Courier New"">=A0=A0= =A0=A0=A0=A0=A0=A0queue_.push_back(bundle);

=A0=A0=A0 }=A0=A0

=A0

=A0<= /p>

=A0

=A0<= /p>

David Zoller

COLSA Corporation

HOSC / C107=A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov

=A0<= /p>

From: dtn-users= -bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Monday, December 17, 2012 6:40 AM
To: David Zoller


Cc: dtn-users@irtf.org
Subject: Re: [dtn-u= sers] A problem with dtntunnel

=A0

Hi David,



T= hanks for your patch, it works! :) Connection is cleanly terminated now.
However, there still seems to be a problem. I run dtntunnel over a lin= k with high latency and about 30% packet loss. The receiving nc sometimes g= ets FIN packet before it receives all the data. If I understand it right, o= n a lossy link bundles may come out of order. In this case the EOF bundle m= ay come too early. I think that the reordering procedure in dtntunnel shoul= d be reviewed. Just for information, pure TCP performs better that DTN over= this link.

2012/12/14 = David Zoller <zol= lerd@gmail.com>

Hi Serg= ey,

I can't stay away from a problem that= is nagging at me so I made some time this morning. Going back to the origi= nal TCPTunnel.cc code, when a bundle is received that has the EOF indicatio= n, an empty bundle with the same EOF flag set needs to be returned to close= the loop (the second FIN). There may still be some loose ends or clean up = needed but try the code below at the end of the while loop.

=A0

I did not do a packet capture but I saw that if the receivin= g nc is not running when you start the sender nc then the receiving dtntunn= el logs an error on the failed connect and returns an EOF bundle which clos= es the sender nc. The EOF bundle could possibly be enhanced to include an e= rror message.

=A0

I agree that some applications could benefit from a "pr= etend to be a receiver" option with the default working as a bidirecti= onal end-to-end TCP connection.

=A0

Thanks for dragging me along kicking and screaming on this l= esson in DTN and TCP tunneling. I've only been using dtntunnel as a one= way UDP transmitter. I see a good fit for the TCP tunnel in an upcoming pr= oject and I'll try to make the pitch to use it. This is a pretty cool l= ittle app.

Best regards,

DZ

=A0

=A0=

=A0

=A0

=A0

484 =A0 =A0 =A0 =A0 =A0 =A0= delete b_recv;

485=A0

486 =A0 =A0 =A0 =A0 =A0 =A0 if (recv_hdr->eof_) {

487 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 log_info("bundle had eof bit set... closing connection");=

488=A0

489 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ( !sock_eof ) {<= /u>

490 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 sock_eof =3D true;

491 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 // send an empty bundle back since we did not initiate the close=

492 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 dtn::APIBundle* b =3D new dtn::APIBundle();

493 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 hdr.eof_ =3D 1;

494 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 hdr.seqno_ =3D ntohl(send_seqno++);=

495 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 memcpy(b->payload_.buf(sizeo= f(hdr)), &hdr, sizeof(hdr));

496 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 b->payload_.set_len= (sizeof(hdr));

497 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 int= err;

498 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ((err =3D tunnel->send_bundle(b, &des= t_eid_)) !=3D DTN_SUCCESS) {

499 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 log_err("error sending final socket closed packet bundle: = %s",

500 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 dtn_strerror(err));=

501 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 tcptun_->kill_connection(this);

=

502 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = exit(1);

503 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 }

<= p class=3D"MsoNormal">504 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 delete b;=

505 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 }

506 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sock_.close();

<= div>

507 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto done;

508 =A0 =A0 =A0 =A0 =A0 = =A0 }

509 =A0 =A0 =A0 =A0 }

51= 0 =A0 =A0 }

511=A0

512 =A0done:

=A0




--
Best re= gards,
Sergey Sireskin

=


--047d7b5d8cd795139604d10c9017-- --047d7b5d8cd795139a04d10c9019 Content-Type: application/x-gzip; name="early_eof.tgz" Content-Disposition: attachment; filename="early_eof.tgz" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hatnfxn90 H4sIALgtz1AAA+1daY/cuBH15/wKAgECDzDu4U1JwCbY+NgMsvAamcknwxj0dGvGje1pdfqw1/8+ RV0tkZSaOmYXTkTDcF98fKoqFovFkhzPd+tvd3HycPXi2RrGHCsh4N+0mf+mrwnFkkkqJecvMCGC yRdIPB+lUzvuD/MdQi92SXJo+92577/TFpf638ebZbybHRbb5fFpO+YYmGCc6tWpf0qVkob+Oafq BcJjkmhq/+f6z9WNXm028He7QusEbZPdAYXQULLL3hD97k8kiKiIMJ9hBmoK0PUH+ILOiAxmZEZn IgwCjP5a+yztGKF36/njHn28+XSJ9vF/EBGcUsYCjC/R19UGiYDDq2R7WCUb+NnTfo8Il/hyP1/8 +ss/L29v0Jf5GnEhiIJxQxQvdghfbpLt5df9Yr6OkQLgdbx5PHxGuEZTEFKlyYFUemF/tZmfWM5K mowTwqSklwiYnGiTjDajSgYmb8lC2Uq8fOd3AYx1l7O+gIxwfgEsI8xlha0eXf9tJ9nAS2FB+/HS gsURzPAg5+hNLcRN1DRanZ3CndVeCg1GyjhRFXYkFeJGeamwu7w+lJYIFxhRPKbMJDYIBkFvkQGz XGQM09FERkLV38S0sCLGYS4OkliVnWVllPS3MqCmCi/CRxMZJUOsTIsr4nhMmZlWRml/KwNmhci4 Gk1kjIv+VqaFFQnB+TNaGRP9RQbURC4yhccTmZQDrEyLK5JkTJmZVsaU6C0yYFaILBjCqc6IBz1E VkhMCytSIqTPaGW87mw7iQyosfFFJnCPiVlamRZXFJAxZWZamSCst8iA2XOILBxgZVpYUSh1JP5s ViZxf18G1Mj4IpNkiC/T4ooIpmMKzTQzSfubmab2HEILeX87S8UVEfjRwGi21dIU5f2lBl+HHaRG /KQGK/mQLYAWWEQggOoutkaCpq0pPsDWgFp/qZEGqcFmu0eioRSaFldEdMgyUGik2dYCzLonGUqp 6WjlGaTGBm03tcBgj87GFJthawHm/SMOTa2Umsf2yVNqPBiQ1EjFFRGhgu6Rrb+tDYnTNLdyO8BG k9qwQC0VWEQkG1Nspq0J0n9DoKkVUpMeCSFPqUk8ILuRiisiKmDd41t/W5MUD1GsZheRgI/J0VSs DIasDppbRPTvn1GKIemZwU0ppiZJCaPdV33qTzHooegy06bJRZRT2n0CN1KEUEJWKYYy6D+DNbcu e/pGVjXbC3vFc6XUtLwixpnsbnt1frSiWExYXWxK9g/oNLkumwfWRMsQWzBg85AKLOIi7JE+qvNj FbEREfAKR0GHWJsm529tAssmWrhGSeEBbiQVWMSDoPNGtc4P3jXOUaBIBoRPKblIBGCy41E0JwRw FP1jdU3Rf0K00KprNhhylJYKLJKM0RE1yxmvzQdGgrC31DRD/x2Or9TgegfET6m8IgV74a5upM3Y TDeiz4f756GB4vjGBkt1j41hmVTVAosCLDsfEbXwEyqkFYqSKkZ7S01z62BsxEtqQEkMcL6pvKJA hZ130wY/UuGnKDcoygHLakouCkPR3Ys0UzTnA3AMB2gWKPbWLGnUbIAHON9UYBHBOJRdw/M2udXD c82xxybn3WkjlvLL0tRjqhc2TxWa8G3YP7mfkqP+6qXBOfUyOQsBtcfC9a4UXHdusCoxyfy4sT6H 9eWexkuDdTrZuyqdav3fLl7Eqy/PUQHYXv8HcqDMrP9TjE/1f79H617/x/hMx8H2DgnCY0UqxltM Nqv8j2K9+w3Twpe8ji6slg211dGFkhDiVQCY8aT0XI1OSd0uAASiTGc3aFjUOeXERUncuwDwRLx8 53cBzPKqZwVdetW8jk2cKZdxcHPSCamkoeVQveik/jTKc6kduQnWxC3Ng9b4sbMFRra+XZnec2uQ RUs4du4ZJSPT6yWyD9VMr4jyDFZXufEmuamAGxzluZ1Ks9i6pNcsVoK7xSaZwFbq6qzY3lXTa90F FhJPauJcVs2WlhlVaIbSX2p1aq6kZE7NTvh5uwvaV1J1On/0kja1Du0U/y0Pm8Nxs4nXdz/P1snj iGO0x38E9oXcjP/S+3+m+O/520cC3kwJDlHNTFClAoWuQPtoGd8fHz+h+XK52jwi/cku3s53+xjt V48bcASf55vlOt79yQAIqUBXh9VTvDMg6t0Q/k09hGwpgkUcU4UeINLMf0EsSC4aOSWH+eEsJfDW HSktF6JGCVuY4GZPM6bAhV8nO418c/3Th+sPb61eilZ7bZLDahF/QjsgkDzB4hAv0WqP0h+aUghF iF1939y+v80+0XZ80IMft7PZzOwewkb41P3quNxe7ZPFr1e44L7YxSDKJdKfxgdkXDGDeYrbAa4/ 3KRdo2iRbB5Wj8ddbGHQMxi6gZXNn/azXXzcx6Co3V1kmoQGOnM1JpDexQCQfVVMeQPd75L5cjHf u4G0kXoCPR3XoLwmIMHOA60T2DTcpdJJ3yMXkCTngXbxU3KIK0gRemVLW3moLaOUyrmZkgq8KZ2Q XECBbAfax4d0Ltz8cvevt/++efvjmzf/MkAICblrRqUXguLNcpusNge0WiL4TXSlf0lpMIN/KiPP t1sTllHhgn1Ijpslin9b7VNiu/gRXu3mOsrSY8BuEMG4a/0ddL67X22WBjLlPKw5wv3nY3aVy+Tr xvgxC2pTpOjicBd53geuO7EdB+Bwl6erUgWAL9WOLN24Ep0OrHRcbR6ST+gxOejIl0t0/w0c9/1R u16zZ0BrZnJYbIthM1d9l3Vzg6GHHfjSisp4XWV/W8Yw8eLV8od2tf4le/kDjP4Kz9I/UZoGKd+l Ib9eJSrZuijL4L2qRdvZL19qLesdwCZB+MK85jBQxjVfgRvdXGnzeVidlq7CTW9X2/gSPSz3EZLV 9SmDg2CKesHBPLnbJJt7MPpfQZ8RqrxBT8ky1r+wwJV4RvDQZJ6bzib+ivQw8SKdMx9/NxV3UO/F J/NyCINpePgMWluWYilmXv4x/o3ED+LhHqOPt68/ZNMzil6Xl2phspA5McEssk8j9BG9EVS+VRjI vkJ+sDwgLrUWQyzjtU4Pa+L2pPu6gq1fbtsmrsDmdM7M5Wn/+J9jfIyLAUrzSV98s2BEB5hvYCpH 8N4/VGKHHEdi3suhsYBxJ4F05SGkIZAiJgEW1CK5Rpy2eCpHCrEX0tmoqoQL+8BZsVUBR9wKa4Zz R1gFHHV7yGY4d5xVwDFvuNZoq4Dj0hfOEXMRhcHApY0q3J62kWRD/FXASS8LbojC0jMAE1ExH8TW cCyD4hCLe02v3IFp/5lYbtgC5V4mvU+3kdfvr2+1b3/9y/v3b1/fXr//yYATuMEHOeFOMBr07c3t j3//+frmH2/fWKhuIRZo5ZK52qBtsl6Da0Iv9X42OR704a4VTIh6ZGsBapA77bHvCpcLFnhFdTSB 9PLx7TK9guNeL03wFyNzAMqcjqdtANgZP0Fc/bCEIG6P4i/xpr70h+CY3QvFyYjAoVOarTvb+bc1 OAv0Z6LNADqhl/DVITnM1xcWrnAHQ+bKsdzNV5s7HVlFaH44xE/bwszSbyB600PvTXhC3J6kDT59 DYsEucpAX+4vssBVf20NYMXDLQOkCOXqh00sJbxmWRR93a0O8ZeXF1pj8PlikZWsWnjSy42maLDE ZiCZHBFG6/jhkH+Uqs5CD1rtLLOIErE0CtDYYr1yWJgK3SFOFmHmscxnQIiTB3S/OmjHpafbYp3s tSmc4s86smRc+QUHOVAeHIC6Ml/R5B1CKaR7rrl9TgVHO5134NJMPEXMlaqAiHdz4yozmzzM70Es 30PArS9PucVlTpb3RbRZ20Wjl4WGtDPUO6sLc4DAc51q0HRdJYLMGJWhqJpldatMFKfurXLaUxEr AmzZKptg38VWmVxY12xNCPR1vtuAsDOh2VsS8LGwVMKLk2XrWb1N9odtskl1n21uvpljwaao62Yh YDMRiLCWLq2oNK0ZcKs07QnbE9NHNavUAvsuVEovrGu20hXnVEp7qDQfK+y6/+M4nElOKG5QKWW6 ZNWl0rwnY2YY3zJLTbDvQqXswrpmYS0zZ1TKuqu0GKsWJnirVIBiZINKW/QpXNfWrM/vT5n8wrpg aWVkzyiT91QmjFVLQ7Up848+O/1faK7z/9vf9fwf9pKc2Of/U/3n79LK0x6l9AYpIL3P/1MAhXUO 0fOwncuFjqqbz/9zSEZ6nv/n/TnuSqnl/L/AdJ2uNZ//571q6aOm8//0h8roG5L6eZ7/+X/WXZ47 Vm4+/88BgiHn/zlGOPj8PwNS1P+0veH8Pwcafv6fA3HuDdRw/p8DDT//z4Gkh7DPnP8XSB7H9u3n /zmQOnNsf/78PwcKzki7/fw/AwmplVxfr/aHeBPvfI5dAxRaiNbWrRmx2+FoMYB47gFCK7lRDkCa fIXpHhl25PIdMGc9hq46ar5g0tFtaDTrNMADze07dBGSdeTTjtbiQHQlkpVAbEdr8SK6HMlKeTeg nXclGs1Xbh7+RJc4WTnjVnJtTkXXOTVPYye5Fs+iS5R8LPe8e2GYUB8kXeeTHzLU9nEWmvIxtuwj wDO7U+qjwfox2M/XN7dv31dPwQow7YXO1jQoEd4/1Gsafs6H/GQi8gbEWkXDa8F+JHZFQxMoCe1j H6cqd7ALNXjD0rmDXrYgqcDOMszWorFn3eObBPUNrT3Kz3Bb+VmKzIlOlpwvP0t/LPXTvYaVn+U4 QedqjezB6QH18adDjkKzcUL7SNhznNYT0eKh3D6zfr5YxFs9V6oHN0sks2STlf8xB6FBy9GV9CpH yYHcx84G0PlqlPwhvm3nLI1odjFKjmYHEu1oDbUoxUOjW6o9XGgNpSjFE5595eZYrF/Zgmuo/bPh HKs10/9bAJiLBdpwrNbEsakQpUBrqaNwUayu2U4jZvZMcSC216HkT0K2twW+lZiEIcItPOYuQRlY LZmDK7dZjwIurN1GJYECV25UZEIIY58UO9a1vue7+SNK9ap+LvhIEzk6uRgKj6rK7LlUNPCIQf7+ DppnVWUGy0KnOfUp7skAub2w9S3uwemqZg3growbtpSFQUOVTH2CRpHmmJedSEfVSfbIJME9KoVO RaquYqH0W7NeKEdXTgfqpbJXxIYLnA5gLIUJXTg1usIEY22lc20KI4oa7llS0VB4U1PYqT6hVBgt FaYfLxI4FSZ1hdR4CpP6JvfnVBgM4F4SBikMSEufgMlWGKOKyzqY4pyer7yqVB+UCmOlwlSghHuG AbqdKOuvMP0f27Tqf6jC9HP5OvvcswoDofdUGGWKGDMsgJ1S6zqTzbBTbUGpMF4qTD/npmGKAbyd zeivMYCzTtBH1RgMYJ1YD9cYoIY+UaatMUNbIUSrrR7AXeoqqqoSsP9zqQqw3bcSdVcVE7OQMGid aiTybtwymN41Eq4YcoT7rjKa7RWm1ZteqM8dLyno8DteMhi3kXS54yXFkbTV27onG2nIgVjgzkRP ++00WUcxcg15iqqsUquxar0zeDsNO1qtdzpA/XbL3rXeGVa7k+3gCNLnleHa7UE+jiDrZtf2/fGO wHF17TGElyMgFmhDRszfEaQwxD2BuziCDId1rmbNO7qr/Ad7kAy8fW/Q3RFkT/5zx0MjOIIMvuHm tjEcQTbAODd9ZFgN+41Btz1kyG4XYwRE3e56SIF5262Np3yex00PGdwINz2MmMrKSPmGCT63KqSp R8upNeRJPNWTCvKPLlCb2tSmNrWpTW1qU5va1KY2talNbWpTm9rUpja1qU1talOb2tSmNrWpTW1q U5va1KbW2P4LZwb7wwCgAAA= --047d7b5d8cd795139a04d10c9019-- From ssireskin@gmail.com Mon Dec 17 06:04:39 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8F3A821F8AFA for ; Mon, 17 Dec 2012 06:04:39 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.046 X-Spam-Level: X-Spam-Status: No, score=-3.046 tagged_above=-999 required=5 tests=[AWL=-0.048, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QazWzaF3gPyf for ; Mon, 17 Dec 2012 06:04:38 -0800 (PST) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by ietfa.amsl.com (Postfix) with ESMTP id 2F73021F8AF4 for ; Mon, 17 Dec 2012 06:04:38 -0800 (PST) Received: by mail-vc0-f182.google.com with SMTP id fy27so4976088vcb.13 for ; Mon, 17 Dec 2012 06:04:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=E1zOvEjdfyjj37N/rNBdWmNl6wYDxTHCJtJSOi3CLAM=; b=cMFjEC/B+OC/JbC+elrr2nmK6jCitvnNOjM752RtE9k16HIt0j1QSmEDtUAoP2oHmy 4YKhdMLEq7s6b5KS3g8yIFazr955UkVSXn5yMBUQJJzI3lfxIjvqQ7szpmknlTwwO+dw +E3FVQVhyBqI0UvsvufJMPNmXeI+X41r8FWDvJ+kcq2Jqh/oOAs8IqH1WpR6p+Z0zfc3 /SN7/ulEGsffhj2svoVsap3Aa3hzyL7xRSzlMVoDQEwU/OQG317913G7ssQMcWmqOURr Cqcfqxgq1eU8xsfJnsDD3ol7APeCzewVaU4EJ4IXujM3QfkI0o0TVR2Mj2GAzO7SuMsy r4kQ== MIME-Version: 1.0 Received: by 10.52.70.205 with SMTP id o13mr20295173vdu.75.1355753077546; Mon, 17 Dec 2012 06:04:37 -0800 (PST) Received: by 10.58.219.65 with HTTP; Mon, 17 Dec 2012 06:04:37 -0800 (PST) In-Reply-To: References: <04E3D99A62496240BCD6A576813E6E31E0BDF5C59C@NDMSSCC05.ndc.nasa.gov> Date: Mon, 17 Dec 2012 17:04:37 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/alternative; boundary=20cf307f32b0971f2904d10cdc7a Cc: David Zoller , "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 14:04:39 -0000 --20cf307f32b0971f2904d10cdc7a Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable David, The problem is probably not with the reordering. Can it be, that "if (recv_hdr->eof_)" is not the right condition to close a connection? Or maybe it is not a sufficient condition? Which connection is recv_hdr about? Can it be a local TCP connection on the receiving side (dtntunnel <-> nc)? If it is, then IMO this condition becomes true when the receiving application doesn't send any data to the sending application, please see line 406 of the original code. 2012/12/17 ssireskin@gmail.com > David, > > I added this code and a log_debug inside the if-clause. According to the > log, this condition doesn't get true. > A have attached my logs and tcpdumps. Please, see dtntunnel_T.log - it's = a > receiving dtntunnel log. > There are 4 bundles after a eof bundle. This doesn't happen with the > original dtntunnel. > > -- > Best regards, > Sergey Sireskin > > > 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov> > > Hi Sergey,**** >> >> I don=92t have your setup and this has not been tested but give this a t= ry. >> In TCPTunnel::Connection::handle_bundle in the while(1) loop add the tes= t >> to see if the next_seqno_ is a match:**** >> >> ** ** >> >> while (1) {**** >> >> iter =3D reorder_table_.find(next_seqno_);**** >> >> if (iter =3D=3D reorder_table_.end()) {**** >> >> break;**** >> >> }**** >> >> ** ** >> >> // add this code:**** >> >> // if seqno does not match then still a bundle missing**** >> >> if (iter->first !=3D next_seqno_) {**** >> >> break;**** >> >> }**** >> >> ** ** >> >> bundle =3D iter->second;**** >> >> log_debug("delivering %zu byte bundle with seqno %d (from reorde= r >> table)",**** >> >> bundle->payload_.len(), next_seqno_);**** >> >> **** >> >> reorder_table_.erase(iter);**** >> >> next_seqno_++;**** >> >> **** >> >> queue_.push_back(bundle);**** >> >> } **** >> >> ** ** >> >> ** ** >> >> ** ** >> >> ** ** >> >> David Zoller**** >> >> COLSA Corporation**** >> >> HOSC / C107 **** >> >> (Office: (256) 544-1820 >> *EMail: david.a.zoller@nasa.gov**** >> >> ** ** >> >> *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *= On >> Behalf Of *ssireskin@gmail.com >> *Sent:* Monday, December 17, 2012 6:40 AM >> *To:* David Zoller >> >> *Cc:* dtn-users@irtf.org >> *Subject:* Re: [dtn-users] A problem with dtntunnel**** >> >> ** ** >> >> Hi David, >> >> >> Thanks for your patch, it works! :) Connection is cleanly terminated now= . >> >> However, there still seems to be a problem. I run dtntunnel over a link >> with high latency and about 30% packet loss. The receiving nc sometimes >> gets FIN packet before it receives all the data. If I understand it righ= t, >> on a lossy link bundles may come out of order. In this case the EOF bund= le >> may come too early. I think that the reordering procedure in dtntunnel >> should be reviewed. Just for information, pure TCP performs better that = DTN >> over this link.**** >> >> 2012/12/14 David Zoller **** >> >> Hi Sergey,**** >> >> I can't stay away from a problem that is nagging at me so I made some >> time this morning. Going back to the original TCPTunnel.cc code, when a >> bundle is received that has the EOF indication, an empty bundle with the >> same EOF flag set needs to be returned to close the loop (the second FIN= ). >> There may still be some loose ends or clean up needed but try the code >> below at the end of the while loop.**** >> >> ** ** >> >> I did not do a packet capture but I saw that if the receiving nc is not >> running when you start the sender nc then the receiving dtntunnel logs a= n >> error on the failed connect and returns an EOF bundle which closes the >> sender nc. The EOF bundle could possibly be enhanced to include an error >> message.**** >> >> ** ** >> >> I agree that some applications could benefit from a "pretend to be a >> receiver" option with the default working as a bidirectional end-to-end = TCP >> connection.**** >> >> ** ** >> >> Thanks for dragging me along kicking and screaming on this lesson in DTN >> and TCP tunneling. I've only been using dtntunnel as a one way UDP >> transmitter. I see a good fit for the TCP tunnel in an upcoming project = and >> I'll try to make the pitch to use it. This is a pretty cool little app.*= * >> ** >> >> Best regards,**** >> >> DZ**** >> >> ** ** >> >> ** ** >> >> ** ** >> >> ** ** >> >> ** ** >> >> 484 delete b_recv;**** >> >> 485 **** >> >> 486 if (recv_hdr->eof_) {**** >> >> 487 log_info("bundle had eof bit set... closing >> connection");**** >> >> 488 **** >> >> 489 if ( !sock_eof ) {**** >> >> 490 sock_eof =3D true;**** >> >> 491 // send an empty bundle back since we did not >> initiate the close**** >> >> 492 dtn::APIBundle* b =3D new dtn::APIBundle();**** >> >> 493 hdr.eof_ =3D 1;**** >> >> 494 hdr.seqno_ =3D ntohl(send_seqno++);**** >> >> 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, >> sizeof(hdr));**** >> >> 496 b->payload_.set_len(sizeof(hdr));**** >> >> 497 int err;**** >> >> 498 if ((err =3D tunnel->send_bundle(b, &dest_eid_))= !=3D >> DTN_SUCCESS) {**** >> >> 499 log_err("error sending final socket closed >> packet bundle: %s",**** >> >> 500 dtn_strerror(err));**** >> >> 501 tcptun_->kill_connection(this);**** >> >> 502 exit(1);**** >> >> 503 }**** >> >> 504 delete b;**** >> >> 505 }**** >> >> 506 sock_.close();**** >> >> 507 goto done;**** >> >> 508 }**** >> >> 509 }**** >> >> 510 }**** >> >> 511 **** >> >> 512 done:**** >> >> ** ** >> >> >> >> >> -- >> Best regards, >> Sergey Sireskin >> >> **** >> > > > --=20 Best regards, Sergey Sireskin --20cf307f32b0971f2904d10cdc7a Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable David,

The problem is probably not with the reordering. Can it be, t= hat "if (recv_hdr->eof_)" is not the right condition to close = a connection? Or maybe it is not a sufficient condition? Which connection i= s recv_hdr about? Can it be a local TCP connection on the receiving side (d= tntunnel <-> nc)? If it is, then IMO this condition becomes true when= the receiving application doesn't send any data to the sending applica= tion, please see line 406 of the original code.

2012/12/17 ssireskin@gmail.com <ssireskin@gmail.com>
David,

I added this code and a log_debug inside the if-clause. Acco= rding to the log, this condition doesn't get true.
A have attached m= y logs and tcpdumps. Please, see dtntunnel_T.log - it's a receiving dtn= tunnel log.
There are 4 bundles after a eof bundle. This doesn't happen with the or= iginal dtntunnel.

--
Best regards,
Sergey = Sireskin


2012/12/17 Zoller, Dav= id A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

Hi Sergey,=

I don=92t have your setup= and this has not been tested but give this a try. In TCPTunnel::Connection= ::handle_bundle in the while(1) loop add the test to see if the next_seqno_= is a match:

=A0<= /p>

=A0=A0=A0 while (1) {

=A0=A0=A0=A0=A0=A0=A0 iter =3D reorder_table_.find(next_seq= no_);

=A0=A0=A0=A0=A0=A0=A0 if (ite= r =3D=3D reorder_table_.end()) {

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;

=A0=A0=A0=A0=A0=A0=A0 }

=A0

= =A0=A0=A0=A0=A0=A0=A0 // add this code:

=A0=A0=A0=A0=A0=A0=A0 // if seqno does not ma= tch then still a bundle missing

=A0=A0=A0=A0=A0=A0=A0 if (iter->first !=3D next_seqno_) {<= /u>

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;

=A0=A0=A0 =A0=A0=A0=A0}

=A0

=A0=A0=A0=A0=A0= =A0=A0 bundle =3D iter->second;

=A0=A0=A0=A0=A0=A0=A0 log_debug("delivering %zu byte b= undle with seqno %d (from reorder table)",

=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 bundle->payload_.len(), ne= xt_seqno_);

=A0=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0reorder_table_.erase(iter);<= u>

=A0=A0=A0=A0=A0=A0=A0 next_seqno_++;

=A0=A0=A0

<= span style=3D"font-size:9.0pt;font-family:"Courier New"">=A0=A0= =A0=A0=A0=A0=A0=A0queue_.push_back(bundle);

From: dtn-users= -bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Monday, December 17, 2012 6:40 AM
To: David Zoller


Cc: dtn-users@irtf.org
Subject: Re: [dtn-users] A pro= blem with dtntunnel

=A0

Hi David,



Thanks for you= r patch, it works! :) Connection is cleanly terminated now.

However,= there still seems to be a problem. I run dtntunnel over a link with high l= atency and about 30% packet loss. The receiving nc sometimes gets FIN packe= t before it receives all the data. If I understand it right, on a lossy lin= k bundles may come out of order. In this case the EOF bundle may come too e= arly. I think that the reordering procedure in dtntunnel should be reviewed= . Just for information, pure TCP performs better that DTN over this link.

2012/12/14 David Zoller = <zollerd@gmail.co= m>

Hi Sergey,=

I can't stay away from a problem that= is nagging at me so I made some time this morning. Going back to the origi= nal TCPTunnel.cc code, when a bundle is received that has the EOF indicatio= n, an empty bundle with the same EOF flag set needs to be returned to close= the loop (the second FIN). There may still be some loose ends or clean up = needed but try the code below at the end of the while loop.

=A0

I did not do a packet capture but I saw that if the receivin= g nc is not running when you start the sender nc then the receiving dtntunn= el logs an error on the failed connect and returns an EOF bundle which clos= es the sender nc. The EOF bundle could possibly be enhanced to include an e= rror message.

=A0

I agree that some applications could benefit from a "pr= etend to be a receiver" option with the default working as a bidirecti= onal end-to-end TCP connection.

=A0

Thanks for dragging me along kicking and screaming on this l= esson in DTN and TCP tunneling. I've only been using dtntunnel as a one= way UDP transmitter. I see a good fit for the TCP tunnel in an upcoming pr= oject and I'll try to make the pitch to use it. This is a pretty cool l= ittle app.

Best regards,

DZ

=A0

=A0=

=A0

=A0

=A0

484 =A0 =A0 =A0 =A0 =A0 =A0= delete b_recv;

485=A0

486 =A0 =A0 =A0 =A0 =A0 =A0 if (recv_hdr->eof_) {

487 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 log_info("bundle had eof bit set... closing connection");=

488=A0

489 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ( !sock_eof ) {<= /u>

490 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 sock_eof =3D true;

491 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 // send an empty bundle back since we did not initiate the close=

492 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 dtn::APIBundle* b =3D new dtn::APIBundle();

493 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 hdr.eof_ =3D 1;

494 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 hdr.seqno_ =3D ntohl(send_seqno++);=

495 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 memcpy(b->payload_.buf(sizeo= f(hdr)), &hdr, sizeof(hdr));

496 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 b->payload_.set_len= (sizeof(hdr));

497 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 int= err;

498 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ((err =3D tunnel->send_bundle(b, &des= t_eid_)) !=3D DTN_SUCCESS) {

499 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 log_err("error sending final socket closed packet bundle: = %s",

500 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 dtn_strerror(err));=

501 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 tcptun_->kill_connection(this);

=

502 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = exit(1);

503 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 }

<= p class=3D"MsoNormal">504 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 delete b;=

505 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 }

506 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sock_.close();

<= div>

507 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto done;

508 =A0 =A0 =A0 =A0 =A0 = =A0 }

509 =A0 =A0 =A0 =A0 }

51= 0 =A0 =A0 }

511=A0

512 =A0done:

=A0




--
Best re= gards,
Sergey Sireskin

=





--
Best regards,
Sergey= Sireskin


--20cf307f32b0971f2904d10cdc7a-- From ccaini@arces.unibo.it Mon Dec 17 11:03:15 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3C12721F86D6 for ; Mon, 17 Dec 2012 11:03:15 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 1.581 X-Spam-Level: * X-Spam-Status: No, score=1.581 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_IT=0.635, HOST_EQ_IT=1.245, MANGLED_MEDS=2.3] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hugfNRR3hDHI for ; Mon, 17 Dec 2012 11:03:14 -0800 (PST) Received: from mail.arces.unibo.it (mail.arces.unibo.it [137.204.143.6]) by ietfa.amsl.com (Postfix) with ESMTP id D315021F86CB for ; Mon, 17 Dec 2012 11:03:13 -0800 (PST) Received: from arces-ef8245339.arces.unibo.it (mars-fw.arces.unibo.it [137.204.143.2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.arces.unibo.it (Postfix) with ESMTP id 8F273407F34C for ; Mon, 17 Dec 2012 20:03:07 +0100 (CET) X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 17 Dec 2012 20:03:05 +0100 To: "dtn-users@irtf.org" From: Caini Carlo In-Reply-To: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=====================_8394921==_" Message-Id: <20121217190307.8F273407F34C@mail.arces.unibo.it> X-Arces-MailScanner-Information: Please contact the ISP for more information X-Arces-MailScanner-ID: 8F273407F34C.29493 X-Arces-MailScanner: Found to be clean X-Arces-MailScanner-From: ccaini@arces.unibo.it Subject: Re: [dtn-users] Use of metadata with DTNSEND and DTNRECV X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 19:03:15 -0000 --=====================_8394921==_ Content-Type: text/plain; charset="us-ascii"; format=flowed Dear Christos and others, I have personally repeated the tests carried out by Luca Tartarini, a thesis student of mine, on DTN2 2.9 and on 2.8+ (not the 2.8 stable version, but one obtained from Mercurial), obtaining the same results of Tartarini. I can therefore confirm the bug, which is on the sending side, i.e. on dtnsend. The bundle with metadata (-M1), if addressed to a node different from the source is not actually transmitted, but is added to the list of pending bundles. Viceversa, the same bundle is transmitted and received correctly if the source and destination are the same. Moreover, without the -M 1 option bundle are sent correctly in both cases. The fact that this bug is not present in stable 2.8 is an important information, but we would like to use the new one, of course. Yours, Carlo Caini At 08.23 11/12/2012, TSELIKIS.Christos wrote: >Content-Language: en-US >Content-Type: multipart/related; > >boundary="_004_A1E9924741E2C74A9ABA10C2DDBAA117013BE7HAIMAIL2010SRV1ha_"; > type="multipart/alternative" > >Hi Luca, >may be is trivial, ok with dtn2.8 (without vm): >root@dtnrouter:~/dtn/usr/bin# ./dtnrecv -v dtn://192.168.1.51.dtn/a >dtnrecv (pid 8945) starting up >opening connection to dtn router... >opened connection to dtn router... >calling parse_eid_string >find registration succeeded, regid 0 >register succeeded, regid 11 >looping forever to receive bundles >dtn_recv [dtn://192.168.1.51.dtn/a]... >bundle spec at 0xBFFA3C04 > >0 extension blocks from [dtn://192.168.1.100.dtn/me]: transit time=0 ms > >1 metadata blocks from [dtn://192.168.1.100.dtn/me]: transit time=0 ms >Metadata Extension Block 0: > type = 1 > flags = 0 >0000000 6d65 7461 | meta > >14 payload bytes from [dtn://192.168.1.100.dtn/me]: transit time=0 ms >0000000 7465 7374 696e 6733 2d63 6169 6e69 | testing3 > > >Freeing metadata block [0].data at 0x08060070 >dtn_recv [dtn://192.168.1.51.dtn/a]... > >With very br, >ctselikis > > >Tselikis Christos >Hellenic Aerospace Industry S.A. >Electronics Engineering Department >e-mail: Tselikis.Christos@haicorp.com >PO Box 23, GR-32009 Schimatari, Greece >Tel. : +30 22620 52561 >Fax: +30 22620 52910 >untitled > > >From: dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] >On Behalf Of Luca Tartarini >Sent: Thursday, December 06, 2012 8:32 PM >To: dtn-users@irtf.org >Subject: [dtn-users] Use of metadata with DTNSEND and DTNRECV > >Dear DTN2 users, I am a student of the University of Bologna and I >am carrying out a thesis on the possible use of metadata for DTN >routing. I have a problem in the use of metadata with the >applications DTNSEND and DTNRECV, DTN2 2.9.0 on Debian. >If I try to send a bundle with metadata from vm1.dtn to vm2.dtn >launching this command on vm1.dtn: >vm1: dtnsend -v -s dtn://vm1.dtn/me -d dtn://vm2.dtn -t m -p >"testing3" -M 1 -S "meta" >while on vm2.dtn dtnrecv is running (vm2: dtnrecv -v dtn://vm2.dtn) >I do not receive anything on vm2.dtn, while in the dtn.log of >vm1.dtn I can see the following two errors: >[1354724317.364108 /dtn/bundle/protocol error] >BundleProtocol::prepare_blocks: 8-prepare returned BP_FAIL on bundle 10 >[1354724317.364128 /dtn/bundle/actions error] >BundleActions::queue_bundle: >prepare_blocks returned NULL on bundle 10 > >Note that: >1) without the -M option the bundle is received correctly (of course >without metadata). >2) if the source and the destination are the same, i.e. if I launch: >vm1: dtnsend -v -s dtn://vm1.dtn/me -d dtn://vm1.dtn -t m -p >"testing3" -M 1 -S "meta" >both the bundle and metadata are received correctly. > >Any help will be greatly appreciated. > >Thanks, Luca Tartarini. > > >_______________________________________________ >dtn-users mailing list >dtn-users@irtf.org >https://www.irtf.org/mailman/listinfo/dtn-users --=====================_8394921==_ Content-Type: image/jpeg; name="800ed6.jpg"; x-mac-type="4A504547"; x-mac-creator="4A565752" Content-ID: <.0> Content-Transfer-Encoding: base64 Content-Disposition: inline; filename="800ed6.jpg" /9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEB AQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQH/2wBDAQEBAQEBAQEBAQEBAQEBAQEBAQEB AQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQH/wAARCAA8AF8DASIA AhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQA AAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3 ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWm p6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEA AwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSEx BhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElK U1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3 uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD++iii igAooqjc3VrYWs9/f3NpbQWqs93dXhGbJVAzycjA3DncByOvSpnOEIuc5xhBJ3nNqMErXbbbSslq 9dg0TV9Pzt5X8i5nJwDz34J/wpCXHYH1xn/Ofwr5B8b/ALc/7MHgmS4gm+Kmk+JNTtgfMsPAdrq/ jyVM8fd8KRXkT/8Af0D64r5G8d/8Ffvg74VjC6H8Kvit4lc/eFwfA/hoA+gefxWSQB38tfTA4r8z zPxY8MsixX1TOuMeFaVS38ONalOakraNU51Y3XV3evZnNic1wODSvKO6Ts739Lb9dWkrtdT9eSwG epwccfQH+o/Ok4PJH5gZ9Pr3HX1FfzreJv8AgvrZ6CP9C/ZT1W8P966+Lugxj6YPhWQ/Q7v1rwrX f+Dlj+w8+f8AsY3lwO2344L/AF+E7e/b1rjwvjR4Z452wfFtN66WUnF/CtFKCaXa/S+zORZ/k2lp p7apPfS29vXpt5n9UFFfyYWf/B1D8NLV/sviH9jX4nWxP8Wh/FPwVqxGOw+0eG/CP49fTjNfRngr /g55/YO1wQjxr4C/aK+HNyy4QXvgbRfEunA8ctceG/FlwD7Zt89uwx9NhOOOFsdbkzuErtW1sne3 ls7K3TV3uylm2Ak0lZt7atL72l/l5n9IdFfLf7NP7XfwY/ay8Ijxh8FtV8b6vobWS3jXfjH4S/Ff 4eqpKFQ1v/wsrwh4P/4SBd+HJ8LfaYlwUZ0Uhh9SV9ThsWsVG+ES5Va7ls1ovv1e2unqel2ffb8H +qCiiue8ReItD8K6DrXibxBqdppOi6Do2r67rWq322PTtO0rw9E9xruryBdxURQb3YZLFYyw3ZKv 1SlyRcntFN66J2V7Xegm7JtdE2n00V9z82f+Cin/AAUV0D9ifQvDPh/w9oFn43+NHj+01W98LeGb i/j0zStE0q3Z0m8aeJGaNsWkUgKbVKStMsyny48rN/N98Qv2uPjZ+0dq39rfGT4jar4mtPvWXhmy J0v4f2DbQpeTQWJ8JyOwUb5HZnYjLMetfn7+0n+1f8Q/28P2r/id8TPh54Q1/wAcz67rQ07wLY7V Xwp4O+G/hxseHj4h19AqlmJLuQqoT4lY8ZxX0x8Mv2O/EE8NrrHxw8cHVLrt4L8F/wDEq0nTvT/i oOR0+vY+pr/M/wCkT4p5hisdmmCxnGayHI5+5T4Vp1IOrVUHycz9nJTjzyTkuZRi23yXS0/O82zb F4tuMLKMZNJJNJpO13brbfr0PZLH4t6f9jOn6Pcf2p2/0LnSf8jjsSKwdV8OfF/xx/yB/D+k6BaX fH9s+J778+uMe2eBX2PpXw6+G/ws8FaD4o8L3/gm1uhrYsL7RzfjVtW0/SPDpOde8Q87j4XG3nKg cjBPzAW/C0HjD403l1qHwn8H/wBvaX9u/wCJ58QdZvtC8AfBHw7rH/ZQfXnH/FF/8J3+or+SsJwt n2a42GE4T4SqVs7m4uNWfNODUnFqW8lqndq77M8VKUrLVv5tdPXy/U/NXxL+zNq2ftHjb4neINTH /Pn4MstD8LaVkc9eF6f7PH1ry3Sv2erfxXrF14f+E/w38Q/EbxTaf8f32Ea54q/s4H/oYvEPiX/i j/DP9f5f03fCH/gmtonjZbLxB8SNQ1T4rzFcmxK6/wDC34AWTHIKr4fzH8WPiO0ZMbDLeC/Ac6iR UjQpvP6a+HfgT8FPgt4ahfVNN8LWui6BZm6sdEttB0Pwt4D0k+HreeUP4a8AeHYZLfeREvlS3ieM vGBldImuHkIjf+1/D/6NvHywUcw484tpcOq0ZOjGFOnFr3ZcrilCOzS1asnutbfRYTh7GWTbXLaL bbvZS9U0r79raXveJ/KP8Bv+CIvx2+OL2+teOb/w/wCBfDF0jNex6FYpqjNtDHA8ROyRpu27Q/hB /G8QYgO6g5r94f2df+CT/wCwZ+x+dE8SeJvCnhPxh48u7nTbCy8YfFwaPqhbWRhEXw9pfiDMNvIS xJRPPnC4kXaVOPqvxV+04jaxofh3wFokctj4o8K6be+Gtbu3Sw1Y6h4nh8Sr4I1nQvBMyC68ReFp JvDkbyyRTQkDV7LIVY5Y3+X9Ij1r4s23hK48YX134ubXNJTwnb3+uWmkaz8QNA8GfGfQ4JtG17XY vChi8K6FP4T+OPgCRIZYBPGngzhxLJC8D/p9NcA8M5jDLuFcqqcR8QKUOab5nwymnG8nZ2u1aSSu 09U9Un6WDwWCwjStzSuldp2jK6tdbKzaTtd6pJptI/XxQIYfKt8W1tbKAD/snoAOw5z7k5JzgU+u F+G1x4nvfAnhG68c2a6b4wfRNJbxJZDb8mrRoiyLkHkiQM7c8FiCOOO6r+ssBi44rBwxFownyL93 F6xdldcvxadNFtofV9rbWVvLTZ+a2fmZ91cW1hBcXFxcW1vbWoa9vLq6YYs8jOSWwBhVJDFlAG7k 5Gf5cf8AgoN/wV6t/i43jD9nz9mS4sz8KNTs9b8I/EH4sXlh/acfj/SHabw34j0T4dpucHw80Uxj m8a7ipw3lLGruH+kv+DgD9s+8+An7O2gfs/eCdcXTfiB+0zdaxoGsXun3gOqaD8INAVG+IEm5pGe N/F27SPBgVVVJILrxQGd5Icj+d/9hf8AYS/ac/bSvLK/+FPhd/DXwzgv/wCztY+MfjyN9P8AANic qpbw6qgj4k+JFZ0WS48IGO3hLoZpYwwJ/A/F/inifMMT/qRwIms5qpKpVV7KM42ac17tuVtvXrrZ 3Pms1xWOf+w5frdWlJavW2l1olazb8mnpodj8OtW8IfDvQrbT9Ig8P8Ag/wzpZ/5c86XpRPT+Y+v 1r6D0Tx9448RWQufBHhgW2l3nT4gfEA/8It4UHP/ADL3h8f8Vd4l9RgtgjJAzX7+/A3/AII3/sjf CzwpJpHjvT9U+O3jzU7I2epfEXx9Ctj/AGDe4iMsnw68OeGnh8NfDZ0YFbe5td/jn98Uk8b3LrGw +svgj+xP8E/gdJZatZWOpeN/HFpHI0Hjvxvt1jVbFcMCmhZiXQfDangxrZQwyk5DOyMFH8yYz6HH E+eY/K6+c5vwxUlKXPXnNVKk48zUqjUYrmq1d+VucoNtppu7j5C4Uxb5bzVrXa3a206a203S2vJH 4q/CX9im7v7HRviV+0ILybR9Q8T+DtC0jxF8ZNDB8KprPiXXPDfh3wynw+/Z+8OMkKSyeKfEA2eL fG8isQ6yELF+8H7IaX4X+CnwS1W9/wCE2s9eYeBPC2j+IP8AhcnxJstLXwFpUniPW7nw7HoPgBom g8J+GPEQuIEMng/wd4MsVdNe8LQB5pLmO3r5b/aQ+Csp+N/xT01te8R+K9b+L37M3xJ8XfCW58Xa 8uqj4ffEP4PeO/D3iQaJ8P08mMeFvD01x4p8L3MsY58zw4SZZct5nzfqq/Ez4tCDQrjxV4m+Jl94 n8MS6h4W1fWbFI9QTQf2g/hR4d+M/wAGblm8MLGq+FvCHxx+DPiDwmC2PLTCMzBMn/S3ws+jtwFw fkGT08uzGMZUYOeKqV6TVVwUKVqmGdvZfxPbUHCb541VC0ZwjVlH4vF8S4zJ8Y8vjk3PJtQhO6lr qrysrq9lNct+VWTle1/qH4cftN+FtM/aDu/DPw81XSNT8HftX/EDwn8QvDN54ki8QaONOePw54k+ HPxi0SPS/ESQTJ4pi8TfB3w//wAUsq27Rv4meQIuBjv/ANpWO38I/FhvGuv6fq2ozxaT8P8AW/A1 8tjqepr5Oga74j0H4weBIUhWZR/wkvwy8Qvrgjk8svJBPNIsm2MyeXePP2afBWi6B4q+Mn7R0uoa H4EutZ1vxUnhHwxa63r3jvRtT+P2h/DWPxBoWkT6A4ufD/i7wn8evD8firwH4v8ABBnnErFmZAst zXog+KGh/EXXte+Dfx51bxz8K7i1134cab8LNIfVRY+KfF2m/ET4Va8n/CE/EHXPD974u8I+Jde8 TS+GviHcy20U6yutuvkSfaQnmvxv8OY+InCcaPBHMs3yajTq1ak1zc1WNSjhm4zTjKu1QnRWIkqE aVOo41ZtVKtSEPXyDN8ZhH9Sz5qPPOXJz/ZUrttxk7K7clkl5SvZaNQTPZNA/Z21PXvDXw7Gr+Kt Q8Paj4U0fWPAtzqPh4aHejx38LP7cttZ8ORpKYpB4fa4g0bRX82FfMjCzwPvWYeX3XiDxz8GfgPJ 4i1LQtEtE1nX/ECXPjOz8AaRBd6tJcJpUviDWte1+S3idnj8O+HH1bxM4l2yPCNsNu0tywbxLx74 21X4mfCb4DeI/DOveIfhD4S+KGg6hYoLXWhpeoeHdY17wLdv4C0fV9ajSWQaMuv6d5N1LEitcl7W M7EfaOM8FeAtQ+M/gfU77w7omhR391428JfHXwn/AG/Z6ofAevaz4i0BvDvxA8F67rmhI4le18T6 D4llZBHMqh/CTSW08a8fzFmduFOIcy4T4Y4Ri+OcmhThKrOzjOXD/DsItxbbXvSk1pfnm3Fxcmfc YbE35fqK5lKKUZTV3JWhJcr+z7rfvKLu3yyjfVaHxE+Nfi7x/b/ETT2K3PgDwrrTanfP4LGu6SdR 8JeHdejPiPwPr+uSGT/hJv8AhL/hede8Wp/wh+WjRUjLlQA/1L+znPOvhHVfBWqyXl5F4E8Tav4c 0vVr5nvBr/g4tD4g8B6qmqrtim8zwjr2gIxDht6FcEOaq+B/2edF0S90LxR4u1C41zxda6Ts1uxs rtk8Dajqx8Ot4YbXh4bkJj8weGv+KaWR1Ie1+VrfJ84/RVtFaadbwRwW1tDZ2araWtlaqALRQn3e OANqAEBVA4xnJNfUeH/BnFOGzN8UcWZs/wC2akW/ZRk1FKdm4qN4xVr2etk07bu/oYWjjU3i8a9b Xik7uN2nZJWS2V1qk1slJ3/n2u/+CROtfte/t3fFX9sX9unVDq3wwstcHhv4A/s06JcSm1X4deG1 fw7oWvfGHXy7BU8XN4fTxh/wg/g6eGNI/FBg8fyzTW0lsP338O+H9A8KaNpfhnwvpGleG9G0OwXT 9I0bSLCPStO0/TYVyItG0W2jEKpGcIFSIANuwXMjBujor9Yo5Ng8A+fDQUZttuclzTvda82+rte9 /Lsb4XDLDX15nLdttt6pu911ttrawUUUV6dl2X3HV6niHjj4S6b428d/CL4gXGq3mlal8JNf8Wah pptIY5IvEOl+JvBMvhjXdE10EkJ5jtBcfLuffoUShQwO3p/A/wANvA/w40LSvDHg/QdN0XTNI0Vd CsYbWJRfRaXBJ5kGl/2uW+0KiT6oJIA8rNvfKHPzJ6R/n86P8/lV/wBp4j6qsNzyUUrNJrlesmm9 LuznNpPROTaSu08P7Gwv13644Jz31V7PTbsvdW1vyPnD9qzwNcfE79nH4y+EdMw+qal4A8XPoYXh j4u8PxSa54dYjjiPxNo0RY5BwDjkgV8B6X8F9U+LvhX4d/EC+1jxf448HfGf9knU7/xh8StUutIf xH4T8d+H/EXhr40fBfxBERb26C48G6/r3ih/AcMIlSxkVZJXkhAlf9i8A9RmsjRbOxsLMafp1hZa bp2mwpb2Vhp9tHaWdvFtY4ighComMYwuF5J27mLH6TJ+JcbkWEWHwjsnUVSMrRUoy5Ix5ozS9pGU Zwo1I8s+W9KKlGS0Pl864Xwec4361ifiUFFJN2fK01dLR2u0r3au7NHyN8FfAVz43/Yu+D/hK9XS 7fU9T+Cvw4kkPibwwuraVY6qPDfh/XCut+HNfyZdlwuJYbjaY33OvzpEte5/CP4fv8NfB9v4dutU TVL2S/1XW9Qv7SwXSdOGp+INbm1/WY9F0VDJFoXh5by4kMNqkzxxpJuJ851D+ok7Y48cZLE49jxx 07+lW1OR+X6gH+tfFZpk+CxfENTiydNf2vL2ilWdvfU53kpJLW97d7Ja7p+1lmAjg8Hlai1K0VTv rqo7Xvv8N31VlZ2uLRRRXf6nqn//2Q== --=====================_8394921==_-- From david.a.zoller@nasa.gov Mon Dec 17 11:40:08 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5A59321F88E7 for ; Mon, 17 Dec 2012 11:40:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.148 X-Spam-Level: X-Spam-Status: No, score=-6.148 tagged_above=-999 required=5 tests=[AWL=-0.150, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AnZJqe4lUfPh for ; Mon, 17 Dec 2012 11:40:05 -0800 (PST) Received: from ndjsnpf02.ndc.nasa.gov (ndjsnpf02.ndc.nasa.gov [198.117.1.122]) by ietfa.amsl.com (Postfix) with ESMTP id C53D221F8889 for ; Mon, 17 Dec 2012 11:40:04 -0800 (PST) Received: from ndmsppt05.ndc.nasa.gov (ndmsppt05.ndc.nasa.gov [198.117.0.104]) by ndjsnpf02.ndc.nasa.gov (Postfix) with ESMTP id 38385A82DC; Mon, 17 Dec 2012 13:40:04 -0600 (CST) Received: from ndmshub05.ndc.nasa.gov (ndmshub05.ndc.nasa.gov [198.117.2.164]) by ndmsppt05.ndc.nasa.gov (8.14.5/8.14.5) with ESMTP id qBHJe311021262; Mon, 17 Dec 2012 13:40:03 -0600 Received: from NDMSSCC05.ndc.nasa.gov ([198.117.2.174]) by ndmshub05.ndc.nasa.gov ([198.117.2.164]) with mapi; Mon, 17 Dec 2012 13:40:03 -0600 From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" To: "ssireskin@gmail.com" Date: Mon, 17 Dec 2012 13:40:04 -0600 Thread-Topic: [dtn-users] A problem with dtntunnel Thread-Index: Ac3cX4lY3Inb0k+YSTq+DJDCGxIWhQALfUZQ Message-ID: <04E3D99A62496240BCD6A576813E6E31E0BDF5C999@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDF5C59C@NDMSSCC05.ndc.nasa.gov> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_04E3D99A62496240BCD6A576813E6E31E0BDF5C999NDMSSCC05ndcn_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2012-12-17_04:2012-12-17, 2012-12-17, 1970-01-01 signatures=0 Cc: David Zoller , "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 19:40:08 -0000 --_000_04E3D99A62496240BCD6A576813E6E31E0BDF5C999NDMSSCC05ndcn_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Sergey, That was my bad. In the patch below I moved the "delete b_recv;" line above= the "if (recv_hdr->eof_) {" block as a short cut to delete it before the p= ossible exit. Move it back down below the if block and you should be okay. Sorry, that was one of those possible loose ends I mentioned :), DZ David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: ssireskin@gmail.com [mailto:ssireskin@gmail.com] Sent: Monday, December 17, 2012 8:05 AM To: Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] Cc: David Zoller; dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel David, The problem is probably not with the reordering. Can it be, that "if (recv_= hdr->eof_)" is not the right condition to close a connection? Or maybe it i= s not a sufficient condition? Which connection is recv_hdr about? Can it be= a local TCP connection on the receiving side (dtntunnel <-> nc)? If it is,= then IMO this condition becomes true when the receiving application doesn'= t send any data to the sending application, please see line 406 of the orig= inal code. 2012/12/17 ssireskin@gmail.com > David, I added this code and a log_debug inside the if-clause. According to the lo= g, this condition doesn't get true. A have attached my logs and tcpdumps. Please, see dtntunnel_T.log - it's a = receiving dtntunnel log. There are 4 bundles after a eof bundle. This doesn't happen with the origin= al dtntunnel. -- Best regards, Sergey Sireskin 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > Hi Sergey, I don't have your setup and this has not been tested but give this a try. I= n TCPTunnel::Connection::handle_bundle in the while(1) loop add the test to= see if the next_seqno_ is a match: while (1) { iter =3D reorder_table_.find(next_seqno_); if (iter =3D=3D reorder_table_.end()) { break; } // add this code: // if seqno does not match then still a bundle missing if (iter->first !=3D next_seqno_) { break; } bundle =3D iter->second; log_debug("delivering %zu byte bundle with seqno %d (from reorder t= able)", bundle->payload_.len(), next_seqno_); reorder_table_.erase(iter); next_seqno_++; queue_.push_back(bundle); } David Zoller COLSA Corporation HOSC / C107 *Office: (256) 544-1820 *EMail: david.a.zoller@nasa.gov From: dtn-users-bounces@irtf.org [mailto= :dtn-users-bounces@irtf.org] On Behalf O= f ssireskin@gmail.com Sent: Monday, December 17, 2012 6:40 AM To: David Zoller Cc: dtn-users@irtf.org Subject: Re: [dtn-users] A problem with dtntunnel Hi David, Thanks for your patch, it works! :) Connection is cleanly terminated now. However, there still seems to be a problem. I run dtntunnel over a link wit= h high latency and about 30% packet loss. The receiving nc sometimes gets F= IN packet before it receives all the data. If I understand it right, on a l= ossy link bundles may come out of order. In this case the EOF bundle may co= me too early. I think that the reordering procedure in dtntunnel should be = reviewed. Just for information, pure TCP performs better that DTN over this= link. 2012/12/14 David Zoller > Hi Sergey, I can't stay away from a problem that is nagging at me so I made some time = this morning. Going back to the original TCPTunnel.cc code, when a bundle i= s received that has the EOF indication, an empty bundle with the same EOF f= lag set needs to be returned to close the loop (the second FIN). There may = still be some loose ends or clean up needed but try the code below at the e= nd of the while loop. I did not do a packet capture but I saw that if the receiving nc is not run= ning when you start the sender nc then the receiving dtntunnel logs an erro= r on the failed connect and returns an EOF bundle which closes the sender n= c. The EOF bundle could possibly be enhanced to include an error message. I agree that some applications could benefit from a "pretend to be a receiv= er" option with the default working as a bidirectional end-to-end TCP conne= ction. Thanks for dragging me along kicking and screaming on this lesson in DTN an= d TCP tunneling. I've only been using dtntunnel as a one way UDP transmitte= r. I see a good fit for the TCP tunnel in an upcoming project and I'll try = to make the pitch to use it. This is a pretty cool little app. Best regards, DZ 484 delete b_recv; 485 486 if (recv_hdr->eof_) { 487 log_info("bundle had eof bit set... closing connection"= ); 488 489 if ( !sock_eof ) { 490 sock_eof =3D true; 491 // send an empty bundle back since we did not initi= ate the close 492 dtn::APIBundle* b =3D new dtn::APIBundle(); 493 hdr.eof_ =3D 1; 494 hdr.seqno_ =3D ntohl(send_seqno++); 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, sizeof(h= dr)); 496 b->payload_.set_len(sizeof(hdr)); 497 int err; 498 if ((err =3D tunnel->send_bundle(b, &dest_eid_)) != =3D DTN_SUCCESS) { 499 log_err("error sending final socket closed pack= et bundle: %s", 500 dtn_strerror(err)); 501 tcptun_->kill_connection(this); 502 exit(1); 503 } 504 delete b; 505 } 506 sock_.close(); 507 goto done; 508 } 509 } 510 } 511 512 done: -- Best regards, Sergey Sireskin -- Best regards, Sergey Sireskin --_000_04E3D99A62496240BCD6A576813E6E31E0BDF5C999NDMSSCC05ndcn_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable




--
Best regards,
Sergey Siresk= in

Sergey,

That was my bad. In the pat= ch below I moved the “delete b_recv;” lin= e above the “if (recv_hdr->eof_) {” blo= ck as a short cut to delete it before the possible exit. Move it back down = below the if block and you should be okay.

Sorry, that was one of those possible loose ends I mention= ed J,

DZ

 

David Zoller

COLSA Corporation

HOSC / C107 

(Office:<= span style=3D'font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F= 497D'> (256) 544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

 <= /o:p>

From: ssireskin@gmail.com [mailto:ssires= kin@gmail.com]
Sent: Monday, December 17, 2012 8:05 AM
To:= Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: Dav= id Zoller; dtn-users@irtf.org
Subject: Re: [dtn-users] A problem = with dtntunnel

 <= /p>

David,

The pr= oblem is probably not with the reordering. Can it be, that "if (recv_h= dr->eof_)" is not the right condition to close a connection? Or may= be it is not a sufficient condition? Which connection is recv_hdr about? Ca= n it be a local TCP connection on the receiving side (dtntunnel <-> n= c)? If it is, then IMO this condition becomes true when the receiving appli= cation doesn't send any data to the sending application, please see line 40= 6 of the original code.

2012/12/17 = ssireskin@gmail.com <ssireskin@gmail.com&g= t;

David,

I added this code and = a log_debug inside the if-clause. According to the log, this condition does= n't get true.
A have attached my logs and tcpdumps. Please, see dtntunne= l_T.log - it's a receiving dtntunnel log.
There are 4 bundles after a eo= f bundle. This doesn't happen with the original dtntunnel.

<= div>


--
Best reg= ards,
Sergey Sireskin

2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] <david.a.zoller@na= sa.gov>

 

= Hi Sergey,

I don’t have you= r setup and this has not been tested but give this a try. In TCPTunnel::Con= nection::handle_bundle in the while(1) loop add the test to see if the next= _seqno_ is a match:

 

&n= bsp;   while (1) {

      = ;  iter =3D reorder_table_.find(next_seqno_);

  &= nbsp;     if (iter =3D=3D reorder_table_.end()) {

            br= eak;

        }

&nb= sp;

        // = add this code:

      = ;  // if seqno does not match then still a bundle missing<= /o:p>

        if (iter->first = !=3D next_seqno_) {

     =        break;

<= span style=3D'font-size:9.0pt;font-family:"Courier New";color:#00B050'>&nbs= p;       }

 

 = ;       bundle =3D iter->second;

 &nb= sp;            =     bundle->payload_.len(), next_seqno_);

&nbs= p;  

        = reorder_table_.erase(iter);

      =   next_seqno_++;

   

<= p class=3DMsoNormal style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:= auto'>  = ;      queue_.push_back(bundle);<= /o:p>

    }  

&n= bsp;

 

=  

 

David Zoller

COLSA Corporat= ion

HOSC / C107 

(Office: (256) 544-1820
*EMail: david.a.zoller@nasa.gov

 

From:<= span style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> dtn-users-bounces@= irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Monday, December 17, 2012 6:40 AM
To: David Zoller


Cc: dtn-users@irtf.org
Subjec= t: Re: [dtn-users] A problem with dtntunnel

=  

Hi David,



Thanks for your patch, it works! :) Connection is cleanly termina= ted now.

However, there still seems to be a problem. I run dtntunnel= over a link with high latency and about 30% packet loss. The receiving nc = sometimes gets FIN packet before it receives all the data. If I understand = it right, on a lossy link bundles may come out of order. In this case the E= OF bundle may come too early. I think that the reordering procedure in dtnt= unnel should be reviewed. Just for information, pure TCP performs better th= at DTN over this link.

2012= /12/14 David Zoller <zollerd@gmail.com>

Hi Sergey,

I can't stay away from a problem that is nagging at= me so I made some time this morning. Going back to the original TCPTunnel.= cc code, when a bundle is received that has the EOF indication, an empty bu= ndle with the same EOF flag set needs to be returned to close the loop (the= second FIN). There may still be some loose ends or clean up needed but try= the code below at the end of the while loop.

 

I did not do a packet capture but= I saw that if the receiving nc is not running when you start the sender nc= then the receiving dtntunnel logs an error on the failed connect and retur= ns an EOF bundle which closes the sender nc. The EOF bundle could possibly = be enhanced to include an error message.

&= nbsp;

I agree that some applications could = benefit from a "pretend to be a receiver" option with the default= working as a bidirectional end-to-end TCP connection.

=

 

Thanks for dragging me a= long kicking and screaming on this lesson in DTN and TCP tunneling. I've on= ly been using dtntunnel as a one way UDP transmitter. I see a good fit for = the TCP tunnel in an upcoming project and I'll try to make the pitch to use= it. This is a pretty cool little app.

Best= regards,

DZ

 

 

 

 

 

484      = ;       delete b_recv;

485&n= bsp;

486          =   if (recv_hdr->eof_) {

487  =               log_info("bundle had= eof bit set... closing connection");

= 488 

489         &= nbsp;       if ( !sock_eof ) {

490                    = ; sock_eof =3D true;

491     &nbs= p;               // send an empty bundle= back since we did not initiate the close

4= 92                     dt= n::APIBundle* b =3D new dtn::APIBundle();

4= 93                     hd= r.eof_ =3D 1;

494       &nbs= p;             hdr.seqno_ =3D ntohl(send_seqn= o++);

495          = ;           memcpy(b->payload_.buf(sizeof(hdr))= , &hdr, sizeof(hdr));

496    =                 b->payload_.set= _len(sizeof(hdr));

497      =               int err;

498                =     if ((err =3D tunnel->send_bundle(b, &dest_eid_)) !=3D= DTN_SUCCESS) {

499       &n= bsp;                 log_err("= error sending final socket closed packet bundle: %s",

500                =                 dtn_strerror(err))= ;

501           &n= bsp;             tcptun_->kill_connection(= this);

502         &nbs= p;               exit(1);

=

503               &nbs= p;     }

504      =               delete b;

<= /div>

505                = ; }

506           =       sock_.close();

507 &nb= sp;               goto done;<= /p>

508             }

509         }

<= /div>

510     }

511&nb= sp;

512  done:

 




-= -
Best regards,
Sergey Sireskin

 

= --_000_04E3D99A62496240BCD6A576813E6E31E0BDF5C999NDMSSCC05ndcn_-- From ssireskin@gmail.com Mon Dec 17 23:40:22 2012 Return-Path: X-Original-To: dtn-users@ietfa.amsl.com Delivered-To: dtn-users@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D54C321F8880 for ; Mon, 17 Dec 2012 23:40:22 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.043 X-Spam-Level: X-Spam-Status: No, score=-3.043 tagged_above=-999 required=5 tests=[AWL=-0.045, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_92=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RpF-LhNrkZur for ; Mon, 17 Dec 2012 23:40:21 -0800 (PST) Received: from mail-vc0-f174.google.com (mail-vc0-f174.google.com [209.85.220.174]) by ietfa.amsl.com (Postfix) with ESMTP id 30CFD21F8856 for ; Mon, 17 Dec 2012 23:40:21 -0800 (PST) Received: by mail-vc0-f174.google.com with SMTP id d16so417260vcd.19 for ; Mon, 17 Dec 2012 23:40:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Tp2OANx6aQxqy67Y4JijljgEGXCfRbwGy1BJKfQcRhY=; b=UpapuRla1eQ7wWXSfdGYQBui5Q+ywxGenM8HW0EFQa40efc38AavrUZmXMHrMDvqEX VciKy94ynRAjxH4mqVExPnHoIpnQsnceIVw8YQvGvgpkD2/FMWB0etsAREQRSZ5wAt+i FBloz29mVk4KaFaxvoxe7Vil/MlW1M0mg5yTrUooSxyEn+QVk71tlgKDt2VnVUVciL15 bxSqmdagzNWVpMj6sDrKD7BT0QA/e534u/HDp54Z/kupYJ36LdCytDoD4Fbhf2usGK4f IGFLd6eusLWnfoydzV4m0f8a9sPTNqqiydGOA1xxD/Jl0Z7jccw1EwvKNlwjjfSY8ziu Oe1g== MIME-Version: 1.0 Received: by 10.220.149.69 with SMTP id s5mr1585491vcv.23.1355816420577; Mon, 17 Dec 2012 23:40:20 -0800 (PST) Received: by 10.58.219.65 with HTTP; Mon, 17 Dec 2012 23:40:20 -0800 (PST) In-Reply-To: <04E3D99A62496240BCD6A576813E6E31E0BDF5C999@NDMSSCC05.ndc.nasa.gov> References: <04E3D99A62496240BCD6A576813E6E31E0BDF5C59C@NDMSSCC05.ndc.nasa.gov> <04E3D99A62496240BCD6A576813E6E31E0BDF5C999@NDMSSCC05.ndc.nasa.gov> Date: Tue, 18 Dec 2012 10:40:20 +0300 Message-ID: From: "ssireskin@gmail.com" To: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" Content-Type: multipart/alternative; boundary=f46d042fd8982112f104d11b9cde Cc: David Zoller , "dtn-users@irtf.org" Subject: Re: [dtn-users] A problem with dtntunnel X-BeenThere: dtn-users@irtf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Users." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2012 07:40:23 -0000 --f46d042fd8982112f104d11b9cde Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David, Now it works like a charm, thank you very much! 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < david.a.zoller@nasa.gov> > Sergey,**** > > That was my bad. In the patch below I moved the *=93delete b_recv;=94* li= ne > above the *=93if (recv_hdr->eof_) {=94* block as a short cut to delete it > before the possible exit. Move it back down below the if block and you > should be okay.**** > > Sorry, that was one of those possible loose ends I mentioned J,**** > > DZ**** > > ** ** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > ** ** > > *From:* ssireskin@gmail.com [mailto:ssireskin@gmail.com] > *Sent:* Monday, December 17, 2012 8:05 AM > *To:* Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] > *Cc:* David Zoller; dtn-users@irtf.org > > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > ** ** > > David, > > The problem is probably not with the reordering. Can it be, that "if > (recv_hdr->eof_)" is not the right condition to close a connection? Or > maybe it is not a sufficient condition? Which connection is recv_hdr abou= t? > Can it be a local TCP connection on the receiving side (dtntunnel <-> nc)= ? > If it is, then IMO this condition becomes true when the receiving > application doesn't send any data to the sending application, please see > line 406 of the original code.**** > > 2012/12/17 ssireskin@gmail.com **** > > David, > > I added this code and a log_debug inside the if-clause. According to the > log, this condition doesn't get true. > A have attached my logs and tcpdumps. Please, see dtntunnel_T.log - it's = a > receiving dtntunnel log. > There are 4 bundles after a eof bundle. This doesn't happen with the > original dtntunnel. **** > > > -- > Best regards, > Sergey Sireskin > > **** > > 2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT] < > david.a.zoller@nasa.gov>**** > > ** ** > > Hi Sergey,**** > > I don=92t have your setup and this has not been tested but give this a tr= y. > In TCPTunnel::Connection::handle_bundle in the while(1) loop add the test > to see if the next_seqno_ is a match:**** > > **** > > while (1) {**** > > iter =3D reorder_table_.find(next_seqno_);**** > > if (iter =3D=3D reorder_table_.end()) {**** > > break;**** > > }**** > > **** > > // add this code:**** > > // if seqno does not match then still a bundle missing**** > > if (iter->first !=3D next_seqno_) {**** > > break;**** > > }**** > > **** > > bundle =3D iter->second;**** > > log_debug("delivering %zu byte bundle with seqno %d (from reorder > table)",**** > > bundle->payload_.len(), next_seqno_);**** > > **** > > reorder_table_.erase(iter);**** > > next_seqno_++;**** > > **** > > queue_.push_back(bundle);**** > > } **** > > **** > > **** > > **** > > **** > > David Zoller**** > > COLSA Corporation**** > > HOSC / C107 **** > > (Office: (256) 544-1820 > *EMail: david.a.zoller@nasa.gov**** > > **** > > *From:* dtn-users-bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] *O= n > Behalf Of *ssireskin@gmail.com > *Sent:* Monday, December 17, 2012 6:40 AM > *To:* David Zoller**** > > > *Cc:* dtn-users@irtf.org > *Subject:* Re: [dtn-users] A problem with dtntunnel**** > > **** > > Hi David,**** > > > > Thanks for your patch, it works! :) Connection is cleanly terminated now. > > However, there still seems to be a problem. I run dtntunnel over a link > with high latency and about 30% packet loss. The receiving nc sometimes > gets FIN packet before it receives all the data. If I understand it right= , > on a lossy link bundles may come out of order. In this case the EOF bundl= e > may come too early. I think that the reordering procedure in dtntunnel > should be reviewed. Just for information, pure TCP performs better that D= TN > over this link.**** > > 2012/12/14 David Zoller **** > > Hi Sergey,**** > > I can't stay away from a problem that is nagging at me so I made some tim= e > this morning. Going back to the original TCPTunnel.cc code, when a bundle > is received that has the EOF indication, an empty bundle with the same EO= F > flag set needs to be returned to close the loop (the second FIN). There m= ay > still be some loose ends or clean up needed but try the code below at the > end of the while loop.**** > > **** > > I did not do a packet capture but I saw that if the receiving nc is not > running when you start the sender nc then the receiving dtntunnel logs an > error on the failed connect and returns an EOF bundle which closes the > sender nc. The EOF bundle could possibly be enhanced to include an error > message.**** > > **** > > I agree that some applications could benefit from a "pretend to be a > receiver" option with the default working as a bidirectional end-to-end T= CP > connection.**** > > **** > > Thanks for dragging me along kicking and screaming on this lesson in DTN > and TCP tunneling. I've only been using dtntunnel as a one way UDP > transmitter. I see a good fit for the TCP tunnel in an upcoming project a= nd > I'll try to make the pitch to use it. This is a pretty cool little app.**= * > * > > Best regards,**** > > DZ**** > > **** > > **** > > **** > > **** > > **** > > 484 delete b_recv;**** > > 485 **** > > 486 if (recv_hdr->eof_) {**** > > 487 log_info("bundle had eof bit set... closing > connection");**** > > 488 **** > > 489 if ( !sock_eof ) {**** > > 490 sock_eof =3D true;**** > > 491 // send an empty bundle back since we did not > initiate the close**** > > 492 dtn::APIBundle* b =3D new dtn::APIBundle();**** > > 493 hdr.eof_ =3D 1;**** > > 494 hdr.seqno_ =3D ntohl(send_seqno++);**** > > 495 memcpy(b->payload_.buf(sizeof(hdr)), &hdr, > sizeof(hdr));**** > > 496 b->payload_.set_len(sizeof(hdr));**** > > 497 int err;**** > > 498 if ((err =3D tunnel->send_bundle(b, &dest_eid_)) = !=3D > DTN_SUCCESS) {**** > > 499 log_err("error sending final socket closed > packet bundle: %s",**** > > 500 dtn_strerror(err));**** > > 501 tcptun_->kill_connection(this);**** > > 502 exit(1);**** > > 503 }**** > > 504 delete b;**** > > 505 }**** > > 506 sock_.close();**** > > 507 goto done;**** > > 508 }**** > > 509 }**** > > 510 }**** > > 511 **** > > 512 done:**** > > **** > > > > > -- > Best regards, > Sergey Sireskin**** > > ** ** > > > > > -- > Best regards, > Sergey Sireskin > > **** > --=20 Best regards, Sergey Sireskin --f46d042fd8982112f104d11b9cde Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi David,

Now it works like a charm, thank you very much!

2012/12/17 Zoller, David A. (MSFC-EO60)[HOSC SERVIC= ES CONTRACT] <david.a.zoller@nasa.gov>

Sergey,

That was my bad. In the p= atch below I moved the =93delete b_recv;=94 line above the =93if (= recv_hdr->eof_) {=94 block as a sh= ort cut to delete it before the possible exit. Move it back down below the = if block and you should be okay.

Sorry, that was one of th= ose possible loose ends I mentioned J,

DZ

=A0

David Zoller

COLSA Corporation<= /u>

HOSC / C107=A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov

=A0<= /p>

From: ssireskin@gmail.= com [mailto:ss= ireskin@gmail.com]
Sent: Monday, December 17, 2012 8:05 AM
To: Zoller, David = A. (MSFC-EO60)[HOSC SERVICES CONTRACT]
Cc: David Zoller; dtn-users@irtf.org


Subject: Re: [dtn-users] A problem with d= tntunnel

=A0

David,

The problem is probably not with the reordering. Can it be, t= hat "if (recv_hdr->eof_)" is not the right condition to close = a connection? Or maybe it is not a sufficient condition? Which connection i= s recv_hdr about? Can it be a local TCP connection on the receiving side (d= tntunnel <-> nc)? If it is, then IMO this condition becomes true when= the receiving application doesn't send any data to the sending applica= tion, please see line 406 of the original code.

2012/12/17 ssireskin@gmail.com <ssireskin@gmail.com>

David,

I added this code and a log_debug inside the if-clause. Acco= rding to the log, this condition doesn't get true.
A have attached m= y logs and tcpdumps. Please, see dtntunnel_T.log - it's a receiving dtn= tunnel log.
There are 4 bundles after a eof bundle. This doesn't happen with the or= iginal dtntunnel.


--
Best regards,
Sergey Sireskin

<= /u>

2012/12/17 Zoller, David A. (MSFC-EO60)[H= OSC SERVICES CONTRACT] <david.a.zoller@nasa.gov>

=A0

Hi Sergey,

I don=92t have your setup and this has not been = tested but give this a try. In TCPTunnel::Connection::handle_bundle in the = while(1) loop add the test to see if the next_seqno_ is a match:<= /u>

=A0<= /p>

=A0=A0=A0 while (1) {

=A0=A0=A0=A0=A0=A0=A0 iter =3D reorder_table_.find(next_seq= no_);

=A0=A0=A0=A0=A0=A0=A0 if (ite= r =3D=3D reorder_table_.end()) {

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;

=A0=A0=A0=A0=A0=A0=A0 }

=A0

= =A0=A0=A0=A0=A0=A0=A0 // add this code:

=A0=A0=A0=A0=A0=A0=A0 // if seqno does not ma= tch then still a bundle missing

=A0=A0=A0=A0=A0=A0=A0 if (iter->first !=3D next_seqno_) {

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;

=A0=A0=A0 =A0=A0=A0=A0}

=A0

=A0=A0=A0=A0=A0= =A0=A0 bundle =3D iter->second;

=A0=A0=A0=A0=A0=A0=A0 log_debug("delivering %zu byte b= undle with seqno %d (from reorder table)",

=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 bundle->payload_.len(), ne= xt_seqno_);

=A0=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0reorder_table_.erase(iter);<= u>

=A0=A0=A0=A0=A0=A0=A0 next_seqno_++;

=A0=A0=A0

<= span style=3D"font-size:9.0pt;font-family:"Courier New"">=A0=A0= =A0=A0=A0=A0=A0=A0queue_.push_back(bundle);

=A0=A0=A0 }=A0=A0

=A0

=A0<= /p>

=A0

=A0<= /p>

David Zoller

COLSA Corporation=

HOSC / C107= =A0

(Office: (256)=A0544-1820
*EMail: david.a.zoller@nasa.gov<= /span>

=A0<= /p>

From: dtn-users= -bounces@irtf.org [mailto:dtn-users-bounces@irtf.org] On Behalf Of ssireskin@gmail.com
Sent: Monday, December 17, 2012 6:40 AM
To: David Zoller


Cc: dtn-users@irtf.org
= Subject: Re: [dtn-users] A problem with dtntunnel

=A0

Hi David,



Thanks for your patch, it works! :) Connection is cl= eanly terminated now.

However, there still seems to be a problem. I run dtntunnel over a link= with high latency and about 30% packet loss. The receiving nc sometimes ge= ts FIN packet before it receives all the data. If I understand it right, on= a lossy link bundles may come out of order. In this case the EOF bundle ma= y come too early. I think that the reordering procedure in dtntunnel should= be reviewed. Just for information, pure TCP performs better that DTN over = this link.

2012/12/14 David Zoller &= lt;zollerd@gmail.com= >

Hi Sergey,<= /u>

I can't stay away from a problem that= is nagging at me so I made some time this morning. Going back to the origi= nal TCPTunnel.cc code, when a bundle is received that has the EOF indicatio= n, an empty bundle with the same EOF flag set needs to be returned to close= the loop (the second FIN). There may still be some loose ends or clean up = needed but try the code below at the end of the while loop.

=A0

I did not do a packet capture but I saw that if the receivin= g nc is not running when you start the sender nc then the receiving dtntunn= el logs an error on the failed connect and returns an EOF bundle which clos= es the sender nc. The EOF bundle could possibly be enhanced to include an e= rror message.

=A0

I agree that some applications could benefit from a "pr= etend to be a receiver" option with the default working as a bidirecti= onal end-to-end TCP connection.

=A0

Thanks for dragging me along kicking and screaming on this l= esson in DTN and TCP tunneling. I've only been using dtntunnel as a one= way UDP transmitter. I see a good fit for the TCP tunnel in an upcoming pr= oject and I'll try to make the pitch to use it. This is a pretty cool l= ittle app.

Best regards,

DZ

=A0

=A0=

=A0

=A0

=A0

484 =A0 =A0 =A0 =A0 =A0 =A0= delete b_recv;

485=A0

486 =A0 =A0 =A0 =A0 =A0 =A0 if (recv_hdr->eof_) {

487 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 log_info("bundle had eof bit set... closing connection");=

488=A0

489 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ( !sock_eof ) {<= /u>

490 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 sock_eof =3D true;

491 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 // send an empty bundle back since we did not initiate the close=

492 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 dtn::APIBundle* b =3D new dtn::APIBundle();

493 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 hdr.eof_ =3D 1;

494 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 hdr.seqno_ =3D ntohl(send_seqno++);=

495 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 memcpy(b->payload_.buf(sizeo= f(hdr)), &hdr, sizeof(hdr));

496 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 b->payload_.set_len= (sizeof(hdr));

497 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 int= err;

498 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ((err =3D tunnel->send_bundle(b, &des= t_eid_)) !=3D DTN_SUCCESS) {

499 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 log_err("error sending final socket closed packet bundle: = %s",

500 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 dtn_strerror(err));=

501 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 tcptun_->kill_connection(this);

=

502 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = exit(1);

503 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 }

<= p class=3D"MsoNormal">504 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 delete b;=

505 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 }

506 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sock_.close();

<= div>

507 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto done;

508 =A0 =A0 =A0 =A0 =A0 = =A0 }

509 =A0 =A0 =A0 =A0 }

51= 0 =A0 =A0 }

511=A0

512 =A0done:

=A0




--
Best re= gards,
Sergey Sireskin

= =A0

<= br>

--
Best regards,
Sergey Sireskin

=