Subject: Pathologic Solaris TCP behavior

From: Wu-chang Feng <[email-masked]>
Date: Mon, 20 Mar 2000 17:43:10 -0800

Has anyone experienced problems with Solaris TCP where you basically
get modem-speed performance over a full duplex ethernet link?  I have
two, factory-installed Netra T1 boxes running Solaris 7.  They're
connected through an ethernet switch, but when I do an ftp between
them, I get a trickle because of packets which are lost after every
6-7 packets.  On the incoming interface of the destination machine
(which is completely unloaded) I get a large number of Ierrs.....

---------------------------------------------------------------------------
pumaweb006 ~ % netstat -i
Name  Mtu  Net/Dest      Address        Ipkts  Ierrs Opkts  Oerrs
Collis Queue 
lo0   8232 loopback      localhost      1527068 0     1527068 0    
0      0     
hme0  1500 p006-be p006-be  34108  0     9366   0     0      0     
hme1  1500 p006    p006     3039955 17817 1164113 0     0      0    
---------------------------------------------------------------------------

Attached is a tcpdump of the ftp connection.....

17:22:32.419072 216.10.101.75.ftp-data > p006.56818: . 131072:132532(1460) ack 1 win 24820 (DF)
17:22:32.419108 p006.56818 > 216.10.101.75.ftp-data: . ack 133992 win 24820 (DF)
17:22:32.419538 216.10.101.75.ftp-data > p006.56818: . 133992:135452(1460) ack 1 win 24820 (DF)
17:22:32.419599 216.10.101.75.ftp-data > p006.56818: P 135452:136344(892) ack 1 win 24820 (DF)
17:22:32.419640 p006.56818 > 216.10.101.75.ftp-data: . ack 135452 win 24820 (DF)
17:22:32.420016 216.10.101.75.ftp-data > p006.56818: . 136344:137804(1460) ack 1 win 24820 (DF)
17:22:32.420136 216.10.101.75.ftp-data > p006.56818: . 137804:139264(1460) ack 1 win 24820 (DF)
17:22:32.420205 p006.56818 > 216.10.101.75.ftp-data: . ack 139264 win 24820 (DF)
17:22:32.420596 216.10.101.75.ftp-data > p006.56818: . 139264:140724(1460) ack 1 win 24820 (DF)
17:22:32.420709 216.10.101.75.ftp-data > p006.56818: . 140724:142184(1460) ack 1 win 24820 (DF)
17:22:32.420777 p006.56818 > 216.10.101.75.ftp-data: . ack 142184 win 24820 (DF)
17:22:35.419098 216.10.101.75.ftp-data > p006.56818: . 139264:140724(1460) ack 1 win 24820 (DF)
17:22:35.419125 p006.56818 > 216.10.101.75.ftp-data: . ack 142184 win 24820 (DF)
17:22:35.419531 216.10.101.75.ftp-data > p006.56818: . 142184:143644(1460) ack 1 win 24820 (DF)
17:22:35.419594 216.10.101.75.ftp-data > p006.56818: P 143644:144536(892) ack 1 win 24820 (DF)
17:22:35.419650 p006.56818 > 216.10.101.75.ftp-data: . ack 143644 win 24820 (DF)
17:22:35.420027 216.10.101.75.ftp-data > p006.56818: . 144536:145996(1460) ack 1 win 24820 (DF)
17:22:35.420144 216.10.101.75.ftp-data > p006.56818: . 145996:147456(1460) ack 1 win 24820 (DF)
17:22:35.420217 p006.56818 > 216.10.101.75.ftp-data: . ack 147456 win 24820 (DF)
17:22:35.420626 216.10.101.75.ftp-data > p006.56818: . 147456:148916(1460) ack 1 win 24820 (DF)
17:22:35.420742 216.10.101.75.ftp-data > p006.56818: . 148916:150376(1460) ack 1 win 24820 (DF)
17:22:35.420815 p006.56818 > 216.10.101.75.ftp-data: . ack 150376 win 24820 (DF)
17:22:38.419135 216.10.101.75.ftp-data > p006.56818: . 147456:148916(1460) ack 1 win 24820 (DF)
17:22:38.419160 p006.56818 > 216.10.101.75.ftp-data: . ack 150376 win 24820 (DF)
17:22:38.419577 216.10.101.75.ftp-data > p006.56818: . 150376:151836(1460) ack 1 win 24820 (DF)
17:22:38.419640 216.10.101.75.ftp-data > p006.56818: P 151836:152728(892) ack 1 win 24820 (DF)
17:22:38.419688 p006.56818 > 216.10.101.75.ftp-data: . ack 151836 win 24820 (DF)
17:22:38.420066 216.10.101.75.ftp-data > p006.56818: . 152728:154188(1460) ack 1 win 24820 (DF)
17:22:38.420183 216.10.101.75.ftp-data > p006.56818: . 154188:155648(1460) ack 1 win 24820 (DF)
17:22:38.420257 p006.56818 > 216.10.101.75.ftp-data: . ack 155648 win 24820 (DF)
17:22:38.420644 216.10.101.75.ftp-data > p006.56818: . 155648:157108(1460) ack 1 win 24820 (DF)
17:22:38.420763 216.10.101.75.ftp-data > p006.56818: . 157108:158568(1460) ack 1 win 24820 (DF)
17:22:38.420839 p006.56818 > 216.10.101.75.ftp-data: . ack 158568 win 24820 (DF)
17:22:41.419183 216.10.101.75.ftp-data > p006.56818: . 155648:157108(1460) ack 1 win 24820 (DF)
17:22:41.419205 p006.56818 > 216.10.101.75.ftp-data: . ack 158568 win 24820 (DF)
17:22:41.419603 216.10.101.75.ftp-data > p006.56818: . 158568:160028(1460) ack 1 win 24820 (DF)
17:22:41.419670 216.10.101.75.ftp-data > p006.56818: P 160028:160920(892) ack 1 win 24820 (DF)
17:22:41.419709 p006.56818 > 216.10.101.75.ftp-data: . ack 160028 win 24820 (DF)
17:22:41.420082 216.10.101.75.ftp-data > p006.56818: . 160920:162380(1460) ack 1 win 24820 (DF)
17:22:41.420201 216.10.101.75.ftp-data > p006.56818: . 162380:163840(1460) ack 1 win 24820 (DF)
17:22:41.420268 p006.56818 > 216.10.101.75.ftp-data: . ack 163840 win 24820 (DF)
17:22:41.420650 216.10.101.75.ftp-data > p006.56818: . 163840:165300(1460) ack 1 win 24820 (DF)
17:22:41.420769 216.10.101.75.ftp-data > p006.56818: . 165300:166760(1460) ack 1 win 24820 (DF)
17:22:41.420840 p006.56818 > 216.10.101.75.ftp-data: . ack 166760 win 24820 (DF)

---

From: Zachary Amsden <[email-masked]>
Date: Mon, 20 Mar 2000 19:45:16 -0800

> (which is completely unloaded) I get a large number of Ierrs.....

The TCP trace you included is completely consistent with ACK loss producing 
unnecessary retransmissions, as far as I can tell.  This is a strong indicator 
that there is a link layer problem, perhaps bad cabling or a cheap switch.  I 
highly doubt that Sun would let such a glaring problem in their stack go 
unnoticed.

-- 
Zachary Amsden  [email-masked]  3-6919  31-2-510  Core Protocols

---

From: Ignacio Goyret <[email-masked]>
Date: Mon, 20 Mar 2000 21:28:29 -0800

>highly doubt that Sun would let such a glaring problem in their stack go 

I agree with Zachary: there must be a link layer problem.

My guess is that one of the links (sun1<->switch or switch<->sun2) is not
set as you expect. I've seen these kinds of errors when one end of the link
thinks is set for half-duplex but the other end thinks it should be full-duplex.
The full-duplex side will ignore collisions but the other end will get mighty
upset.

---

From: Wu-chang Feng <[email-masked]>
Date: Tue, 21 Mar 2000 09:53:46 -0800

Thanks everyone for all of the responses.  They were right on target.  Turning off auto negotiation
on the switch worked like a charm!  :)

Wu