Protocol Priorities:

Why the Nice Solution May Not Be So Nice

Douglas M. Clapp
CS-522 Project
University of Colorado, Colorado Springs
Fall, 2000

Abstract

The problem examined here is manifested when one uses a mix of protocols, such as ftp and telnet, from the same workstation or personal computer and especially if the network connection is slow. In many cases, the ftp connection seems to hog the connection's bandwidth to the detriment of responsiveness on the other protocol. This is the reverse of the desired result, which is ftp data transfer in the background and responsive telnet in the foreground.

The obvious solution is to adjust the priorities of the local ftp client relative to the telnet client so that it will receive less of the workstation's compute resources relative to the telnet connection.

This paper examines in more detail how this solution can be implemented in a Unix-based workstation and presents experimental evidence that it does not work. It will discuss the reasons underlying this failure and present some avenues for future research.

The paper is organized into four major sections: Introduction, Experimental Setup, Implementation and Experimentation (which presents the setup used for experimentation, how the prioritization was implemented, and experimental data resulting), Conclusions and Explanations (what happened, reasons why this scheme does not work and some possibilities for more research), and References.

Introduction

Many people use a computer at home. This may be a Microsoft Windows-based personal computer, a Macintosh, or s Unix-based PC/workstation. One common element is the network connection - a dialup modem. Although cable modem and broadband broadcast are becoming more and more widely available, they are by no means yet universal. One common problem shared by many of these home computer users is how to share their limited bandwidth among the applications they use. In particular, they would like to have downloads of large files (ftp or http downloads) occur in the background, using only the bandwidth not being used for interactive sessions (telnet, http). In this paper the emphasis is on the mix of ftp and telnet, but the results probably apply in some way to mixtures containing http as well.

The obvious way to manage this mix of protocols is to control the priority of the client applications involved. In this case, that would be the ftp client and the telnet client program running on the local PC/workstation. For a Unix-based computer, the priority of an application can be controlled to some degree by use of the "nice" command. Darrell Stossell (ref x) suggests this in his paper and presents quite a bit of discussion of the underlying protocol stack routines and data structures involved. He suggests the voluntary use of the "nice" command by users to lower the protocol of their ftp client. He also mentions the possibility of modifying the priority of the ftp daemon (server) on the ftp serving machine. Apparently he never actually tried to implement and test these mechanisms.

This paper will describe the setup used to test the idea of using the "nice" command to control the relative priorities of the ftp and telnet clients on a home workstaion, how "nice" was applied, and some of the experiments performed using this setup. Then it will show the results of experiments performed. Finally, the paper will discuss the results and what they mean, draw conclusions from them, and suggest future avenues of research.


Experimental Setup

The overall configuration used for the experiments consists of a local (home) PC/workstation with dialup modem connection to the network at UCCS. Connections are then made to computers in the Engineering building and to computers in the main campus computer center. There are also connections to other computers outside the campus network, using the Internet. In more detail, the major components are:

Transport protocols are shown as PPP and ethernet. Since the author has complete control over the local FreeBSD computer, this is known. The ethernet is inferred, based on evidence such as output from "netstat" and "ifconfig" on the campus computers. This may be 10 Mbps ethernet or 100 Mbps fast ethernet (more likely the latter from some of the observer transfer speeds between machines in the CS lab). Once the connection leaves the UCCS campus, the transport mechanism is unknown to the author.

Implementation

Implementing control using the "nice" command has two possible components. One is the client - the user's machine. The other is the server.

Client:

Stossel's suggestion on the client side depends on user cooperation. In other words, the user would be expected to always use the nice command in conjunction with something like ftp, as in "nice ftp host." It is not difficult to make this behavior mandatory by "wrapping" the ftp command. This involves creating a shell script which replaces the ftp binary file. The binary is moved to a different location, from which it is invoked by the script. The script explicitly uses the nice command to lower the priority of the ftp command. For the experiments described herein, however, the explicit "nice" command was used. This allowed easy alternation between running the ftp client "niced" and "unniced."

Server:

Stossel suggests that persuading users to "nice" their executions of the ftp client would be easy to achieve but that controlling the incoming ftp connections would be difficult or impossible. In fact, this may be even easier than controlling the client or outgoing connections. An incoming ftp request first encounters the inetd daemon, which looks up the requested port (in the /etc/services file) and then invokes the ftp daemon. To "nice" the daemon involves merely modifying the way it is invoked by inetd. This can be done be substituting a "wrapper" script for the ftp binary (executable) file, moving the actual binary file to some other location. The "wrapper" script would invoke the daemon using "nice". In fact, this was done on the author's machine. The entry in the /etc/inetd.conf file was changed to 

ftp  stream  tcp  nowait  root  /tmp/ftpd     ftpd  -l

and the file /tmp/ftpd looks like: 

#!/bin/sh
/usr/bin/nice -20 /usr/libexec/ftpd
Between the two, an incoming ftp connection request results in invoking the ftp daemon (ftpd) with a "nice" value of 20, which is added to its priority (effectively lowering its execution priority, since lower priority values mean higher priority). One downside to using this technique, which was also observed by Stossel, is that all connections use the "niced" ftp daemon. This may not always be desirable - there may be preferred or priority customers for the server whose ftp sessions should not be "niced."

As it happens, the observations indicate this technique may not work on the server anyway. Using the program "top" to monitor system performance showed that ftpd ran with a nice value of 0 and during actual data transfer ftpd ran with a negative priority.

Planned Experimentation

The experiments planned for this project involved various combinations of ftp transfers from local machine to local machine and local machine to/from a remote machine. Another variable to be used in combination with the local/local or local/remote dimension was the choice of "nice" values for the ftp client, the ftp server, neither, or both. The initial idea was to simply watch the transfer time for a given file. Later, in local/remote interactions, the "ping" command was used to give an indication of how much delay would be incurred by an interactive session. Other areas for experimentation involved modifying the MTU size for the interface to see what effect this would have on transfer time and possibly on the observed packet size at the receiver.

Actual Experimentation

Data from the experimental runs performed are included as Appendix A. One set of runs involved local-to-local ftp (using the loopback link). The transfer was monitored with "tcpdump." The MTU (maximum transfer unit) was set to 16K bytes. Without "niceing" the ftp command, the observed transfer rate was 1.14 MB/sec; with "nice" the rate was 1.45 MB/sec. Adding the "time" command to the above changed the rates somewhat. Without nice the rate was 0.899 MB/sec; with nice it was 1.41 MB/sec. Removing the tcpdump gave the following: 1.42 MB/sec and 1.68 MB/sec without nice, 1.52 MB/sec and 1.73 MB/sec with nice. The interesting point of this set of runs is that the "niced" ftp transfers seem to transfer data faster, implying that "niceing" the ftp command results in taking more bandwidth, not less. This is not only not intuitive, it is also counter to the aims of the techniques being tested.


Another set of transfers was done using ftp between the local machine (named "miraloma") and a server on campus (brain). One addition to the experimental setup was to run a "ping" command, pinging brain from miraloma to assess the delay that would be observed by an interactive session such as telnet. Different transfers gave average rates of 1.33 KB/sec, 1.34 KB/sec, 1.5 KB/sec. Meanwhile, the ping showed delays before and after the transfers of 210 msec. During the transfers, delay was typically 12 seconds. These transfers all involved running the ftp client on miraloma and using the "get" command to initiate the transfer. Another transfer was from brain to miraloma, but was initiated from brain using the "put" command. This meant the receiving side (miraloma) was running the ftpd (ftp daemon), which ran "niced." In this case, the delay also ran about 12 seconds during the transfer.

In addition to the ftp transfers, delay was measured for several http downloads. During two downloads from cs.uccs.edu, the delay varied between 9 and 15 seconds. Two more files were downloaded from an internet web host and again the delay was about 9 seconds, with a few intervals of 5-6 seconds.

Results

To summarize, the local/local transfers showed little influence from use of the "nice" command - indeed, the influence seemed to be negative to the goals of the study. The "niced" ftp client transferred data at a faster rate than the "unniced" ftp client, implying it used more, not less bandwidth. In the transfers (both ftp and http) involving remote servers (and in the one case where the local machine was the server with the client on the remote machine), the delay was measured at 9 to 12 seconds. The tcpdump program, when used to monitor the remote transfer, showed 1460 bytes of actual data transferred in each TCP message.

Explanations and Conclusions

Originally, the project documented in this paper was aimed at showing how to use the Unix "nice" command to control the relative priorities of ftp and telnet client programs, thus keeping the ftp data transfers in the background and allowing the telnet sessions to maintain their responsiveness. Initial informal experiments seemed to show that "niceing" the ftp client had no effect (and as the local-to-local experiments showed, might even have negative effects. The thrust then shifted to an attempt to discover what lay behind the observed phenomena. In the process, the author read many of the documents behind the Internet Protocols (Request for Comment - RFC and Standard - STD).

The original premise that controlling the use of bandwidth can be done by using "nice" is based on the idea that controlling a program's access to the processor can control its access to the network interface. This would be true if the bandwidth- hog program also hogged the CPU. However, CPU speed and power has increased much faster than network speeds. This means the potential efficacy of the "nice" technique has been reduced with each new generation of faster CPU. IBM's first PC used the Intel 8088 processor which ran at about 4 MHz. At the same time, local area networks were commonly running 10 Megabits/second ethernet. Many networked computers still use 10 Mbit/second ethernet, although this is being replaced by fast ethernet at 100 Mbits/second. That is only a factor of ten increase in network speed. Meanwhile, PC processor speeds have increased many times over. PCs for home use are commonly advertised with clock speeds of 500 MHz or 600 MHz, and business desktops may be 800 MHz. That is a factor of 125 to 200 over the 4 MHz 8088. In addition, the CPU now does much more with each clock cycle, so the effective increase in processing power is even more than the 200 mentioned above. Thus a job that might have tasked the 4 MHz 8088 is unlikely to do so with its more recent descendents. If the CPU is used at home with dial-up access, the likelihood is that line speed will not exceed 56 Kilobits/second, not even up to the original ethernet speeds, and even less likely to task the capabilities of the CPU. Thus, unless there is a heavy processing load on the computer aside from processing the data to/from the network, the competing communications programs (ftp, telnet, etc.) will all take the CPU whenever they are ready, regardless of their priority. Using "nice" to modify their relative priority will have no effect on their access to the network connection bandwidth.

Another likely explanation of the observed behavior lies in the relative sizes of blocks of data transferred with each application program call to the protocol stack. In general, block sizes differ between ftp and telnet. Telnet transmits a character or a line at a time, while ftp sends a larger block of data. This means if the system has started sending a block of ftp data, no telnet data can be sent until the ftp block has completed transmission, regardless of the priority of the respective processes. What is the blocksize used by ftp? The IP size field is 16 bits, which allows 64K or 65536 bytes in an IP packet. Search of ftpd man pages and source for two different implementations of a daemon for ftp (see references for wu-ftp and publicfile) did not reveal a hard size limit. The conclusion is that 64K is possible. Other calculations indicate that 8K is more likely. Consider that most of the transfers demonstrated delays to the ping of about 9-12 seconds. Each of the original blocks of data, whether from an internet host or from one of the campus hosts (blanca or brain, for example), eventually entered an ethernet segment. The MTU for ethernet is 1500 bytes. If the ethernet packet contains an encapsulated TCP PDU, as when it transfers ftp data, the actual data in the packet would be reduced by 20 bytes for the IP header and another 20 bytes for the TCP header. This leaves the actual data payload at 1460 bytes, which is what tcpdump showed when it was used to monitor such transfers. A transfer was made between blanca and redcloud, both in the CS lab at UCCS. Traceroute showed the connection between contained one hop - that is, there is no intermediate router between the two. The delay between the two runs normally about 150-200 msec. During the transfer, one interval showed a delay as high at 1500 msec. The transfer itself ran at 5,600,000 bytes/sec=44,800,000 bits/sec (assuming here that 1K=1000, and by the way one of the observations implying that the CS backbone uses fast ethernet). One 1500 byte packet takes 1500/5600000 sec = .267 msec and 1500 microsec allows transmission of 5.6 packets. Assuming no options, the IP headeroccupies 20 bytes and the TCP header occupies 20 bytes, so the data payload is 1460 bytes. At the transmission speed for this transfer, 1500 microsec. allows 5.6*1460=8176 bytes, or about 8K (8192) bytes (where here 1K=1024). This is an argument that the ftp transfer started out as 8K blocks of data. Ignoring any effects of PPP itself, 5.6 packets at 14.4 Kbps will take 5.6*1500*8/14400 seconds=4.7 seconds. The typical delay of 9 seconds corresponds to 9*14400/8=16200 bytes, which is almost 16384 (16K); 16K is also a likely block size for ftp to use. The theory here is that once the block of data has been committed to the server's IP protocol stack, any other application is effectively blocked until that block of data has been transmitted. This would seemingly account for the observed delays at the home end of the connection.

Even if the packets were transmitted from the host with the desired priority, there is no guarantee this would last all the way to the receiver. The reason for this statement is the probablility of some ethernet segment in the route. IP itself provides for some prioritization (precedence and type of servic), although it may not be implemented by routers along the network. In addition, if the path contains ethernet links, any priority settings in the IP packets may be essentially ignorecd. This situation may be changing (see further the Hall articles), but for now the inclusion of ethernet in the routing may effectively negate any prioritization from the IP part of the routing.

This raises the issue of how to set IP priorities, anyway. Typically, Precedence and TOS would be set by application, but the question is how? There is no description in the ftp man page of how to get the ftp client or the ftp server to to set a lower priority for data transfers. In the wu-ftpd source code, there is some indication that it can throttle its data stream for bandwidth control, but the mechanism for requesting that through the control command structure is not readily apparent.

One possibility considered but ultimately not actually part of the experimentation is to adjust IP/TCP parameters. This can be done using the sysctl command in FreeBSD. After some thought, the conclusion is that it might help when the data transfer comes directly from one host and the interactive session is directly connected to another. Where the routing to both is indirect, as in the setup used for the experiments, changing window size parameters and such would likely not affect either connection preferentially over the other and so would not make much difference.

One nagging observation came out of the experiments - that is, in localhost-localhost ftp transfers, niceing the ftp client seems to make the data move faster. The natural question is; why? One theory is that lowering the priority of the client may allow preferential dispatch of the protocol stack routines.

Internet Protocol (IP) contains a Type of Service (TOS) field, as well as a precedence field. From what is found in several of the RFCs, however, TOS seems to have more influence on routing from node to node. That is, it influences which route a packet will take. What it does not do is determine which packet will be sent next from the given host or router. In addition, there was no mention at all of the precedence field in the FTP RFC.

In the final analysis, the original goal of keeping interactivity in one session (telnet) while running the other (ftp data transfer) in the background seems no more attainable than before. One definite conclusion, however, is that in terms of a solution to this problem the "nice" command is not nice at all.

References

  1. Hall, Eric, "Bringing Prioritization Services to Ethernet" in Network Computing, available at www.networkcomputing.com/914/914ws1.html
  2. Hall, Eric, "Implementing Prioritization On IP Networks" in Network Computing, available at www.networkcomputing.com/915/915ws1.html
  3. Request for Comments 791, Internet Protocol , available at http://www.ietf.org/rfc, as are all the following Requests for Comments
  4. Request for Comments 793, Transmission Control Protocol
  5. Request for Comments 854, Telnet Protocol Specification
  6. Request for Comments 855 , Telnet Option Specifications
  7. Request for Comments 959, File Transfer Protocol (FTP)
  8. Request for Comments 1058, Routing Information Protocol
  9. Request for Comments 1122, Requirements for Internet Hosts -- Communication Layers
  10. Request for Comments 1323, TCP Extensions for High Performance
  11. Request for Comments 1349, Type of Service in the Internet Protocol Suite
  12. Request for Comments 1375, Suggestion for New Classes of IP Addresses
  13. Request for Comments 1661, The Point-to-Point Protocol (PPP)
  14. Request for Comments 1662, PPP in HDLC-like Framing
  15. Request for Comments 1700, Assigned Numbers
  16. Request for Comments 2212, Specification of Guaranteed Quality of Service
  17. Request for Comments 2330, Framework for IP Performance Metrics
  18. Request for Comments 2475, An Architecture for Differentiated Services
  19. Request for Comments 2481, A Proposal to add Explicit Congestion Notification (ECN) to IP
  20. Request for Comments 2616, Hypertext Transfer Protocol -- HTTP/1.1
  21. Request for Comments 2702, Requirements for Traffic Engineering Over MPLS
  22. Request for Comments 2817, Upgrading to TLS Within HTTP/1.1
  23. Request for Comments 2914, Congestion Control Principles
  24. Request for Comments 2964, Use of HTTP State Management
  25. Request for Comments 2965, HTTP State Management Mechanism
  26. Source code for publicfile 0.52, available at http://pobox.com/~djb/publicfile.html
  27. Source code for wu-ftpd-2.6.1, available at http://www.wu-ftpd.org
  28. Stossel, Darrell, "Methods For Controlling Network Usage On A Linux PC", currently available at URL: http://owl.uccs.edu/~destosse/522.htm

Appendix A

Raw Results Captured From Experiments 

transfer local/local, loopback, with tcpdump running:

without nice, 16K MTU:

ftp> get so51a_lnx_01_n7.tar.gz /dev/null
9072280 bytes received in 7.62 seconds (1.14 MB/s)

with nice, 16K MTU:

ftp> get so51a_lnx_01_n4.tar.gz /dev/null
9910227 bytes received in 6.54 seconds (1.45 MB/s)

without nice, 16K MTU, time:

ftp> get so51a_lnx_01_n4.tar.gz /dev/null
9910227 bytes received in 10.76 seconds (899.17 KB/s)
ftp> bye
0.0u 0.3s 0:47.16 0.7% 106+914k 14+0io 29pf+0w

with nice, 16K MTU, time:

ftp> get so51a_lnx_01_n7.tar.gz /dev/null
9072280 bytes received in 6.14 seconds (1.41 MB/s)
0.0u 0.2s 0:36.67 0.7% 138+1183k 18+0io 34pf+0w

--------------------------------------------------------

no tcpdump, local/local ftp, 16K MTU:

without nice:

ftp> get so51a_lnx_01_n7.tar.gz /dev/null
9072280 bytes received in 6.10 seconds (1.42 MB/s)
0.0u 0.2s 0:39.97 0.7% 110+901k 18+0io 26pf+0w

with nice:

ftp> get so51a_lnx_01_n7.tar.gz /dev/null
9072280 bytes received in 5.69 seconds (1.52 MB/s)
0.0u 0.2s 0:27.71 1.2% 132+1076k 28+0io 29pf+0w

with nice:

ftp> get so51a_lnx_01_n4.tar.gz /dev/null
9910227 bytes received in 5.45 seconds (1.73 MB/s)
0.0u 0.2s 0:34.55 0.8% 151+1210k 18+0io 28pf+0w

without nice:

ftp> get so51a_lnx_01_n4.tar.gz /dev/null
9910227 bytes received in 5.62 seconds (1.68 MB/s)
0.0u 0.2s 0:27.47 1.0% 172+1419k 21+0io 28pf+0w

----------------------------------------------------------------

http transfer of research.html from cs.uccs.edu

128.198.162.68.80 > 128.198.66.14.1057: tcp 0
128.198.162.68.80 > 128.198.66.14.1057: tcp 1460 (DF) [tos 0x10]
128.198.162.68.80 > 128.198.66.14.1057: tcp 1400 (DF) [tos 0x10]
128.198.162.68.80 > 128.198.66.14.1057: tcp 0 [tos 0x10]
128.198.162.68.80 > 128.198.66.14.1057: tcp 0 (DF) [tos 0x10]
128.198.162.68.80 > 128.198.66.14.1057: tcp 0 (DF) [tos 0x10]

from wc on server:     85     405    2860 research.html
wc on file as transferred: 
      88     414    2859

----------------------------------------------------------------

revised research.html:

tcpdump: listening on tun0
128.198.162.68.80 > 128.198.66.14.1069: tcp 0
128.198.162.68.80 > 128.198.66.14.1069: tcp 1460 (DF) [tos 0x10]
128.198.162.68.80 > 128.198.66.14.1069: tcp 1378 (DF) [tos 0x10]
128.198.162.68.80 > 128.198.66.14.1069: tcp 0 [tos 0x10]
128.198.162.68.80 > 128.198.66.14.1069: tcp 0 (DF) [tos 0x10]

the transfer:

miraloma# telnet cs.uccs.edu 80 | wc
GET /~dmclapp/research.html
Connection closed by foreign host.
      93     428    2846

after transfer on local machine:

miraloma# ll res*ml
-rw-r--r--  1 dmclapp  user  2838 Oct 30 20:08 research.html

-----------------------------------------------------------------
ftp pushed file from brain to miraloma, while from miraloma pinging 
brain 1/sec

delay ran about 220 ms before transfer started, built up to about 12 sec
then stayed at that while transfer took place, then dropped back to 
about 220 ms again after transfer finished

-------------------------------------------------------------------
http:

	downloaded 1100+ ppt file from cs while pinging cs 1/sec
	ping time started at 220 ms, ramped up to 9 sec, then 15 sec,
	back to 9 sec, up to 16, back to 9, then up to 12-15
	then plateau again at 12, then 15-16, then ramp back to 220 ms

	download 55K html file from cs while pinging 1/sec
	ping started at 220 ms
	ramped up to 9 sec
	ramped back to 220 ms

----------------------------------------------------------------------

changed MTU on serial link to 4096 but tcpdump showed tcp length to be
1460 on http, ftp transfers

----------------------------------------------------------------------
ftp brain, with 1/sec ping to 128.198.66.66:

ftp> get mail0009.txt.gz
local: mail0009.txt.gz remote: mail0009.txt.gz
200 PORT command successful.
150 Opening BINARY mode data connection for mail0009.txt.gz (128.198.66.39,49153) (593922 bytes).
100% |**************************************************|   580 KB    00:00 ETA
226 Transfer complete.
593922 bytes received in 434.61 seconds (1.33 KB/s)
ftp> 
 
during transfer ping delay ran about 12 secs - ramped up from 210 ms and 
quickly back to 21 ms when transfer complete, CPU 95% or more idle

-------------------------------------------------------------------------

miraloma# traceroute brain
traceroute to brain.uccs.edu (128.198.1.130), 30 hops max, 40 byte packets
 1  annex1.uccs.edu (128.198.66.66)  168.132 ms  168.479 ms  158.974 ms
 2  brain.uccs.edu (128.198.1.130)  178.949 ms  168.550 ms  168.976 ms
miraloma# 

miraloma# traceroute blanca
traceroute to blanca.uccs.edu (128.198.162.60), 30 hops max, 40 byte packets
 1  annex1.uccs.edu (128.198.66.66)  177.937 ms  160.161 ms  217.205 ms
 2  dhcat0-rsm.uccs.edu (128.198.1.252)  168.517 ms  168.729 ms  159.002 ms
 3  blanca.uccs.edu (128.198.162.60)  278.067 ms  188.715 ms  178.991 ms
miraloma# 


--------------------------------------------------------------------------

nice ftp brain, with 1/sec ping to 128.198.66.66:

ftp> get pics.tar.gz
local: pics.tar.gz remote: pics.tar.gz
200 PORT command successful.
150 Opening BINARY mode data connection for pics.tar.gz (128.198.66.39,49155) (380662 bytes).
100% |**************************************************|   371 KB    00:00 ETA
226 Transfer complete.
380662 bytes received in 277.28 seconds (1.34 KB/s)
ftp> 

during transfer, ping delay rose quickly to about 12 sec, quickly ramped
down to 210 ms. when transfer finished, CPU 95% or more idle

---------------------------------------------------------------------------

from brain, ftp to miraloma (annexp39), with 1/sec ping to 128.198.66.66,
ftpd on miraloma running niced?, CPU 95% or more (typically 97%) idle,
ping delay ramped slowly up (brain maybe loaded) to 9 secs, down to 
about 4, then up to 12 secs, then connection dropped and dialup didn't 
answer attempts to reconnect

ftp> put pics.tar.gz
200 PORT command successful.
150 Opening BINARY mode data connection for 'pics.tar.gz'.

--------------------------------------------------------------------------

ftp from brain to miraloma(annexp41), ftpd on miraloma niced,
ping 1/sec to 128.198.66.66, delay ramped quickly from about 210 ms
to about 12 sec, then quickly back to about 210 ms when transfer complete

brain.uccs.edu> ftp annexp41
Connected to annexp41.uccs.edu.
220 miraloma FTP server (Version 6.00) ready.
Name (annexp41:dmclapp): 
331 Password required for dmclapp.
Password:
230 User dmclapp logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> !ls pic*gz
pics.tar.gz
ftp> !ll pic*gz
ll: Command not found.
ftp> !ls -l pic*
-rw-r--r--   1 dmclapp  thesis    380662 Nov 17 20:19 pics.tar.gz
ftp> cd bin
250 CWD command successful.
ftp> put pics.tar.gz
200 PORT command successful.
150 Opening BINARY mode data connection for 'pics.tar.gz'.
226 Transfer complete.
380662 bytes sent in 2.5e+02 seconds (1.5 Kbytes/s)
ftp> 

---------------------------------------------------------------------------

http transfers from IETF (http://www.ietf.org/rfc/rfcNNNN.txt)
284K and 235K, 1/sec ping to 128.198.66.66,
ping delay about 9 sec,sometimes 5-6 sec, 
back to 210 ms when transfer complete

--------------------------------------------------------------------------

login to redcloud (2), ping blanca, ftp blanca, on redcloud:

ftp> get testfile
local: testfile remote: testfile
200 PORT command successful.
150 Opening BINARY mode data connection for testfile (15968633 bytes).
226 Transfer complete.
15968633 bytes received in 2.78 secs (5.6e+03 Kbytes/sec)
ftp> pwd
257 "/users/server/students/dmclapp" is current directory.
ftp> !pwd
/tmp
ftp> bye
221-You have transferred 31844351 bytes in 4 files.
221-Total traffic for this session was 31848019 bytes in 4 transfers.
221-Thank you for using the FTP service on blanca.uccs.edu.
221 Goodbye.
redcloud.uccs.edu> 

on redcloud, ping delay normally 150-200 usec, during transfer one delay
as high as 1.5 msec=1500 usec

redcloud.uccs.edu> traceroute blanca
traceroute to blanca.uccs.edu (128.198.162.60), 30 hops max, 38 byte packets
 1  blanca (128.198.162.60)  0.309 ms  0.207 ms  0.194 ms
redcloud.uccs.edu> 

5.6e+03 Kbytes/sec=5,600,000 bytes/sec=44,800,000 bits/sec (assuming
here that 1K=1000)
1 1500 byte packet takes 1500/5600000 sec = .267 msec
and 1500 usec allows xmsn of 5.6 packets
assume no options, IP header=20 bytes, TCP header=20 bytes, so
data payload=1460 bytes
so 1500 usec allows 5.6*1460=8176 bytes, about 8K (8192) bytes (where
here 1K=1024)
--------------------------------------------------------------------------

traceroute to freebsd.org:

 1  annex1.uccs.edu (128.198.66.66)  167.999 ms  159.418 ms  159.886 ms
 2  dhcat0-rsm.uccs.edu (128.198.1.252)  169.651 ms  169.570 ms  159.808 ms
 3  ucd-uccs.colorado.edu (204.132.228.5)  169.774 ms  169.491 ms  169.832 ms
 4  frgp-officepark.cudenver.edu (132.194.3.6)  169.734 ms  169.636 ms  169.750 ms
 5  * acr2-serial2-2-0-0.denverden.cw.net (208.172.161.205)  170.242 ms  169.578 ms
 6  acr2-loopback.chicagochd.cw.net (208.172.2.62)  209.905 ms  189.603 ms  199.802 ms
 7  exodus-comm-inc.chicagochd.cw.net (208.172.1.214)  189.749 ms  199.585 ms  199.821 ms
 8  bbr02-g1-0.okbr01.exodus.net (216.34.183.66)  199.763 ms  199.640 ms  189.738 ms
 9  bbr02-p0-0.sntc04.exodus.net (216.32.132.150)  239.783 ms  239.430 ms  239.857 ms
10  bbr02-p0-0.sntc05.exodus.net (209.1.169.142)  239.705 ms  249.577 ms  249.879 ms
11  dcr02-g2-0.sntc05.exodus.net (64.56.192.4)  249.697 ms  249.580 ms  249.808 ms
12  csr02-ve241.sntc05.exodus.net (64.56.192.43)  239.776 ms  259.586 ms  249.857 ms
13  g2-1.bas2-m.sc5.yahoo.com (64.56.207.150)  249.835 ms  239.545 ms  249.822 ms
14  hub.freebsd.org (216.136.204.18)  239.772 ms  239.515 ms  239.822 ms