Chapter 5. UUCP and USENET

Also in 1976, Mike Lesk at AT&T developed UUCP (UNIX-to-UNIX copy). 1 V2 was implemented in 1977. 2

UUCP meant that information could be directed around the network (as it was). It also meant that one could establish a telephone connection and transmit information across that (relatively expensive) link. Two years later, three graduate students in North Carolina (Tom Truscott, Jim Ellis, and Steve Bellovin) took the next step.

Tom Truscott had an early interest in chess. While a student at Duke in 1974, he devised a chess program (Duchess) and played against Ken Thompson's Belle. Duchess lost on time. (In competitive chess, each side has a given time to make its next move; Duchess exceeded that time due to a core dump.) But Truscott competed in every ACM computer chess tournament from 1974 through 1980. He also attended the 1976 UNIX Users Group meeting at Harvard (1-2 April) and the 1978 meeting at Columbia (24-28 May), where he met Ken and others.3 In 1979, Truscott went to the Labs as a summer student and, on his return to Duke, arranged for a UUCP link. (He also attended the USENIX meeting in Toronto [20-23 June], to which we'll return in the next chapter.)

When he returned to Duke, he found that Jim Ellis had installed V7 on the Computer Science PDP 11/70. They employed the auto-dialer capacity to dial up two other Duke computers and one at the University of North Carolina. Ellis and Truscott then called a meeting to discuss their idea -- to have something like the mailing lists on the ARPAnet for computer sites that weren't on the ARPAnet. Steve Bellovin, then a graduate student at the University of North Carolina, attended and then wrote the first Netnews program -- three pages of shell script (later rewritten in C). The first implementation was between the Duke and UNC Computer Science departments; the Duke Medical Center Department of Physiology was added at the beginning of 1980. In January 1980, Ellis and Truscott went to Boulder, CO, and announced their Netnews design at the USENIX meeting (January 29 - February 1).

The first version of Netnews was fairly simple and efficient. It periodically checked the "last saved" time-stamp of each file in a specified directory, and then sent any file updated since the last check to another computer using UUCP across a modem connection.

Tom Truscott and Steve Daniel (also a graduate student at Duke) then rewrote the program to create what was called Netnews Version A. Since Netnews was designed for UNIX at a university, it was automatically categorized as public domain software under the conditions of the AT&T UNIX license, which greatly facilitated its subsequent use and adoption. This implementation appeared on the 1980 USENIX distribution tape, which was distributed at the Newark, DE, meeting (June 17-20). Duke University then invited other sites to join the network, which was made easier by the fact that the software was free, starting the first Usenet expansion -- to 15 sites. But one of those was Berkeley, which resulted in an explosive growth spurt.

That connection was the responsibility of Armando Stettner, then with DEC. Someone at the Delaware meeting complained about the inordinate cost of the long-distance telephone connections needed to get news to the West Coast. Armando spoke to Bill Shannon and they said that if they could get a news feed to decvax (in New Hampshire), they'd pick up the Berkeley phone bill. [Armando later supplied the first news feeds to Europe, Japan, and Australia, too.] The network soon spread to universities across North America, quickly establishing a critical mass of useful information, which made it even more popular.

In under a year, Usenet grew to 100 sites and over 25 articles per day (the original protocol had been optimized for 1-2 messages per day). Mike Lesk had never contemplated such uses of uucp. Truscott, Ellis and Bellovin had never imagined such popularity. The system collapsed.

In 1982, Netnews Version B was developed by Mark Horton (a graduate student at Berkeley) and Matt Glickman (a high school student) to increase efficiency so that USENET could cope with the increasing loads on the growing network. Horton continued to maintain the system till 1984, when Rick Adams at the Center for Seismic Studies took over maintenance of Version B, releasing 2.10.2. There were 135 news groups at the end of March. Version 2.11 of Netnews was released in late 1986.

In 1989, Netnews Version C was developed by Henry Spencer and Geoff Collyer at the University of Toronto again increasing efficiency.

By the late 1980s there were 20,000 newsgroups. Looking at what was current became nearly impossible. The remedy was "searching." As Mike O'Dell put it: "The Internet is now a rich fabric of resources and capabilities, and it is no longer possible to simply know all the places where interesting stuff is available. We now need tools with which to discover and navigate this world-wide treasure trove" (Computing Systems 5.4 [Fall 1992] p. 373).

Searching in a Distributed Environment

By the beginning of 1992, there were a number of "resource discovery systems" (much of this section is drawn from the special issue of Computing Systems cited above).

The earliest of these were search engines which pointed at specific, dedicated servers: whois, which responded to queries about people, network numbers and domains across the Internet; and X.500 (ISO DIS 9594-1 [1988]), a distributed directory look-up service.

The next tool was archie, developed by Alan Emtage and Peter Deutsch when they were at McGill University in Montreal. archie maintained a directory of materials available on about 1100 UNIX archives accessible by anonymous FTP. In 1992 it contained about 2.6 million files with 150 gigabytes of information.

Gopher (from the University of Minnesota's "Golden Gophers") provided a simple menu-driven interface to data searching.

Prospero (Neuman, 1992) was an "enabling technology," allowing users to create their own views of information in a distributed file system.

WAIS, wide-area information service, developed by Brewster Kahle and colleagues at Thinking Machines, was a network of over 70 servers world-wide, offering access to over 300 databases by natural language keyword search.

There were also knowbots (developed at the CNRI) and Netfind and a host of others.

But the "winner" was the World Wide Web (WWW), invented by Tim Berners-Lee at CERN, and first published in Electronic Networking in Spring 1992. We will return to the Web in about a dozen years.

1Lesk earned a Ph.D. in Chemical Physics and worked in the UNIX group, where he wrote tbl, refer, lex, and UUCP. He went on to Bellcore; was head of a Division at the National Science Foundation (1998-2002); and is currently Professor of Library and Information Science at Rutgers University.

2 UUCP has a rich history; see Chapter 15 of Casting the Net.

3 This was the meeting at which the name was changed to USENIX, spurred by a letter from an AT&T lawyer stating that Western Electric had not granted permission to use "UNIX" in "UNIX User Group."