Blog

Usenet History: Usenet Growth and B-News

When you think of Usenet, you have to focus on the computer-based global network. The primary purpose of Usenet thrived from its very name “use of the internet”. The major discussion forums on Usenet have been the news providers, news discussion, and information exchange.

Tracing history, we reach Reed College, University of Oklahoma, and UC Berkeley. All the top-tier educational institutions with their own researches on stake and an invitation to the future of networking. Out of these, Berkeley was the most progressive – being on ARPANET.

The curious minds of Jim Ellis and Tom Truscott around 1979 were shuffling along with this idea of “Usenet”. It was their graduation time, and naturally, their minds were sprawling in ideas to change the world and the way it functions. They were doubtful but braver. Another factor was the lucky timing. The idea would have slipped in thin air had there not been supportive successors at Berkeley.

Growth of Usenet

Berkeley was the ground, where the initial thinkers of Usenet met Mary Horton (a meticulous Ph.D. student). She, alongside Bellovin, brought TCP/IP to the Usenet Lab Trials. They experimented with various network lines. They tried to combine mailing lists into Usenet groups. This was indeed a revolutionary idea – unforeseen at that time. The groups were divided based on interests – quite logical as interests are a prime way to connect and be a part of a social group.

With the potential of actual traffic on the Usenet Network, it was an exciting way to share interests and connect with like-minded people. There were two noted divisions – SF LOVERS and HUMAN NETS. SF LOVERS stood for science fiction enthusiasts – which were quite a lot. These systems were dedicated to understand and debate on science fiction. HUMAN NETS was a little more complex than SF LOVERS. It was based on the general networking attitude of people. The primary purpose was just to find people. To connect with people who you could talk to about things that are interesting and that are substantial.

Usenet Backing

All these activities were permissible given the grounds of Berkeley. Many other such budding networks were often shut down, either through wrong execution or through authority. Usenet was non-commercial. It did not want to raise any dollars, and thus it was not perceived as any threat to any entity.

B-news

After sites became active on the network, Usenet rose dramatically. There were more newsgroups, articles, and information spots. But, with these came the B-news. The Usenet server is dedicated to news, with advanced improvements. A very impressive feature of the B-news was to read articles in a randomized order. It even encouraged messages. The human psyche was the same though – even messages then resulted in fake news and pranks. So there was more control demanded on this feature.

An improvement over A-news was the directory storage of B-news. The directors were able to form a hierarchical structure and introduce the concept of subdirectories. This was great for databases and further discussions. B-news was, thus, significantly better than A-news. The news showed the latest news and kept updating on that loop. It did not analyze or provide the space to analyze. B-news worked on every loop-hole of A-news. It even encouraged more advancement.

Usenet Consequences and Negative Notions

Usenet was indeed quite beneficial and like a layer of freshness in those times of technological uprising. There were, though, some negative notions as well. The site was unable to carry the load at times – crashing and not loading. There was some mischief around the messages and emailing systems too. The original purpose of the network was fading in the chaos of connections and information overload.

With the latest changes in technology, many negative consequences of the Usenet are repaired and sought after. They have merged with a more-refined and foul-proof systems. There are new servers as well.

Usenet History: The Public Announcement

UseNet history

Usenix was a relatively small and informal organization in the eighties. Its members wanted to launch Usenet in a Usenix meeting in 1980. Those were the days when the organization held its meetings at universities instead of big conference halls and meeting rooms of plush hotels. The meeting was held in Boulder. While Steven Bellovin was not present in the meeting, it was attended by Jim Ellis and Tom Truscott.

Bellovin felt that apart from announcing about Usenet, they required non-experimental code while his prototype was not adequate to succeed. However, he does not exactly remember the exact deficiencies present in his C version. At the same time, he thinks that one of them could be the code’s inability to configure, which neighboring websites would get what newsgroups.

Issues while declaring Usenet

Incidentally, Stephen Daniel came up with the code, which was referred to as “A-news”. A crucial alteration was the feature to exist with multiple hierarchies instead of simply having the original “NET.” or “NET”. The production version also had the possibility to configure which hierarchies or groups a website would get.

The members believed that the configuration should be in a file instead of being included in an array within the code. This particular point was not always considered. For instance, UUCP had an array for listing the commands that remote websites were allowed to execute.

To execute the rnews command, a system administrator had to alter the source code before recompiling. While looking back, this procedure looked like a wrong decision. However, in those days, some argued that it was justifiable to do so. After all, there are very few commands at that time.

Sorting the problems

The group decided to sort out this problem and came up with a mail-to-rnews program. It means a sending website would be able to email articles instead of directly trying to execute rnews. A clock-driven system was designed for retrieving the email messages and then transferring them to rnews.

 The system had to be clock-driven as there was no other procedure available in those days to get the email directly delivered to a file or a program. As such, the A-news’ remote site configuration file is required to have a command for the execution.

The group wanted the first articles to cover issues such as trouble reports, general requests for help, and bug fixes. At that time, there was a strong positive culture to offer mutual assistance among developers not only in companies like Usenix but also in the world of the IBM mainframe.

Another proposal was to locate source code that appeared interesting without flooding them to the network. The reason for that was the chances of software being bulky and phone calls at that time were costly. In those times, the phone rates at nighttime were around 0.50 USD for 3 minutes.

It was also a time when at 30 bytes/per second or 300 bps, one could transmit a maximum of 5400 bytes. If protocol overhead had to be taken into account, the conservative estimate was about 3000 bytes or one kilobyte every minute.

So, the source to UUCP is approximately 120KB at 1 KB per second, which is 20 USD or 2 hours. If inflation is to be adjusted, that is more than 60 USD in today’s time. Also, most people do not wish to have most packages.

Lack of adequate bandwidth

Another issue was Duke had just a couple of autodialers. So, they did not have the necessary bandwidth to transfer large files to several places. If they tried to do so, it would stop all news transfers to other websites. Rather, the suggestion was that perhaps Duke could act as a central repository. The software might be retrieved on demand. UUNET adopted this model later on.

However, the most interesting point is that the announcement did not discuss any possibility of non-technical use. There were no hobby discussions, social discussions, or political discussions. They did not think people wanted to discuss issues with someone they did not meet ever.

Usenet History: Authentication and Norms

Usenet is a computer-based international distributed discussion system. The particular Unix-to-Unix Copy (UUCP) dial-up network scaffolding was used to create it. This idea was devised by Tom Truscott and Jim Ellis in 1979, and the company was founded in 1980. At the time users could read and post messages (known as articles or postings and together referred to as news) to around one or more newsgroups. In many ways, Usenet is similar to a bulletin board system (BBS), and it was the forerunner of the commonly used Internet forums. These discussions were threaded, similar to the web forums and bulletin boards we see today, except that these posts were saved chronologically on the server.

Authentication & Security:                                     

Usenet was now in function. Well, what now? Truscott and his group were aware that it would need some sort of an administering system. This also meant that they soon realized that they lacked in the authentication arena as well. Truscott and his group needed an authentication system for sites, users, and even posts. Even though they were aware that they would need something like a public cryptic key to authenticate it, Truscott and his group weren’t aware of how a site’s public key could be authenticated.

They could’ve used a certificate issued by a certificate authority, but they didn’t know it at the time. Even if they had known, there was no way for them to get it done as there were no such authorities present at the moment. Also, they couldn’t simply create one themselves while also creating Usenet. Their knowledge in this subject ran very thin, and they had questions that they could not quite find answers to. For example, what exactly was secure, or what would be the ideal length of the key?

This shortcoming brought them to consider alternate options: What if their sites could authenticate each other in a peer-based authentication system? They considered the idea of this neighborhood authentication for a while but were ultimately met with failure because of loopholes in its logic. In order to spoof an authentication, a user had to simply fake its path line to claim to be a couple of neighboring sites away. This could potentially cause security concerns.

Security was quickly becoming a roadblock that kept them stagnant for a while. They realized that their approach to authentication had several loopholes in them, though anyone with the slightest knowledge of Usenet could get through.

To understand this flaw in the system, it is important to note that messages on Usenet were sent using a generic remote execution mechanism. This means that the site ran a command to read the next-in-line computer that had the ‘rnews’ command running. This is exactly what was concerning. If someone knew about this detail in the algorithm, they could easily just manipulate the neighbor’s key to get an authentication.

At the time, it wouldn’t have probably caused a big issue because the number of users was very low as the internet was only used by a handful of people. But making something that only gave off a perception of security instead of real security wouldn’t have been very ethical to do.

Norms:

In principle, detecting misconduct and determining who perpetrated it will be simple. Like UNIX, the uucp system isn’t built to avoid excess consumption problems. Which uses of the internet are actually misuse, and what should be done regarding them, would be revealed via experiences.

Abuse of the internet may be quite dangerous. They may be talked about, sought for, and sometimes even coded against, just like conventional misdeeds, but only experiences can reveal what counts. Uucp gives a certain level of security. This operates as a regular user with stringent access restrictions. It is reasonable to argue that it does not offer a higher hazard than a call-in line. But the main issue was that they had no idea what to do or what other difficulties would arise. They were concerned about security to some level. They actually encountered their earliest hackers in 1971, when some behavior resulted in a console alert. This ordeal prompted them to inspect the punch cards for the code in question — but it was far from their major concern. They made a fast-and-easy password guesser and let a few individuals know that their passwords were terrible and could lead to potential hacking.

Usenet History: Implementation and User Experience

Before you begin to understand the implementation of Usenet, you must know about two critical things which contributed. First, the University of North Carolina Computer Science department had a Unix machine equipped with a slow, small disk, a slow-performing CPU, and most importantly system with a low RAM. This machine is supposed to be slower than most time-sharing machines for 1979. Duke CS had a relatively faster computer, the 11/70. Since it was the first implementation, UNC’s offering of 11/45 had to be used. During 1979, there was no Internet, and departments were not connected to the ARPANET, so logging in remotely was not an option. Using a dial-up network, billed per minute, costed daytime charges. A speed of 9600 bps could be topped if connected via the Gandalf port selector.

The second important thing to bear in mind is that the first implementation would involve few experiments. The first-ever public announcement of Usenet read out the problems faced during the implementation. Many amateurs worked together on this plan, but it was time to get started. Once Usenet was made available, a committee could be formed and later that committee could use the net to begin analyzing what the problems were. A network protocol had not been designed before. A few experiments had to be carried out to get things right. Do keep in mind that Tom Truscott and Jim Ellis had programming experience and were experienced system admins. Tom had communications software experience and had been programming kernel-level software for about 14 years.

Implementation of Usenet

The strategy implemented for developing the Usenet was rapid prototyping. The first version of Netnews software was implemented as a Bourne Shell Script. The script had features such as cross-posting and multiple newsgroups and was 150 lines long.

But why use a shell script for programming? The simple reason is: Compiling a program took a very long time. The more the waiting time to compile made Tom start something new. Most of the code made use of string-handling concepts and C programming was not good for string-handling. You could have written a string library, but it was time-consuming as the compilation speeds were low. With shell script, you could try out new things and develop the code incrementally. The shell script is slow to use in production and didn’t run that quickly when executed. It was not much of a hindrance as it was not a production program. It was a prototype intended to be used to create a file format. Once everything was in place, it was re-written in C programming language.

Implementation details

Regrettably, the script version and the C version of the implementation are not available today. However, Tom remembered a few of the implementation details. The subscribed newsgroups of a user were being saved in an environmental variable set in the .profile file. There were commands to retrieve all the articles the particular user had read previously. The retrieval was possible through $HOME/ .netnews to mark with the current time. On successful exit, only the last read time was getting saved. The script was not written to have the capabilities to read out of an order, to skip articles to read them later or to even stop reading midway in an article. The limitation was due to an assumption error—only a couple of articles would be read per day. The incoming traffic today is 60 tebibytes per day. The prediction was off by many orders of magnitude.

Other implementations

Another implementation was not to display cross-posted articles more than once. The cross-posted articles appeared as a single file linked from multiple directories. This technique was not only helpful to find duplicates but also saved adequate disk space. At that time, disk space was quite expensive.

Few other points worth knowing: Redundancy of a global coordinator as each line had an article ID with the site name, period, and a sequence number. The filenames were the articles’ IDs that had a character limitation. A database may have helped to store the files. But a single database of all news would require a locking mechanism which was hard to achieve on a 7th Edition Unix. Pipes had to be created before the processes, so the file system had to be relied upon. The UI resembled the 7th Edition mail command—it was simple and worked seamlessly for exchanging low-volume mails.

Usenet History: File Format

The designers of the file format over the wire knew that it would not be perfect at their first attempt. The first decision they took was that the transmitted file’s first letter would be “A” for this version.

Why were email style headers not used in the beginning?

Many people might ask that why email-style headers were not used initially, though it was later used for HTTP. The key reason for not doing so was that many of them did not have any exposure to such protocols during that time. The author admits that even he got to know about Internet Protocol after receiving a copy of a workbook two years later. It was because of the USENET that he was aware of such protocols.

The designers instead chose the minimalist style, which was influenced by the seventh edition of Unix. Had they been aware of the Internet that was known as ARPANET those days, they would have avoided it deliberately. A shell script was the first version of their code. They felt it was easier to deal with complete lines as single entities. Also, continuation lines, optional white space, and parse headers, which enabled arbitrary case was definitely simpler.

Issue of duplicate articles

They also had the issue of how to handle duplicate articles. The designers felt that an article ID was an absolute necessity so that duplicate detection would be allowed. At that time, they decided to have the article ID as the remaining part of the 1st line after the letter A.

The designers also wished to minimize the costs for transfers. At that time, article transmissions were carried out by costly, dial-up connections. So, transmitting a file that was not required also required them to spend a lot of bucks. As such, articles then had to have a series of systems to indicate that the article was already seen.

This information comprised a string of hostnames and exclamation points that separated them. The last element was the user’s login name who posted it.

A pertinent question could be why that particular format was chosen rather than something like blanks or commas as separators. The format chosen by the designers was used by UUCP for email.

Today, the scenario has changed entirely as there is full connectivity over the web. Things are no longer done in the same way. Rather, a party would transmit a series of article IDs. The party would then ask for the ones that have not been seen.

It is interesting to note that the designers had contemplated something of that sort but then decided to reject it. After all, they were using dial-up connections that were infrequent to relay articles. Alternatively, the count of loops and as such, duplicate articles received did not appear to be high.

In the original scheme of plans, the Duke would poll several sites once per night. If Duke sent a list of articles to the sites during that call, they were not allowed to request for it until the succeeding night. Also, they would not get those articles until the following night.

However, such a delay was not acceptable. The designers instead decided to have the possibility of transmitting unnecessary text. There would be some additional transmissions on certain occasions. However, it was felt that such a volume would be acceptable. It was an era before MP3 and JPG formats. So, we are talking about only text articles. Thus, these would be relatively tiny and inexpensive as well.

It was obvious that the article’s title and date would also have to be sent. The library routines asctime() and ctime() were used to generate the time and dateline. The designers had made up their minds from the start that there was a requirement to have articles in multiple categories i.e. newsgroups. However, there was just a single relayed newsgroup called NET in the original design. There were no differences between various types of non-local articles.

Why was there cross-posting of articles?

Finally, there was one more interesting thing to note. They were aware from the beginning that some articles could be a part of multiple categories. Hence, they supported the cross-posting of articles in different newsgroups from the start. Although some people considered cross-posting to be impolite, the feature was intentionally included from the beginning.

Gets Exclusive Content & Expert Advice

Subscribe to our marketing newsletter to get the latest tips and advice delivered to your inbox each month!

Email Address*

Connect With Us

Featured Posts