The Psychology Of Browsing

As one browses the Web, one is drawn to areas that provoke, as well as areas that are of personal interest. What is interesting?

Zudfunck thinks it's all about sex. Many browsers would tend to agree. That would not explain, however, the popularity of sites like those on the Yahoo Buzz Index. The variety of sites causing buzz cannot be explained away by just that data point.

The key drivers do include sex, money & news. More generally, though, they are representative of the zeitgeist, a la Google.

(From Merriam-Webster, the general intellectual, moral, and cultural climate of an era)

In the end, though, who we are, drives what we browse.

Web Agents, play a collaborative role in this effort. A good web agent would be an extension of our identity.

An early, and effective web agent was Lycos. This searched the entire web, not just http - a good 1994 paper on Lycos can be found here - still relevant, makes good reading on Lycos implementation

Interesting breakdown of sites from 1994:


142132 http
102910 ftp
84143 gopher
4314 news
1396 telnet
379 mailto
244 wais
13 rlogin


Another agent mentioned in the paper cited, is called WebAnts. This seems to have some promise as a collaborative browsing agent:

To address these issues, the WebAnts project is developing a cooperative explorer agent; that is, an explorer (or ant) which will share results with other ants and not duplicate their efforts. Another system that employs multiple agents is the WebCrawler [Pinkerton 94], although it is unclear as to how distributed these agents are. The WebAnts model could be used for either searching or indexing, although at the moment, we are concentrating on searching, until we have settled on a design for the index.

The paper had some hope for the future, not yet realized:

The Future of the Web
We categorize Web agents into two broad classes: those that ``pull'' information out, such as the WebCrawler and Lycos, and those that ``push'' information back out, of which the best example is fictional. In Cold as Ice, [Sheffield 92] describes ``faxes,'' which can be defined as expert systems that can oversee experiments, answer questions, and replace many of the functions of secretaries and assistants.

It is this second kind of agent that we hope to build in the future, building on our success with TinyMUD agents such as Julia [Mauldin 94]. Instead of looking for answers, a proxy agent might read newsgroups looking for questions relevant to its owner's areas of interest, might send email to people working on a problem that the its owner has several publications that might be relevant, and might sort, prioritize and even answer email.