Stigmergy and the World-Wide Web

Joe Gregorio

or how we all became ants building a nest we can't even see.

Weblogs, Neighborhoods, and Google

Weblogs, Neighborhoods, and Google are all phenomena of the World-Wide Web. All of these are fairly new and they are all very powerful. Weblogs are successfully taking on large publishers on their fact checking. A minor shift in Googles ranking algorithms creates huge ripples. Warchalking swept through the web and onto street corners in a matter of days. Is there any connection between them? I want to convince you that they are all intertwingled, all the result of the same phenomenon, and that we have much more interesting and powerful phenomenon on the horizon.

Weblogs

I have a weblog. You are reading it. Just a slim page among the billions on the internet. I stop by here several times a week and write up recent ideas, observations or just point to plain old weird stuff I have found on the internet. Stop and think about that for a minute. If this weren't on the web, what would be the closest analogy for what I am doing? Writing my thoughts and ideas down on a chalkboard on a street corner is the closest I can come. Imaging a street with chalkboards mounted at random points along the thoroughfare. I write my stuff on my chalkboard. Other people have their own chalkboards on the same street. Sometimes I walk by their boards, read what they have to say and if I find it interesting then either I scribble a little note on their board (if their weblog has a comment system) otherwise I run back to my board and scribble a note about what I read 'over there' and what I think about it. If I find something interesting happening on the street I go back to my board and write about it. We never speak directly. We only communicate through the boards. Writing on our own boards, or scribbling even smaller notes on other peoples boards is the only way we converse.

Now if you saw this happy little analogy playing out on the street you would do what any sane person would do; have everyone involved committed to a psychiatric ward. For pete's sake, you'd exclaim, just talk to each other or use a phone. But this is exactly what we're doing when we use our weblogs, communicating that is... not getting committed. If we want to talk so badly why not use email or pick up the phone? The weblog has got to be the single most inefficient mechanism for communication that has even been invented. Webloggers should be committed en masse. There's only one problem: It works.

Neighborhoods

I have a weblog. You are reading it. Just a slim page among the billions on the internet, oh wait, I already said that. But anyway look at the people in my neighborhood, as defined by Google. Or maybe the neighborhood found by Mark Pilgrim's Recommended Reading. Google and Mark's tool do a very good job of picking out similar sites with the vast majority of their recommendations already in my news aggregator. I can check them several times a day with my news aggregator while I still have to visit a few throwbacks manually. They write about stuff that is interesting to me, I hopefully do the same for them, and we often link among ourselves or a group of us will all link to the same story we found on the internet. There is no geographic grouping that supports this grouping, these people are from all over the planet. Billions of pages, millions of users and yet here is a tight clustering of people with similar interests, a grouping of people that know about each other.

Google

Google is the premier search engine in the world. Here is their description of their basic PageRank algorithm:

PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important." [Google Technology]

Memes

A meme is a unit of intellectual or cultural information that survives long enough to be recognized as such, and which can pass from mind to mind. They can be carried by word of mouth, dead trees, e-mail, or the web. On the web, in particular on weblogs, memes are tracked by by links to particular sites or stories, thus the rise of blogdex, daypop and popdex. Memes just in the realm of newspapers are now tracked by news.google.com which does an excellent jobs of summarizing the news the of the day automatically, no human internvention required, by collectively scanning thousands of news websites and pointing to the stories that are being talked about the most.

In a similar vein periodical bursts of activity will swirl through the weblogs. Examples of such bursts of activity include the road to RSS 2.0, RSD [More on RSD], neighborhood explorers, TrackBack, XHTML Syndication, and StructuredText (aka Textile aka WikiML). Is this all just serendipity?

Why?

Why do memes spread so easily on the web? Why is Google's PageRank algorithm so successful? Why do Neighborhoods form? Why does communicating through a weblog work at all? What is stigmergy?

What is stigmergy?

Stigmergy, a term coined by French biologist Pierre-Paul Grasse[2] is interaction through the environment.

Self-Organization in social insects often requires interactions among insects: such interactions can be direct or indirect. Direct interactions are the "obvious" interactions: antennation, trophallaxis (food or liquid exchange), mandibular contact, visual contact, chemical contact (the odor of nearby nestmates), etc. Indirect interactions are more subtle: two individuals interact indirectly when one of then modifies the environment and the other responds to the new environment at a later time. Such an interaction is an example of stigmergy. [1, p.14]

Examples from nature

Ant Trails
Ants are the premier example of organisms that use stigmergy. Did you ever drop a bit of food on the ground outside and return to later find it swarming with ants, all following a single narrow trail back to their nest? Why do they settle on just one path? How do they find the food and get everyone to use that same path? The answer is stigmery. When an ant finds food it heads back to the nest leaving a pheremone trail, a trail of scent that will evaporate over time. Other ants that come across the trail will follow it and if they find food will also head back to the nest also leaving a trail, re-enforcing the scent. The more food, the more ants, the stronger the scent on the trail. Once the food is gone the trail evaporates. Not only do ants use stigmeric communciation to find the food, they also end up finding the shortest route to the food. If multiple trails are found then the shortest route will dominate because the shortcut will result in more trips per ant per unit of time and thus the scent on that trail will be stronger and draw more ants.
Termite nest-building
Termites in Africa can build incredible nests that are up to 30 feet tall that contain many tunnels and chambers. These tall complex structures are used to regulate temperature, humidity and oxygen in the nest which is all carefully controlled to promote the growth of the fungus which the ants eat. The construction of the termite mound is controlled through stigmergy, in particular the construction of columns and arches is controlled by the evaporation of pheromones. The ants place pheremones on the balls of mud they use for construction. The new balls are placed near existing pheromone concentrations. As construction proceeds and the columns get taller the pheromones near the bottom evaporate. If two columns are built near each other then the concentration of pheremones at the top of the columns will cause the two to be joined into an arch.
Ant corpse-gathering
Ants are dying all the time and their little corpses are gathered into a single burial mound. This may seem symbolic but it is functional and easily coordinated through stigmergy, in this case the stigmeric communication is not through pheromones but the corpses themselves. Ants pick up their fallen brethren and put them in piles, with a preference for larger piles over smaller piles. Ants can also pick a corpse from an already formed pile and move it to another pile, again preferring to drop corpses in larger piles over smaller piles. Many ants will perform these actions over time and over time all the piles end up merged into a single large pile.

Web = Stigmergy

And now finally we get the point. The whole point of this is the connection between the web and stigmergy. The World-Wide Web is the first stimeric communication medium for humans.

The telephone and email don't count as stigmeric communication since they are only readable by the people on either end of the phone call, or the e-mail. In order for an environment to support stigmeric communciation the messages must be readable by everyone. Radio and TV don't count since they are a read-only medium as far as most people are concerned. In order for an environment to support sitgmery everyone has to be able to not only read it but to be able to write into it also.

Oh sure, we have had books and newspapers, but for the vast majority of people the only avenue they have to 'write-back' into that environment is in the 'letter-to-the-editors' department. Now we have groups.yahoo.com, K5, Slashdot, and weblogs. All avenues for anyone to enter into the conversation. Avenues for anyone to place a mub ball here or an ant corpse there. Yes, I know, not everyone has access to the internet, and it certainly isn't free for most folks, but the point is that it is the most accessible system yet and prices are falling and more people are getting on everyday.

Note that some other internet based communciation forms have moved onto the internet. Many mailing lists now have web accessible archives. Groups.yahoo.com takes it even further and allows posting from the web to the list as well. Another notable migration has been of usenet newsgroups onto the web via the much lauded groups.google.com. In both of these cases the mailing list and newsgroups were enhanced from being able to be searched and linked to from the web.

Now that we know web is a stigmeric communciation medium and that we've seen some of the power that nature has gotten out of stigmergy the answers to our earlier questions become rather easy.

Weblogs

Why does communicating through a weblog work? Stigmergy.

Using a weblog is communcicating through stigmergy. Just like an ant, as I blog I leave a trail of information and links to other information I find interesting.

Google

Why is Google's PageRank algorithm so good? It is just following the Ant Trails.

If links represent a dropping of pheremone then Google is just following the trails laid down to the tastiest morsels.

Neighborhoods

Why do Neighborhoods form? Ant Corpse Piles.

Just like Ant Corpse Piles, if I link to you and you link to me that brings our weblogs closer together. The more we talk about similar stuff the more likely we are to cross link to each other. The more links to each other and the more links from us to similar material on the web the closely Google thinks we are related. The habit of 'welcoming' new bloggers with simlar interests by linking to their site with a welcome message only grows the pile.

Memes

Why do memes spread so effectively on the web? Stigmergy.

Because they are travelling through a stigmeric medium. They can live on the internet where anyone can find them either intentionally, by using Google to follow the trail, or serendipitiously by the idea moving into a receptive neighborhood.

Summary

The World-Wide Web is human stigmergy. The web and it's ability to let anyone read anything and also to write back to that environment allows stigmeric communication between humans. Some of the most powerful forces on the web today, Google and weblogs are fundamentally driven by stigmeric communication and their behaviour follows similar natural systems like Ant Trails and Nest Building that are accomplished using stigmergy. The web is new. In the context of written human history is barely a blink of an eye. Yet as new as the web is, it is already showing it's ability to support complex human interactions that mimic natural systems use of stigmergy. And were just getting started...

Further Reading

Some of the things I have described follow from fairly simple interactions. Here are some of the things Ants can accomplish using stigmergy:
A Superorganism's Fuzzy Boundaries
Structure of Macrotermes mounds

The classic paper on why the web is successful is Clay Shirky's In Praise of Evolvable Systems.

Learning to view the world in a decentralized way is hard. Here's some reasons why and how to break out of the habit: Learning About Life

Bibliography - Dead Tree References

[1] E. Bonabeau, M. Dorigo, and G. Theraulaz, Swarm Intelligence: From Natural to Artificial Systems, Oxford University Press, New York, US, 1999.
[2] Grasse, P.P. (1959). La reconstruction du nid et les coordinations inter-individuelles chez Bellicosi-termes natalensis et Cubitermes sp. La theorie de la stigmergie: Essai d'interpretation des termites constructeurs. In Ins. Soc., 6, 41-83.
Don't over look the work of Peter Small at: http://www.stigmergicsystems.com/

Posted by Joe 10 on 2003-05-06

Excellent article.

Posted by randy on 2003-05-10

Nice analogies. Here are a couple of links that might be interesting to those enjoying the concept of Stimergy: http://alfa.ist.utl.pt/~cvrm/staff/vramos/ref_42.html http://alfa.ist.utl.pt/~cvrm/staff/vramos/ref_39.html http://alfa.ist.utl.pt/~cvrm/staff/vramos/ref_29.html ... and finally what if an image habitat could be seen as an stigmergic interactive environment emerging a "species" of collective conscience ?! http://alfa.ist.utl.pt/~cvrm/staff/vramos/KAFKA_2_RED_ANTe.zip http://alfa.ist.utl.pt/~cvrm/staff/vramos/MC2.html best, v.

Posted by Vitorino RAMOS on 2003-06-04

This comment really only pertains to the statement "The World-Wide Web is the first stigmergic communication medium for humans." I comment because I disagree with this statement. I believe that there are other mediums which humans use, and have used for generations. I give two examples.

The first is in the process of navigation and erosion. For many many years in our history we have formed routes from one place to another. While walking through the bush, or veld, or savanna wherever one finds similar environments, people find paths. Many of these paths were formed, not through the central planning of an agency, but rather through the distributed and additive effects of the actions of many. Where there are no paths people plan for themselves a route which will require a minimum effort, as more and more people walk a similar route, a path is worn into the terrain, and this makes that chosen path a route with the least effort.

Actually, it exhibits many of the same qualitative features of the ant trails explained in Bonabeau et. al. - including bifurcation, and other similar effects, like continued use of a longer path when shorter options become available, etc.

A number of necessary features for stigmergy are present (depending on the definition you choose to pick), including the multi-cast features, communication that effects the environment for an extended period, the autocatalitic effects, and the continued communication as a side-effect of some other effort.

Another example, perhaps just as old, and even more damaging in some cases, is the process with which people choose illegal dumping grounds. A necessary feature is some degree of seclusion, or at least shelter from the prying eyes of the law. However, we find that once a place has been chosen by some offender, their dump adds a stronger justification to others for that site.

As more people dump illegally on that site, more opportunistic offenders become aware of that site simply because it looks like a dumping ground. Again we have a positive feedback mechanism. The analogy to the operations that the Belicositermes Natalensis and Cubitermes perform is nest construction is obvious. What probably differs (although who can really tell what goes on in the mind of an illegal dumper) is the complexity of the rules involved for choosing a site... such things as seclusion, distance from police, types of refuse already there, distance to an escape route, etc. These factors do differ - but, the communication mechanism that causes those sites that are more favourable to be promoted automatically is no different.

Anyway, I do feel that although blogs and such are wonderful methods for spreading and influencing ideas, I do have serious doubts about human stigmergic interaction occurring therein first. Perhaps the examples I cite are more specific than the whole range of ideas that a the web now allows to be transmitted: any idea that can be successfully expressed is a candidate for the WWW. And that is the amazing bit, at least in my mind.

Posted by Dylan Shell on 2003-08-31

You might want to take a look at Peter Small's recent work. He's interested in using such comments for aggregating information.

http://webseitz.fluxent.com/wiki/PeterSmall

Posted by Bill Seitz on 2003-09-15

Well!
The Google Pagerank is not to be trusted cause it's has it weakness. High PR doesn't give a top position automaticly at google search engine. It's the quality inbound links who really counts.
/danny

Posted by Danny on 2004-08-14

Great. Just what the world needs. Another buzzword. Stigmergy.

Posted by anonymous on 2004-09-03

The term stigmery has been around since the 50's, and was widely used in the 60's by ethologists and computer scientists alike. It's far from a "new" buzzword.

Posted by Chris on 2004-10-27

This is a cool site.richa

Posted by richa on 2005-07-13

Greetings:

Can yo subscribe me to your blog? Quite interesting.  I am here to read up on stigmergy, now interested in your blog in general. Thank you, ldmf@att.net

Posted by L. D. Misek-Falkoff on 2005-08-23

comments powered by Disqus