The ultimate grid

[TankGirl]

Scientists and other specialists have been working for some time to design and set up a new computing and communication platform called ‘the scientific grid’. Their idea is to hook together a number of powerful supercomputers from around the planet with dedicated high-speed lines. With proper control hardware and software this grid of supercomputers can be configured to act as a single virtual ‘hypercomputer’ with horsepower far exceeding that of any of its components. The researchers can then use their new number crunching beast to tackle the very hardest jobs on fields like astronomy, biology, chemistry and physics.

Besides the scientific grid, there is also another grid - even more exciting and interesting - in the making. This grid is also a computing and communication platform but of a different architecture and composition. Instead of supercomputers it will be powered by consumer level workstations, and instead of dedicated lines it will use mere domestic broadband connections over public Internet. Its strength does not come from raw power or raw speed but from big numbers. By binding together millions and millions of computers it will be a powerful beast on its own.

There’s a fundamental difference between how the scientific grid and the ‘domestic’ grid are designed and built. The scientific grid is managed in a systematic, organized and professional fashion by the participating organizations and nominated persons - just as you would expect from any important international co-operative project. The domestic grid, in comparison, evolves in a wild and uncontrolled, almost organic fashion. It does not have any committees or organizations to steer its growth; there is rather a Darwinian battle for best solutions. Who builds it? Anybody who decides to join. Who provides the software? Anybody who is willing and up to the job. Who decides what solutions and protocols will prevail? Anybody and everybody by their own choices and selections in their own sphere of influence. So instead of being built and organized, the domestic grid rather builds itself and organizes itself - just like living systems tend to do.

The greatest difference between the two grids, however, comes from a special computing device attached to each computer in the domestic grid. This gadget, even if literally ancient in its design and circuitry, is nevertheless sophisticated and powerful enough to give the domestic grid all sorts of fancy capabilities that the scientific grid can not have by its design. The attached device is nothing less than the human brain, kindly made available by the living human intelligence residing in it.

***

Now what does it mean to have a grid of computers with brains attached to them?

Your first association might be a Matrix-like hardware connector plugged into the back of your skull. No need for anything so intrusive and unpleasant! The familiar keyboard, mouse and monitor will do just fine – after all, we have been successfully using them to link our brains to computers for decades. To be effective, the brain-computer link does not even need to pass that many bits per second – it’s all about the meaning and value of those bits. The brain is a poor number cruncher but a wonderful distiller of meaning and value from whatever data it digests.

There is no need (nor possibility) for any particular brain to be available all of the time for the grid to work – after all, we need our precious brains for all sorts of things during our days. The computer and the software are there to do all the mechanical round-the-clock housekeeping and bit shuffling. It is enough that a meaningful fraction of the brain population is available at any given time. Should an average grid user spend say half an hour each day at her computer on grid-related activities, in a grid of ten million computers that would sum up to an average of 200,000 active brains online at any given time. We are thus talking about some serious brain capacity here, comparable to the resources of the largest multinational corporations. Naturally this set of brains would and could not run a corporation – but it could well run a community.

***

You may have heard about Moore’s Law. Gordon Moore, a co-founder of Intel, predicted in 1965 that the data density on integrated circuits would double every year. His prediction turned out to be surprisingly insightful, even if the doubling cycle has been closer to 18 months since late 1970s. The main thing here is that the growth is not linear but exponential. In other words, the power of microprocessors and the capacity of memory chips is not only growing – it is accelerating. The same applies to hard drive capacity. Line speeds have grown at somewhat slower pace but grow they do, and as a net result we have a technological infrastructure that accelerates in capacity.

Accelerates towards what? What do we do with these faster processors? What will flow through these faster lines? With what do we fill up the ever larger hard drives?

Well, for starters, we will fill them up with whatever works of art and culture there already exist. This process is already well under way thanks to p2p, and as a result there exist massive private music and movie collections competing head-to-head with what the multinational corporations can offer. As our Jack Spratts pointed out recently, when multi-terabyte drives become commonplace, we are in a situation where all pre-existing digitized content is easily available to everybody, everywhere, anytime. This will inevitably mark the end of the scarcity paradigm, and the commercial content providers will just have to adapt to it if they want to stay in the business.

When the ‘catching up with the past’ is done, the focus will naturally shift to what is new. It is not too daring to predict that all sources of public content (news, live broadcasts, music and movie releases) become eventually freely and easily available to everybody. The existing p2p technology is already giving good hints on how to build distribution networks that adapt swiftly to the changing interests and spontaneous selections of people. More channel capacity can be automatically allocated to whatever content happens to be popular at any given moment, and this can be done smoothly in a decentralized fashion with no central authority needed to watch over it. BitTorrent, a pioneer in this type of technology, can already provide fresh TV episodes with a one day delay to anybody with a broadband. Just imagine the impact of this delay shrinking to one hour, to one minute, to one second.

***

Not only will receiving new content get faster and easier; the same applies to pushing it around. The technical threshold for anybody to set up their own program channel – be it a music playlist, a public webcam, a movie feed or whatever – will become lower and lower. This process has also been going on for some time, and even many in this little community have personal experience of broadcasting something on Internet. Bandwidth capacity is still a problem but it is a passing one. In a few years or a decade we will be in a situation where anybody will be technically able to broadcast even to the whole planet (should the planet be interested to hear) without having to worry about things like bandwidth or transmission costs.

When being able to get something ceases to be an issue, the challenge will be to find out what is worth getting. When being able to give something ceases to be an issue, the challenge will be to find interested receivers. And here the brainpower will show its strength and flexibility.

Peer comments and recommendations have always been an important source of information on the more social p2p networks, influencing strongly what people download and listen to. They are good examples of how valuable a few bytes of metadata can be when its source is a living human intelligence with access to personal memory, experience, knowledge and understanding. Recommendations may be given in informal chats or declared more formally; in any case the brain is smart enough to understand the message and to evaluate it from its own unique viewpoint.

When recommendations and other suggestive metadata is combined to social factors like trust and personal interests, it is possible to build all sorts of chains of influence and fields of interaction between groups and individuals. This way the brains will not only affect the movements of the content on the grid but also automatically rewire the underlying network infrastructure to match their tastes and preferences. The beauty of it is that there is no need whatsoever for a single dimension of quality or a single definition of popularity. All program sources being similarly available to everybody, any custom catering of them would be similarly valuable and relevant, even if some caterings might be more popular than others. The customer would always be right by definition.

Where will such a grid eventually evolve? What will become of it? It is impossible to say. Having millions of human brains as its components it will be an immensely sophisticated, unpredictable system with its fate in its own hands. You could characterize it doing massively parallel multitasking on the substance of human culture, but the outcome of that processing can hardly be guessed. It will be an open-ended adventure to live through and to find out.

- tg

[JackSpratts]

The truly mind boggling aspect of this, as you’ve pointed out, is the sheer accelerating madness of the process. Things have been fairly exciting and fun up until now as we’ve harnessed the power of the grid to do things we’ve normally done for ourselves but were too lazy to do seriously or we’ve used it to do things we’ve always wanted to do but really couldn’t (like retrieving every extant song for instance). Amazing feats of power from our perspective of course but mere trifles for coming generations. From now on real changes are in store for us as the grid mutates increasingly unconventionally and, like evolution, starts doing things no one ever imagined. This is when it gets interesting, and more than a little awe-inspiring.

There is only one chance in life to exist "before something." Before printing, before books, before TV, before medicine. We, all of us living right now, are the last to be alive "before the grid." We must bear witness to this time, so those coming after can hear of lives lived in disconnected oneness, with the hardships and the freedoms that implies. They may not comprehend our past but we should not fail to report it, regardless of how little it resembles their present.

That’s how massive a change I foresee.

- js.

[sepia]

It already amazes me, the brain grid.
As a child, computers were unheard of, they were special things in secret places like NASA.
When I was in my teens, I asked my parent if I could take a programming course--1978. I was told this whole home pc thing was never going to happen and that computer programming was not a sensible goal, or even a class to be bothered with. My step-father felt it was something men in the gov't need worry about (I guess he failed to recall that one of the main computer languages was developed by a female military officer.)
Now, I go to sites, talk with people, talk with my kids friends, and they are all well versed in all things related to the computer--the only thing that seems to frighten them (yet the only thing I love) is working with binary numbers. Children in elementary school can work their way around computers easily. I think this is fantastic. It would seem that despite gov't reports, to me, our children are increasing their learning potentials and are smarter--at least many of them.
The whole Matrix/Logan's Run thing is a frightening thought to me. But, reality is an odd thing, not always what we think it is.

Your write up is excellent TankGirl. Thank you, gives me something to ponder and reread again after a bit more pondering.