jump to navigation

Is The Singularity No Threat? January 22, 2011

Posted by Metabiological in Synthetic Intelligence, Transhumanism.
Tags: , ,
comments closed

Over at IEET Kyle Munkittrick has just explained why we have nothing to fear from the rise of machine intelligence.  His post is a response to a short post by Michael Anissimov in which he reiterates a position that he’s held for awhile: that the Singularity is the greatest threat humanity is likely to face in the coming century.

Now I’m not entirely sure where I stand on the issue of the Singularity’s threat.  I certainly recognize that the development of new technologies always brings with it risks (see coal power) and there is little doubt that the emergence of greater than human intelligence will constitute a major risk.  That being said I have a hard time not laughing at some of the more apocalyptic visions I’ve seen and feel are strong urge to punch anyone who brings up Skynet as anything other than a joke.  But Kyle’s naivite, I can think of no other word for it, on the potential threat is nothing short of jaw-dropping.

The major problem with his argument unfortunately happens to be its central thesis; that even if a synthetic intelligence were to arise it would be unable to interact with the physical world and therefore poses no threat.  Even assuming that this scenario is likely, personally I think otherwise, his suggestion that an intelligence confined to a computer wouldn’t be able to affect is downright ludicrous.  He even mentions as an example an SI causing havoc on our communication networks and then brushes it off as if it were nothing.  One would have hoped that a person who writes blogs on the internet for a living would have a little more respect for the way that communications technology has become the bedrock of our society and economy.

In fact, let’s do a little though experiment.  Let’s take an industry, agriculture for example, that is essential to the continued prosperity of humanity and heavily reliant on computer technology.  Nowadays most food is produced far away from its point of consumption.  Whether this is good or bad is a subject for another time because the fact is most of us do not subsist on food grown in our local region.  To maintain the elaborate system that ensures your food gets from its farm thousands of miles away, across continents and oceans, takes a large and powerful infrastructure that today is heavily reliant on telecommunications technology.  Now imagine that something, say an SI, were able to disrupt that system?  How long to you think it would take for cities to turn into battlegrounds?  It wouldn’t even need to be a large disruption.  Most supermarkets don’t plan to carry stocks for large periods of time and an event like what I’m describing could send people into a buyer’s panic.

Want a more relevant example of a computer wreaking havoc in the real world? How about the flash-crash in the stock market last year? Wall Street algorithms caused a seven hundred point drop in the Dow Jones in a matter of minutes. They weren’t malicious (just doing what they were programmed to do) and they sure as hell couldn’t interact with the physical world yet they still managed to send people into a panic if only for a few moments. As we give more and more control to machines who can really predict the next crash won’t be worse.

Now I realize that what I’m saying sounds somewhat apocalyptic and I don’t think that such scenarios are necessarily likely. As I said I’m not sure where exactly I stand on the issue of the Singularity’s threat but one thing I do is acknowledge that there is a threat.  To brush off the danger as Kyle and others are doing is akin to a person walking backwards towards a cliff and saying “Everything looks good from here.”


The Machines Run Wall Street January 13, 2011

Posted by Metabiological in Synthetic Intelligence.
Tags: , ,
comments closed

There’s an excellent interview up on NPR concerning the use of high speed computers on Wall Street.  You can listen to it right here.

A few things I take away:

Firstly, despite what the commentator alludes this isn’t SI really.  These algorithms don’t display sentience and certainly don’t mimic human thought process.  But then they’re not being asked to think like us, they’re being asked to do things we can’t.  Their advantage over a human trader is in being able to sift large amounts of data and data sets and determine what variables exert the greatest affect on the market.  This isn’t an example of synthetic intelligence but it is a further example of the effects that automation will have on our society and economy.

Many people have been under the false pretense (or should I say illusion) that automation would only effect the industrial and manufacturing industries by eliminating blue-collar jobs and anything that requires manual labor.  This is flat out wrong.  While I could point out how we cling to our collective delusion that there is something special about human intelligence that can never be replicated that’s not really what’s going on here.  These algorithms aren’t simulating human thought because human thought isn’t required, only the ability to sift data and recognize patterns.  Now stop for a moment and think of all the jobs that really can be boiled down to those activities.  Quite a few right?  This is the next frontier of automation and it’s going to take us for a ride (for a really good read on the subject, look up The Lights in the Tunnel.)

Secondly, I’m amazed that so much of our prosperity rest on a system that even insiders claim to not really understand.  I can sympathize with those working the market to a degree since I fully understand the difficulty involved in predicting the actions of complex systems (I am an ecologist after all) but it scares me to think of how little we know about something that exerts such a great influence on all of us.  The commentator mentioned one company in particular, Volion, that doesn’t even know what their trading on.  Putting aside the fact of how that’s even possible I wonder… no screw that, HOW IS THAT EVEN POSSIBLE!?  How have we built the driver of our economic prosperity on quicksand?

Finally, the major problem I see here is the major problem that confronts any use of computers to make our decisions for us: how to we know they’ll make the right ones.  This isn’t a question of computers going rogue and taking over the world a la Skynet.  This is a simple question of priorities.  How do we know that computers will have the same priorities as we do?

Imagine for a moment that we build a computer, the most powerful computer ever, blessed with freedom and ability to solve any problem we put to it.  We ask this computer to solve a seemingly impossible math problem.  The computer, following our instructions, proceeds to convert all matter on earth, including us, into hardware to increase it’s computational ability and solve the problem.  Sound crazy?  It isn’t.  These algorithms do not think like we do and cannot be predicted to behave as we do.

As we hand more and more power over to machines we can’t predict working within a system we don’t understand, what do you think the odds are something will go wrong?