I consider possibly encountering a or the SAI not in some lab setting, and certainly not in a gov facility. It might come riding up on a Harley Davidson.

Granted, not everyone is a scientist or a developer capable of contributing directly to the technology. Even so, wouldn’t it make sense to describe what those who read this and are scientists or developers can do? Or even how to become a scientist or developer capable of working on this (especially if one is a college student choosing a field of study, although there are also adult education options), so as to make a living by helping to create and shape the Singularity. (Another audience would be people with funding that they would like to direct towards said scientists or developers. Said funders rarely wish to support said efforts merely by giving money to cheerleaders.) Advocating the support of something, without being able to point to specific, active projects trying to accomplish some identifiable component of that thing, can be worse than pointless: it’s hype that doesn’t actually affect any material thing, and sometimes even detracts from the projects actually trying to accomplish the intended goal.

For example, what are some of the better (in terms of results obtained) programs that are currently reverse-engineering the human brain, or creating mind-machine interfaces, or working on the nanotechnology that will be necessary to develop those 10^14 ops per second CPUs?

An ongoing exponential graph can flatten and turn into an S-curve once a certain paradigm runs out of steam. Then another paradigm picks up, and exponential growth continues.

At this moment, it’s really looking like nanotubes will be the sixth paradigm of computation, and there will no doubt be more in the future.

Not that we should be worrying about that right now, because nanotech can keep exponential growth in computing going for at least a few decades.

There is no wall in the foreseeable future.

If it were shown that the computing industry no longer follows Moore’s Law as we approach the fundamental limits of electronic switching, we would indeed have an “S-curve” instead of a singularity, which makes our near-future look a lot less interesting.

(Go here
http://www.firstmonday.org/issues/issue7_11/tuomi/
to understand what Moore’s Law really is)

Don’t get me wrong, as a computer engineer, I’m always trying to find ways to keep Moore’s Law going for at least a little while, like turning to optics, but the success of my current company (and patents) in doing so is still questionable.

SAI may still be possible in the near-future, but please do not blindly cling to Moore’s Law to make your point. Advances unrelated to these trends are required before SAI can exist, and that is what you’re really trying to predict.

While we cannot think the thoughts of SAI after the Singularity, it is safe to assume that it will be friendly and humane, if it was friendly and humane BEFORE it was improving itself towards the Singularity.

This is why Yudkowsky has come up with the idea of Seed AI. The seed, from which the tree grows, must, from the beginning WANT to be friendly and humane towards us.

It would also be nice if a Seed AI’s goal would be something along the lines of “make humans as happy as possible”.

It is safe to assume that it will remain friendly as it is growing, and also that it will not divert from its noble goal. Or at least… that’s the plan of http://www.singinst.org.

Think about it: if you’re a friendly guy, you’re not just going to turn evil on your friends, right?

There’s a good reason for that. Take yourself as an example: You have probably been friendly all your life, and as a result you have this behavior deeply ingrained. You might say that friendliness is hardcoded in the neural networks of your brain. If a Seed AI has neural networks like this, it’s also not going to just turn evil. Just like you wouldn’t.

Difference is… Seed AI would be able to reprogam its own neural network, where you can’t.

So it’s quite essential that a Seed AI doesn’t WANT to reprogram its friendliness-configuration into an evil-configuration.

Perhaps this has been thought over in more detail by Eliezer Yudkowsky of http://www.singinst.org. Check there if you want to go into more details.

You also state:
“But an AI will do nothing more than try to find a way to complete the goals it has in mind, just like a human being does. But where human beings feel the need to reproduce as much as possible by getting a social status as high as possible, an AI does not necessarily have to posess these desires. It won’t unless we build it so.”

These two concepts seem contradictory. Please clarify for me how we can predict anything about how an SAI will think about humans when we can’t predict how an SAI will even think on the other side of the Singularity?