So, how do you get motivated? You need a goal function, i.e. something that gives you a general direction. I agree that this is pre-defined in humans, and it is based on physical sensations. There are some sensations you would rather have and some you would rather avoid. On a slightly higher level you’ve got emotions, which some people argue are translated internally into the same signals (which is why you often get strong physical sensations when you’re having strong emotions of some sort).

On a neurophysiological level, you’ve got a huge network of nerve cells in which you want some parts to be active and others to be inactive. This is a fairly vague goal and I think that it is difficult to model using typical artificial neural networks as used by everyone. After all, these are simply functions in which it is clearly defined which neurons are inputs and which are outputs, and what you do is “teach” them what kind of output should be generated for a given input. Quite different, isn’t it?

So in my opinion, artificial neural networks are not suited for developing AGI. There are other things the brain does that artificial neural networks currently cannot, such as re-wiring and development of new cells, possibly according to some sort of strategy. These things are not well understood at all. Additionally, artificial neurons are dumbed-down copies of neurons. Real neurons translate inputs into outputs according to an arbitrary function, whereas people tend to use the inputs of an artificial neural network as a weighted sum that is passed through a linear or sigmoid activation function to decide whether the neuron fires.

I think we need a different model of the brain before we can ever hope to succeed in the quest of AGI. Artificial neural networks do have competition, mainly from the area of statistics. I believe that statistical approaches are great tools for weak AI but will not manage to achieve the degree of adaptability that the brain possesses. Another thing that seems to be the latest hype is the so-called Hierarchical Temporal Memory, but that again seems to be tailored to simple pattern recognition (and I haven’t heard anything about it so far, except for buzzwordy marketing).

So one hurdle that I see is finding the right kind of approach to the problem in the first place. This includes finding a model that can grow itself in a meaningful way (even if we do not understand the meaning). The next would be to find a suitable goal function, possibly based on pre-defined pain/pleasure signals. Another would be what you said: supplying a world that is complex enough to generate enough useful knowledge for the machines to process (and that in itself should be a huge project).

To me it seems like we cannot “invent” AGI unless we copy ourselves (or find some other way to achieve AGI by accident). If we actually manage to do so, I imagine that this AGI will suffer from the same insufficiencies as humans do (as stated at the end of the posting, I will elaborate on that in a follow-up), and I can’t help but wonder if the quest of copying ourselves is worth it in the end.

Also, if you compare the machine to the brain, please compare it to an infant’s brain. An adult brain has had the advantage of many years of access to all sorts of “training data” and, also, got feedback from other humans. That’s a typical part of learning. While the human brain is preprogrammed to some extent (also, learning probably begins long before birth), many of the things we are capable of habe been taught. I don’t see why we need to make it harder for computers by not teaching them as well.

The ultimate goal is probably some software brain, that, running on a computer inside a robot with similar physics and sensors to those of human beings, is given the freedom and the time to “grow up”, just like children do, with the result of it learning many of the things that are completely normal for human beings. Still, we would have to allow for some differences concerning world view. Just like people with disabilities recognize the world differently, a robot will. If we just could replace a brain with a computer by making it run on whatever the body provides and connect it to the complete set of sensory input, that is, the nerves, things might look a bit different.

In the end, I think, AGI will fail mainly because it doesn’t have nearly as much input and also not as great a variety of input as humans do.