Well, there's a heck of a lot artificial intelligence can't do yet, but in this post I'm going to give an example of a conversation that no computer program can have. There's no existing theory that explains how humans are capable of this either.
a: What is the capital of Spain?
b: Madrid.
a: Why did you say Madrid?
b: Because you asked what the capital of Spain is.
a: Why did you answer?
b: Because we're having a conversation.
a: How did you know the capital of Spain?
b: I looked it up on Google.
a: Why did you do that?
b: Because you wanted to know, and I didn't know already.
a: Why did you want to find the answer to my question?
b: Because I felt like it.
a: Why did you feel like it?
b: I like to be helpful.
a: Why do you like to be helpful?
b: It makes me feel good.
a: Why do you like to feel good?
And so forth. Humans can go on like this forever, explaining their feelings and notions in ever-increasing detail, as abstractly as necessary. We don't really know why we feel certain ways, but we can guess based on our history and experience and make ourselves understood to each other -- because our brains all work in the same general way. We may not be able to verbalize it all, but when someone says "you know what I mean", we do.
Computers don't. No computer program can explain its own workings to you. Why? Because you'd have to create code that did the explaining, and then you'd have to create code that explained that code, and so forth. The series of "why?" questions can go on uniquely forever. No one knows how we handle it, and no one knows how to make a computer do it.
but what if the next answer was
b:I don't know, it just does.
For AI to be useful (or humans for that matter), I don't think they need to be explain their actions ad-infinitum. Most humans can't, that's for sure. And the driving down in the example is probably quite good enough for most AI.
I disagree that the conservation given will never be able to held by an AI. An AI having motivations driven by emotions is not an unreasoneable request. We could make a basis of motivations, in this case capably answering questions for a human. As such the answer to the question "Why did you want to find the answer to my questions?" could have been answered. "Becuase I am motivated and rewarded by answering questions from humans."
If the next question is "Why are you motivated?" the answer could be "Because I am, in the same way a human is."
Once we get down to "I think, therefore I am", it's proably far enough for 99% of uses.(Apologies to philosophers for been reduced to 1%, but I am looking at the total picture, and there really isn't that many of you.)
It would appear that field of AI is getting rather stagnant these days. People are happy with silly robots and such trivia but missing the research on the big questions, such as why people do things, how emotions can break deadlocks in rational thought and providing AI with motivations and complex behaviours for doing things. Very little research seems to tackle the question of providing AI with reasons for existance such as self preservation and working towards a complex goal.
Lets face it, most humans can't answer the last question, "Why do like like to feel good?", why should an AI! Is such a reductionist approach really going to give the golden answer when it hasn't for human conciousness. Maybe it's time to hop off the reductionist approach and look at humans as complex systems. And AI too...
Nigel: You're mistaken in thinking the field is stagnant, there's progress, but it's tough going.
As for the conversation above, the key isn't in the answers, those are easy to generate. The tough part is generating the answers in the same way humans do. Even your example of "I don't know, I just do" is fraught with complexity.
Check out Hubert Dreyfus' essays and books for a counterpoint to the AI panaceas on the market.
I agree with you, Michael, that no AI will ever be able to hold conversation, and you give a very interesting argument for it (the reductio ad infinitum).
Nevertheless, I disagree that "there is no existing theory that explains how humans are capable of this either." As a matter of fact, there IS a theory - it's just one that everyone has forgotten about, because they are too wrapped up in their absurdist Cartesian and Kantian worlds to have noticed. Plato & Aristotle both had a theory of the soul that is, as yet, totally unparalleled in terms of its ability to explain human action - with reference to the present question, they maintain that all natural beings (men, animals, plants) have a certain act of being. Their nature (what it is to be that KIND of thing) determines that act of being, and determines it so that their acts work towards the perfection of their being. Men have, therefore, certain desires that incline them towards the perfection of their being, such as the desire for food, knowledge, sex, etc etc.
Now, since being is determined (that is, since it receives a determinate form, in virtue of which it is a "this thing" instead of a "that thing"), the being of a particular substance has a first principle of its existence that simply IS - it does not receive any natural account. Therefore, the answer to the question "why do men seek the good" is simply this: being is, and non-being is not. To be good is to be the fulfillment of being. Since being is contrary to non-being, it cannot direct itself towards non-being; rather, it must direct itself towards being. But being towards which another being is directed is called "the good."
If someone asks "why" to this final position, it can only be for one of two reasons: either they fail to see that this principle is a FIRST principle & therefore undoubtable, or they disagree that being and non-being are contrary - which is intellectually absurd.
Achilles: I definitely believe in spirits and/or souls, and I tentatively agree that they're necessary for biological intelligence. That doesn't mean that a computer with sufficient capability wouldn't "acquire" a soul in the same way a newly conceived human does, for instance. (I'm not saying I believe that, I'm just throwing it out there.)
I don't buy the argument that humans inherently seek what is good, unless you only take "good" to mean that which is immediately best for the individual human.