New Scientist has an article about a software system called ChatNannies that purports to engage pedophiles in chat room conversations in an effort to catch them "grooming" children for real-life meetings. There's a transcript of one such conversation, and the creator claims the program is so effective that no one has caught on yet.
As an expert in artificial intelligence I'm extremely skeptical, and I'll explain why.
First, there are details in the article that just don't make sense.
The nanniebots do such a good job of passing themselves off as young people that they have proved indistinguishable from them. In conversations with 2000 chatroom users no one has rumbled the bots, [Jim] Wightman [the author] says. ...Wightman currently has 100,000 bots chatting away undetected in chatrooms - the most he can generate on the four internet servers at his IT practice. He would like to build more but funding is the sticking point, as he does not want anyone to profit financially from his technology.
He's got 100,000 bots running, but only 2000 conversations in which the bot has gone undetected. That's a miserably low success rate, and actually quite believable. I suspect these numbers were intended to mean something else, but what?
Then there's Wightman's reluctance to reveal details of the system to anyone.
One of its tricks is to use the internet itself as a resource for its information on pop culture. Wightman will not reveal how it judges what is reliable information and what not. He does say, however, that each bot has dozens of parameters that are assigned at random, to give each one a different "personality". ..."Some companies have offered fantastic sums of money, but all want technology ownership. And that's something that isn't going to happen," he says. Instead, he hopes eventually to get financial support from government-run organisations that focus on child protection.
If this is a fraud, it would be a lot easier and safer to profit from government hand-outs than to actually risk revealing the "system" to technically-savvy investors. This is why reputable scientists publish the details of their research.
Interestingly, the transcript does include a few hints that it's likely machine-generated. In this script, "B" is the purported machine and "A" is the human.
B - pancake day! i love pancakes...mmmm so tasty A - yeah me too, but i forget every damn year B - did you forget this year?
The response by B is very script-ish. Notice also that B's responses are longer on average than A's. That's a sign of a poor (i.e., standard) conversation routine. It's very hard to generate complex sentence structures that sound natural.
Here's another interaction, with my comments.
B - oh cool. did you watch robocop 2 last night? A - what side was it on? B - sky one A - we haven't got sky A - but i've seen it before A - it wasn't as good as robocop B - i agree, though it was cool in places.
Canned response, ok.
A - did you watch robocop last night B - yes, i just said i did! A - no you said you watched robocop 2 not robocop - so which one was it? B - robocop 2 - pedant!
Interesting confusion of tokens. The system splits the "robocop 2" token on its own, or assumes that A is using shorthand for the same token. This leads to confusion for the robot, which is fine, but there's no way it could be smart enough to untangle the subsequent miscomprehension. The usage of "pedant" after a dash as an exclamation feels made up. It's not a very natural chat construction, particularly for a child, and I can't imagine a robot could so easily identify the source of confusion and label it so appropriately.
A - not robocop or robocop 3 or robocop the series B - it was definitely robocop 2, the one with kain the second robocop in it. i haven't seen robocop 3 or the series.
If the system is genuine, this is a remarkable feat of comprehension. Most humans would be confused by this point.
Anyway, this conversation could be machine generated, but I suspect it's not representative of how any real system interacts with humans on a consistent basis.
Beyond all this, the creator claims the software can reliably detect pedophiles based on non-sexual conversations? No way. Human children and parents can't even do that face-to-face, and we're finely tuned to pick up on vocal, physical, and conversational cues that aren't present in text chats.
Furthermore, he promotes his "utterly free service" on message boards. He makes incredible claims on his homepage:
The most technologically advanced AI construct ever conceived and built. The NannieBot spawns and controls a large number of virtual internet users, whose behaviour is indistinguishable from humans interacting on the internet. The first AI construct to effortlessly pass the 'Turing Test', after more than 13 hours of conversation the AI was still undiscovered!
The only thing is, I don't see the catch. He asks for sponsors and donations, but he doesn't directly charge money for the software. Of course, the software isn't released yet, and they're auctioning off the first public chat with their robot on eBay. Maybe he really is hoping some government will fund his project?
Any system can occasionally hit a home run, but the claims in this article are not credible, in my opinion. Go here to chat with some of the best existing real chatbots; none of them are anywhere near the capabilities claimed by Wightman.
(HT: GeekPress.)
Update:
Via Apothecary's Drawer and Waxy.org I see that a grad student at MIT named Cameron Marlow managed to secure an exclusive interview with one of the NannieBots. From the transcipt he's posted it's virtually certain he was talking to a human posing as a robot.
The secret to a good scam is knowing how far you can go before you cross the line into absurdity. Jim Wightman doesn't have a clue.
See, this is why I don't post a lot about AI. No one cares! But everyone wants to read about dumb lawyer stuff....
Hey, I liked it!!
As a real life AI expert, do you have any recommendations on fictional books/movies that do get the science right? I read Richard Powers Galatea 2.2 and thought it very interesting.
JP: Hmmm, good question. The problem is that right now the science isn't very exciting from a non-technical perspective; the state of the art isn't substantial enough to support a work of fiction. Any depiction of artificial intelligence that's interesting enough to base a book around will be science-fiction/fantasy, and could only speculate on the future of the field.
That said, all of Asimov's robot books are excellent, as is Heinlein's "The Moon Is a Harsh Mistress" -- one of my favorites. I'm named after the AI computer, so you can't beat that.
Good call dude!
The guy is using the written media's eaggerness to report "breaking news" and the fact that in the AI field is easy to fool people since there is not much knolwedge about AI out there aside the movies. I read this article and was intrigued to read that one "consultant" wrote alone some software that is able to do what he claims, knowing that only to write the conversation engine, it would take a team of researchers at MIT that don't do anything else at work and none could predict the time it would take them to deliver. I agree that there are geniuses out there, the odds that one able to do such work alone is not w/ MIT or ISS or so, is extremly low. As long as he does not let experts perform accurate tests on his tech and publish the results, he's a "false prophet". I recall a guy that photographed some designer PC cases (in magenta) some time ago and claimed that he developed a new chip technology that lets chips work like a brain and autoconfigure themselves. I agree that this would be something in the right direction, however i haven't heared from that guy again.
I have to admit that Whightman is very smart when he is chosing:
- a field that not many are savy about;
- an approach that looks very cool (bots, chatrooms, etc.);
- a field that is as sensitive as the pedophiles in chatrooms, which is everyone's concern and increasing one's readiness to salute any little piece of work that would help protect the children;
- the small garage inventor image, that in deed has a well motivated reputation (HP, Apple, etc);
- to make it sound like he has Mother Theresa inside that tells him to not let the big bad corporations use this for profits - i'm sure that corps would be more than happy to get a license to use the software without tech owenership;
I'm sad to see that a serious publication bites into this the way the New Scientist did.
I'm glad your column reached such a rank in the Google search.
Well done. I too was *very* sceptical when I read it in my (paper) copy of New Scientist at the weekend. That's why I googled "Chatnannies" and found your page thinking I wasn't the only sceptic. I couldn't tell the bot in the sample conversation because of the "Pedant" and the Robocop 2, 3 parts of the conversation. Very very suspicious...
Hey thanks guys, it's nice that people have found this and read it. I emailed all the various departments at New Scientist but haven't heard back from them yet.
JimW, the ChatNannies creator, has added an apology to the site, in which he explains that there are 3 concepts going on here:
1. ChatNannies - the web site, intended to be a focus for human activity regarding the monitoring and reporting of chatrooms.
2. LiveNannie! - a piece of software that controls access to chatrooms (based on information gathered by the ChatNannies site).
3. NannieBots - the bots themselves. Unfortunately, "Unless you have Terabytes of storage space, ultra quick Dell PowerEdge servers and a big fat internet connection (and a bulging wallet), you cannot 'use' a Nanniebot!"
There are some things about this enterprise that simply don't add up. Sorry to sound like an old cynic but I predict tears before bedtime...
Wightman's March 24, 10:32 am, posting to the Waxy.org discussion is priceless.
Speaking of Richard Powers and Galatea 2.2, I’d like to step back from this discussion for an instant (and I apologise if this is off-topic for this post). I agree, the bot described in the New Scientist article is a hoax, but when this blows over, what does this event suggest?
I’ve written a modest bot who identifies himself as a computer program and challenges visitors to prove to him that they are human. Despite his clumsy responses, his inability to learn, his no or low AI status, his insistence that he is a machine (he says, “Do you think machines like imitating humans? No Loebner Prize for me!”) his transcripts are full of accusations that he is, indeed, human, a hoax!
Some visitors prefer to imagine that they are engaging with a human, even an implausible inhuman human typing behind the curtain 24/7, than to come to terms with the experience of engaging with a machine in conversation.
Except as an incentive to build better bots, the Turing Test is beside the point; other people talking to bots aren’t even paying attention to the “Is it human? Is it fooling me?” question. Who cares? We’ve adjusted to dealing with not-real time, not-real space, and now we’re fumbling with not-real humans.
Richard Powers (cited in a post above) in a discussion with Bruno Latour (published in Common Knowledge), pointed out that bots and the Turing Test churn up “…the mind’s anxiety about the authenticity of its own mental map.” Powers cuts right to the core of what upsets us about the Nanniebots; he asks,
“Am I, myself, capable of passing the Turing test honestly, or do I do so only by cheating?”
Funny, this came out the same week as GrokItBot.
I thought I was being clever adding a Bayesian guesser that screened stuff before passing it off to what is effectively an ALICE bot. It works well and might constitue a small step towards improving AI bots in general.
http://www.suttree.com/code/GrokItBot/
Then nanniebot pops up and it's hard not to feel cynical. All those brains writing AIML sets vs. nanniebot...
Superb article, it's nice to know that there are people out there with the knowledge and intelligence to see through this kind of thing.
I know next to nothing about AI but the sentence constructs I've seen from "nanniebot" just look utterly beyond the reach of current tech... Great point made about even humans not being able to detect a paedophile face to face!
David Vannen (I make music, I'm not a scientist!)
PS - Very thought-stirring stuff about our coming to terms with "AI" by pweil... I've got to say that nothing annoys me more than a machine that tries to interact with me like a human! I like to think I can do machine better than a machine can do human, for the near future anyway... ;-)
Hi, there,
I thought it best to let the dust settle a little before I responded to your queries regarding our AI technology.
I'm sure you'd agree that the evils that all our children potentially face on the internet need all the ideas and work we can get. We don't pretend to have all the answers - we're only at the beginning - but if our work stimulates others to do even better then we shall feel that we've done our bit. And if your ideas help, then many thanks. So, to your points
"He's got 100,000 bots running, but only 2000 conversations in which the bot has gone undetected. That's a miserably low success rate, and actually quite believable. I suspect these numbers were intended to mean something else, but what?"
Actually the reporting on this is slightly inaccurate - and because of this the analysis of it is too. The figure of 2000 is a low estimate figure of how many volunteer users we had concurrently testing the bots in one chatroom. This was way before any media interest, back in the really early testing phases. In total we clocked well over 150,000 hours of testing between us in various different configurations using different strategies. This was a 'try it and see' approach to forging the algorithms that we use to spot paedophiles using only their use of words, sentence structure, and other 'written word' linguistic qualities.
The other inaccuracy in the report was that while we have over 100,000 spawned instances of the bots running at any one time (at full capacity), in reality only about 1/3 of these survive to chat in chatrooms. I've noted elsewhere that we are at a really early stage of development still (15 years on) and we still get regular crashes for the most innocuous of reasons...so most bots either crash before they choose a chatroom to monitor, or crash by resolving the hostname incorrectly, or get locked by the connection, or drop out once they get to the room because of a variety of problems. Obviously we are working at this, and progress is slow (because I have a day job too) but we hope to get all this ironed out in time for demonstrations.
"If this is a fraud, it would be a lot easier and safer to profit from government hand-outs than to actually risk revealing the "system" to technically-savvy investors. This is why reputable scientists publish the details of their research."
Once again, I'm afraid we have some inaccurate reporting. I didn't tell the New Scientist at any point that we were only relying on government handouts - nor that that we cared particularly where sponsorship came from - only that we were looking for sponsorship and investment to accelerate the growth and development of both the ChatNannies site and the AI.
"If the system is genuine, this is a remarkable feat of comprehension."
Thank you, We've worked very hard to get this AI to where it is, working on it for 15 years or more.
"Beyond all this, the creator claims the software can reliably detect pedophiles based on non-sexual conversations? No way. Human children and parents can't even do that face-to-face, and we're finely tuned to pick up on vocal, physical, and conversational cues that aren't present in text chats."
Well as I mentioned earlier, we have done a hell of a lot of testing to back this claim up - its not something we would claim lightly. We aren't actually trying to detect paedophiles per se however...we are comparing the linguistic structure of an average 14 year old with that of any number of users currently in a chatroom. If we get values which indicate a significant rise above that average 14 year old level of complexity in linguistics, there are algorithms to spot this happening. The reality is that the 'scores' given to different age groups, and those things that indicate an extension of age beyond this score, were built by repetitive testing and chatting with human users, so it really is nothing exciting in terms of advancement. In fact I am shortly contributing to a paper here in the UK for someone at Edinburgh University about this very topic.
Since I too am quite sceptical I can fully understand your viewpoint. I'm sorry that we as yet have not provided the 'killer' evidence that we would love to provide - the truth is, as I brushed upon before, we simply were not ready for this level of media attention and so are not as far along with development as we would like to demonstrate to members of the public or press. It is however, as stated elsewhere, our intention to enter the Loebner prize to prove in an open yet tightly controlled forum that our work is genuine and corroborates the evidence we have presented thus far.
In the meantime, I am very happy to answer any queries you have about any parts of the AI in which you are particularly interested; please understand however that to answer a question such as 'how does it work' might take a little more than a few hours :-)
Best wishes
Jim
I've read all the comments on the ChatNannies concept, and frankly I think you're all missing the point.
It doesn't matter one bit if the replies at present seem scripted. Not one bit.
What does matter is that someone is out there trying to protect vulnerable teenagers. It's a damn good shot, in my opinion.
So let's hear some congratulations to Jim for his efforts, and let's see all you critics show him how it REALLY ought to be done - I'm sure he'd be only too delighted
I stand by my theory that there's no AI whatsoever, and that Jim was behind the keyboard the entire time for both transcripts. Jim: If you submit a bot that manages to make the final eight for the Loebner prize, I'll gladly retract everything I wrote.
JW: I've responded.
TM: You're obviously JW in disguise. I'm not an idiot; I know how to use tracert.
AB: I want source code. That's all, it's very simple.