The most talked-about feature debuted during Google’s I/O developer conference this week is “Duplex,” an AI system that can mimic actual human speech with eerie, accurate results. The point of the system, at least in Google’s onstage demos, was to use Duplex as a concierge service, calling phone numbers and booking appointments for you by literally speaking with the human on the other end. It’s like having a secretary, only it’s a computer that sounds like a person.
Just tell Google what you want and it makes the call in the background (hence the name Duplex), and then notifies you of the outcome. Theoretically, this system will allow businesses who don’t have online infrastructure to access a generation of customers that, conditioned by services like Seamless, dreads making an actual call and dealing with another person. The hypothetical economic benefits of such a system are obvious and substantial. It’s also, in the classic Google mode, a huge boon to daily convenience. And boy, is it impressive technology — and worrying.
To some extent, these worries are more cultural than technical. The Duplex demos showed the system completing a task, like booking an appointment, and the dialogues stuck to discussing the program objective. How does Duplex work when the conversation goes off on a tangent, or if there’s an emotional component (let’s say your preferred stylist is out because of a family emergency)? Duplex inverts what we usually want out of our software. It is a complex system for performing dead-simple tasks. Google told the Verge that “it can only converse in ‘closed domains’ — exchanges that are functional, with strict limits on what is going to be said.” As of now, Duplex can only book hair appointments, make restaurant reservations, and ask a store’s holiday hours. At that point, you’re taking what should be a relatively simple, personable action and making it artificial and complicated. Not to be a grouch, but it’s not hard to pick up the phone and call if that’s really the only option left.
At its heart, the system that Duplex proposes is imbalanced. The system of making reservations or appointments over the phone isn’t a power struggle, but for it to work well everyone has to be coming from the same place, with the same restrictions on time and effort. The introduction of automation upsets this balance. Now it’s one person tapping a button and the other performing conversation. It’s not clear what sort of safeguards are in place to ensure that, for instance, the human answering the phone is not dealing with an overload of robocalls. After all, these places don’t have online reservations systems; how can Google know if they’re all booked up? Google told Wired that it is limiting the number of background calls users can place per day, and putting in safeguards to make sure a single user can’t spam a single number (it’s got a bit of experience identifying the habits of spammers).
But beyond the cultural anxiety Duplex sets off, there’s an ethical problem. Even with safeguards in place, there is a blatant deception at the heart of Duplex. At no point in its demos did Google identify itself as an automated call. That nondisclosure is bad enough on its face, but the robot voice also displayed verbal tics in its conversations, saying things like “uh” and “mm-hmm.” This is not a program that is designed to make it easy for a person to interact with a computer, like voice-powered assistants are. This is a program designed to mask the fact that someone is interacting with a computer at all. It obliterates the idea of informed consent, where one party doesn’t understand that their voice is being piped to a server center for analysis (Google is a company headquartered in California, a state with two-party wiretap laws). Compounding the privacy issues, the Verge reports that a human will take over if the automated voice can no longer handle the call.
I guess the good news is that, for now, Google is one of just a handful of companies capable of developing a system like Duplex, which is reliant on incomprehensible amounts of data for AI training (thanks, centralization). A spammer sitting at their computer can’t just run the program on their own. That also means that — again, only for now — we have to trust Google to be a responsible steward of these systems. In an era when tech companies can act without consequence (or at least, without a hit to their bottom line) that’s a worthwhile concern to probe. We’ve placed a lot of trust in developers at our biggest tech companies to transform the world in responsible ways, and so far that trust has not quite been rewarded in full. All of this might be Luddite hand-wringing, but if I come across as overly skeptical, it’s because neither Google nor any of the developers in attendance at the demo seemed skeptical in the slightest.
