To test the Muse personal AI agent, which is a feature of Meta, the company is introducing a new human concierge system where human workers answer some of the calls Meta’s AI system makes on behalf of the user. The experiment also illustrates the increasing complexity of AI assistants that are created to answer questions and independently perform daily tasks. Meanwhile, the test has sparked a few privacy concerns among Meta staff members and the possibility of processing sensitive information when users enter into interactions that they’d expect to be entirely automated.
Shortly after announcing phone calling capabilities for Muse publicly last week, the Facebook and Instagram parent company has notified staff of the experiment. Some of the calls that the AI assistant can trigger may also have to go to human contractors, who serve as a back-office concierge. It is currently being tested internally and if/when it is made available, it will be rolled out gradually.
The concept is part of a wider trend in how tech firms are designing their personal AI bots. Most chatbots that exist are text-output generators or answerers, whereas newer AI agents are being engineered to enact real-world activities. Muse is already able to send emails, buy tickets, search for travel and more. That’s expanded with its calling feature, letting the assistant talk to businesses directly on the phone.
The feature is supposed to streamline everyday tasks for users. A user might ask Muse to dial a business to find out if a specific product is in stock, ask a contractor for an estimate or make an appointment. The AI is meant to take over the conversation, rather than have the user sit on hold or explain their request to the voiceless person.
In the human concierge experiment, there’s another layer in that process. While the Muse can act as an AI assistant for the user, some of these calls may require the participation of human workers to handle the conversation. Such an approach could help Meta cover cases where the AI fails to meet a complex request, it gets a strange answer or where a conversation “needs judgment” that is not currently supported by the AI.
But there is also a privacy concern with people being involved. During discussions internally, regarding the testing, the possibility of potentially exposing sensitive information to third-party personnel in contact centers was discussed. Personal information, account information, appointment information, or any other information that is present in a phone conversation may be provided to an AI agent when users give instructions for a call, but may not be intended to be heard by a human being.
The experiment will help the company to gain a better understanding of how the technology performs in real-world scenarios and to develop protections before releasing it more widely,” Meta said. Employees’ response to the test has been “overwhelmingly positive,” the company said in a statement via Meta spokesperson Daniel Roberts, adding it aims to use the test to “get feedback so we can implement safety and privacy protections and improve features before releasing them publicly.
Roberts added: “We’re continuing to work with merchants to make this potential calling feature better and better, and will only roll it out when it’s ready, with the proper disclosures. The comments suggest that Meta is still working out how the feature will work and what information users would get if they were to be used.
The experiment follows the attention of Muse amongst consumer. According to market intelligence firm Sensor Tower, the AI assistant is reportedly making it past the 2.5-million download mark in the U.S. since its release. It’s popular in its early days, so it’s giving Meta a big user base to experiment with new features, and it also makes privacy and security measures more crucial as the service grows in size.
Muse is part of Mark Zuckerberg’s Meta’s project of creating “personal superintelligence. The plan is to develop AI models that will be able to accomplish more complex tasks for the billions of users who interact with Meta’s platforms and services.
The firm has made great strides in securing these abilities. At the time of Muse’s launch, Meta stated that each AI agent will be deployed via its own secure virtual machine. This is a cloud-based computing space that is designed to keep one user’s AI session separate from others. The company also claims to have password sensitive data in separate secure storage.
These safeguards are especially critical when an AI agent starts to perform actions on behalf of a user. An assistant who writes an e-mail message carries one kind of risk, whereas an agent who sends the e-mail, makes a purchase, books a meeting or calls a business may impact the user’s finances, privacy, and personal relationships.
The phone calling feature is thus an important test of how autonomous AI systems can interface with the offline environment. While conversations with humans can be more unpredictable, with people asking follow-up questions, asking for clarification or even reacting in ways that an automated system may not expect. In some of these scenarios, an AI agent might need a human backup, but it also requires transparency and protection in regards to the information shared in the call.
