Operator: The AI Agent that Simulates Keyboard and Mouse Clicks
OpenAI is introducing a research preview called Operator, an AI agent that simulates keyboard and mouse clicks in a browser, reading the screen, and performing actions. This technology has the potential to revolutionize the way we interact with websites and automate repetitive tasks.
How Operator Works
Operator uses a model called CUA (Computing-Using Agent), which dictates how the AI talks to websites. Unlike other AI models, Operator doesn’t use APIs or extract text from the DOM. Instead, it views an actual web page in a live browser running in the cloud, reading the context directly off the screen.
Partnerships and Limitations
Operator has partnered with several websites, including Instacart, DoorDash, Etsy, OpenTable, Tripadvisor, AP, Priceline, StubHub, Thumbtack, Target, Uber, and more. These partnerships are unclear, but they may be affiliate deals, agreements to notify Operator of website changes, or additional modeling for those sites. Until we have a better understanding of these partnerships, we won’t know the scope of what Operator can do.
Guardrails and Privacy
OpenAI has given serious consideration to issues of privacy and guardrails. Operator knows when to pause and ask for human intervention, and it can also be controlled by the human user. Additionally, the human user can opt out of allowing website interactions to be used as training data for the AI.
Site-Specific Custom Instructions
Operator allows users to create site-specific custom instructions on a site-by-site basis. This can be useful for tasks that require specific settings or preferences, such as booking a hotel room with a free breakfast.
Baby Steps
Operator feels like baby steps at this time. For example, I’d love to tell an AI to go through my inbox and find all the press releases and assign them to one label. This is both a complex task and one that’s got quite a long runtime. As such, it’s way beyond the scope of what Operator can do.
Conclusion
Operator is an interesting technology that has the potential to automate repetitive tasks and make our lives easier. However, it’s still in its early stages, and there are limitations to what it can do. As the technology develops, we can expect to see more advanced capabilities and a wider range of applications.
FAQs
Q: What is Operator?
A: Operator is an AI agent that simulates keyboard and mouse clicks in a browser, reading the screen, and performing actions.
Q: How does Operator work?
A: Operator uses a model called CUA (Computing-Using Agent), which dictates how the AI talks to websites. It views an actual web page in a live browser running in the cloud, reading the context directly off the screen.
Q: What are the limitations of Operator?
A: Operator is still in its early stages, and there are limitations to what it can do. It may not be able to perform complex tasks or handle long-running tasks.
Q: Is Operator secure?
A: OpenAI has given serious consideration to issues of privacy and guardrails. Operator knows when to pause and ask for human intervention, and it can also be controlled by the human user. Additionally, the human user can opt out of allowing website interactions to be used as training data for the AI.
Q: Can I use Operator for free?
A: No, Operator requires a Pro account, which costs $200 per month. However, users of the $20-per-month Plus plan will eventually be able to use Operator.
