OpenAI: Meet ‘Operator’, a web-enabled AI agent that performs tasks for You

OpenAI on Thursday introduced Operator, its first artificial intelligence (AI) agent, which can “go to the web to perform tasks for you”. It marks the latest entry into the agents segment by a major player, following the likes of Google and Salesforce. ET explains what Operator can do, how it works and who can access it.What can Operator do?

Users can ask Operator to carry out a range of repetitive browser tasks such as filling out forms, ordering groceries and even creating memes, OpenAI said in a blog post.Some who have access shared on social media that they tried using the agent to order dinner ingredients based on pictures and recipes, schedule a barber appointment by checking Google calendar availability, plan a trip by parsing recommendations on Reddit that would be within budget, among other tasks.

OpenAI is collaborating with firms including food delivery app DoorDash, ecommerce site eBay, grocery delivery platform Instacart, taxi aggregator Uber, sports and entertainment ticket booking app StubHub to ensure conformity with their terms of service agreements.

“It (Operator) has limitations and will evolve based on user feedback,” OpenAI said.

It added, however, that the agent has produced state-of-the-art results, setting new benchmarks when evaluated for full computer use tasks (38% success rate on the OSWorld benchmark) and web-based tasks (58% and 87% success rates on WebArena and WebVoyager benchmarks, respectively).

How does it work?

Operator processes raw pixel data to understand what’s happening on the screen and uses a virtual mouse and keyboard to complete actions. It can recognise buttons, menus and text fields people see on a screen.

It does not need to use back-end application programming interfaces (APIs) to interact with platforms.

The agent is powered by a new model called Computer-Using Agent. This combines the vision capabilities of its most advanced generative AI model GPT-4o with advanced reasoning through reinforcement learning.

The ability to use the same interfaces and tools that humans interact with on a daily basis broadens the utility of AI, helping people save time on everyday tasks while opening up new engagement opportunities for businesses, the company said.

OpenAI CEO Sam Altman said during the launch livestream that AI agents are “going to be a big trend in AI and really impact the work people can do, how productive they can be, how creative they can be, what they can accomplish”.

Who is able to access it?

Operator is currently a research preview, available to Pro users in the United States.

The company plans to expand access to Plus, Team and Enterprise users and integrate Operator’s capabilities into ChatGPT in the future.

It will also be available in other countries “soon”, Altman said during the livestream. “Europe will, unfortunately, take a while,” he added.