Operator relies a “Computer-Using Agent” model that combine GPT-4o’s vision capabilities with “advanced reasoning through reinforcement learning” to be able to interact with GUI’s, OpenAI says. “Operator can ‘see’ (through screenshots) and ‘interact’ (using all the actions a mouse and keyboard allow) with a browser, enabling it to take action on the web without requiring custom API integrations,” according to OpenAI.
Operator can use reasoning to “self-correct,” and if it gets stuck, it will give the user control. It will also ask the user to take over when a website asks for sensitive information like login credentials and “should” ask for a user to approve actions like sending an email. OpenAI also says that the Operator has ben designed to “refuse harmful requests and block disallowed content.”
OpenAI posted:Today, we’re releasing Operator, an agent that can go to the web to preform tasks for you. Using it’s own browser, it can look at a webpage and interact with it by typing, clicking and scrolling. It is currently a research preview, meaning it has limitations and will evolve based on user feedback. Operator is one of our first agents, which hare AIs capable of doing the work for you independently — you give it a task and it will execute it.
Operator can be asked handle a wide variety of repetitive browser tasks such a filling out forms, ordering groceries, and even creating memes. The ability to use the same interfaces and tools that humans interact with on a daily basis broadens the utility of AI, helping people save time on everyday tasks while opening up new engagement opportunities for business.
To ensure a safe and iterative rollout, we are starting small. Starting today, Operator is available to Pro users in the U.S. at operator.chatgpt.com. This research preview allows users to learn from our users and the broader ecosystem, refining and improving as we go. Our plan is to expand to Plus, Team, and Enterprise users and integrate these capabilities into ChatGPT in the future.
CNBC reported: OpenAI is taking its ChatGPT chatbot to the next level, adding a feature to automate tasks such as planning family vacations, filling out forms, making restaurant reservations and ordering groceries.
The tool, announced on Thursday, is called Operator. OpenAI describes it as “an agent that can go to the web to perform tasks for you,” and added that it is trained to interact with “the buttons, menus, and text fields that people use daily” on the web.
It can also ask follow-up questions to further personalize the tasks it completes, such as login information for other websites. Users can take control of the screen at any time.
In my opinion, it sounds like OpenAI has the potential to help those who have difficulty with computers, and to make it easier for people to find what they want online.