The best AI agents are terrible freelancers – for now

Comply with ZDNET: Add us as a preferred source on Google.

ZDNET's key takeaways

• High AI brokers fail at freelance work, in keeping with a brand new examine.
• The examine assessed Gemini 2.5 Professional, GPT-5, and different brokers.
• Near half of the US workforce did freelance work in 2025.

In the event you're a contract employee and you've got been pressured in regards to the prospect of shedding your job to AI, you'll be able to relaxation straightforward — at the very least in the intervening time.

In response to a brand new study performed by Scale AI and the Middle for AI Security, probably the most cutting-edge AI brokers are presently solely capable of automate lower than 3% of the duties required from the typical unbiased contractor, “failing to finish most initiatives at a stage that might be accepted as commissioned work in a practical freelancing surroundings,” the authors wrote.

Additionally: Want better ChatGPT responses? Try this surprising trick, researchers say

The Distant Labor Index

The examine, posted to the preprint server arXiv on Thursday and but to be peer-reviewed, establishes a testing benchmark for AI programs, which it calls the Distant Labor Index (RLI). 

The benchmark serves as a qualitative framework for measuring the flexibility of AI programs to carry out economically invaluable work at a time when some tech leaders have been making sweeping claims in regards to the disruptive influence AI can have on the labor market. Anthropic CEO Dario Amodei stated in Could, for instance, that the expertise might replace up to half of all white-collar jobs throughout the subsequent 5 years.

Because the identify suggests, the RLI is particularly designed to evaluate AI's potential to automate distant, freelance work. As anybody who has ever spent a stint as a freelancer can attest, it is a mode of labor that requires a excessive diploma of self-sufficiency and group, amongst different abilities. It has additionally change into fairly in style: A latest survey discovered that simply shy of 73 million People carried out freelance work in 2025, representing practically 43% of the total US workforce as of August.

AI and economically invaluable labor

The brand new examine assessed the efficiency of six industry-leading AI brokers, together with Google's Gemini 2.5 Professional, OpenAI's GPT-5, and Anthropic's Sonnet 4.5. 

Agents, which — not like extra restricted chatbots — are capable of work together with digital instruments (corresponding to an online browser) and carry out complicated, multi-step duties, are extensively positioned by tech builders as a vital evolutionary step towards the event of synthetic normal intelligence (AGI).

Additionally: AI is more likely to transform your job than replace it, Indeed finds

AGI is an imprecisely outlined time period: Consultants debate what it will imply for a pc to have true “normal intelligence,” and if such a feat is even doable. Nonetheless, one of many extra widespread definitions for AGI that will get thrown round in tech circles is a system that may match or outperform people on any economically invaluable job. 

If we take that definition as a place to begin, the brand new RLI examine suggests we're possible a good distance away from constructing true AGI. Every of the six fashions examined within the examine is “removed from able to autonomously performing the various calls for of distant labor,” in keeping with the authors.

The fashions had been evaluated throughout 23 classes of freelance work, together with graphic design, product design, computer-aided design (CAD), and recreation growth. These classes and their attendant talent necessities had been recognized by the researchers utilizing freelance platforms like Upwork, “grounding the benchmark in financial worth and capturing the range and complexity of actual distant labor markets.”

Additionally: The best free AI courses and certificates for upskilling in 2025 – and I've tried them all

Fashions had been fed a venture transient together with any obligatory recordsdata to finish their remaining deliverables, which had been then manually assessed by the researchers compared to deliverables for a similar venture created by human freelancers. The objective, in keeping with the researchers, was to search out out “whether or not an AI deliverable completes the venture at the very least in addition to the human gold commonplace — particularly, whether or not the deliverable could be accepted by an affordable shopper because the commissioned work.” 

The brokers had been then in contrast utilizing an Elo metric. Manus scored the very best, with an automation price of two.5%, adopted by Grok 4 and Claude Sonnet 2.5, each of which had a rating of two.1%.

The takeaway

Common narratives round AI automation could make human labor really feel extra unidimensional than it's in actuality. Because the AI {industry} strives to develop programs that may match or surpass the human mind, we more and more admire the mind's exceptional flexibility, dynamism, and complexity.

Some jobs are more amenable to automation than others, however most require an amalgamation of technical and interpersonal abilities, and due to this fact are extra difficult than the AI programs of right now can deal with.

Additionally: These jobs face the highest risk of AI takeover, according to Microsoft

Even right now's most superior AI programs, that are designed to be general-purpose brokers, are solely able to performing a slender subset of the duties required by most human staff. Because the authors of the brand new RLI examine wrote of their report, the failure of industry-leading brokers to automate lower than 3% of the duties required by the typical freelancer reveals “a stark hole” separating the promise and precise, demonstrable capabilities of AI. That is very true contemplating that the RLI does not seize many facets of most freelancers' day-to-day work lives, corresponding to speaking and negotiating with shoppers.

Then once more, these are early days. The capabilities of brokers are increasing quickly, and the biggest tech builders are investing billions in coaching new, extra superior fashions. It is doable that in 5 or ten years, corporations can be hiring AI freelancers. However for now, contractors do not appear to have any actual motive to concern the AI job reaper.

Get the morning's high tales in your inbox every day with our Tech Update newsletter.