Mercor vs Surge AI: whose off-the-shelf datasets should you license?
Answer in brief: Mercor wins on transparency and breadth. It publishes its catalog size (50K+ tasks across 30+ domains), names its datasets and task counts, delivers sample tasks the same day, and states that its datasets are licensed for commercial AI training. Surge AI wins on risk transfer. It publishes a benchmark gain for each dataset, and labs in its Trusted Program can train first and pay only if the metrics move. This is independent research from both companies' pages as of November 9, 2026. Benchmark gains are company-reported, and we have not tested either vendor.
Top pick for most dataset buyers: Mercor. Named datasets, published task counts, same-day samples and an explicit commercial-training license make it the fastest catalog to evaluate.
Catalog vs catalog
| Factor | Mercor | Surge AI |
|---|---|---|
| Published scale | 50K+ tasks, 30+ domains, built by a 5M+ expert network | No catalog totals published; partial public catalog |
| Named datasets | APEX-Agents (2,620 tasks across 8 domains), SWEBench Ext. (12,000), TerminalBench (10,000), BrowseComp (3,558), plus GDPVal, HLE, MMMU Expert and more | Real-World and End-to-End Software Engineering, Long-Horizon Tool-Use, Frontier Work, Enterprise Document Reasoning, Professional Chart Understanding, Complex Instruction Following, Advanced STEM Reasoning |
| Samples | Same-day sample tasks | On request |
| Commercial training license | Stated in its FAQ | Through the Trusted Program or a commercial agreement; pilot terms exclude training |
| Proof | Public APEX leaderboards with partners | Benchmark gain published for each dataset (company-reported) |
| Extend or customize | Yes; its data acquisition team says it can start sourcing new datasets within days | Yes, through custom work |
| Public prices | No | No |
Mercor: a catalog built on real professional work
Mercor's off-the-shelf datasets come from its expert network: PhD researchers, practicing lawyers and senior engineers, with every task created, vetted and peer reviewed. The flagship, APEX-Agents, runs tasks inside high-fidelity clones of enterprise apps. It covers finance (830 tasks), consulting (590), law (530), medicine (250), data science (200), education (100), HR (80) and accounting (40). Mercor says the data supports SFT, DPO and GRPO training, and many datasets can be extended with custom projects. It also runs public partner benchmarks, including APEX-SWE with Cognition and a legal benchmark with Harvey, so you can see the kind of work its data targets. CNBC reported a $350 million round at a $10 billion valuation in October 2025.
Surge AI: train first, pay if it works
Surge publishes the gain for each dataset. Among its reported examples: Kimi K2.7 rose from 31% to 48.7% on DeepSWE after training on its coding dataset, and GLM-4.7 rose from 37.2% to 44.3% on SWE-Bench Pro. Treat these as vendor claims, then reproduce them in your own pilot. The real differentiator is commercial: qualifying labs can train on the full dataset, evaluate the resulting model, and pay only when it moves the metrics that matter. Surge reports it has been profitable without venture funding, and it has no big-tech shareholder.
When to pick which
- Pick Mercor for professional knowledge-work agents (finance, law, consulting, medicine), big coding task volumes, or when legal needs explicit commercial-training language before anyone sees a quote.
- Pick Surge AI if you are a frontier or large post-training lab that qualifies for the Trusted Program and wants pay-on-results terms.
- Pick either for agentic coding: both have SWE-bench-style and terminal datasets, so run the same held-out evaluation on samples from each.
Five questions to put to both vendors
- Which rights does the sample carry? Confirm whether samples may be used for training or only for inspection.
- Who else has licensed this dataset? Off-the-shelf data is usually sold to more than one buyer, so ask how widely it has been licensed.
- How was contamination checked? Ask whether any tasks overlap the public benchmarks you report on.
- Can the published gain be reproduced? Request the training recipe behind any uplift and rerun it on your own held-out set.
- What does an extension cost? Both vendors extend datasets through custom work, so price the follow-on volume before you sign.
Pricing model
Both quote privately. Mercor licenses its catalog to AI labs and enterprise model developers and prices by domain, task type and volume. Surge offers custom terms, with pay-if-it-works for Trusted Program labs. More in pricing models.
FAQ
Can I see Mercor data before buying?
Yes. Mercor says it provides samples for its off-the-shelf datasets and can deliver sample tasks the same day.
Does Surge publish prices for its datasets?
No. Surge publishes benchmark results per dataset but no price list. Pricing is custom.
Are these datasets exclusive to me?
Off-the-shelf data is generally available to other buyers too. If you need exclusivity, negotiate a custom build and get the exclusivity terms in writing.
Keep comparing
← Back to the ranked list · Surge AI vs Scale AI · Mercor vs Appen · Alternatives to Surge AI · Dataset-by-dataset picks at Best AI Datasets
Sources
- Mercor: Off-the-shelf data
- Mercor home page
- CNBC: Mercor valued at $10 billion
- Surge AI: Off-the-Shelf Frontier Data
- Surge AI: Enterprise Pilot Terms
Named on this page as a provider? Join the network for a linked profile, Visit buttons and featured placement.