Databricks Classic vs. Serverless vs. Cazpian: The 2026 Compute Cost Showdown
"Which platform is cheapest for Spark?" is one of the most common questions data teams ask — and one of the most misleading. The honest answer is: it depends entirely on your workload shape.
A platform that saves you thousands on large nightly batch jobs might quietly waste thousands on your fleet of small ETL runs. The billing model that looks transparent at first glance might hide costs in cold starts, minimum increments, bundled premiums, or idle compute you never asked for.
In this post — Part 3 of our compute cost series — we price three realistic workload scenarios on Databricks Classic, Databricks Serverless, and Cazpian, and keep raw Amazon EMR in the table as the AWS floor. Real pricing. Real math.
Two changes since this post first ran. Databricks Serverless is now a first-class column, because that is where Databricks is steering every customer, and it is the most expensive way to run the same work. And Cazpian's side is now arithmetic on measured runs rather than a model: the full TPC-DS at 1 TB for $6.49, 100 concurrent jobs in 67.5 seconds on a $2/hour pool, a 500-job ramp on the same pool, and an ARM64 A/B that measured the same pool shape at $1.60/hour. The billing section reflects how Cazpian is actually sold: a flat annual subscription per deployment, with the infrastructure on your own AWS bill.
First: Understand the Billing Models
Before running any numbers, you need to understand how each platform charges you. The billing model is not just a pricing page — it is the structural reason one platform costs more than another for your specific workload.
Databricks Classic
Databricks bills in Databricks Units (DBUs). A DBU is a unit of processing capability per hour, and the rate depends on the workload type:
- Jobs Compute: ~$0.15-0.40/DBU-hour (varies by instance and cloud)
- All-Purpose Compute: ~$0.40-0.65/DBU-hour (interactive workloads)
Each VM instance maps to a specific DBU count. A 4-core i3.xlarge might be 1 DBU; a 16-core i3.4xlarge might be 4 DBUs. You pay DBU cost plus the underlying cloud infrastructure cost.
Key cost drivers: Cold-start time on job clusters (2-5 minutes billed), DBU markup over raw infrastructure cost, minimum billing per job.
Databricks Serverless
Serverless is the model Databricks now defaults new workspaces to and is actively moving existing customers onto. The VM disappears from your cloud bill and the DBU price bundles the infrastructure, at a much higher rate (Premium tier, AWS, list):
- Serverless Jobs: $0.35/DBU-hour
- Serverless SQL warehouses: $0.70/DBU-hour
- Serverless notebooks / all-purpose: $0.75/DBU-hour
What you gain: no cluster to manage, start-up in seconds, per-second billing, so the cold-start waste of Classic largely disappears. What you give up: any control over the hardware, and any visibility into what a DBU is buying. Databricks does not publish a DBU-per-vCPU mapping for serverless; consumption is derived from the workload by Databricks itself. Throughout this post we assume 0.25 DBU per vCPU-hour, the same assumption our published 1 TB report uses, and we say "estimated" wherever that assumption is doing the work. We would rather show you a receipt: if you run one of these workloads on serverless, send us the bill and we will print it here.
Key cost drivers: The bundled rate itself — 2-5× the Classic DBU price — and which serverless product the work lands on. SQL and notebooks cost twice what jobs do.
Amazon EMR (reference floor)
EMR bills as a surcharge of roughly 15-25% on top of EC2 instances, with you managing cluster lifecycle, Spark versions, and Spot interruptions. We keep it in the tables as the "raw AWS plus a thin markup" floor: the cheapest a per-job-cluster model can be if your platform team does all the work.
Cazpian
Cazpian deploys the whole platform — control plane, compute pools, catalog, BI, and AI — inside your own AWS account, and charges a flat annual software subscription per deployment. There is no per-job markup, no DBU, no credit. Two things land on your books:
- Your AWS bill, at AWS prices: the Fargate or EC2 the compute pools run on, S3, and any Bedrock usage. This is what the scenarios below measure.
- The subscription, a flat annual fee by band (users and concurrent vCPU). It does not move with job count, and it is not in the per-job math below — see the software fee before you compare totals.
A Compute Pool is a warm Spark cluster with AutoStop. Jobs land on it in seconds instead of provisioning their own cluster, so a pool amortises its cost across every job it runs. The measured shape we quote throughout is a tier-S pool: a 4 vCPU driver and three 8 vCPU workers on Fargate in us-east-1, which costs $2.00/hour on x86 and $1.60/hour on ARM64 at public on-demand rates.
Key cost drivers: How many hours a pool is awake (AutoStop decides), pool size, and nothing else.
The Three Scenarios
We modeled three workload profiles that represent the majority of real-world Spark usage. For each one, we calculate the monthly cost using publicly available list pricing as of 2026 for Databricks and EMR, and measured pool rates and throughput for Cazpian.
Common assumptions across all scenarios:
- AWS us-east-1 region, Databricks Premium tier
- On-demand pricing (no reserved instances, savings plans, or committed-use discounts); Fargate for Cazpian unless stated
- Serverless DBUs estimated at 0.25 DBU per vCPU-hour of the equivalent Classic cluster
- Standard Spark configurations (no exotic tuning)
- 30-day month; "business hours" means about 264 pool-hours a month (12 hours on 22 working days)
Scenario A: The Small ETL Fleet
Profile: A data team running hundreds of lightweight transformation jobs daily — CSV/JSON ingestion, incremental loads, dimension table refreshes, data quality checks.
| Parameter | Value |
|---|---|
| Jobs per day | 300 |
| Average input size | 2 GB |
| Average job runtime (actual work) | 2 minutes |
| Average cold-start overhead | 3.5 minutes (Classic), ~0 (Serverless) |
| Cluster per job | 1 driver + 2 workers (m5.xlarge), 12 vCPU |
Databricks Classic
Instance cost: m5.xlarge = $0.192/hr x 3 nodes = $0.576/hr
DBU cost: ~2 DBU x $0.25/DBU-hr = $0.50/hr
Total rate: $1.076/hr = $0.0179/min
Per job billed time: 3.5 min cold start + 2 min runtime = 5.5 min
Per job cost: 5.5 x $0.0179 = $0.099
Monthly: 300 jobs x 30 days x $0.099 = $891/month
Of that $891, $574 is cold-start overhead (63.5%). Your actual compute work costs $317.
Databricks Serverless (estimated)
12 vCPU x 0.25 = 3 DBU/hr
Serverless Jobs: 3 x $0.35 = $1.05/hr -> 2 min = $0.035/job
9,000 jobs = $315/month
Serverless SQL: 3 x $0.70 = $2.10/hr -> 2 min = $0.070/job
9,000 jobs = $630/month
Serverless does exactly what Databricks says it does here: it deletes the cold-start line. On per-second billing, tiny sporadic jobs are cheap, and Serverless Jobs is the lowest number in this scenario. The same jobs run from a notebook or a SQL warehouse cost twice as much.
Cazpian (measured basis)
Small jobs do not get their own cluster. They land on one warm tier-S pool, and the pool is the cost.
Tier-S pool (driver + 3 workers), Fargate us-east-1:
x86 $2.00/hr -> business hours (264 h) = $529/month
ARM64 $1.60/hr -> business hours (264 h) = $423/month
Per job: the pool absorbed 100 concurrent jobs in 67.5 s (measured),
and 300 concurrent jobs in 212.6 s. Marginal cost of a job: cents.
That is the conservative number: the pool stays awake through the working day whether or not a job is running, and AutoStop turns it off overnight. If the same 300 jobs arrive in batches instead of a trickle, the measured 300-job round costs about $0.12 of pool time, and the monthly figure collapses to whatever hours the pool is actually awake.
Scenario A Summary
| Platform | Monthly Cost | Cold-Start Waste | Cost per Job |
|---|---|---|---|
| Databricks Classic | $891 | $574 (63.5%) | $0.099 |
| Databricks Serverless Jobs (est.) | $315 | $0 | $0.035 |
| Databricks Serverless SQL (est.) | $630 | $0 | $0.070 |
| EMR (floor) | $432 | $216 (50%) | $0.048 |
| Cazpian | $423 (ARM64) - $529 (x86) | $0 | $0.047 - $0.059 (pool share) |
The honest read: for a sparse trickle of two-minute jobs, Serverless Jobs' per-second billing beats a warm pool that is awake all day — Cazpian is 34-68% above it on infrastructure alone, while sitting 41-53% under Classic and 16-33% under Serverless SQL. Cazpian's advantage in this scenario is not the compute line. It is that the number does not move: at 500 jobs/day, Classic rises to $1,485 and Serverless Jobs to $525, while the Cazpian pool, which digested 500 jobs in 229 seconds with throughput still climbing, stays at $423-529. At 1,000 jobs/day (Classic $2,970, Serverless Jobs $1,050) budget a second pool at most: $846-1,058. And none of it is metered — see the software fee.
Scenario B: The Large Nightly Batch
Profile: A data engineering team running a handful of heavy transformation and aggregation jobs each night — rebuilding fact tables, running large joins, producing analytics-ready datasets.
| Parameter | Value |
|---|---|
| Jobs per day | 8 |
| Average input size | 200 GB |
| Average job runtime | 45 minutes |
| Average cold-start overhead | 4 minutes (Classic), ~0 (Serverless) |
| Cluster per job | 1 driver + 8 workers (r5.2xlarge), 72 vCPU |
Databricks Classic
Instance cost: r5.2xlarge = $0.504/hr x 9 nodes = $4.536/hr
DBU cost: ~4 DBU x 8 workers x $0.25/DBU-hr = $8.00/hr
Total rate: $12.536/hr = $0.209/min
Per job billed time: 4 min cold start + 45 min runtime = 49 min
Per job cost: 49 x $0.209 = $10.24
Monthly: 8 jobs x 30 days x $10.24 = $2,458/month
Cold-start overhead here is only $200/month (8%). For large jobs, the cold start is a small fraction of total billed time. This is where Databricks' optimized Photon engine and runtime can deliver real value.
Databricks Serverless (estimated)
72 vCPU x 0.25 = 18 DBU/hr
Serverless Jobs: 18 x $0.35 = $6.30/hr -> 45 min = $4.73/job
240 jobs = $1,134/month
Serverless SQL: 18 x $0.70 = $12.60/hr -> 45 min = $9.45/job
240 jobs = $2,268/month
With no cold start to delete, serverless is simply the bundled rate times the runtime. Serverless Jobs comes in under Classic because the VM cost is inside the DBU; Serverless SQL, the product an analytics team would actually be using, lands back where Classic was.
Cazpian (measured basis)
The pool shape here is the one we benchmarked at 1 TB: four 2XL workers (64 worker vCPU), which ran all 103 TPC-DS queries against a terabyte in 63 minutes for $6.49 on Fargate. Eight 45-minute jobs run back to back on it each night, then AutoStop.
2XL x 4 pool, Fargate us-east-1: $5.19/hr (measured basis)
Per night: 8 jobs x 45 min = 6 h x $5.19 = $31.14
Monthly (x86 Fargate): $934 ($3.89 per job)
Monthly (ARM64, -20%): $747 ($3.11 per job)
Monthly (EC2 spot, $1.63/hr): $293 ($1.22 per job)
Holding a 200 GB job at 45 minutes on this pool is a conservative assumption: the same hardware finished a full terabyte of TPC-DS in 63 minutes with Native Query Acceleration doing 87% of the operators. Pool start is a few minutes once a night, not once a job.
Scenario B Summary
| Platform | Monthly Cost | Cost per Job |
|---|---|---|
| Databricks Classic | $2,458 | $10.24 |
| Databricks Serverless Jobs (est.) | $1,134 | $4.73 |
| Databricks Serverless SQL (est.) | $2,268 | $9.45 |
| EMR (floor) | $1,090 | $4.54 |
| Cazpian (Fargate x86) | $934 | $3.89 |
| Cazpian (ARM64) | $747 | $3.11 |
| Cazpian (EC2 spot) | $293 | $1.22 |
Cazpian saves 62-70% vs. Classic, 59-67% vs. Serverless SQL, and 18-34% vs. Serverless Jobs on on-demand Fargate, and sits at or under the EMR floor; on spot the gaps widen to 88%, 87%, and 74%. For large batch workloads the savings come from the absence of a bundled premium and from an engine that does not need the extra hardware — not from cold-start elimination, which matters less when jobs run for 45 minutes.
Important nuance: if your large jobs rely on Databricks-specific features like Photon or Delta Lake optimizations, and those features meaningfully reduce your runtime, the premium may be justified. Always benchmark with your actual workloads. Ours are published, with the harness.
Scenario C: The Mixed Workload
Profile: A typical enterprise data platform running a mix of everything — hundreds of small ETL jobs during the day, medium-sized transformation jobs hourly, and large batch jobs at night. Multiple teams share the platform.
| Workload Tier | Jobs/Day | Avg Input | Avg Runtime | Cluster Size |
|---|---|---|---|---|
| Small ETL | 250 | 2 GB | 2 min | 1+2 m5.xlarge (12 vCPU) |
| Medium transforms | 40 | 15 GB | 12 min | 1+4 m5.2xlarge (40 vCPU) |
| Large batch | 6 | 250 GB | 50 min | 1+8 r5.2xlarge (72 vCPU) |
Databricks Classic
| Tier | Per Job | Jobs/Month | Monthly Cost |
|---|---|---|---|
| Small (5.5 min billed) | $0.099 | 7,500 | $743 |
| Medium (16 min billed) | $0.514 | 1,200 | $617 |
| Large (54 min billed) | $11.29 | 180 | $2,032 |
| Total | $3,392 |
Databricks Serverless (estimated)
| Tier | DBU/hr | Serverless Jobs ($0.35) | Serverless SQL ($0.70) |
|---|---|---|---|
| Small (2 min) | 3 | $0.035 x 7,500 = $263 | $0.070 x 7,500 = $525 |
| Medium (12 min) | 10 | $0.70 x 1,200 = $840 | $1.40 x 1,200 = $1,680 |
| Large (50 min) | 18 | $5.25 x 180 = $945 | $10.50 x 180 = $1,890 |
| Total | $2,048 | $4,095 |
This is the scenario that shows what the serverless migration really costs. The same platform that runs for $3,392 on Classic runs for $4,095 on Serverless SQL — 21% more — once the ETL, the hourly transforms, and the nightly batch are all on the bundled rate. Only a shop that keeps every workload strictly on Serverless Jobs comes out ahead of Classic.
Cazpian (measured basis)
Three pools, each awake only as long as its work: the tier-S pool through business hours for the small jobs, a medium pool (1 driver + 4 x 8 vCPU workers, about $2.61/hr on Fargate by the same public arithmetic) for 8 pool-hours a day, and the 2XL x 4 pool for 5 pool-hours a night.
| Tier | Pool | Pool-hours / month | x86 Fargate | ARM64 |
|---|---|---|---|---|
| Small (250/day) | tier-S, business hours | 264 | $529 | $423 |
| Medium (40 x 12 min) | medium, 8 h/day | 240 | $626 | $501 |
| Large (6 x 50 min) | 2XL x 4, 5 h/night | 150 | $779 | $623 |
| Total | $1,934 | $1,547 |
Scenario C Summary
| Platform | Monthly Cost | Annual Cost | Annual vs. Serverless SQL |
|---|---|---|---|
| Databricks Serverless SQL (est.) | $4,095 | $49,140 | — |
| Databricks Classic | $3,392 | $40,704 | -17% |
| Databricks Serverless Jobs (est.) | $2,048 | $24,576 | -50% |
| EMR (floor) | $1,658 | $19,896 | -60% |
| Cazpian (x86) | $1,934 | $23,208 | -53% |
| Cazpian (ARM64) | $1,547 | $18,564 | -62% |
In mixed workloads Cazpian lands next to the EMR floor and next to the best case Databricks can offer (everything on Serverless Jobs, nothing interactive), at roughly half of Classic and 40% of Serverless SQL. The operational realities differ: EMR needs a platform team, Databricks Serverless needs you to accept whatever the DBU meter reads, and Cazpian ships the platform into your account and upgrades it quarterly.
The real comparison at this tier is not just price — it is total cost of ownership. Factor in the engineering time your team spends managing EMR infrastructure, and Cazpian's effective cost drops further.
The Full Picture
| Scenario A (Small ETL) | Scenario B (Large Batch) | Scenario C (Mixed) | |
|---|---|---|---|
| Databricks Serverless SQL (est.) | $630/mo | $2,268/mo | $4,095/mo |
| Databricks Classic | $891/mo | $2,458/mo | $3,392/mo |
| Databricks Serverless Jobs (est.) | $315/mo | $1,134/mo | $2,048/mo |
| EMR (floor) | $432/mo | $1,090/mo | $1,658/mo |
| Cazpian (x86 Fargate) | $529/mo | $934/mo | $1,934/mo |
| Cazpian (ARM64) | $423/mo | $747/mo | $1,547/mo |
| Cazpian vs. Serverless SQL | -16% to -33% | -59% to -67% | -53% to -62% |
| Cazpian vs. Classic | -41% to -53% | -62% to -70% | -43% to -54% |
| Cazpian vs. Serverless Jobs | +34% to +68% | -18% to -34% | -6% to -24% |
Three things the table cannot show. First, the Cazpian rows are pool-hours, so they are flat under load: the pool that costs $529 a month at 300 jobs a day costs $529 at 500. Second, every Databricks row scales with usage by design — that is what a meter is for — and the serverless rows scale at the bundled rate. Third, every Cazpian number is raw AWS infrastructure; the Databricks rows already contain its software fee (the DBU), and Cazpian's does not, which brings us to the part most comparisons skip.
The Software Fee
Cazpian's software is a flat annual subscription per deployment, banded by provisioned users and concurrent vCPU. It is not a markup on the numbers above and it does not change when you add jobs, but it is real money, so put it in your total.
The band is set by the whole estate, not one team's ETL. If Cazpian is running only Scenario A for a single team, the subscription can cost more than the compute it saves — that team should keep its warm pool inside a platform bought for more than that. The subscription pays for itself when it replaces the per-unit meter on several things at once: the ETL fleet, the BI serving that would otherwise run on a $0.70 serverless warehouse, the dev and test environments nobody wants to meter, and the AI Studio usage. Current bands and prices are on the pricing page; the billing overview explains how users and concurrent vCPU are counted.
Where Each Platform Wins
Databricks Classic is strongest when:
- You are heavily invested in the Databricks ecosystem (Unity Catalog, Delta Lake, Photon)
- You run primarily large, long-running jobs where cold-start overhead is negligible
- You are willing to pay the DBU premium for ecosystem convenience
Databricks Serverless is strongest when:
- Your jobs are tiny, sporadic, and strictly on Serverless Jobs — per-second billing with no idle is hard to beat for a trickle
- You want zero infrastructure decisions and accept the meter as the price of that
EMR is strongest when:
- You have a strong platform engineering team that can manage infrastructure
- You can leverage Spot instances effectively (can reduce costs by 60-70%)
- You already have mature AWS infrastructure and tooling
Cazpian is strongest when:
- Your platform runs a mix — ETL, BI, ad-hoc, AI — and you want one flat price instead of three meters
- You run a high volume of small-to-medium jobs where cold starts or bundled premiums dominate cost
- You need Iceberg-native operations with built-in compaction and file hygiene
- You want data sovereignty — everything, including the control plane, in your VPC
- Your team's time is better spent on data work than infrastructure management
What These Numbers Do Not Capture
Cost comparisons like this are useful but incomplete. Here are factors that the spreadsheet does not show:
Engineering Time
EMR requires a platform team to manage. Cluster configurations, AMI updates, Spark version upgrades, Spot interruption handling, scaling policies, log management — this is real work that costs real salary dollars. If your platform team spends 20 hours/month on EMR operations at a blended $100/hour, that is $2,000/month that does not appear in your cloud bill but absolutely appears in your budget.
The Serverless Meter
Serverless removes decisions, and with them the levers. You cannot pick a cheaper instance family, cannot ride a Spot market, cannot pin a pool size, and cannot audit what a DBU bought. Every efficiency Databricks finds in its serverless fleet is theirs to keep or pass on. The estimates in this post assume they pass a fair share on; check your own invoice.
Small File Debt
Platforms that do not manage output file sizes create downstream costs. Thousands of small files slow every query, increase S3 LIST operation costs, and eventually require compaction jobs that themselves consume compute. Cazpian's built-in write coalescing prevents this debt from accumulating.
Vendor Lock-In Cost
Databricks pipelines built on Delta Lake, Unity Catalog, and Photon create switching costs. If you ever need to move, the migration effort can be measured in months and engineering quarters. Cazpian is built on Apache Iceberg — an open standard. Your tables are readable from Athena, Snowflake, Trino, or any Iceberg-compatible engine without migration.
Scaling Economics
These scenarios assume fixed job volumes. In practice, job counts grow. A warm pool's cost scales with hours awake, not with jobs; a per-job model, Classic or Serverless, scales with jobs by definition. The 500-job ramp on a single $2/hour pool is the measured version of that sentence.
How to Run Your Own Comparison
Every organization's workload is different. Here is how to build your own cost model:
Step 1: Profile your workloads. Export 30 days of job history. For each job, capture: input size, runtime, cluster configuration, and cold-start duration. Group jobs into small (under 10 GB), medium (10-100 GB), and large (over 100 GB).
Step 2: Calculate your current cost per tier. For each tier, compute the average cost per job including cold-start overhead, instance costs, and any platform surcharges (DBUs, EMR markup). If you are already on serverless, use the DBU figures from your invoice — that is the only place the real mapping exists.
Step 3: Model the alternatives. Apply each platform's pricing model to your actual workload profile. Do not use vendor TCO calculators — they are designed to make the vendor look good. Use the raw pricing and your real numbers. For Cazpian, that is pool-hours times the public Fargate or EC2 rate, plus the subscription band your estate falls into.
Step 4: Factor in operations. Estimate the engineering hours your team spends on infrastructure management, cluster tuning, and incident response. Add that to the platform costs that require self-management.
Step 5: Consider the trajectory. Where is your job volume headed in 12 months? Model the cost at 2x and 5x current volume. The platform that scales most efficiently might not be the cheapest today.
The Bottom Line
There is no universally cheapest Spark platform. But there is a cheapest platform for your workload.
If your data platform looks like most — dominated by hundreds of small-to-medium jobs with a handful of large batch runs, and analysts querying all day — the math consistently favors a platform that eliminates cold-start overhead without replacing it with a bundled meter.
That is the architecture Cazpian was built around. Not because warm pools are a clever optimization, but because the way most teams actually use Spark — many small jobs, running frequently, processing modest data volumes, beside people running queries — demands a compute model designed for that reality, and a price that does not climb with it.
Your workload has changed. Your compute model should change with it. Your bill should not.
Want a cost comparison built on your actual workload data? Contact the Cazpian team — we will analyze your job history and show you exactly where your budget is going and how much you can save. Or run the published benchmarks yourself and check our arithmetic.