I Moved a Production Durable Function to the Durable Task Scheduler

Thu, 23 Jul 2026 00:00:00 +0000

I have used Durable Functions for years. Until recently, I had never used the Durable Task Scheduler.

That distinction matters. Durable Functions are the programming model: orchestrators, activities, durable timers, replay and all the constraints that come with them. The Durable Task Scheduler (DTS) is the managed backend that persists their state and dispatches their work.

Historically, our Durable Functions used the default Azure Storage provider. Queues moved the work, Tables held instance state and history, and Blobs helped coordinate workers. It worked. I had no dramatic outage forcing me to replace it.

Then Microsoft made the DTS Consumption SKU generally available. The pitch was difficult to ignore: a purpose-built high-performance backend, no storage infrastructure to operate, identity-based security, an excellent dashboard out of the box, and pay-per-action pricing with no idle cost.

The final push, though, was a conversation. During Microsoft Build in San Francisco I had a shift at the Cloud Apps booth, where Nick Greenfield, a senior product manager on the Azure Functions team, walked me through the Durable Task Scheduler. In hindsight, he was too humble about it. Nothing in that conversation prepared me for how good the dashboard and the performance would turn out to be.

So I moved a real two-phase production workflow to it.

What this workflow actually does

The workflow distributes documents and, when requested, sends them to an external bookkeeping system. A Service Bus message can ask for either phase or both:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


Service Bus message
 │
 ▼
Starter function
 ├── validate the tenant and batch references
 ├── build a deterministic instance ID
 └── reject duplicate starts
 │
 ▼
Orchestrator
 ├── Phase 1: document distribution
 │ ├── load lightweight batch metadata
 │ ├── page items by serialized byte size
 │ ├── fan out deliveries
 │ └── use durable timers for email throttling
 │
 └── Phase 2: bookkeeping
 ├── load lightweight batch metadata
 ├── page document references
 ├── group work by provider/company
 ├── send entries
 └── persist per-document results

Bookkeeping only starts if distribution succeeds, or when no distribution phase was requested. SQL remains the business source of truth. DTS coordinates the process; it does not become a second business database.

This is a good Durable Functions workload because it combines fan-out/fan-in, external systems, rate limits, retries, partial failures and live progress. It is also exactly the sort of workflow where orchestration infrastructure becomes visible when it is slow or awkward to operate.

The migration itself was almost boring

For a .NET isolated Function App, the visible code change was small. I added the Azure-managed provider extension:

1
2
3


<PackageReference
 Include="Microsoft.Azure.Functions.Worker.Extensions.DurableTask.AzureManaged"
 Version="1.9.0" />

Then I selected the provider in host.json:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


{
 "extensions": {
 "durableTask": {
 "hubName": "document-workflows",
 "storageProvider": {
 "type": "azureManaged",
 "connectionStringName": "DURABLE_TASK_SCHEDULER_CONNECTION_STRING"
 }
 }
 }
}

The connection uses identity-based authentication and stays outside source control. The task hub name above is deliberately generic; the real resource and environment identifiers are irrelevant to the example.

That is the pleasant part of the migration: the orchestrator and activity programming model does not change. Microsoft describes DTS as a drop-in provider for existing Durable Functions applications, although instance history cannot be migrated between storage providers. That makes the deployment boundary important: finish any old work you need to retain before pointing the app at the new, empty task hub.

The provider switch was easy. Making the workflow a good citizen of the new backend took more thought.

The 1 MB limit changed how I thought about batches

DTS limits orchestrator inputs and outputs, activity inputs and outputs, external events and custom status to 1 MB each. A fixed page size such as “take 500 documents” is unsafe: 500 tiny objects might fit while 50 large ones might not.

We page by the actual UTF-8 JSON size instead:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


public const long MaxPayloadBytes = 1_048_576;
public const long ItemBudgetBytes = 800_000;

public static List<T> TakeWithinBudget<T>(
 IReadOnlyList<T> ordered,
 long maxBytes = ItemBudgetBytes)
{
 var page = new List<T>();
 long size = 2; // []

 foreach (T item in ordered)
 {
 long itemBytes =
 JsonSerializer.SerializeToUtf8Bytes(item).Length + 1;

 if (page.Count > 0 && size + itemBytes > maxBytes)
 break;

 page.Add(item);
 size += itemBytes;
 }

 return page;
}

The 800 KB working budget leaves room for the result envelope, cursor metadata and JSON framing. The orchestrator receives a page, fans out the relevant activities, waits for the page, records coarse progress, then asks for the next one.

This also keeps orchestration history smaller. Durable state is not a convenient place to carry entire entity graphs.

Durable timers make throttling honest

One delivery channel permits 750 emails per hour. The original temptation was to load everything, split it into chunks of 750, and put a delay between the chunks.

Inside an orchestrator, the delay must be durable:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


const int emailWaveSize = 750;

while (moreEmails)
{
 await SendNextByteBoundedWave(emailWaveSize);

 if (moreEmails)
 {
 context.SetCustomStatus(new
 {
 Phase = "Distribution",
 Channel = "Email",
 WaitingForRateWindow = true
 });

 await context.CreateTimer(
 context.CurrentUtcDateTime.AddHours(1),
 CancellationToken.None);
 }
}

The Function App is not burning compute for an hour. The orchestration is checkpointed and wakes when the durable timer fires. More importantly, the dashboard says that it is deliberately waiting for a rate window instead of merely looking stuck.

Retry only after making the activity safe to retry

Distribution calls are naturally narrow: one item goes to one delivery activity. Transient operations get bounded retries.

Bookkeeping is more dangerous. An external system may accept an invoice immediately before the Function host dies. Replaying the activity can then submit the same invoice twice.

We did not solve that with a larger retry number. We first made the activity resumable:

Already resolved documents are skipped on the next attempt.
Per-document state is persisted as work progresses.
Unresolved documents are moved out of the temporary “in progress” state when an attempt fails.
Batch counters are recomputed from persisted document statuses instead of trusted as an in-memory total.
Retries are selective and bounded: three attempts, starting at 30 seconds with exponential backoff.
The remaining crash window relies on idempotency at the receiving provider.

That last point is important. Durable Functions give you reliable orchestration, not magical exactly-once delivery to an external system. If an API changes state and does not accept an idempotency key, there is always an uncomfortable edge between “the remote call succeeded” and “we persisted that it succeeded.”

Another small migration lesson: exceptions are data now

One of the first DTS failures was not in the business logic at all. An activity result tried to carry a live Exception object back to the orchestrator.

System.Exception is not a useful cross-process contract and does not round-trip through the isolated worker’s JSON serializer. The result now carries an error string instead:

1
2
3
4
5
6


public sealed class ActivityResult
{
 public bool Success { get; init; }
 public object? Result { get; init; }
 public string? Error { get; init; }
}

If the orchestrator needs a failure, it creates a new exception from that serialized message. This makes the boundary explicit and avoids pretending an in-memory object can survive a durable hop.

The dashboard is not a side feature

This may be my favourite part of DTS.

The scheduler includes a dashboard where I can filter orchestrations, inspect their status and duration, see payloads and custom status, open their history, and view a timeline or flow diagram. I can also pause, resume, terminate, restart or raise an event when the orchestration supports it.

Our custom status is intentionally small and coarse: phase, channel, total items, dispatched items, whether the workflow is waiting for the next email window, and a capped error message. We update it once per page, not once per document.

That gives the dashboard useful business context without turning custom status into another event stream.

The timeline is particularly useful. In the captured example, the entire orchestration took just over a minute and the chart made it immediately obvious that almost all of that time belonged to the external bookkeeping call, not the scheduler.

That is much more useful than staring at a pile of storage queues and wondering whether the workflow is progressing.

What production looks like so far

From the DTS cutover through 19 July, this task hub contained only this workflow:

Measurement	Result
Orchestration instances	596
Completed	576
Failed	20
Completion rate	96.6%
Billable scheduler actions	9,339
Orchestrator actions	4,968
Activity actions	4,365
Timer actions	6
Peak actions in one day	7,478
Peak pending activities	60
Peak pending orchestrators	1
Connected workers	1

The 20 failures are workflow outcomes, not 20 scheduler outages. They include failures surfaced by our own activities and external integrations, which is exactly what I want the dashboard to make visible.

The usage pattern also explains why Consumption fits. Most days are quiet, then a batch arrives and the workload spikes. One worker handled the measured load. The activity queue briefly reached 60 pending items, while the orchestrator queue barely formed at all and drained again.

What did it cost?

The Consumption SKU charges for dispatched actions. TotalActions is the billable metric and is the sum of orchestrator, activity, timer and entity actions.

At the Microsoft retail price for our Azure region on 20 July 2026, $0.0039 per action, the scheduler calculation is simple:

1
2
3


9,339 actions × $0.0039 = $36.42

$36.42 ÷ 596 orchestrations ≈ $0.061 per orchestration

So the managed scheduler, persisted orchestration state and built-in dashboard cost roughly six cents per orchestration during this measured period.

That is an estimate based on Microsoft retail pricing, not an invoice. It excludes Azure Functions compute, taxes, currency conversion and any agreement-specific discounts. The Function App is billed separately, and Durable orchestrator replays can also count as Function executions on the Consumption hosting plan.

Still, for this workload, roughly $36 for the scheduler is an easy decision.

Is it actually faster?

Microsoft’s own early benchmark found DTS roughly five times faster than the Azure Storage provider for a deliberately simple sequential workload. Microsoft also warns that the result was from an early preview configuration and should be treated as a rough comparison, not a universal promise.

Our production system feels much faster after the switch.

That is the strongest claim I can defend.

The batches before and after the migration contained wildly different numbers and kinds of documents. We also changed paging and retry behaviour around the same time, and our tracing configuration was not identical on both sides of the cutover. A chart comparing those request durations would look scientific while proving very little.

So I am not publishing one.

The honest version is: Microsoft has controlled benchmark evidence for higher backend throughput, and our subjective production experience agrees with its direction. We do not have an apples-to-apples benchmark of our own workflow.

Consumption is the right SKU, for now

The Consumption SKU currently supports up to 500 dispatched actions per second, retains orchestration data for up to 30 days, and has no base fee or idle charge. That matches our bursty workload well.

Dedicated is a different trade-off. It provides reserved capacity, up to 2,000 actions per second per Capacity Unit, up to 90 days of retention, and high availability when configured with three Capacity Units. If this workflow becomes continuously busy, requires longer history, or needs the higher SLA, I will revisit the choice.

Right now, buying dedicated capacity for a worker that spends most of its day idle would be solving the wrong problem.

What I would carry into the next migration

Keep durable payloads small from the start. Page by serialized bytes, not an arbitrary item count.
Make external effects idempotent before adding retries. Retry policies cannot repair an unsafe activity contract.
Use custom status as a dashboard summary. A few meaningful checkpoints beat thousands of status writes.
Build deterministic instance IDs. They make Service Bus redelivery and duplicate detection manageable, but remember the 100-character DTS limit.
Keep tracing consistent across the cutover. Otherwise your before-and-after telemetry will not be comparable.
Do not confuse orchestration failures with scheduler failures. A managed backend should expose your application failures clearly, not make them disappear.

The Durable Task Scheduler did not change how I write orchestrators. It changed how little infrastructure I have to think about while they run.

For a few lines of configuration, we got a purpose-built backend, identity-based connectivity, a useful operational dashboard and a Consumption model that matches our bursts. The code still has to respect determinism, payload limits, idempotency and the realities of unreliable external systems.

That seems like the right division of responsibility.

Thanks to Nick Greenfield for the conversation at Build that finally got me to take the leap.

Sources

Featured image: Durable Task Scheduler dashboard screenshot from Microsoft’s public preview announcement
The Durable Task Scheduler Consumption SKU is generally available
Durable Task Scheduler overview
Durable Task Scheduler dashboard
Action metrics and billing
Durable Task Scheduler billing
DTS throughput benchmark
Microsoft Azure Retail Prices API

Azure on ND Teknik