Compute Options
By the end of this lesson
Choose between virtual machines, containers and functions.
Compute is the part of a cloud account that runs your code. Storage holds bytes, a database answers queries, and compute is where the employees API actually executes when a request arrives.
There are three shapes to choose from, and every provider offers all three under different names: a whole machine you look after, a container the platform runs for you, and a function the platform starts when something happens. They differ in how much you operate, how you pay, and what kind of work they suit. The question that separates them is not which is most modern. It is how your work arrives — steadily, in bursts, or as long jobs.
The words you need before the comparisons make sense:
- Virtual machine
- A rented computer with its own operating system. You install what you need and keep it patched. Azure Virtual Machines, Amazon EC2.
- Container image
- Your application packaged with the dependencies it needs, as a single versioned file. The same image runs on your laptop and in production, which removes a large class of it-worked-on-my-machine problems.
- Container platform
- A managed service that runs your image, restarts it when it crashes and adds copies when traffic rises. Azure Container Apps or App Service, AWS ECS on Fargate or App Runner.
- Function
- A single piece of code the platform runs in response to an event — an HTTP request, a file landing in storage, a message on a queue. Azure Functions, AWS Lambda.
- Cold start
- The extra delay on the first request after a platform has no warm instance ready. It has to allocate capacity and load your code before your first line runs.
- Scale to zero
- Running no instances, and therefore paying nothing for compute, when no work is arriving. The reason functions are cheap for occasional work and the reason cold starts exist.
First choice: do you want a machine, or do you want a platform to run your image?
| Virtual machine | Container on a managed platform | |
|---|---|---|
| Called | Azure Virtual Machines, Amazon EC2 | Azure Container Apps or App Service, AWS ECS on Fargate or App Runner |
| What you hand over | Nothing. You log in and set it up | A container image and a port |
| Patching the operating system | Yours, every month, forever | The provider's |
| Reproducing the environment | A document, a script, or somebody's memory | The image. It is the same bytes everywhere |
| Adding capacity | Build another machine and keep it identical to the first | Raise a replica count or a scaling rule |
| Billing | Per hour the machine exists, busy or idle | Per replica per second, and some platforms scale to zero |
| Suits | Software with host-level requirements, or a legacy application that expects a full machine | Most web services, including the employees API |
Second choice, and the one people get wrong: a container that stays running, or a function billed per invocation.
| Container that stays running | Function billed per request | |
|---|---|---|
| Start-up delay | Paid once at deploy. Requests hit a warm process | Cold start on the first request after idle. Commonly tens of milliseconds to a few seconds depending on runtime and package size |
| Cost at steady high load | Predictable, and usually lower per request | Higher. You pay per invocation and per millisecond, and at volume that adds up |
| Cost at low or bursty load | You pay for idle replicas | Close to nothing while nothing happens |
| Maximum run time | As long as you like | Capped by the platform. Minutes, not hours |
| In-process state and caching | Works. A warm process can hold a cache between requests | Unreliable. An instance can vanish between calls, so treat every call as starting fresh |
| Database connections | A pool per replica, and the replica count is something you control | Awkward. Many short-lived instances can exhaust a database connection limit |
| Suits | The employees API itself — steady traffic, a warm connection pool, predictable latency | Reacting to an event: resize an uploaded photo, process a queued message, run a nightly export |
// The platform calls this method. Your code does not start a web server,
// does not own the schedule, and cannot assume the previous call ran here.
public class PhotoUploadedHandler
{
private readonly IThumbnailStore _thumbnails;
private readonly ILogger<PhotoUploadedHandler> _logger;
public PhotoUploadedHandler(IThumbnailStore thumbnails, ILogger<PhotoUploadedHandler> logger)
{
_thumbnails = thumbnails;
_logger = logger;
}
public async Task HandleAsync(string photoKey, Stream photo, CancellationToken ct)
{
var thumbnailKey = photoKey.Replace("photos/", "thumbnails/");
// Writing the same key twice produces the same result, so a repeated
// delivery is harmless. That property is not optional here.
await _thumbnails.WriteAsync(thumbnailKey, photo, ct);
_logger.LogInformation("Thumbnail written. Key={ThumbnailKey}", thumbnailKey);
}
}- There is no host, no port and no route. The platform owns the trigger, which is the real difference between a function and a small web service.
- The handler is written so that running it twice for one upload changes nothing. Event platforms generally promise to deliver an event at least once, not exactly once, so a retry after a timeout can call you again with the same photo. Code that appends a row or increments a counter would double it.
- Nothing is cached in a field between calls. The instance holding this object may be discarded at any point, so anything you want to keep has to go to storage or a database.
- The log line uses named placeholders rather than an assembled sentence. That keeps the employee-facing values as separate fields you can filter on, which the monitoring lesson later in this module builds on.
A workable default for a new service: package it as a container image and run it on a managed container platform. You get reproducible environments, the provider patches the host, scaling is a setting, and you have not painted yourself into an unusual runtime.
Reach for functions when work arrives as discrete events and finishes quickly — a thumbnail to generate, a message to process, a report to kick off. Reach for a virtual machine when something about the host is a stated requirement rather than a preference.
Most real systems use two of the three. The employees API on a container platform, with a function handling photo uploads, is a reasonable shape and it is the one this course assumes from here on.
Summary
- Compute comes in three shapes: a machine you operate, a container a platform runs, and a function triggered by an event
- A container on a managed platform is a reasonable default for a service with steady traffic
- Functions scale to zero and cost little when idle, at the price of cold starts and a hard execution time limit
- Functions are the wrong choice for long-running work, steady high load, and tight latency commitments
- Never keep state on local disk or in memory between calls — instances are replaced routinely
Practice
Attempt each one before opening the solution. Getting it wrong first is how the idea sticks.
Think about it
Pick the compute for four jobs
Choose a compute shape for each, and say what would change your mind: the employees API serving around 40 requests per second during working hours; a nightly job that reconciles the employee directory with the payroll system and takes about an hour; resizing each uploaded photo; a licensed reporting tool that must be installed on Windows with a specific driver.
Show solution
The API is steady traffic, so a container on a managed platform. Warm processes, a connection pool that stays open, no cold starts on the paths users touch.
The hour-long reconciliation exceeds function time limits, so run it as a scheduled container job. If you had to use functions, you would split the work into queued chunks — which is more moving parts for no benefit unless the volume is genuinely unpredictable.
Photo resizing is event-shaped and short, so a function. It costs nothing between uploads and the platform handles a burst when someone uploads fifty photos at once.
The reporting tool needs a host-level dependency, so a virtual machine. That is a stated requirement, which is the bar for taking the operating system back.
What would change your mind: if API traffic became a handful of requests a day, a function would be cheaper and the cold start would not matter. If photo uploads became constant, a container consuming a queue would cost less.
Try it yourself
Measure a cold start
Deploy a function that does nothing except return the current time. Call it, wait long enough for the platform to scale it to zero, then call it again and compare the two response times.
If you cannot deploy one, write down what you would measure and what number would make you rule functions out for a user-facing endpoint.
Show solution
The second call is the interesting one, and the gap is usually much larger than people assume from reading a pricing page. It varies by runtime, package size and platform, which is exactly why measuring beats guessing.
The number that matters is the delay a user would notice on the slowest path, not the average. Averages hide cold starts because most calls are warm, so look at the worst few percent.
If your latency target is a few hundred milliseconds and cold starts land above it, your options are keeping instances warm, moving to a container, or accepting that a small share of users wait. All three are legitimate; picking one without measuring is not.
Knowledge check
Nothing is recorded and there is no score. The explanation appears either way.
Saved in this browser only.