Skip to main content
ANVISoftware Solutions
Lesson 5 of 17Intermediate16 min

Compute Options

By the end of this lesson

Choose between virtual machines, containers and functions.

Compute is the part of a cloud account that runs your code. Storage holds bytes, a database answers queries, and compute is where the employees API actually executes when a request arrives.

There are three shapes to choose from, and every provider offers all three under different names: a whole machine you look after, a container the platform runs for you, and a function the platform starts when something happens. They differ in how much you operate, how you pay, and what kind of work they suit. The question that separates them is not which is most modern. It is how your work arrives — steadily, in bursts, or as long jobs.

The words you need before the comparisons make sense:

Virtual machine
A rented computer with its own operating system. You install what you need and keep it patched. Azure Virtual Machines, Amazon EC2.
Container image
Your application packaged with the dependencies it needs, as a single versioned file. The same image runs on your laptop and in production, which removes a large class of it-worked-on-my-machine problems.
Container platform
A managed service that runs your image, restarts it when it crashes and adds copies when traffic rises. Azure Container Apps or App Service, AWS ECS on Fargate or App Runner.
Function
A single piece of code the platform runs in response to an event — an HTTP request, a file landing in storage, a message on a queue. Azure Functions, AWS Lambda.
Cold start
The extra delay on the first request after a platform has no warm instance ready. It has to allocate capacity and load your code before your first line runs.
Scale to zero
Running no instances, and therefore paying nothing for compute, when no work is arriving. The reason functions are cheap for occasional work and the reason cold starts exist.

First choice: do you want a machine, or do you want a platform to run your image?

 Virtual machineContainer on a managed platform
CalledAzure Virtual Machines, Amazon EC2Azure Container Apps or App Service, AWS ECS on Fargate or App Runner
What you hand overNothing. You log in and set it upA container image and a port
Patching the operating systemYours, every month, foreverThe provider's
Reproducing the environmentA document, a script, or somebody's memoryThe image. It is the same bytes everywhere
Adding capacityBuild another machine and keep it identical to the firstRaise a replica count or a scaling rule
BillingPer hour the machine exists, busy or idlePer replica per second, and some platforms scale to zero
SuitsSoftware with host-level requirements, or a legacy application that expects a full machineMost web services, including the employees API

Second choice, and the one people get wrong: a container that stays running, or a function billed per invocation.

 Container that stays runningFunction billed per request
Start-up delayPaid once at deploy. Requests hit a warm processCold start on the first request after idle. Commonly tens of milliseconds to a few seconds depending on runtime and package size
Cost at steady high loadPredictable, and usually lower per requestHigher. You pay per invocation and per millisecond, and at volume that adds up
Cost at low or bursty loadYou pay for idle replicasClose to nothing while nothing happens
Maximum run timeAs long as you likeCapped by the platform. Minutes, not hours
In-process state and cachingWorks. A warm process can hold a cache between requestsUnreliable. An instance can vanish between calls, so treat every call as starting fresh
Database connectionsA pool per replica, and the replica count is something you controlAwkward. Many short-lived instances can exhaust a database connection limit
SuitsThe employees API itself — steady traffic, a warm connection pool, predictable latencyReacting to an event: resize an uploaded photo, process a queued message, run a nightly export
A function that reacts to a photo upload
C#
// The platform calls this method. Your code does not start a web server,
// does not own the schedule, and cannot assume the previous call ran here.
public class PhotoUploadedHandler
{
    private readonly IThumbnailStore _thumbnails;
    private readonly ILogger<PhotoUploadedHandler> _logger;

    public PhotoUploadedHandler(IThumbnailStore thumbnails, ILogger<PhotoUploadedHandler> logger)
    {
        _thumbnails = thumbnails;
        _logger = logger;
    }

    public async Task HandleAsync(string photoKey, Stream photo, CancellationToken ct)
    {
        var thumbnailKey = photoKey.Replace("photos/", "thumbnails/");

        // Writing the same key twice produces the same result, so a repeated
        // delivery is harmless. That property is not optional here.
        await _thumbnails.WriteAsync(thumbnailKey, photo, ct);

        _logger.LogInformation("Thumbnail written. Key={ThumbnailKey}", thumbnailKey);
    }
}
  • There is no host, no port and no route. The platform owns the trigger, which is the real difference between a function and a small web service.
  • The handler is written so that running it twice for one upload changes nothing. Event platforms generally promise to deliver an event at least once, not exactly once, so a retry after a timeout can call you again with the same photo. Code that appends a row or increments a counter would double it.
  • Nothing is cached in a field between calls. The instance holding this object may be discarded at any point, so anything you want to keep has to go to storage or a database.
  • The log line uses named placeholders rather than an assembled sentence. That keeps the employee-facing values as separate fields you can filter on, which the monitoring lesson later in this module builds on.

A workable default for a new service: package it as a container image and run it on a managed container platform. You get reproducible environments, the provider patches the host, scaling is a setting, and you have not painted yourself into an unusual runtime.

Reach for functions when work arrives as discrete events and finishes quickly — a thumbnail to generate, a message to process, a report to kick off. Reach for a virtual machine when something about the host is a stated requirement rather than a preference.

Most real systems use two of the three. The employees API on a container platform, with a function handling photo uploads, is a reasonable shape and it is the one this course assumes from here on.

Summary

  • Compute comes in three shapes: a machine you operate, a container a platform runs, and a function triggered by an event
  • A container on a managed platform is a reasonable default for a service with steady traffic
  • Functions scale to zero and cost little when idle, at the price of cold starts and a hard execution time limit
  • Functions are the wrong choice for long-running work, steady high load, and tight latency commitments
  • Never keep state on local disk or in memory between calls — instances are replaced routinely

Practice

Attempt each one before opening the solution. Getting it wrong first is how the idea sticks.

Think about it

Pick the compute for four jobs

Choose a compute shape for each, and say what would change your mind: the employees API serving around 40 requests per second during working hours; a nightly job that reconciles the employee directory with the payroll system and takes about an hour; resizing each uploaded photo; a licensed reporting tool that must be installed on Windows with a specific driver.

Show solution

The API is steady traffic, so a container on a managed platform. Warm processes, a connection pool that stays open, no cold starts on the paths users touch.

The hour-long reconciliation exceeds function time limits, so run it as a scheduled container job. If you had to use functions, you would split the work into queued chunks — which is more moving parts for no benefit unless the volume is genuinely unpredictable.

Photo resizing is event-shaped and short, so a function. It costs nothing between uploads and the platform handles a burst when someone uploads fifty photos at once.

The reporting tool needs a host-level dependency, so a virtual machine. That is a stated requirement, which is the bar for taking the operating system back.

What would change your mind: if API traffic became a handful of requests a day, a function would be cheaper and the cold start would not matter. If photo uploads became constant, a container consuming a queue would cost less.

Try it yourself

Measure a cold start

Deploy a function that does nothing except return the current time. Call it, wait long enough for the platform to scale it to zero, then call it again and compare the two response times.

If you cannot deploy one, write down what you would measure and what number would make you rule functions out for a user-facing endpoint.

Show solution

The second call is the interesting one, and the gap is usually much larger than people assume from reading a pricing page. It varies by runtime, package size and platform, which is exactly why measuring beats guessing.

The number that matters is the delay a user would notice on the slowest path, not the average. Averages hide cold starts because most calls are warm, so look at the worst few percent.

If your latency target is a few hundred milliseconds and cold starts land above it, your options are keeping instances warm, moving to a container, or accepting that a small share of users wait. All three are legitimate; picking one without measuring is not.

Knowledge check

Nothing is recorded and there is no score. The explanation appears either way.

A reporting job takes 45 minutes to run. Why is a serverless function a poor fit?
Why should a function handler be safe to run twice with the same input?

Saved in this browser only.