Changing server count to follow load
Why capacity changes over time
An application may have few requests at night and many during a promotion. Keeping servers sized for peak load all the time is expensive. Autoscaling adds instances as traffic grows and removes them after it drops.
A controller estimates needed capacity and orders server startup or shutdown. Here decisions use incoming flow and target utilization. This is a simplified algorithm, not an exact reproduction of a cloud platform.
Minimum, maximum and target utilization
Minimum is the number kept ready even with low traffic, a reserve for the start of a burst. Maximum caps server count. It permits growth but does not mean those servers are already running.
Target utilization defines the headroom the controller seeks. A server limited to 300 requests/s at 50% target is budgeted for 150. At 600 requests/s, the controller requests four servers if maximum allows. Actual service depends on servers already ready.
Why autoscaling responds late
Startup takes time. An instance cannot process requests while preparing. If traffic rises sharply, the controller can request enough servers immediately, but capacity is still insufficient until they are ready. Requests lost at this point are not queued in this lesson.
Thus three counts differ: desired servers from calculation, starting servers still preparing, and ready servers already working. Reserve bridges startup delay but raises cost. Increasing maximum alone cannot protect the first seconds.
What happens after traffic drops
Removing servers immediately after a brief drop is risky: load may rise again. A stabilization window requires the scale-down recommendation to persist. Longer windows reduce hasty shutdowns but retain costs longer.
Watch the full experiment timeline: startup losses, ready capacity at peak and extra instances after decline. Both ready and starting servers incur costs here. A good policy handles the specified burst and returns to a reasonable instance count by the experiment’s end.
Task
Base flow is 300 req/s, rising to 900 req/s from second 10 to 30. A server handles 300 req/s, but new instances do not start instantly. Choose reserve and scaling policy.
60 one-second steps. Ready servers serve 300 req/s each. Desired uses incoming flow and target utilization. Already ordered servers count, avoiding duplicate orders every step. Starting instances are billed but add no capacity until readyAt. Scale-down requires a stable low recommendation.
- Find the point where traffic has risen but ready servers have not appeared yet.
- Add servers to set the available maximum; configure reserve and target utilization.
- Compare startup and stabilization effects on losses, final server count and cost.