Cold starts is a phenomenon that has been around for years.  During winter, you notice that your car’s engine often takes more time to start.  IT is no different.  When services are not invoked for a considerable time, they take a little longer to respond.  The duration of such a cold start delay is very much depending on the technology under the hood.  Serverless technology is more sensitive for cold starts, due to its event-driven nature.  All of this is very well explained over here, by Mikhail Shilkov.

How to avoid cold starts?

Each technology has some little tweaks you can do, to reduce the impact of cold start.  In the end, the best way to avoid it, is to keep it warm by pinging it each X minutes.  Recently, I read the following statement on Twitter: “If you need to keep your serverless services warm, serverless is not the right hosting model for you”.  Although this statement might be valid is some scenarios, I tend to disagree in general.  Serverless is a well-suited and resource-friendly way to handle unpredictable and burst load, which often implies periods of no activity at all.  Even if your use case does not allow a long delay for a first call after an idle period, it might be the cost-effective and scalable solution you need.  Don’t forget that other hosting models also suffer from cold start, however less.

How to keep your Azure Services warm?

There are several solutions to keep your Azure Functions in a consumption plan warm, you can find them on different forums.  The best way to do this – IMHO – is by pinging a health endpoint within your Azure Function through Azure Monitor.

First things first.  What is a health endpoint?  This is an extra API endpoint that can be interrogated any time.  The implementation of the endpoint can vary from scenario to scenario.  The simplest solution is just returning an HTTP 200.  Does this have value?  Definitely!  If you ping the health endpoint and it returns an HTTP 200, this means that:

• Your function is up and running (e.g. after a deployment)
• The API gateway in front of it is well configured
• Security validations, such as certificate validation and API keys, work correctly
• Network routes are configured the right way
• …

A more advanced implementation of a health endpoint also checks the availability of its dependencies, like other messaging services or databases.

Via Azure Monitor’s Availability Testing feature, you can easily ping your service on frequent intervals.  If the service is not available, your IT department can be alerted proactively, before small problems become big.  If you leverage release annotations throughout your CI/CD pipelines, you can also link decreased availability to specific releases that happened.  Talking about automated releases: if you have a health endpoint implemented, consider implementing a release gate that verifies if your deployment was successful.  All this combined, is a real devops win/win situation!

What if this is not sufficient?

The solution described above is perfect to avoid cold starts after idle time.  However, when your serverless function is scaling out aggressively, you might also notice delays on the first request of a freshly provisioned worker.  If you want to overcome this, you should definitely look at the Azure Functions premium plan.  Within this hosting model, there’s always a small buffer of warm workers available to handle your requests.

Conclusion!

Apply a devops mindset to turn the need of keeping your serverless functions warm into an opportunity to have proactive monitoring!  Win-win situation, for sure!

Toon