Announcing KEDA: bringing event-driven containers and functions to Kubernetes

Event-driven architectures are a natural evolution of microservices, enabling a flexible and decoupled design, and are increasingly being adopted by enterprise customers. Fully managed serverless offerings like Azure Functions are event–driven by design, but we have been hearing from customers about gaps in these capabilities for solutions based on Kubernetes. Scaling in Kubernetes is reactive, based on the CPU and memory consumption of a container.

In contrast, services like Azure Functions are acutely aware of event sources and therefore able to scale based on signals coming directly from the event source, even before the CPU or memory are impacted. We set out to bring the benefits of event-driven architectures and the productivity of functions to Kubernetes. The result is KEDA – Kubernetes-based event-driven autoscaling.

Microsoft and Red Hat partnered to build this open source component to provide event-driven capabilities for any Kubernetes workload. KEDA enables any container to scale from zero to potentially thousands of instances based on event metrics like the length of a Kafka stream or an Azure Queue. It also enables containers to consume events directly from the event source instead of decoupling with HTTP. KEDA can drive the scale of any container and is extensible to add new event sources.   

Because Azure Functions can be containerized, you can now deploy functions to any Kubernetes cluster while maintaining the same scaling behavior you would have in the Azure Functions service. For workloads that may span the cloud and on-premises, you can now easily choose to publish across the Azure Functions service, in a cloud-hosted Kubernetes environment, or on-premises. Our partnership with Red Hat enables Azure Functions to run integrated within OpenShift, providing the productivity and power of serverless functions with the flexibility and control to choose where and how to host it. The same application can move seamlessly between environments without any changes to development, code, or scaling. 

Bringing rich events into Kubernetes workloads 

KEDA provides a FaaS-like model of event-aware scaling, where deployments can dynamically scale to and from zero based on demand. KEDA also brings more event sources to Kubernetes. When we think about event–driven scenarios, the first event source we often consider is an HTTP event. However, we’ve learned from running Azure Functions that only 30% of our executions are coming from HTTP events, while the majority come from other event sources. We wanted to make sure that those other 70% of event executions were handled in a natural way in Kubernetes with KEDA. 

While other Kubernetes offerings have attempted to bring events from other sources, they usually involve converting the event to an HTTP request, which means data and context is lost in the process and direct communication with the event source is limited. KEDA unlocks rich connectivity directly to the event sources, enabling rapid and proactive scaling while preserving direct interaction with the event. KEDA provides a key component for event-driven and cloud native apps, and it also works seamlessly alongside Virtual Kubelet and AKS Virtual Nodes, letting you to run your event-driven apps on serverless containers. 

Running Azure Functions in any environment

Thousands of customers have already been using Azure Functions as the serverless compute service to process billions of events every day. Azure Functions provides an unmatched development experience, allowing you to develop and debug functions locally on any platform using editors like Visual Studio, Visual Studio Code, and more. While Azure Functions provides a fully managed serverless service, many customers want the freedom to run serverless in an open environment they can manage.

With the release of KEDA, any team can now deploy function apps created using those same tools directly to Kubernetes. This allows you to run Azure Functions on-premises or alongside your other Kubernetes investments without compromising on the productive serverless development experience. The open source Azure Functions runtime is available to every team and organization, and brings a world-class developer experience and programming model to Kubernetes.

The combination of flexible hosting options and an open source toolset gives teams more freedom and choice. If you choose to take advantage of the full benefits of a managed serverless service, you can shift responsibility and publish your apps to Azure Functions. If it makes sense for your organization or scenario, you can choose to run those same apps in Kubernetes with KEDA. While the hosting may vary, the developer experience and scaling remains constant.

You can publish your first Azure Function to Kubernetes by following this simple tutorial.

Compatible with every event source

KEDA will automatically detect new deployments and start monitoring event sources, leveraging real-time metrics to drive scaling decisions. Today KEDA and supporting tooling works with Kafka, Azure Queues, Azure Service Bus, RabbitMQ, HTTP, and Azure Event Grid / Cloud Events. More triggers will continue to be added in the future including Azure Event Hubs, Storage, Cosmos DB, and Durable Functions. KEDA event sources are extensible and we look forward to many additions from the community.

KEDA is open source on GitHub and open to community contributions. We look forward to hearing feedback from the community and working together to make event-driven container scale available to every developer and for every event type. If you want to learn more about KEDA, register today for our webinar later in May where we can share more on the benefits of event-driven and serverless architectures.

Questions or feedback on this news? Please let us know in the comments below.

Jeff Hollan

Principal PM Manager, Azure Serverless

See more articles from this author