Microsoft open sources Trill, a powerful query processor for analytics at incredible speeds

In today’s demanding business environment, processing massive amounts of data each millisecond is becoming a common business requirement. We are excited to be announcing that an internal Microsoft project known as Trill—for processing “a trillion events per day”—is now being open sourced.

Trill started as a research project at Microsoft Research in 2012, and has been extensively described in research papers, including at VLDB and in the IEEE Data Engineering Bulletin. The roots of Trill’s language lie in Microsoft’s previous system – called Microsoft StreamInsight. Both systems are based off an extended query and data model that extends the relational model with a time component.

Here are just a few of Trill’s powerful new capabilities that we are excited to be enabling for the developer community:

• As a single-node engine library, any .NET application, service, or platform can easily use it and start processing queries;
• A temporal query language allows users to express complex queries over real-time and/or offline data sets; and,
• Trill’s high performance across its intended usage scenarios means users get results with incredible speed and low latency. For example, filters operate at memory bandwidth speeds – up to several billion events per second, while grouped aggregates operate at 10-100 million events per second.

Trill is already becoming a key component of some of the largest mission-critical streaming pipelines within Microsoft, powering internal applications such as Bing Ads, Azure Stream Analytics, and Halo. We are incredibly excited to be offering the power of the IStreamable abstraction to all customers the same way that IEnumerable and IObservable are available, and hope that Trill and IStreamable will become the basis for streaming or temporal processing for future open source offerings.

To learn more about the open sourcing of Trill, visit the announcement on Azure blog.

James Terwilliger

Principal Software Engineer

See more articles from this author