{"id":77079,"date":"2019-05-29T09:00:08","date_gmt":"2019-05-29T16:00:08","guid":{"rendered":"https:\/\/cloudblogs.microsoft.com\/opensource\/?p=77079"},"modified":"2025-06-27T06:54:23","modified_gmt":"2025-06-27T13:54:23","slug":"how-to-use-trill-for-impression-feedback-part-1","status":"publish","type":"post","link":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/","title":{"rendered":"How to use Trill for impression feedback (part 1)"},"content":{"rendered":"\n

On the Microsoft BingAds team, one of my primary responsibilities is the development and maintenance of the FastBI pipeline \u2013 the system responsible for all revenue coming from the Bing search engine.<\/p>\n\n\n\n

We have been working on streaming technologies for the last five years, combining the scale and stability of our internal Cosmos compute platform with the expressivity and power of Trill as a processing engine, to produce a world-class streaming compute solution that is capable of handling some of our most important workloads.<\/p>\n\n\n\n

In this 2-part series, I\u2019ll introduce one of several challenging scenarios that we faced in the FastBI pipeline, followed by a detailed discussion of how we address it. I\u2019ve streamlined and simplified things a bit in the interest of making things quicker to understand, but for the most part, this is representative of the kinds of problems we face on a daily basis when modeling our system temporally.<\/p>\n\n\n\n

Note: this post assumes you have read Trill 101<\/a> and Trill 102<\/a> blog posts by my colleague, James Terwilliger.<\/em><\/p>\n\n\n\n

Without further ado, let\u2019s dig on in!<\/p>\n\n\n\n

Scenario<\/h1>\n\n\n\n

User activity on the Bing search engine and other search providers powered by Bing causes ad requests to be made and logged to an impression log. Search providers have the option of making modifications to the ads that were returned, including things like reordering ads within a block, moving ads between blocks, or dropping them entirely. These modifications are collected and logged as a separate feedback signal that will be processed by the Bing Ads processing pipeline. To get a view of the final, served ad slate, the feedback stream must be joined to the impression stream.<\/p>\n\n\n\n

One of the other major responsibilities of the pipeline is fraud detection. How fraud detection works is beyond the scope of this article; the important thing here is that the pipeline computes various statistics about the input traffic as one of several inputs that determines whether traffic is considered billable or not. Some of these statistics are very time-sensitive \u2013 the closer the events are processed to the actual order in which they occurred, the better.<\/p>\n\n\n\n

Requirements<\/h1>\n\n\n\n

The above business scenario translates to the following technical requirements.<\/p>\n\n\n\n

    \n
  • Output impressions as soon as they join successfully (and not before)<\/li>\n\n\n\n
  • Don\u2019t output an impression more than once<\/li>\n\n\n\n
  • Don\u2019t drop any impressions<\/li>\n\n\n\n
  • Output impressions immediately as they arrive so they can contribute to statistics<\/li>\n\n\n\n
  • Don\u2019t count an impression more than once in statistics<\/li>\n\n\n\n
  • Feedback can arrive up to 10 minutes after the associated impression<\/li>\n\n\n\n
  • Delivery of the input logs can be delayed by up to 30 minutes on either side<\/li>\n<\/ul>\n\n\n\n

    As stated, some of these requirements seem mutually exclusive \u2013 we can\u2019t output an impression until it is joined, but we must output the impression immediately for statistics; we can\u2019t output an impression until it is joined, but we can\u2019t drop any impressions (what about impressions that never receive feedback?).  This seems like a problem.<\/p>\n\n\n\n

    Fear not, dear reader, for not all is lost. What we really need is the ability to distinguish between these cases so that downstream queries can filter according to their specific requirements. This query, then, needs to be able to provide answers to the following questions for each event:<\/p>\n\n\n\n

      \n
    • Is the impression done receiving feedback?<\/li>\n\n\n\n
    • Should the impression contribute to statistics?<\/li>\n<\/ul>\n\n\n\n

      Let\u2019s look at each of these requirements more closely.<\/p>\n\n\n\n

      Is the impression done receiving feedback?<\/strong> An impression is considered done as soon as it receives feedback, which matches perfectly with the behavior of an inner join.  But what if feedback never arrives? This implies the operator must produce a second stream; namely, we need an expired event<\/em> to be produced at the end of the impression\u2019s lifetime only if it never received feedback previously<\/em>. We\u2019ll examine how to accomplish this later, but for now, the combination of these two streams contains all the impressions in the input, some with feedback and some that are expired.<\/p>\n\n\n\n

      Should the impression contribute to statistics?<\/strong> An impression must contribute to statistics immediately upon arrival. This means the query needs to produce output as soon as the impression arrives, which suggests a combination of an antijoin (when feedback is not present) and an inner join (when feedback is present).<\/p>\n\n\n\n

      However, there is one problem here \u2013 in the case where feedback arrives after the impression, both the antijoin and inner join will produce an output, and only one of them may contribute to statistics. This means we need a way to identify whether the operator produced an antijoin before the inner join or not.  We can record this in the event we produce so that the statistics query can properly filter incoming impressions to avoid double-counting.<\/p>\n\n\n\n

      The following diagram illustrates several possible ways that impression and feedback signals can arrive and the expected output of this operator for each one. For clarity, the join lifetime of the input events is shown as a dotted region; the input events are originally points.<\/p>\n\n\n\n

      \"illustrates<\/figure>\n\n\n\n

      *Due to the imprecise nature of the diagram, it is difficult to show exactly what these cases are intended to illustrate. The Way before<\/em> and Way after<\/em> cases represent when impression and feedback events overlap by the absolute minimum amount, 1 tick. In the Way before<\/em> case, the end time of the feedback is one tick larger than the start time of the impression, and in the Way after<\/em> case, the start time of the feedback is one tick smaller than the end time of the impression.<\/p>\n\n\n\n

      Don\u2019t worry too much about the event descriptors yet (Joined, Stats, Done). We\u2019ll cover those later. For now, focus on the events we produce and their lifetimes.<\/p>\n\n\n\n

      Creating the operator<\/h2>\n\n\n\n

      Now that we\u2019ve defined the problem, let\u2019s work on the solution. We are going to create a user-defined operator that abstracts away the complexity of this join into a single, logical operator. Let\u2019s start with the signature. We need to have both the impression and feedback streams, so we\u2019ll start off by supplying those as arguments.<\/p>\n\n\n

      \ninternal static IStreamable JoinFeedbackToImpressions(\n    this IStreamable impressions,\n    IStreamable feedback)\n{\n    throw new NotImplementedException();\n}\n<\/pre><\/div>\n\n\n
        \n
      1. A stream of impressions that successfully join to the feedback stream<\/li>\n\n\n\n
      2. A stream of impressions that never joined to the feedback stream (expired)<\/li>\n\n\n\n
      3. A stream of impressions immediately upon arrival, whether joined or not<\/li>\n<\/ol>\n\n\n\n
        Let\u2019s tackle these one at a time.<\/p>\n\n\n\n
        1. A stream of impressions that successfully join to the feedback stream<\/h3>\n\n\n\n
        This stream is the easiest \u2013 an inner join produces exactly what we want. We also must consider the allowed delays of the system \u2013 up to 30 minutes on either side, plus 10 additional minutes on the feedback side. We can update the implementation of our method like so:<\/p>\n\n\n
        \ninternal static IStreamable JoinFeedbackToImpressions(\n    this IStreamable impressions,\n    IStreamable feedback)\n{\n-   throw new NotImplementedException();\n+   return impressions\n+       .SetDuration(TimeSpan.FromMinutes(40))\n+       .Join(\n+           feedback.SetDuration(TimeSpan.FromMinutes(30)),\n+           imp  => imp.RGUID,\n+           fdbk => fdbk.RGUID,\n+           (imp, fdbk) => new ImpressionsWithFeedback(imp, fdbk))\n+       .ToPointStream()\n+       ;\n}\n<\/pre><\/div>\n\n\n
        Note: in the above code snippet, \u201c-\u201d indicates that a line is removed and \u201c+\u201d indicates that a line is added<\/em><\/p>\n\n\n\n
        This query defines a simple inner join between the impression and feedback streams on \u201cRGUID\u201d \u2013 short for \u201crequest GUID\u201d \u2013 which serves as the join key between all logs in the Bing Ads pipeline. When the join succeeds, a new ImpressionsWithFeedback<\/strong> instance is created from the impressions and feedback that joined. When we are done, we convert the events to points since we don\u2019t care about their lifetime after this.<\/p>\n\n\n\n
        Let\u2019s see how this result compares to our original goal. The output produced by the query is in color; missing events are in gray.<\/p>\n\n\n\n
        \"diagram<\/figure>\n\n\n\n
        Not bad, but we\u2019ve got a bit of a way to go yet.<\/p>\n\n\n\n
        2. A stream of impressions that never joined to the feedback stream (expired)<\/h3>\n\n\n\n
        The next stream is a bit more complicated because it requires us to look into the past to determine whether or not to output an event. There are two possible outcomes, depending on what happened previously.<\/p>\n\n\n\n
          \n
        1. The impression joined at some point to its feedback. In this case, we\u00a0do not<\/em>\u00a0want to output anything.<\/li>\n\n\n\n
        2. The impression never joined to its feedback. In this case, we want to produce a point event that coincides with the impression\u2019s end time.<\/li>\n<\/ol>\n\n\n\n
          Let\u2019s see how we can accomplish this by combining other temporal primitives.<\/p>\n\n\n\n
          When you see a pattern like \u201coutput an event only when some condition is satisfied,\u201d this often indicates that an antijoin should be used. The antijoin operator suppresses output of the events in a stream based on the content of another stream, so the basic strategy is to produce the desired output for every event in the input stream and then to use an antijoin to selectively suppress the ones which are not needed.<\/p>\n\n\n\n
          Side note: there is some similarity in function between the Where and LeftAntiJoin\/RightAntiJoin operators in that they both cause events to be dropped from their input. However, there are some important differences.<\/p>\n\n\n\n
            \n
          • The Where operator is a scalar (payload-based) operator that filters events according to a user-supplied predicate function. If the filter is not satisfied, the entire event is dropped.<\/li>\n\n\n\n
          • The LeftAntiJoin\/RightAntiJoin operators are temporal operators that suppress the output of an event as long as it matches an event in another stream. Notably, a matching event does not cause the input event to be dropped completely. It instead is \u201cturned off\u201d for the duration of the match, after which it can become active again.<\/li>\n<\/ul>\n\n\n\n
            In this case, since we need to suppress events based on whether they joined successfully in the past, we must use an antijoin.<\/p>\n\n\n\n
            Here is how we can modify the query to produce a stream of expired events.<\/p>\n\n\n
            \ninternal static IStreamable JoinFeedbackToImpressions(\n    this IStreamable impressions,\n    IStreamable feedback)\n{\n+   \/\/ set the lifetime of the input streams\n+   impressions = impressions.SetDuration(TimeSpan.FromMinutes(40));\n+   feedback    = feedback   .SetDuration(TimeSpan.FromMinutes(30));\n+\n+   \/\/ produce a stream of successful joins\n-   return impressions\n-       .SetDuration(TimeSpan.FromMinutes(40))\n+   IStreamable joined = impressions\n        .Join(\n-           feedback.SetDuration(TimeSpan.FromMinutes(30)),\n+           feedback,\n            imp  => imp.RGUID,\n            fdbk => fdbk.RGUID,\n            (imp, fdbk) => new ImpressionsWithFeedback(imp, fdbk))\n        .ToPointStream()\n        ;\n+\n+   \/\/ produce a stream of \"expired\" point events that coincide with the end time\n+   \/\/ of the impression stream, and suppress those which had a successful join\n+   IStreamable expired = impressions\n+       .PointAtEnd()\n+       .LeftAntiJoin(\n+           joined.SetDuration(TimeSpan.FromMinutes(40)),\n+           l => l.RGUID,\n+           r => r.RGUID)\n+       .Select(imp => new ImpressionsWithFeedback(imp, null))\n+       ;\n+\n+   \/\/ union together the two streams and return the result\n+   return joined.Union(expired);\n}\n<\/pre><\/div>\n\n\n
            Note: in the above code snippet, \u201c-\u201d indicates that a line is removed and \u201c+\u201d indicates that a line is added<\/em><\/p>\n\n\n\n
            There are a lot of changes, so let\u2019s talk about them.<\/p>\n\n\n\n
            First, I pulled out the SetDuration calls so that they could be reused without being repeated. For simplicity\u2019s sake, I reused the same local variable for each alteration. I could have instead defined a new IStreamable<T><\/strong> variable for each one; the result is the same.<\/p>\n\n\n\n
            Second, instead of returning the join result immediately, I created a joined stream that contains the result of the join. This is used in the expired event computation and is also returned at the end of the method.<\/p>\n\n\n\n
            Third, I computed a stream of expired events. As explained above, this creates an expired event for every impression (impressions.PointAtEnd()) and then suppresses those which are preceded by a successful join (LeftAntiJoin(joined\u2026)).  But what about the SetDuration operator being applied to joined on the right side of the antijoin?<\/p>\n\n\n\n
            It may help to visualize what is going on here. The below chart shows how each case will behave in this subquery:<\/p>\n\n\n\n
            \"chart<\/figure>\n\n\n\n
            As you can see, the lifetime of the events in joined changes significantly depending on the relative position of the input events in time. To ensure that we correctly suppress the appropriate expired events, the lifetime of the events in the filtering stream must always overlap with the events in the expired stream.<\/p>\n\n\n\n
            In the worst case, this means that the join lifetime needs to be extended by 40 minutes, which happens to be the same as the lifetime of the impressions going into the join. (This is not a coincidence; proof of this is left as an exercise for the reader.)<\/p>\n\n\n\n
            Notice that after changing the duration of the joined events, all of them overlap with the point at the end of the impression\u2019s lifetime. When these two streams are antijoined, this will cause the output to be suppressed in all cases except for No feedback<\/em>, which is exactly what we want.<\/p>\n\n\n\n
            After making the above changes, we add the expired events to the output by unioning the joined and expired streams.<\/p>\n\n\n\n
            Now let\u2019s look again at the result after making these changes.<\/p>\n\n\n\n
            \"diagram<\/figure>\n\n\n\n
            \u201cWait a minute,\u201d you say, \u201cthat looks almost exactly the same as it did before!\u201d You are right \u2013 they are almost <\/em>the same. The only difference is an additional event in the No feedback<\/em> case which coincides with the end of the impression, the expired event.<\/p>\n\n\n\n
            This brings us to the end of Part 1. In Part 2, we will show how to find out impressions that don\u2019t ever have a joined feedback. We will then put all 3 cases together into a single query that answer our original question.<\/p>\n\n\n\n
            Questions in the meantime? Let us know in the comments?<\/p>\n","protected":false},"excerpt":{"rendered":"
            On the Microsoft BingAds team, one of my primary responsibilities is the development and maintenance of the FastBI pipeline \u2013 the system responsible for all revenue coming from the Bing search engine.<\/p>\n","protected":false},"author":5562,"featured_media":95467,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"msxcm_post_with_no_image":false,"ep_exclude_from_search":false,"_classifai_error":"","_classifai_text_to_speech_error":"","footnotes":""},"post_tag":[2272],"content-type":[340],"topic":[2238],"programming-languages":[],"coauthors":[609,621],"class_list":["post-77079","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","tag-microsoft","content-type-tutorials-and-demos","topic-ai-machine-learning","review-flag-1593580428-734","review-flag-1593580415-931","review-flag-1593580419-521","review-flag-1-1593580432-963","review-flag-2-1593580437-411","review-flag-3-1593580442-169","review-flag-alway-1593580310-39","review-flag-exclu-1593580297-613","review-flag-never-1593580314-283","review-flag-new-1593580248-669"],"yoast_head":"\nHow to use Trill for impression feedback (part 1) | Microsoft Open Source Blog<\/title>\n<meta name=\"description\" content=\"In this 2-part series, I\u2019ll introduces a challenging scenario faced by the BingAds team, followed by a detailed discussion of how Trill addressed it.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to use Trill for impression feedback (part 1) | Microsoft Open Source Blog\" \/>\n<meta property=\"og:description\" content=\"In this 2-part series, I\u2019ll introduces a challenging scenario faced by the BingAds team, followed by a detailed discussion of how Trill addressed it.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/\" \/>\n<meta property=\"og:site_name\" content=\"Microsoft Open Source Blog\" \/>\n<meta property=\"article:published_time\" content=\"2019-05-29T16:00:08+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-27T13:54:23+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2024\/06\/CLO20b_Garrett_Preeti_Jayesh_Aline_001_v2.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1170\" \/>\n\t<meta property=\"og:image:height\" content=\"640\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Daniel Musgrave, Zhong Chen\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@OpenAtMicrosoft\" \/>\n<meta name=\"twitter:site\" content=\"@OpenAtMicrosoft\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel Musgrave, Zhong Chen\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 min read\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/\"},\"author\":[{\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/author\/daniel-musgrave\/\",\"@type\":\"Person\",\"@name\":\"Daniel Musgrave\"},{\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/author\/zhong-chen\/\",\"@type\":\"Person\",\"@name\":\"Zhong Chen\"}],\"headline\":\"How to use Trill for impression feedback (part 1)\",\"datePublished\":\"2019-05-29T16:00:08+00:00\",\"dateModified\":\"2025-06-27T13:54:23+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/\"},\"wordCount\":2082,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2024\/06\/CLO20b_Garrett_Preeti_Jayesh_Aline_001_v2.webp\",\"keywords\":[\"Microsoft\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/\",\"url\":\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/\",\"name\":\"How to use Trill for impression feedback (part 1) | Microsoft Open Source Blog\",\"isPartOf\":{\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2024\/06\/CLO20b_Garrett_Preeti_Jayesh_Aline_001_v2.webp\",\"datePublished\":\"2019-05-29T16:00:08+00:00\",\"dateModified\":\"2025-06-27T13:54:23+00:00\",\"description\":\"In this 2-part series, I\u2019ll introduces a challenging scenario faced by the BingAds team, followed by a detailed discussion of how Trill addressed it.\",\"breadcrumb\":{\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#primaryimage\",\"url\":\"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2024\/06\/CLO20b_Garrett_Preeti_Jayesh_Aline_001_v2.webp\",\"contentUrl\":\"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2024\/06\/CLO20b_Garrett_Preeti_Jayesh_Aline_001_v2.webp\",\"width\":1170,\"height\":640},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/opensource.microsoft.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How to use Trill for impression feedback (part 1)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/#website\",\"url\":\"https:\/\/opensource.microsoft.com\/blog\/\",\"name\":\"Microsoft Open Source Blog\",\"description\":\"Open dialogue about openness at Microsoft \u2013 open source, standards, interoperability\",\"publisher\":{\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/opensource.microsoft.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/#organization\",\"name\":\"Microsoft Open Source Blog\",\"url\":\"https:\/\/opensource.microsoft.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2019\/08\/Microsoft-Logo.png\",\"contentUrl\":\"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2019\/08\/Microsoft-Logo.png\",\"width\":259,\"height\":194,\"caption\":\"Microsoft Open Source Blog\"},\"image\":{\"@id\":\"https:\/\/opensource.microsoft.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/OpenAtMicrosoft\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How to use Trill for impression feedback (part 1) | Microsoft Open Source Blog","description":"In this 2-part series, I\u2019ll introduces a challenging scenario faced by the BingAds team, followed by a detailed discussion of how Trill addressed it.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/","og_locale":"en_US","og_type":"article","og_title":"How to use Trill for impression feedback (part 1) | Microsoft Open Source Blog","og_description":"In this 2-part series, I\u2019ll introduces a challenging scenario faced by the BingAds team, followed by a detailed discussion of how Trill addressed it.","og_url":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/","og_site_name":"Microsoft Open Source Blog","article_published_time":"2019-05-29T16:00:08+00:00","article_modified_time":"2025-06-27T13:54:23+00:00","og_image":[{"width":1170,"height":640,"url":"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2024\/06\/CLO20b_Garrett_Preeti_Jayesh_Aline_001_v2.png","type":"image\/png"}],"author":"Daniel Musgrave, Zhong Chen","twitter_card":"summary_large_image","twitter_creator":"@OpenAtMicrosoft","twitter_site":"@OpenAtMicrosoft","twitter_misc":{"Written by":"Daniel Musgrave, Zhong Chen","Est. reading time":"9 min read"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#article","isPartOf":{"@id":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/"},"author":[{"@id":"https:\/\/opensource.microsoft.com\/blog\/author\/daniel-musgrave\/","@type":"Person","@name":"Daniel Musgrave"},{"@id":"https:\/\/opensource.microsoft.com\/blog\/author\/zhong-chen\/","@type":"Person","@name":"Zhong Chen"}],"headline":"How to use Trill for impression feedback (part 1)","datePublished":"2019-05-29T16:00:08+00:00","dateModified":"2025-06-27T13:54:23+00:00","mainEntityOfPage":{"@id":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/"},"wordCount":2082,"commentCount":0,"publisher":{"@id":"https:\/\/opensource.microsoft.com\/blog\/#organization"},"image":{"@id":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#primaryimage"},"thumbnailUrl":"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2024\/06\/CLO20b_Garrett_Preeti_Jayesh_Aline_001_v2.webp","keywords":["Microsoft"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/","url":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/","name":"How to use Trill for impression feedback (part 1) | Microsoft Open Source Blog","isPartOf":{"@id":"https:\/\/opensource.microsoft.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#primaryimage"},"image":{"@id":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#primaryimage"},"thumbnailUrl":"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2024\/06\/CLO20b_Garrett_Preeti_Jayesh_Aline_001_v2.webp","datePublished":"2019-05-29T16:00:08+00:00","dateModified":"2025-06-27T13:54:23+00:00","description":"In this 2-part series, I\u2019ll introduces a challenging scenario faced by the BingAds team, followed by a detailed discussion of how Trill addressed it.","breadcrumb":{"@id":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#primaryimage","url":"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2024\/06\/CLO20b_Garrett_Preeti_Jayesh_Aline_001_v2.webp","contentUrl":"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2024\/06\/CLO20b_Garrett_Preeti_Jayesh_Aline_001_v2.webp","width":1170,"height":640},{"@type":"BreadcrumbList","@id":"https:\/\/opensource.microsoft.com\/blog\/2019\/05\/29\/how-to-use-trill-for-impression-feedback-part-1\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/opensource.microsoft.com\/blog\/"},{"@type":"ListItem","position":2,"name":"How to use Trill for impression feedback (part 1)"}]},{"@type":"WebSite","@id":"https:\/\/opensource.microsoft.com\/blog\/#website","url":"https:\/\/opensource.microsoft.com\/blog\/","name":"Microsoft Open Source Blog","description":"Open dialogue about openness at Microsoft \u2013 open source, standards, interoperability","publisher":{"@id":"https:\/\/opensource.microsoft.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/opensource.microsoft.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/opensource.microsoft.com\/blog\/#organization","name":"Microsoft Open Source Blog","url":"https:\/\/opensource.microsoft.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/opensource.microsoft.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2019\/08\/Microsoft-Logo.png","contentUrl":"https:\/\/opensource.microsoft.com\/blog\/wp-content\/uploads\/2019\/08\/Microsoft-Logo.png","width":259,"height":194,"caption":"Microsoft Open Source Blog"},"image":{"@id":"https:\/\/opensource.microsoft.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/OpenAtMicrosoft"]}]}},"msxcm_display_generated_audio":false,"msxcm_animated_featured_image":null,"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Microsoft Open Source Blog","distributor_original_site_url":"https:\/\/opensource.microsoft.com\/blog","push-errors":false,"_links":{"self":[{"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/posts\/77079","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/users\/5562"}],"replies":[{"embeddable":true,"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/comments?post=77079"}],"version-history":[{"count":1,"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/posts\/77079\/revisions"}],"predecessor-version":[{"id":97751,"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/posts\/77079\/revisions\/97751"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/media\/95467"}],"wp:attachment":[{"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/media?parent=77079"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/post_tag?post=77079"},{"taxonomy":"content-type","embeddable":true,"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/content-type?post=77079"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/topic?post=77079"},{"taxonomy":"programming-languages","embeddable":true,"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/programming-languages?post=77079"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/opensource.microsoft.com\/blog\/wp-json\/wp\/v2\/coauthors?post=77079"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}