{"id":84863,"date":"2021-02-18T09:00:03","date_gmt":"2021-02-18T17:00:03","guid":{"rendered":"https:\/\/cloudblogs.microsoft.com\/opensource\/?p=84863"},"modified":"2025-06-23T10:51:30","modified_gmt":"2025-06-23T17:51:30","slug":"create-privacy-preserving-synthetic-data-for-machine-learning-with-smartnoise","status":"publish","type":"post","link":"https:\/\/opensource.microsoft.com\/blog\/2021\/02\/18\/create-privacy-preserving-synthetic-data-for-machine-learning-with-smartnoise\/","title":{"rendered":"Create privacy-preserving synthetic data for machine learning with SmartNoise"},"content":{"rendered":"\n

Watch<\/span> our webinar<\/span> on <\/span>Open Data Science Conference<\/span><\/a> <\/span><\/p>\n\n\n\n

Read <\/span>the white paper on SmartNoise Differential Privacy<\/span> machine learning case studies<\/span><\/a><\/p>\n\n\n\n

The COVID-19 pandemic demonstrates the tremendous importance of sufficient and relevant data for research, causal analysis, government action, and medical progress. However, for understandable data protection considerations, individuals and decision-makers are often very reluctant to share personal or sensitive data. To ensure sustainable progress, we need new practices that enable insights from personal data while reliably protecting individuals’ privacy.<\/p>\n\n\n\n

Pioneered by Microsoft Research and their collaborators, differential privacy is the gold standard for protecting data in applications that prepare and publish statistical analyses. Differential privacy provides a mathematically measurable privacy guarantee to individuals by adding a carefully tuned amount of statistical noise to sensitive data or computations. It offers significantly higher privacy protection levels than commonly used disclosure limitation practices like data anonymization. The latter increasingly shows vulnerability to re-identification attacks\u2014especially as more data about individuals become publicly available.<\/p>\n\n\n\n

SmartNoise is jointly developed by Microsoft and Harvard’s Institute for Quantitative Social Science (IQSS) and the School of Engineering and Applied Sciences (SEAS) as part of the Open Differential Privacy (OpenDP) initiative<\/a>. The platform’s initial version was released in May 2020<\/a> and comprises mechanisms for providing differentially private results to users of analytical queries to protect the underlying dataset. The SmartNoise system includes differentially private algorithms, techniques for managing privacy budgets for subsequent queries, and other capabilities.<\/p>\n\n\n\n

\"Workflow<\/figure>\n\n\n\n

<\/p>\n\n\n\n

Check out the SmartNoise website<\/a> to learn more. The code for the updated open source differential privacy platform is available on GitHub<\/a>.<\/p>\n\n\n\n

Privacy-preserving synthetic data<\/h2>\n\n\n\n

With the new release of SmartNoise, we are adding several synthesizers that allow creating differentially private datasets derived from unprotected data.<\/p>\n\n\n\n

A differentially private synthetic dataset is generated from a statistical model based on the original dataset. The synthetic dataset represents a “fake” sample derived from the original data while retaining as many statistical characteristics as possible. The essential advantage of the synthesizer approach is that the differentially private dataset can be analyzed any number of times without increasing the privacy risk. Therefore, it enables collaboration between several parties, democratizing knowledge, or open dataset initiatives.<\/p>\n\n\n\n

While the synthetic dataset embodies the original data’s essential properties, it is mathematically impossible to preserve the full data value and guaranteeing record-level privacy at the same time. Usually, we can’t perform arbitrary statistical analysis and machine learning tasks on the synthesized dataset to the same extent as it is possible with the original data. Therefore, the type of downstream job should be considered before the data is synthesized.<\/p>\n\n\n\n

For instance, the workflow for generating a synthetic dataset for supervised machine learning with SmartNoise looks as follows:<\/p>\n\n\n\n

\"High<\/figure>\n\n\n\n

<\/p>\n\n\n\n

Various techniques exist to generate differentially private synthetic data, including approaches based on deep neural networks, auto-encoders, and generative adversarial models.<\/p>\n\n\n\n

The new release of SmartNoise includes the following data synthesizers:<\/p>\n\n\n\n

Supported Synthesizer <\/span><\/th>Overview <\/span><\/th><\/tr>
Multiplicative Weights Exponential Mechanism (MWEM)<\/span> <\/span><\/td>\n
    \n
  • Achieves Differential Privacy by combining Multiplicative Weights and Exponential Mechanism techniques<\/span> <\/span><\/li>\n
  • A relatively simple but effective approach<\/span> <\/span><\/li>\n
  • Requires fewer computational resources, shorter runtime<\/span> <\/span><\/li>\n<\/ul>\n<\/td><\/tr>
Differentially Private Generative Adversarial Network (DPGAN)<\/span> <\/span><\/td>\n
    \n
  • Adds noise to the discriminator of the GAN to enforce Differential Privacy<\/span> <\/span><\/li>\n
  • Has been used with image data and electronic health records (HER)<\/span> <\/span><\/li>\n<\/ul>\n<\/td><\/tr>
Private Aggregation of Teacher Ensembles Generative Adversarial Network (PATEGAN)<\/span> <\/span><\/td>\n
    \n
  • A modification of the PATE framework that is applied to GANs to preserve Differential Privacy of synthetic data<\/span> <\/span><\/li>\n
  • Improvement of DPGAN, especially for classification tasks<\/span> <\/span><\/li>\n<\/ul>\n<\/td><\/tr>
DP-CTGAN<\/span> <\/span><\/td>\n
    \n
  • Takes the state-of-the-art CTGAN for synthesizing tabular data and applies DPSGD (the same method for ensuring Differential Privacy that DPGAN uses)<\/span> <\/span><\/li>\n
  • S<\/span>uited for tabular data, avoids issues with mode collapse<\/span> <\/span><\/li>\n
  • Can lead to e<\/span>xtensive training times<\/span> <\/span><\/li>\n<\/ul>\n<\/td><\/tr>
PATE-CTGAN<\/span> <\/span><\/td>\n
    \n
  • Takes the state-of-the-art CTGAN for synthesizing tabular data and applies PATE (the same method for ensuring Differential Privacy that PATEGAN uses)<\/span> <\/span><\/li>\n
  • S<\/span>uited for tabular data, avoids issues with mode collapse<\/span> <\/span><\/li>\n<\/ul>\n<\/td><\/tr>
Qualified<\/span> Architecture to Improve Learning (QUAIL)<\/span> <\/span><\/td>\n