Privacy problems and attacks in Machine Learning

July 19,2024

last update: 2024-11-19

Privacy problems and attacks in Machine Learning

• Membership inference attacks: These aim to determine if a particular data point was used to train a model. Researchers study how to detect and mitigate such attacks to protect individuals’ privacy.
• Model inversion attacks: These attempt to reconstruct training data from model outputs. Understanding these vulnerabilities helps in developing more robust privacy-preserving techniques.
• Data reconstruction in federated learning: This examines how malicious participants could potentially extract private information from other participants during collaborative training.
• Model extraction attacks: These involve extracting a model’s architecture or parameters, which could be used to launch further attacks or to steal intellectual property.
• Data Poisoning in Edge ML: This is a problem where the data that is being used to train the model is poisoned by an attacker. This is a huge problem in edge ML where the data is not curated and is coming from a variety of sources.
• Differential privacy techniques: Researchers explore how to add controlled noise to data or model outputs to preserve privacy while maintaining utility.
• Secure multi-party computation: This involves developing protocols for multiple parties to jointly compute on private data without revealing individual inputs. Homomorphic encryption: This allows computations on encrypted data, potentially enabling privacy-preserving machine learning on sensitive information. Privacy-preserving record linkage: Techniques to securely combine datasets from multiple sources without compromising individual privacy.

Papers and other readings

• Membership Inference Attacks Against Machine Learning Models

  • MIA is an attack vector against ML models where the original dataset is inferred from the model’s output distribution. This leverages a very curious failure of ML models where they overfit to previously seen data. Based on the confidence scores from the models it is possible to rebuild the original dataset. This has privacy impact in that, it is possible to rebuild a dataset from a model that was trained on it. This is a huge privacy violation.
  • In this paper the authors train an ML model to look at the target model’s predictions on the inputs it trained on versus the input it did not train on.

• Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures

  • This paper showcase attack is done on Facial Recognition Models, where the attacker is able to figure out the original face on ML models with different ML models