數(shù)據(jù)與模型安全 課件 第9周：隱私攻擊和防御

PrivacyAttacks

&

Defenses姜育剛，馬興軍，吳祖煊SalvadorDalí,“ThePersistenceofMemory,”1931Recap:

week

8Data

Extraction

Attack

&

DefenseModel

Stealing

AttackFuture

ResearchThis

WeekMembership

Inference

AttackDifferential

PrivacyMembership

Inference

AttackDifferential

PrivacyMembership

Inference

AttackMembership

Inference

Attack推理一個輸入樣本是否存在于訓練數(shù)據(jù)集中Shokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."

S&P,2017.存在？Privacy

and

Ethical

Problems

MIA

could

cause

the

following

harms:Leak

private

info:

someone

has

been

to

some

place

or

having

an

unspeakable

illness

Expose

info

about

the

training

dataMIA

sensitivity

also

indicates

data

leakage

riskAn

Early

WorkHomer,Nils,etal."ResolvingindividualscontributingtraceamountsofDNAtohighlycomplexmixturesusinghigh-densitySNPgenotypingmicroarrays."

PLoSgenetics

4.8(2008):e1000167.判斷個人基因是否出現(xiàn)在一個復雜的混合基因里可用于調查取證MIA：The

Most

Well-known

WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."

S&P,2017.0Black-box

attack

pipelineNeeds

probability

vectorMIA：The

Most

Well-known

WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."

S&P,2017.Train

k

shadow

models

on

disjoint

datasetsSample

a

number

of

subsets

from

DTrain

a

model

on

each

of

the

subsetTake

one

model

as

the

targetTake

the

rest

models

as

shadow

modelsMIA：The

Most

Well-known

WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."

S&P,2017.Different

ways

to

get

the

training

data：Random

SynthesisData

synthesisPhase

1:

searching

for

high

confidence

data

points

in

the

data

spacePhase

2:

samplesyntheticdatafromthesepointsRepeat

the

above

for

each

class

cPhase

1:每次只改變已找到的高置信度樣本的k個特征MIA：The

Most

Well-known

WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."

S&P,2017.Statistics-basedsynthesisPrior

knowledge:The

marginal

distribution

w.r.t.

each

classPhase

1:

sample

according

to

the

statisticsMIA：The

Most

Well-known

WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."

S&P,2017.We

could

also

assume

the

attacker

can

access

NoisyRealdata:

real

but

noisyVery

similar

to

the

real

datasetBut

with

a

few

features

(10%

or

20%)

are

randomly

resetMIA：The

Most

Well-known

WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."

S&P,2017.Finally:

training

the

inference

model”in”:

in

the

training

set”out”:

:

in

the

test

setTrain

the

inference

model

with

dataset:

(prob1,

”in”),

(prob2,

”in”),

(prob3,

”out”)

(prob4,

”out”)MIA：The

Most

Well-known

WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."

S&P,2017.How

well

can

MIA

perform?數(shù)據(jù)集：CIFAR-10、CIFAR-100、Purchases、Locations、Texashospitalstays、MNIST、UCIAdult(CensusIncome).White-box

MIANasr

et

al.“Comprehensiveprivacyanalysisofdeeplearning:Passiveandactivewhite-boxinferenceattacksagainstcentralizedandfederatedlearning.”

S&P,2019.

Hu,Hongsheng,etal."Membershipinferenceattacksonmachinelearning:Asurvey."

ACMComputingSurveys(CSUR)

54.11s(2022):1-37.White-boxvs

Black-boxWhite-box

MIANasr

et

al."Comprehensiveprivacyanalysisofdeeplearning:Passiveandactivewhite-boxinferenceattacksagainstcentralizedandfederatedlearning."

S&P,2019.抽取特征：概率、中間層激活、梯度無監(jiān)督設置下的重構損失推理結果Limitations

of

MIAConstructing

shadow

modelsAssuming

access

to

some

data

or

prior

knowledgeOverfitting

is

a

mustLimited

to

classification

modelsLimited

to

small

modelsAddressing

Limitations

of

MIASalemetal."ML-Leaks:ModelandDataIndependentMembershipInferenceAttacksandDefensesonMachineLearningModels."

NDSS,2019.Model

and

Data

Independent

MIAAddressing

Limitations

of

MIALong,Yunhui,etal."Apragmaticapproachtomembershipinferencesonmachinelearningmodels."

EuroS&P,2020.Attacking

non-overfitting

DNNsFocusing

on

minimizingfalsepositives目標問題：樣本A/B在哪個模型的訓練數(shù)據(jù)里？Addressing

Limitations

of

MIALeino

&

Fredrikson."StolenMemories:LeveragingModelMemorizationforCalibratedWhite-BoxMembershipInference."

USENIXSecurity,2020.More

practical

white-box

threat

modelThe

adversary

only

knows

the

model

but

not

the

data

distribution利用詭異的獨家記憶進行成員推理Training

imagesInternal

explanations

Pink

background

explanation

of

Tony

BlairAddressing

Limitations

of

MIAHayes,Jamie,etal."Logan:Membershipinferenceattacksagainstgenerativemodels."

arXivpreprintarXiv:1705.07663

(2017).Extension

to

generative

models充分利用判別器的判別能力：高置信度的大概率來自原始訓練數(shù)據(jù)集Metric-guided

MIAYeom,Samuel,etal.“Privacyriskinmachinelearning:Analyzingtheconnectiontooverfitting.”

CSF,

2018.

Salemetal."ML-Leaks:ModelandDataIndependentMembershipInferenceAttacksandDefensesonMachineLearningModels."

NDSS,2019.Metric

based

Anomaly

detection預測正確性：預測正確的就是成員預測損失：高于訓練樣本平均損失的是成員預測置信度：有概率接近1的是成員預測熵：低概率熵的是成員修正預測熵：不同類別區(qū)別考慮A

Summary

of

Existing

MIAsUsed

DatasetsImage:CIFAR-10,CIFAR-100,MNIST,Fashion-MNIST,YaleFace,ChestX-ray8,SVHN,CelebA,ImageNetTabulate:Adult,Foursquare,Purchase-100,Texas100,Location,etc.Audio:LibriSpeech,TIMIT,TED

Text:Weibo,TweetEmoInt,SATED,Dislogs,Redditcomments,Cora,

Pubmed,CitesserHu,Hongsheng,etal.“Membershipinferenceattacksonmachinelearning:A

survey.”

ACMComputingSurveys,

2022.A

Summary

of

Existing

MIAsTargetmodels:Onimage:Multi-layerCNN+1or2FC(>5papersused2-4layersCNN)Alexnet,ResNet18,ResNet50,VGG16,VGG19,DenseNet121,Efficient-netv2,EfficientNetB0GAN:InfoGAN,PGGAN,WGANGP,DCGAN,MEDGAN,andVAEGANOntabulate

data:FConlymodelsOntext:Multi-layerCNN,multi-layerRNN/LSTM,

transformers(e.g.,BERT,GPT-2)Onaudio:Hybridsystem:HMM-DNNmodelEnd-to-end:Multi-layerLSTM/RNN/GRUMLaaS(Online):GooglePredictionAPI,AmazonMLMembership

Inference

AttackDifferential

PrivacyDifferential

PrivacyFinite

Difference

and

Derivativeh

tends

to

be

small(zero)通過函數(shù)在某一點隨微小擾動的變化可以估計在這一點的梯度如果對數(shù)據(jù)集進行微小擾動呢？Differential

PrivacyFinite

Difference

->

Differential

Privacy數(shù)據(jù)集的微小變化會導致多大的算法輸出變化？

函數(shù)

輸入值

Differential

Privacy

數(shù)據(jù)集的微小變化會導致多大的算法輸出變化？

Differential

Privacy

Dwork,Cynthia."Differentialprivacy:Asurveyofresults."

ICTAMC,Heidelberg,2008.Properties

of

DPMcSherry,FrankD.“Privacyintegratedqueries:anextensibleplatformforprivacy-preservingdataanalysis.”

ACM

SIGMOD,2009.How

to

Obtain

a

Differentially

Private

Model?思考：如何讓自己的聲音不被發(fā)現(xiàn)？？Measuring

SensitivityNissimandAdam.“Smoothsensitivityandsamplinginprivatedataanalysis.”

STOC,2007.Noise

Models幾種噪聲添加機制拉普拉斯機制(Laplacian)高斯機制(Gaussian)指數(shù)機制：離散->

概率；確定->不確定The

Laplace

Mechanism拉普拉斯機制(Laplace

Mechanism)

SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyThe

Laplace

Mechanism拉普拉斯機制(Laplace

Mechanism)

SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyThe

Laplace

Mechanism

SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyLaplace

vs.

Gaussian

SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyDP

+

Deep

Learning問題：在哪里添加噪聲？輸入空間模型空間輸出空間輸入空間DP差分隱私預處理訓練數(shù)據(jù)dp-GAN

pipelineZhang

et

al.“Differentiallyprivatereleasingviadeepgenerativemodel(technicalreport).”

arXiv:1801.01594

(2018).輸入空間DP隨機平滑Randomized

Smoothing隨機平滑：可驗證對抗防御Cohen,Jeremy,ElanRosenfeld,andZicoKolter."Certifiedadversarialrobustnessviarandomizedsmoothing."

ICML,2019.用隨機噪聲填充輸入空間，得到對抗魯棒性邊界模型空間DPAbadi,Martin,etal.“Deeplearningwithdifferentialprivacy.”

CCS,

2016.差分隱私平滑模型參數(shù)：DP-SGD算法DP-SGD性能SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyDP-SGD性能SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyDP-SGD性能SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyMore

Practical

Solution?1:

Training

on

public

dat