參考講稿圖像理解face dnn

1、Deep Learning For FaceXuejin ChenFace Recognition Timeline889092949698100102Face Recognition AccuracyDeepIDCVPR1497.45DeepFaceCVPR1497.25TL Joint BayesianICCV1396.33high-dim LBPCVPR1395.1792.42Joint BayesianECCV12ConvNet-RBMICCV1392.5292.5893.03CMD+SLBPTR11Fisher vector facesBMVC1393.30Tom-vs-PeteBM

2、VC12human97.5399.6599.6099.53DeepID3arXiv 15Tencent2014FaceNetCVPR 2015Face Recognition Rate Deep Learning Face Representation from Predicting 10,000 Classes, CVPR 14 Yi Sun DeepFace: Closing the Gap to Human-Level Performance in Face Verification, CVPR 14 Deep Learning Face Representation from Pred

3、icting 10,000 Classes Yi Sun, Xiaogang Wang, Xiaoou Tang, CUHK, CVPR 14 Deep hidden IDentity features (DeepID) High-level feature representation Can be generalized into other tasks Feature: the last hidden layer Weekly aligned faces Deep Learning Face Representation from Predicting 10,000 Classesk =

4、 3 for color patches k = 1 for gray patchesDeep Learning Face Representation from Predicting 10,000 Classes Convolutional layer Max-pooling layer Number of feature mapsKernel size ReLU nonlinearityDeep Learning Face Representation from Predicting 10,000 ClassesDeep Learning Face Representation from

5、Predicting 10,000 Classes The last hidden layer of DeepID is fully connected toboth the third and fourth convolutional layers (after max-pooling) It sees multi-scale features, features in the fourth convolutional layer are more global than those in the third one The fourth convolutional layer contai

6、ns too few neurons and becomes the bottleneck for information propagationDeep Learning Face Representation from Predicting 10,000 ClassesDeep Learning Face Representation from Predicting 10,000 Classes Feature Extraction Five facial landmarks Globally aligned by similarity transformation Extract 60

7、patches: 10 regions, 3 scales, rgb+gray Train 60 ConvNets, each one extract two 160-D features (horizontally flipped) The total length of DeepID is 19200 (160 x2x60) for the final face verificationDeepFaceCVPR 2014DeepFace: Face Alignment with an 3D model(a) The detected face(b) 2D-aligned crop (a)

8、Mesh with 67 fiducial points(b) Reference 3D template(e) Triangle visibility(f) Piecewise affine warpping (g) The frontalized crop(h) A new viewDeepFace: Learning Representation DeepFace: Learning Representation Convolution layers to extract low-level featuresDeepFace: Learning Representation Locall

9、y Connected Layers Different regions of an aligned image have different local statisticsDeepFace: Learning Representation Fully Connected Layers Correlations between features captured in distant parts The output of the last fully-connected layer is fed to a K-way softmaxDeepFace: Learning Representa

10、tion The features produced by this network are very sparse (75% of the feature components in the topmost layers are exactly zero). ReLU activation function Drop in F7DeepFace: Learning Representation Fully Connected Layers DatasetsSocial Face Classification (SFC) 4.4 million labeled faces 4,030 peop

11、le Each has 8001200 faces The LFW dataset 13,323 web photos 5,749 celebrities 6,000 face pairs in 10 splits.Results on SFC The capacity of the network can well accommodate the scale of 3M training image, and the network scales comfortably to more persons. Lower error could be obtained by More traini

12、ng data Deeper networkTable 1. Comparison of the classification errors on the SFC w.r.t.training dataset size and network depthResults on the LFW datasetDeepFace-single: 3d aligned, rgb inputDeepFace-align2D: 2D-aligned RGB imagesDeepFace-gradient: gray + gradient magnitude & orientationDeepFace

13、-Siamese: learn a verification metric by fine-tuning the Siameses (shared) pre-trained feature extractorDeepFace-Ensemble: all four networks togetherOverall, the DeepFace runs at 0.33 seconds per imageaccounting for image decoding, face detection and alignment, the feedforward network, and the final

14、 classification output.FaceNet CVPR 2015FaceNet Directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity Representational efficiency: State-of-the-art face recognition performance using only 128-bytes per faceFaceNet:

15、 Model Architecture End-to-end learning of the whole system FaceNet: Triplet lossHow to select triplets?Fast convergence Contribute to training FaceNet: Triplet Selection Two options Generate triplets offline every n steps using the most recent network checkpoint and computing the argmin and argmax

16、on a subset of the data. Generate triplets online This can be done by selecting the hard positive/negative exemplars from within a mini-batch.FaceNet: Triplet Selection Generate triplets online Use large mini-batches a minimal number of exemplars of any one identity is present in each mini-batch 40 per identify, use all anchor-positive pairs Randomly sample negative faces to the mini-batch Select semi-hard exampl