Initial commit of stubbed README.
This commit is contained in:
Brandon Amos
parent
caa2caa6be
commit
4eb080dce2
|
|
@ -0,0 +1,154 @@
|
|||
# FaceNet
|
||||
|
||||
This is a Python and Torch implementation of the CVPR 2015 paper
|
||||
[FaceNet: A Unified Embedding for Face Recognition and Clustering](http://www.cv-foundation.org/openaccess/content_cvpr_2015/app/1A_089.pdf)
|
||||
by Florian Schroff, Dmitry Kalenichenko, and James Philbin at Google
|
||||
using publicly available libraries and datasets.
|
||||
Torch allows CPU and CUDA
|
||||
|
||||
**Crafted by [Brandon Amos](http://bamos.github.io) in the
|
||||
[Elijah](http://elijah.cs.cmu.edu) research group at
|
||||
Carnegie Mellon University.**
|
||||
|
||||
---
|
||||
|
||||
The following example shows the workflow for a single input
|
||||
image of Sylvestor Stallone from the publicly available
|
||||
[LFW dataset](http://vis-www.cs.umass.edu/lfw/person/Sylvester_Stallone.html).
|
||||
|
||||
1. Detect faces with a pre-trained models from
|
||||
[dlib](http://blog.dlib.net/2014/02/dlib-186-released-make-your-own-object.html)
|
||||
[OpenCV](http://docs.opencv.org/master/d7/d8b/tutorial_py_face_detection.html).
|
||||
2. Transform the face for the neural network.
|
||||
This repository uses dlib's
|
||||
[real-time pose estimation](http://blog.dlib.net/2014/08/real-time-face-pose-estimation.html)
|
||||
with OpenCV's
|
||||
[affine transformation](http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/warp_affine/warp_affine.html)
|
||||
to try to make the eyes and nose appear in
|
||||
the same location on each image.
|
||||
3. Use a deep neural network to represent (or embed) the face on
|
||||
a 128-dimensional hypersphere.
|
||||
The embedding is a generic representation for anybody's face.
|
||||
Unlike other face representations, this embedding has the nice property
|
||||
that a larger distance between two face embeddings means
|
||||
that the faces are likely not of the same person.
|
||||
This trivializes clustering, similarity detection,
|
||||
and classification tasks.
|
||||
|
||||
|
||||
|
||||
# Help Wanted!
|
||||
|
||||
As the following table shows, the forefront of deep learning research
|
||||
is driven by large private datasets.
|
||||
In face recognition, there are no open source implementations or
|
||||
models trained on these datasets.
|
||||
If you have access to a large dataset, we are very interested
|
||||
in training a new FaceNet model with it.
|
||||
Please contact Brandon Amos at [bamos@cs.cmu.edu](mailto:bamos@cs.cmu.edu).
|
||||
|
||||
| Dataset | Public | #Photos | #People |
|
||||
|---|---|---|---|
|
||||
| [DeepFace](https://research.facebook.com/publications/480567225376225/deepface-closing-the-gap-to-human-level-performance-in-face-verification/) (Facebook) | No | 4.4 Million | 4k |
|
||||
| [Web-Scale Training...](http://arxiv.org/abs/1406.5266) (Facebook) | No | 500 Million | 10 Million |
|
||||
| FaceNet (Google) | No | 100-200 Million | 8 Million |
|
||||
| [FaceScrub](http://vintage.winklerbros.net/facescrub.html) | Yes | 100k | 500 |
|
||||
| [CASIA-WebFace](http://arxiv.org/abs/1411.7923) | Yes | 500k | 10k |
|
||||
|
||||
# Real-Time Web Demo
|
||||
See [our YouTube video](TODO) of using this in a real-time web application
|
||||
for face recognition.
|
||||
The source is available in [demos/www](/demos/www).
|
||||
|
||||
TODO: Screenshot
|
||||
|
||||
# Cool demo, but I want numbers. What's the accuracy?
|
||||
Even though the public datasets we trained on have orders of magnitude less data
|
||||
than private industry datasets, the accuracy is remarkably high and
|
||||
outperforms all other open-source face recognition implementations we
|
||||
are aware of on the standard
|
||||
[LFW](http://vis-www.cs.umass.edu/lfw/results.html)
|
||||
benchmark.
|
||||
We had to fallback to using the deep funneled versions for
|
||||
152 of 13233 images because dlib failed to detect a face or landmarks.
|
||||
|
||||
TODO: ROC Curve
|
||||
|
||||
This can be generated with the following commands from the root `facenet`
|
||||
directory, assuming you have downloaded and placed the raw and
|
||||
deep funneled lfw data from [here](http://http://vis-www.cs.umass.edu/lfw/)
|
||||
in `./data/lfw/raw` and `./data/lfw/deepfunneled`.
|
||||
|
||||
1. Install prerequisites as below.
|
||||
2. Preprocess the raw `lfw` images, change `8` to however many
|
||||
separate processes you want to run:
|
||||
`for N in {1..8}; do ./util/align-dlib.py data/lfw/raw align affine data/lfw/dlib-affine-sz:96 --size 96 &; done`.
|
||||
Fallback to deep funneled versions for images that dlib failed
|
||||
to align:
|
||||
`./util/align-dlib.py data/lfw/raw align affine data/lfw/dlib-affine-sz:96 --size 96 --fallbackLfw data/lfw/deepfunneled`
|
||||
3. Generate representations with `./batch-represent/main.lua -outDir evaluation/lfw.nn4.v1.reps -model models/facenet/nn4.v1.t7 -data data/lfw/dlib-affine-sz:96`
|
||||
4. Generate the ROC curve from the `evaluation` directory with `./lfw-roc.py --workDir lfw.nn4.v1.reps`.
|
||||
This creates `roc.pdf` in the `lfw.nn4.v1.reps` directory.
|
||||
|
||||
# Setup
|
||||
|
||||
## Check out git submodules
|
||||
Clone with `--recursive` or run `git submodule init && git submodule update`
|
||||
after checking out.
|
||||
|
||||
## Download the models
|
||||
Run `./models/download_models.sh` to download pre-trained FaceNet
|
||||
models on the combined CASIA-WebFace and FaceScrub database.
|
||||
This also downloads dlib's pre-trained model for face landmark detection.
|
||||
|
||||
## With Docker
|
||||
TODO
|
||||
|
||||
This repo can be deployed as a container with [Docker](https://www.docker.com/)
|
||||
for CPU mode:
|
||||
|
||||
```
|
||||
./models/download_models.sh
|
||||
sudo docker build -t facenet .
|
||||
sudo docker run -t -i -v $PWD:/facenet facenet /bin/bash
|
||||
cd /facenet
|
||||
TODO
|
||||
```
|
||||
|
||||
To use, place your images in `facenet` on your host and
|
||||
access them from the shared Docker directory.
|
||||
|
||||
## By hand
|
||||
TODO
|
||||
|
||||
Dependencies:
|
||||
+ [torch7](https://github.com/torch/torch7)
|
||||
+ [dpnn](https://github.com/nicholas-leonard/dpnn)
|
||||
+ TODO
|
||||
|
||||
Optional dependencies:
|
||||
+ CUDA 6.5+
|
||||
+ [cudnn.torch](https://github.com/soumith/cudnn.torch)
|
||||
|
||||
|
||||
# Usage
|
||||
## Existing Models
|
||||
TODO
|
||||
|
||||
# Training new models
|
||||
TODO
|
||||
[Atcold/torch-TripletEmbedding](https://github.com/Atcold/torch-TripletEmbedding)
|
||||
|
||||
# Licensing
|
||||
This source is copyright Carnegie Mellon University
|
||||
and licensed under the [Apache 2.0 License](./LICENSE).
|
||||
Portions from the following third party sources have
|
||||
been modified and are included in this repository.
|
||||
These portions are noted in the source files and are
|
||||
copyright their respective authors with
|
||||
the licenses listed.
|
||||
|
||||
Project | Modified | License
|
||||
---|---|---|
|
||||
[Atcold/torch-TripletEmbedding](https://github.com/Atcold/torch-TripletEmbedding) | No | MIT
|
||||
[facebook/fbnn](https://github.com/facebook/fbnn) | Yes | BSD
|
||||
Loading…
Reference in New Issue