Annotated Faces on TV Dataset
Main author: Timothee Cour, NEC Labs. Contributors: Ben Sapp, Ben Taskar, University of Pennsylvania.
download the whole dataset tv_data.tar.gz or a small version with just 1 TV-show tv_data_small.tar.gz to get started.
The dataset contains approximately 3,000 face images extracted from 8
episodes of the TV-show LOST, annotated with ground-truth names, along
with approximately registered faces and frame information if one wants
to re-extract the faces from the videos.
It also contains automatically extracted names using a screenplay aligned with the video closed captions.
See our Convex Learning from Partial Labels Toolbox for learning from the ambiguous supervision provided by the screenplay.
Please cite the following dataset and paper if you use it in your work:
[Learning from Ambiguously Labeled Images. Timothee Cour, Ben Sapp,
Chris Jordan, Ben Taskar. IEEE International Conference on Computer
Vision and Pattern Recognition (CVPR), 2009]
[Annotated faces on TV dataset. Timothee Cour, Ben Sapp, Ben Taskar.]
How to read the dataset
Each file (lost_season1_disc*.mat) contains annotated face data for one TV episode. The data can be read as follows using matlab:
>> load lost_season1_disc2_1.mat
movie
movie =
faces: [1x453 struct]
movieName: 'lost_season1_disc2_1'
>> movie.faces(1)
image_registered: [90x60x3 uint8]
rhull: [6 403 145 542]
number: 328
name: 'Young Jack'
image: [90x90x3 uint8]
Each face contains the following data:
number: frame number
name: groundtruth name
rhull: bounding box in the format (y1,y2,x1,x2)
image: cropped image, obtained as frame(rhull(1):rhull(2),rhull(3):rhull(4),:), where frame is the frame# number in the movie
image_registered: registered image using detected eyes, nose and mouth for the registration.
in rare cases, the name _ERROR corresponds to a non-face image.
The file lost_with_screenplay_supervision.mat contains some of the faces from all those episodes, along with ambiguous labels automatically extracted using a screenplay, using the technique presented in the above paper.
Additionally, the file fiw_data.mat contains the first 50 faces for the top 10 most frequent labels from the faces in the wild dataset, which we used in our above paper. The faces in the wild dataset is presented in the following paper:
[Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller.
Labeled Faces in the Wild: A Database for Studying Face Recognition in
Unconstrained Environments. University of Massachusetts, Amherst,
Technical Report 07-49, October, 2007.]