Image Inpainting for Content Localization

In our last post, we trained StyleGAN2 over a corpus of hundreds of thousands theatrical posters we scraped from sites like IMDb. Then we explored image retrieval applications of StyleGAN2 after extracting embeddings by projecting our image corpus onto the learned latent factor space. Image retrieval techniques can form the basis of personalized image recommendations as we use content similarity to generate new recommendations. Netflix engineers posted about testing the impact on user engagement from artwork produced by their content creation team....

 · 3 min · Terry Rodriguez & Salma Mayorquin

Applying GAN Latent Factors for Image Retrieval

GANs consistently achieve state of the art performance in image generation by learning the distribution of an image corpus. The newest models often use explicit mechanisms to learn factored representations for images which can be help provide faceted image retrieval, capable of conditioning output on key attributes. In this post, we explore applying StyleGAN2 embeddings in image retrieval tasks. StyleGAN2 To begin, we train a StyleGAN2 model to generate theatrical posters from our image corpus....

 · 3 min · Terry Rodriguez & Salma Mayorquin

Deepfake Detection With NVIDIA TLT 3.0 and DeepStream SDK

Last year, over 2 thousand teams participated in Kaggle’s Deepfake detection video classification challenge. For this task, contestants were provided 470 GB of high resolution video and required to submit a notebook which predicts whether each sample video file has been deepfaked with a 9 hour run-time limit. Since most deepfake technology performs a faceswap, contestants concentrated around face detection and analysis. Beginning with face detection, contestants could develop an image classifier using the provided labels....

 · 5 min · Terry Rodriguez & Salma Mayorquin

Movie Trailer Similarity for Recommendation

Intro In a previous post, we discussed scraping a movie poster image corpus with genre labels from imdb and learning image similarity models using tensorflow. In this post, we extend this idea to recommend movie trailers based on audio-visual similarity. Data We started by scraping IMDB for movie trailers and their genre tags as labels. Using Scrapy, it is easy to build a text file of video links to then download with youtube-dl....

 · 4 min · Terry Rodriguez & Salma Mayorquin

Scraping Smarter with Content Filtering

Scrapy is a powerful web scraping framework and essential tool for building machine learning datasets. For sites with simple structure, scrapy makes it easy to curate a dataset after launching a spider. Check out the tutorials in scrapy’s documentation. To train a poster similarity model, we first gathered hundreds of thousands of movie posters. More concretely, when scraping IMDb.com, we may be interested in gathering posters from <img> tags under <div> tags with the class "poster"....

 · 3 min · Terry Rodriguez & Salma Mayorquin