Movie posters signal the style and content of the films. By including the visual information, the movie recommend system can better filter preferences according to user's choices. A deep learning recommend system, Neural Collaborative Filtering [1], obtain a great performance by using neural networks to learn user-item embedding features. The authors propose three networks, which are Generalized Matrix Factorization (GMF), Multi-Layer Perceptron (MLP), and Neural Matrix Factorization (NeuMF). This repo include visual information into the models and propose VGMF and VNeuMF. The following image is the model architechture of VNeuMF:
The authors of NCF published a nice implementation written in tensorflow(keras), and LaceyChen17 provides the implementation written in pytorch. This repo implement VGMF and VNeuMF model based on LaceyChen17's code.
[1] He, Xiangnan, et al. "Neural collaborative filtering." Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 2017.
The Movielens 1M Datasets is used to test the repo.
We scrape the movie poster images from IMDB website and use Resnet152 (pretrained on ImageNet) to obtain our movie poster embedding features. The correspondence movieId to imdbId is in the file links.csv. (or you can just download the embeddings with download.py)
- Clone this repository:
$ git clone --recursive https://github.com/yyyjoe/neural-collaborative-filtering.git-
Go to src directory.
-
Download movie poster embedding features:
$ python download.py- Train the model (checkpoints saved in
./checkpoints/MODEL_NAME/)
# python train.py --model MODEL_NAME(gmf, vgmf, neumf, vneumf)
$ python train.py --model gmf- You can modify the training setting in
config_factory.py
All models are trained from scratch with constant learning rate. Better performance can be achieved with careful tuning hyper params.
Experiments' results with num_negative_samples = 4 and dim_latent_factor=4 are shown as follows
