Spring 2020 - Thu 3:00-6:00 PM, Peking University

This course covers the fundamentals, research topics and applications of deep generative models.

Schedule

Fundamentals


Week 1	Introduction	Lecture 1: Introduction Lecture 2: Data Representation Lecture 3: Mathematic Foundation & Basic Concept
Week 2	Autoregressive Models	Lecture 4: Sequential Models - Recurrent Neural Networks Lecture 5: Autoregressive Models 1 Lecture 6: Autoregressive Models 2
Week 3	Variational Autoencoders	Lecture 7: From Autoencoder to VAE Lecture 8: Variational Autoencoder Lecture 9: VAE Variants
Week 4	Normalising Flow Models	Lecture 10: Normalising Flow Background Lecture 11: Normalising Flow Models Lecture 12: Deep Learning Development
Week 5	Generative Adversarial Networks	Lecture 13: Introduction of GAN Lecture 14: Understanding GAN Lecture 15: Selected GANs
Week 6	Practice	Lecture 16: Variational Autoencoders Lecture 17: Generative Adversarial Networks Lecture 18: More: WGAN, LSGAN, VAE-GAN … GAN/VAE DCGAN CycleGAN SRGAN

Research & Application


Week 7	Evaluation of Generative Models	Lecture 19: Sampling Quality Lecture 20: Density Evaluation & Latent Representation Lecture 21: Practice
Week 8	Energy-based Models	Lecture 22: Hopfield Network Lecture 23: Boltzmann Machine Lecture 24: GANs
Week 9	Discreteness in Generative Models	Lecture 25: Discrete Sequence Generation Lecture 26: Discrete Latent Variables Lecture 27: Generating Graphs
Week 10	Challenges of Generative Models	Lecture 28: High-dimensional Data Generation Lecture 29: Learning Large Encoder Lecture 30: Other Challenges
Week 11	Applications of Generative Models	Lecture 31: Image Synthesis, Translation and Manipulation Lecture 32: X Learning Lecture 33: Advanced Topics
Week 12	Generative Model Variants	Lecture 34: GLO, IMLE, GLANN Lecture 35: Discussion Lecture 36: Practice

Practices


Week 13	Paper Reading	_{11-1 Multi-source Domain Adaptation for Semantic Segmentation, NeurIPS 2019 11-2 Generating Natural Language Adversarial Examples on a Large Scale with Generative Models, arxiv 2020 11-3 StructureNet, SIGGRAPH Asia 2019 11-4 GANimation: Anatomically-aware Facial Animation from a Single Image, ECCV 2018 11-5 Reusing Discriminators for Encoding: Towards Unsupervised Image-to-Image Translation, CVPR 2020 13-1 Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders, ACL 2017 13-2 Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation, NeurIPS 2018 13-3 PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows, ICCV 2019 13-4 Pretraining Text Encoders As Discriminators Rather Than Generators, ICLR 2020 13-5 ClusterGAN, AAAI 2019 13-6 Unified Language Model Pre-training for Natural Language Understanding and Generation, NeurIPS 2019 13-7 LSGAN: Least Squares Generative Adversarial Networks, ICCV 2017 & PAMI 2019 13-8 LeakGAN: Long Text Generation via Adversarial Training with Leaked Information, AAAI 2018 13-9 Semantic Photo Manipulation with a Generative Image Prior, SIGGRAPH 2019 13-10 StyleGAN: A Style-Based Generator Architecture for Generative, CVPR 2019}
Week 14	Paper Reading	_{14-1 Reconstruction of 3D Porous Media From 2D Slices, arXiv 2019 14-2 Levenshtein Transformer, NeurIPS 2019 14-3 PF-Net Point Fractal Network for 3D Point Cloud Completion, CVPR 2020 14-4 ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks, ECCV 2018 14-5 Synthesizing Programs for Images using Reinforced Adversarial Learning, ICML 2018 14-6 Self-supervised GAN, NeurIPS, 2019 14-7 A Closer Look at the Optimization Landscapes of GANs, ICLR 2020 14-8 Junction Tree Variational Autoencoder for Molecular Graph Generation, ICML 2018}
Week 15	Group Projects	_{15-1 CVAE-GPT2 Architecture for Diverse Responses Generation torch 15-2 MolLVAE: de novo drug design with hierarchical latent representation torch 15-3 CVAE Image Captioning torch 15-4 Attention-Constrained Transformer torch 15-5 Improving Least Squares GAN tensorlayer 15-6 Neural Machine Translation with Permutation Language Modeling torch 15-7 Personalizing Adaptation for Meta- Learning-based Dialogue Generation torch 15-8 IM-Net1 and Sampling Optimization torch 15-9 More Closer Look at the Optimization Landscapes of GANs tensorlayer}
Week 16	Group Projects	_{16-1 Application CGAN on Addictive Manufacturing torch 16-2 An effort of solving mode collapse in MolGAN tensorflow 16-3 GANimation on TensorLayer and thoughts about a better dataset tensorlayer 16-4 NB Latent Space Clustering in GAN tensorflow 16-5 A Experiment on Point Cloud Completion torch 16-6 Unsupervised Vid2Vid Translation torch 16-7 Animation Generation with Speech Signal tensorflow 16-8 Generating Natural Language Adversarial Examples tensorflow 16-9 Multi-source Domain Adaptation for Semantic Segmentation tensorflow 16-10 CT to MR translation: Stroke Detection torch 16-11 Super Resolution tensorlayer 16-12 SeqGAN in Text Generation torch}

Course Staff

Instructor: Hao Dong
Teaching Assistants: Jialei Huang, Junning Shao, Ruihai Wu, Zhijian Duan, Mingdong Wu, Xiaotian Li, Guanqi Zhang, Yihao Zhao, Hangrui Bi, Bingcan Zhao, Haoqi Yuan

Feedback

For questions, please discuss on the Wechat group. You can also email Dr. Dong at hao.dong@pku.edu.cn.