Spring 2021 - Thu 3:10-6:00 PM, Peking University
This course covers the fundamentals, research topics and applications of deep generative models.
Schedule
Fundamentals
| Week 1 | Introduction | Lecture 1: Introduction Lecture 2: Data Representation Lecture 3: Mathematic Foundation & Basic Concept |
| Week 2 | Autoregressive Models | Lecture 4: Sequential Models - Recurrent Neural Networks Lecture 5: Autoregressive Models 1 Lecture 6: Autoregressive Models 2 |
| Week 3 | Variational Autoencoders | Lecture 7: From Autoencoder to VAE Lecture 8: Variational Autoencoder Lecture 9: VAE Variants |
| Week 4 | Normalising Flow Models | Lecture 10: Normalising Flow Background Lecture 11-12: Normalising Flow Models |
| Week 5 | Generative Adversarial Networks | Lecture 13: Introduction of GAN Lecture 14: Understanding GAN Lecture 15: Selected GANs |
| Week 6 | Practice | Lecture 16-18: Practice: VAE and GAN Lecture 16-18: Demo Code |
Research & Application
| Week 7 | Evaluation of Generative Models | Lecture 19: Sampling Quality Lecture 20: Density Evaluation & Latent Representation Lecture 21: Practice |
| Week 8 | Energy-based Models | Lecture 22: Hopfield Network Lecture 23: Boltzmann Machine Lecture 24: Energy-based GANs |
| Week 9 | Challenges of Generative Models | Lecture 25: High-dimensional Data Generation Lecture 26: Learning Large Encoder Lecture 27: Other Challenges |
| Week 10 | Applications of Generative Models | Lecture 28: Image Synthesis, Translation and Manipulation Lecture 29: X Learning Lecture 30: Advanced Topics |
Practices
| Week 11 | Paper Reading | EMNLP2020: Learning VAE-LDA Models with Rounded Reparameterization Trick ICLR2021: Zero-shot Synthesis with Group-Supervised Learning CVPR2020: Analyzing and Improving the Image Quality of StyleGAN CVPR2020: Interpreting the Latent Space of GANs for Semantic Face Editing CVPR2020:Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models CVPR2021: PISE: Person Image Synthesis and Editing with Decoupled GAN NIPS2020: A Causal View of Compositional Zero-Shot Recognition ACL2017: Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders |
| Week 12 | Paper Reading | ICLR2021： A Distributional Approach to Controlled Text Generation Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space ICLR2021: Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering CVPR2021 Few-shot Image Generation via Cross-domain Correspondence ICLR2021 On Self-Supervised Image Representations for GAN Evaluation JCP：Physics-informed semantic inpainting: Application to geostatistical modeling StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery ECCV2020: Contrastive Learning for Unpaired Image-to-Image Translation |
| Week 13 | Paper Reading | ICLR20: Plug and Play Language Models: A Simple Approach to Controlled Text Generation ECCV2020: Unpaired Image-to-Image Translation using Adversarial Consistency Loss NIPS2018: A Unified Feature Disentangler for Multi-Domain Image Translation and Manipulation CVPR2021: Domain Generalization via Inference-time Label-Preserving Target Projections ICLR2020: low-resource knowledge- grounded dialogue generation CoCon: A Self-Supervised Approachfor Controlled Text Generation CVPR2020：PointAugment an Auto-Augmentation Framework for Point Cloud Classification Cell System 20 A Generative Neural Network for Maximizing Fitness and Diversity of Synthetic DNA and Protein Sequences |
| Week 14 | Group Projects | Taming transformers for High-Resolution Image Synthesis Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One Syntax-Guided Grammatical Error Correction Model SRFlow: Learning the Super-Resolution Space with Normalizing Flow GAN-BERT on Virus host prediction Improved Image2StyleGAN Controlable Sentence Simplification Implicit Normalizing Flows |
| Week 15 | Group Projects | Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation PointAugment: an Auto-Augmentation Framework for Point Cloud Classification Neural Text Generation with Part-of-Speech Guided Softmax Adapt iterative back translation in AMR Parsing Contrastive Learning for Unpaired Image-to-Image Translation Abstractive Dialog Summarization Efficient Upscaling of Geologic Model based on Theory-guided Encoder-Decoder CycleGAN for Domain Adaptation in Medical Imaging |
| Week 16 | Group Projects | GAN-CodeBERT Story of Face Swapping Hippop-Transformer: Towards Rhymed Chinese Lyric Generation Low/Zero-Resource Knowledge-Grounded Dialogue Generation Transfer Learning with Domain Transfer Networks BSP-CVAE for mesh generation Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models Exploring Degree Control of General-Text-Driven Manipulation of StyleGAN Imagery GraphLVAE Hierarchical and Spatial VAE |
Course Staff
- Instructor: Hao Dong
- Teaching Assistants: Churan Wang churanwang@pku.edu.cn
Feedback
For questions, please discuss on the Wechat group. You can also email Dr. Dong at hao.dong@pku.edu.cn.
Others
