专栏首页机器学习与生成对抗网络GAN的图像修复:多样化补全

GAN的图像修复:多样化补全

2019 cvpr:Pluralistic Image Completion

https://arxiv.xilesou.top/pdf/1903.04227.pdf

https://github.com/lyndonzheng/Pluralistic-Inpainting

  • 对于每个输入(masked input),大多数图像补全方法只能产生一种结果(尽管有许多合理的其它可能结果)。基于学习的方法里,通常每个标签只有一个ground true(目标参照图象GT)。即便从有条件VAE采样、仍然会多样性不足。本文提出了一种多元化图像补全方法。
  • 提出了一种具有两个平行路径的概率学习框架。一个是重建路径(网络),它只利用对应一个的GT去获取缺失区域的先验信息,并完成重建;另一个是生成路径(网络),它将条件先验耦合于重建路径所获得的分布。在对抗方式下完成训练,训练完后只用生成路径。
  • 提出了一种新的、建模长短区域关系的注意力机制,以提高图像一致性。
  • 在建筑、人脸(Celeba-HQ)、ImageNet等数据集上不仅取得了更好的补全效果,在多样性上也合理、令人信服。

方法

  • 定义为原始完整图像,是被遮挡的(掩masked)图像,则经典的图像补全方法是去学习映射,它们是确定性的。
  • 本文还定义表示的“反”,也就是它仅仅是由被遮挡部分构成,而本文的目标是从采样去恢复。

  • 损失函数

进一步地:

  • 分布正则化(参照VAE/CVAE)

对于重建路径:

对于生成路径:

(这里的KL,个人感觉不应该带负号啊??

  • 图像外表匹配

对于重建路径,约束重建图像和目标参考GT相似:

对于生成路径,约束生成和GT相似:

  • 对抗损失

在特征层面的约束和LSGAN损失:


  • 网络结构之attention设计

在前面网络整体的图所示,decoder之前的红色模块即为本文所提出的attention设计:

部分效果


GAN图像修复/补全生成-相关阅读:

001 (2020-04-8) Attentive Normalization for Conditional Image Generation

https://arxiv.xilesou.top/pdf/2004.03828.pdf

002 (2020-03-27) GAN-based Priors for Quantifying Uncertainty

https://arxiv.xilesou.top/pdf/2003.12597.pdf

003 (2020-02-6) Automatic Segmentation and Visualization of Choroid in OCT with Knowledge Infused Deep Learning

https://arxiv.xilesou.top/pdf/2002.04712.pdf

004 (2020-02-7) Local Facial Attribute Transfer through Inpainting

https://arxiv.xilesou.top/pdf/2002.03040.pdf

005 (2020-02-5) Domain Embedded Multi-model Generative Adversarial Networks for Image-based Face Inpainting

https://arxiv.xilesou.top/pdf/2002.02909.pdf

006 (2020-02-6) Image Fine-grained Inpainting

https://arxiv.xilesou.top/pdf/2002.02609.pdf

007 (2020-02-4) Pixel-wise Conditioned Generative Adversarial Networks for Image Synthesis and Completion

https://arxiv.xilesou.top/pdf/2002.01281.pdf

008 (2020-01-11) Symmetric Skip Connection Wasserstein GAN for High-Resolution Facial Image Inpainting

https://arxiv.xilesou.top/pdf/2001.03725.pdf

009 (2019-12-31) Erase and Restore Simple Accurate and Resilient Detection of $L 2$ Adversarial Examples

https://arxiv.xilesou.top/pdf/2001.00116.pdf

010 (2019-12-18) Unsupervised Adversarial Image Inpainting

https://arxiv.xilesou.top/pdf/1912.12164.pdf

011 (2020-02-15) Image Outpainting and Harmonization using Generative Adversarial Networks

https://arxiv.xilesou.top/pdf/1912.10960.pdf

012 (2020-03-31) Image Processing Using Multi-Code GAN Prior

https://arxiv.xilesou.top/pdf/1912.07116.pdf

013 (2019-12-9) Environment reconstruction on depth images using Generative Adversarial Networks

https://arxiv.xilesou.top/pdf/1912.03992.pdf

014 (2019-12-5) Blind Inpainting of Large-scale Masks of Thin Structures with Adversarial and Reinforcement Learning

https://arxiv.xilesou.top/pdf/1912.02470.pdf

015 (2020-03-19) PiiGAN Generative Adversarial Networks for Pluralistic Image Inpainting

https://arxiv.xilesou.top/pdf/1912.01834.pdf

016 (2019-11-2) Pixel-wise Conditioning of Generative Adversarial Networks

https://arxiv.xilesou.top/pdf/1911.00689.pdf

017 (2019-10-23) Facial Expression Restoration Based on Improved Graph Convolutional Networks

https://arxiv.xilesou.top/pdf/1910.10344.pdf

018 (2019-09-27) Region-wise Generative Adversarial ImageInpainting for Large Missing Areas

https://arxiv.xilesou.top/pdf/1909.12507.pdf

019 (2019-12-23) Physics-informed semantic inpainting Application to geostatistical modeling

https://arxiv.xilesou.top/pdf/1909.09459.pdf

020 (2019-08-19) Boundless Generative Adversarial Networks for Image Extension

https://arxiv.xilesou.top/pdf/1908.07007.pdf

021 (2019-08-19) Fully Automated Image De-fencing using Conditional Generative Adversarial Networks

https://arxiv.xilesou.top/pdf/1908.06837.pdf

022 (2019-08-16) The Angel is in the Priors Improving GAN based Image and Sequence Inpainting with Better Noise and Structural Priors

https://arxiv.xilesou.top/pdf/1908.05861.pdf

023 (2019-08-14) Faster Unsupervised Semantic Inpainting A GAN Based Approach

https://arxiv.xilesou.top/pdf/1908.04968.pdf

024 (2019-10-29) Generative Modeling by Estimating Gradients of the Data Distribution

https://arxiv.xilesou.top/pdf/1907.05600.pdf

025 (2019-06-14) Single-Channel Signal Separation and Deconvolution with Generative Adversarial Networks

https://arxiv.xilesou.top/pdf/1906.07552.pdf

026 (2019-06-6) How to make a pizza Learning a compositional layer-based GAN model

https://arxiv.xilesou.top/pdf/1906.02839.pdf

027 (2019-09-28) Fashion Editing with Adversarial Parsing Learning

https://arxiv.xilesou.top/pdf/1906.00884.pdf

028 (2019-06-3) GazeCorrection Self-Guided Eye Manipulation in the wild using Self-Supervised Generative Adversarial Networks

https://arxiv.xilesou.top/pdf/1906.00805.pdf

029 (2019-05-23) Generative Imaging and Image Processing via Generative Encoder

https://arxiv.xilesou.top/pdf/1905.13300.pdf

030 (2020-01-30) Training Generative Adversarial Networks from Incomplete Observations using Factorised Discriminators

https://arxiv.xilesou.top/pdf/1905.12660.pdf

031 (2019-09-27) Invertible generative models for inverse problems mitigating representation error and dataset bias

https://arxiv.xilesou.top/pdf/1905.11672.pdf

032 (2020-03-6) PEPSI++ Fast and Lightweight Network for Image Inpainting

https://arxiv.xilesou.top/pdf/1905.09010.pdf

033 (2019-05-6) The Missing Data Encoder Cross-Channel Image Completion\\with Hide-And-Seek Adversarial Network

https://arxiv.xilesou.top/pdf/1905.01861.pdf

034 (2019-07-10) Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting

https://arxiv.xilesou.top/pdf/1904.07475.pdf

035 (2019-04-15) A deep learning framework for quality assessment and restoration in video endoscopy

https://arxiv.xilesou.top/pdf/1904.07073.pdf

036 (2019-03-28) GANs-NQM A Generative Adversarial Networks based No Reference Quality Assessment Metric for RGB-D Synthesized Views

https://arxiv.xilesou.top/pdf/1903.12088.pdf

037 (2019-11-6) Implicit Generation and Generalization in Energy-Based Models

https://arxiv.xilesou.top/pdf/1903.08689.pdf

038 (2019-03-10) A Sinogram Inpainting Method based on Generative Adversarial Network for Limited-angle Computed Tomography

https://arxiv.xilesou.top/pdf/1903.03984.pdf

039 (2019-10-23) GAN-based Projector for Faster Recovery with Convergence Guarantees in Linear Inverse Problems

https://arxiv.xilesou.top/pdf/1902.09698.pdf

040 (2019-02-10) Cross-spectral Face Completion for NIR-VIS Heterogeneous Face Recognition

https://arxiv.xilesou.top/pdf/1902.03565.pdf

041 (2019-01-25) Diversity-Sensitive Conditional Generative Adversarial Networks

https://arxiv.xilesou.top/pdf/1901.09024.pdf

042 (2019-01-9) Detecting Overfitting of Deep Generative Networks via Latent Recovery

https://arxiv.xilesou.top/pdf/1901.03396.pdf

043 (2019-01-11) EdgeConnect Generative Image Inpainting with Adversarial Edge Learning

https://arxiv.xilesou.top/pdf/1901.00212.pdf

044 (2019-07-30) Exploiting the Inherent Limitation of L0 Adversarial Examples

https://arxiv.xilesou.top/pdf/1812.09638.pdf

045 (2019-11-27) Face Completion with Semantic Knowledge and Collaborative Adversarial Learning

https://arxiv.xilesou.top/pdf/1812.03252.pdf

046 (2018-12-3) Semantic Image Inpainting Through Improved Wasserstein Generative Adversarial Networks

https://arxiv.xilesou.top/pdf/1812.01071.pdf

047 (2019-04-9) Context Encoding Chest X-rays

https://arxiv.xilesou.top/pdf/1812.00964.pdf

048 (2019-02-26) Void Filling of Digital Elevation Models with Deep Generative Models

https://arxiv.xilesou.top/pdf/1811.12693.pdf

049 (2019-01-15) SEIGAN Towards Compositional Image Generation by Simultaneously Learning to Segment Enhance and Inpaint

https://arxiv.xilesou.top/pdf/1811.07630.pdf

050 (2020-01-11) On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs

https://arxiv.xilesou.top/pdf/1811.07104.pdf

051 (2018-10-20) Improved Techniques for GAN based Facial Inpainting

https://arxiv.xilesou.top/pdf/1810.08774.pdf

052 (2018-10-15) Adversarial Inpainting of Medical Image Modalities

https://arxiv.xilesou.top/pdf/1810.06621.pdf

053 (2019-02-15) Empty Cities Image Inpainting for a Dynamic-Object-Invariant Space

https://arxiv.xilesou.top/pdf/1809.10239.pdf

054 (2018-09-6) Dense Pose Transfer

https://arxiv.xilesou.top/pdf/1809.01995.pdf

055 (2018-08-29) Chest X-ray Inpainting with Deep Generative Models

https://arxiv.xilesou.top/pdf/1809.01471.pdf

056 (2018-08-25) Painting Outside the Box Image Outpainting with GANs

https://arxiv.xilesou.top/pdf/1808.08483.pdf

057 (2019-07-18) Cross-view image synthesis using geometry-guided conditional GANs

https://arxiv.xilesou.top/pdf/1808.05469.pdf

058 (2018-08-6) X-GANs Image Reconstruction Made Easy for Extreme Cases

https://arxiv.xilesou.top/pdf/1808.04432.pdf

059 (2018-11-2) Latent Convolutional Models

https://arxiv.xilesou.top/pdf/1806.06284.pdf

060 (2018-06-13) Self-Supervised Feature Learning by Learning to Spot Artifacts

https://arxiv.xilesou.top/pdf/1806.05024.pdf

061 (2018-07-17) AVID Adversarial Visual Irregularity Detection

https://arxiv.xilesou.top/pdf/1805.09521.pdf

062 (2018-05-21) Unsupervised Deep Context Prediction for Background Foreground Separation

https://arxiv.xilesou.top/pdf/1805.07903.pdf

063 (2018-06-4) An Unsupervised Approach to Solving Inverse Problems using Generative Adversarial Networks

https://arxiv.xilesou.top/pdf/1805.07281.pdf

064 (2019-07-20) Generative Steganography by Sampling

https://arxiv.xilesou.top/pdf/1804.10531.pdf

065 (2018-04-25) Generative Model for Heterogeneous Inference

https://arxiv.xilesou.top/pdf/1804.09858.pdf

066 (2018-05-14) The Reincarnation of Grille Cipher A Generative Approach

https://arxiv.xilesou.top/pdf/1804.06514.pdf

067 (2018-04-3) Correlated discrete data generation using adversarial training

https://arxiv.xilesou.top/pdf/1804.00925.pdf

068 (2018-03-27) Structural inpainting

https://arxiv.xilesou.top/pdf/1803.10348.pdf

069 (2018-05-10) Digital Cardan Grille A Modern Approach for Information Hiding

https://arxiv.xilesou.top/pdf/1803.09219.pdf

070 (2018-03-27) Image Inpainting using Block-wise Procedural Training with Annealed Adversarial Counterpart

https://arxiv.xilesou.top/pdf/1803.08943.pdf

071 (2018-03-20) Patch-Based Image Inpainting with Generative Adversarial Networks

https://arxiv.xilesou.top/pdf/1803.07422.pdf

072 (2018-07-5) Generating Realistic Geology Conditioned on Physical Measurements with Generative Adversarial Networks

https://arxiv.xilesou.top/pdf/1802.03065.pdf

073 (2018-08-30) Deep Structured Energy-Based Image Inpainting

https://arxiv.xilesou.top/pdf/1801.07939.pdf

074 (2017-12-20) Context-Aware Semantic Inpainting

https://arxiv.xilesou.top/pdf/1712.07778.pdf

075 (2017-12-11) GibbsNet Iterative Adversarial Inference for Deep Graphical Models

https://arxiv.xilesou.top/pdf/1712.04120.pdf

076 (2018-08-6) Discriminative Region Proposal Adversarial Networks for High-Quality Image-to-Image Translation

https://arxiv.xilesou.top/pdf/1711.09554.pdf

077 (2017-11-26) Semantically Consistent Image Completion with Fine-grained Details

https://arxiv.xilesou.top/pdf/1711.09345.pdf

078 (2017-11-17) Improving Consistency and Correctness of Sequence Inpainting using Semantically Guided Generative Adversarial Network

https://arxiv.xilesou.top/pdf/1711.06106.pdf

079 (2019-04-2) Perceptual Adversarial Networks for Image-to-Image Transformation

https://arxiv.xilesou.top/pdf/1706.09138.pdf

080 (2017-12-6) r-BTN Cross-domain Face Composite and Synthesis from Limited Facial Patches

https://arxiv.xilesou.top/pdf/1706.00556.pdf

081 (2017-09-1) Adversarial Inverse Graphics Networks Learning 2D-to-3D Lifting and Image-to-Image Translation from Unpaired Supervision

https://arxiv.xilesou.top/pdf/1705.11166.pdf

082 (2017-10-12) CVAE-GAN Fine-Grained Image Generation through Asymmetric Training

https://arxiv.xilesou.top/pdf/1703.10155.pdf

083 (2016-11-21) Context Encoders Feature Learning by Inpainting

https://arxiv.xilesou.top/pdf/1604.07379.pdf

本文分享自微信公众号 - 机器学习与生成对抗网络(AI_bryant8),作者:bryant8

原文出处及转载信息见文内详细说明,如有侵权,请联系 yunjia_community@tencent.com 删除。

原始发表时间:2020-04-19

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

我来说两句

0 条评论
登录 后参与评论

相关文章

  • 强数据所难!SSL(半监督学习)结合GAN如何?

    众所周知,标注工作往往费时耗力甚至不可行。而类似深度学习等的机器学习方法,在缺乏带标签数据下,效果大打折扣甚至无法施展。

    公众号机器学习与生成对抗网络
  • 2020年7月90篇GAN论文汇总

    001 (2020-07-30) Unsupervised Disentanglement GAN for Domain Adaptive Person Re...

    公众号机器学习与生成对抗网络
  • 2020年6月100篇最新GAN论文汇总

    据不完全统计,GAN在CVPR2020上超115篇之多;其中,可看到GAN在朝着无监督/自监督/弱监督/半监督、少样本/单样本/零样本、多模态、3D、可解释性/...

    公众号机器学习与生成对抗网络
  • 阿里社招面经 (已拿 offer)

    问题比较多,而且很多面试题都是跟个人项目相关的,项目相关的问题借鉴意义不大,所以这里总结一些与项目无绝对关系的问题,欢迎围观~

    winty
  • 那些有意思的网站(二)

    这是一个苹果公司档案网站(非官方),收集了苹果公司从成立以来的所有广告、海报和宣传材料 https://www.applearchive.org/

    苏生不惑
  • 【专知荟萃22】机器阅读理解RC知识资料全集(入门/进阶/论文/综述/代码/专家,附查看)

    机器阅读理解(Reading Comprehension)专知荟萃 入门学习 进阶论文 综述 Datasets Code 领域专家 入门学习 深度学习解决机器阅...

    WZEARW
  • 如何在Ubuntu和Debian上安装R.

    R是包含了数据可视化功能的用于统计分析的编程语言。该语言通过Comprehensive R Archive Network(CRAN)拥有高度的可扩展性,该网络...

    魔法少女伊莉雅
  • 自然语言处理(NLP)入门指南

    大数据文摘
  • 【超全资源】自然语言处理(NLP)入门学习资源清单(部分资料下载)

    Melanie Tosik目前就职于旅游搜索公司WayBlazer,她的工作内容是通过自然语言请求来生产个性化旅游推荐路线。回顾她的学习历程,她为期望入门自然语...

    新智元
  • 【独家】自然语言处理(NLP)入门指南

    致谢 钟崇光博士参与了数据派THU于6月5日、THU数据派于6月8日发布的《循序渐进提升Kaggle竞赛模型精确度,以美国好事达保险公司理赔为例》一文的校对工作...

    数据派THU

扫码关注云+社区

领取腾讯云代金券