GANs, mutual information, and possibly algorithm selection?

If you ask deep learning people "what is the best image generation model now", many of them would probably say "generative adversarial networks" (GAN). The original paper describes an adversarial process that asks the generator and the discriminator to fight against each other, and many people including myself like this intuition. What lessons can we learn from GAN for better approximate inference (which is my thesis topic)? I need to rewrite the framework in the language I'm comfortable with, hence I decided to put it on this blog post as a research note.

Suppose there's a machine that can show you some images. This machine flips a coin to determine what it will show to you. For heads, the machine shows you a gorgeous paint from human artists. For tails, it shows you a forged famous paint. Your job is to figure out whether the shown image is "real" or not.

gan 优化如下:

To summarize, GAN can be viewed as an augmented generative model which is trained by minimizing mutual information. This augmentation is smart in the sense that it uses label-like information ss that can be obtained for free, which introduces supervision signal to help unsupervised learning.

Recently I started to think about automatic algorithm selection. Probably because I'm tired of my reviewers complaining on my alpha-divergence papers "I don't know how to choose alpha and you should give us a guidance". Tom Minka gives an empirical guidance in his divergence measure tech report, and same for us in recent papers. I know this is an important but very difficult topic for research, but at least not only myself have thought about it, e.g. in this paper the authors connected beta-divergences to tweedie distributions and performed approximate maximum likelihood to select beta. Another interesting paper in this line is the "variational tempering" paper which models the annealing temperature in a probabilistic model as well. I like these papers as the core idea is very simple: we should also use probabilistic modeling for algorithmic parameters. Perhaps this also connects to Bayesian optimization but I'm gonna stop here as the note is already a bit too long.

本文由zdx3578推荐。

原文发布于微信公众号 - CreateAMind(createamind)

原文发表时间:2016-11-11

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏技术小黑屋

How to Create Facebook Key Hash

When I create a new application on Facebook, I meet the problem. Facebook asks m...

1043
来自专栏程序猿DD

Spring Boot使用@Async实现异步调用:ThreadPoolTaskScheduler线程池的优雅关闭

上周发了一篇关于Spring Boot中使用 @Async来实现异步任务和线程池控制的文章:《Spring Boot使用@Async实现异步调用:自定义线程池》...

6047
来自专栏菩提树下的杨过

mac:在当前文件夹打开terminal终端

System Preferences -> Keyboard -> Shortcuts -> Services -> New Terminal at Folde...

2298
来自专栏数据和云

实践真知:使用ASM和文件系统的数据库在AIO上有何不同?

张大朋(Lunar)Oracle 资深技术专家 Lunar 拥有超过十年的 ORACLE SUPPORT 从业经验,曾经服务于ORACLE ACS部门,现就职...

2934
来自专栏Python与爬虫

佛系编程[如何创建一个安全可靠的应用程序]

推荐最近在GitHub上很火的一个项目,按照介绍,你也可以创建一个安全可靠的应用程序 项目地址在>>> nocode https://github.com/k...

4359
来自专栏潇涧技术专栏

Gradle Plugin for Android Development User Guide 2

Gradle Plugin for Android Development User Guide (2)

931
来自专栏bboysoul

linux编译安装apache

wget http://mirrors.ustc.edu.cn/apache/httpd/httpd-2.4.25.tar.gz tar -zxvf http...

2393
来自专栏程序猿DD

Spring Cloud Feign的文件上传实现

在Spring Cloud封装的Feign中并不直接支持传文件,但可以通过引入Feign的扩展包来实现,本文就来具体说说如何实现。 ? 服务提供方(接收文件) ...

41611
来自专栏Netkiller

怎样制作RPM包

怎样制作RPM包 摘要 我在网上找RPM包的制作例子几乎都是C源码编译安装然后生成RPM包, 而我的程序不是C写的很多时候是脚本语言如Python, PHP 甚...

6906
来自专栏菩提树下的杨过

spring-boot 速成(8) 集成druid+mybatis

spring-boot与druid、mybatis集成(包括pageHelper分页插件), 要添加以下几个依赖项: compile('mysql:my...

9279

扫码关注云+社区

领取腾讯云代金券