合成动态视频效果及声音合成

Synthesizing Dynamic Textures and Sounds by Spatial-Temporal Generative ConvNet

左面是原始视频，右面是合成的效果。

图片大小限制，更多可访问 http://www.stat.ucla.edu/~jxie/STGConvNet/STGConvNet.html

或 http://weibo.com/3164120327/Ewsk7eWcR 阅读原文访问

官方网站还有声音效果展示

Abstract

Dynamic textures are spatial-temporal processes that exhibit statistical stationarity or stochastic repetitiveness in the temporal dimension. In this paper, we study the problem of modeling and synthesizing dynamic textures using a generative version of the convolution neural network (ConvNet or CNN) that consists of multiple layers of spatial-temporal filters to capture the spatial-temporal patterns in the dynamic textures. We show that such spatial-temporal generative ConvNet can synthesize realistic dynamic textures. We also apply the temporal generative ConvNet to the one-dimensional sound data, and show that the model can synthesize realistic natural and man-made sounds.

http://www.stat.ucla.edu/~jxie/STGConvNet/STGConvNet_file/doc/STGConvNet.pdf

相关：

https://deepmind.com/blog/wavenet-generative-model-raw-audio/

Generative Temporal Models with Memory https://arxiv.org/abs/1702.04649