- 【课程】Open Source Deep Learning Curriculum
2.【论文&代码】Deep multi-task learning with low level tasks supervised at lower layers
In all previous work on deep multi-task learning we are aware of, all task supervisions are on the same (outermost) layer. We present a multi-task learning architecture with deep bi-directional RNNs, where different tasks supervision can happen at different layers. We present experiments in syntactic chunking and CCG supertagging, coupled with the additional task of POS-tagging. We show that it is consistently better to have POS supervision at the innermost rather than the outermost layer. We argue that this is because “lowlevel” tasks are better kept at the lower layers, enabling the higher-level tasks to make use of the shared representation of the lower-level tasks. Finally, we also show how this architecture can be used for domain adaptation.
3.【博客】GANs will change the world
It’s New Year’s 2017, so time to make predictions. Portfolio diversification has never been me, so I’ll make just one.
Generative Adversarial Networks — GANs for short — will be the next big thing in deep learning, and GANs will change the way we look at the world.
4.【训练GAN技巧】starter from "How to Train a GAN?" at NIPS2016
While research in Generative Adversarial Networks (GANs) continues to improve the fundamental stability of these models, we use a bunch of tricks to train them and make them stable day to day.
Here are a summary of some of the tricks.
5.【代码】Hierarchical Temporal Memory (HTM)
The Numenta Platform for Intelligent Computing (NuPIC) is a machine intelligence platform that implements the HTM learning algorithms. HTM is a detailed computational theory of the neocortex. At the core of HTM are time-based continuous learning algorithms that store and recall spatial and temporal patterns. NuPIC is suited to a variety of problems, particularly anomaly detection and prediction of streaming data sources.