#### Problem addressed: 
Variational learning of Bayesian networks

#### Summary: 
This paper present a generic method for learning belief networks, which uses variational lower bound for the likelihood term.

#### Novelty:
Uses a re-parameterization trick to change random variables to deterministic function plus a noise term, so one can apply normal gradient based learning

#### Drawbacks:
The resulting model marginal likelihood is still intractible, may not be very good for applications that require the use of actual values of the marginal probablities

#### Datasets:
MNIST, Frey face

#### Additional remarks:
Experimentally compared with wake sleep algorithm on logliklihood lower bound as well as estimated marginal likelihood

#### Resources:
Implementation: https://github.com/y0ast/Variational-Autoencoder

#### Presenter:
Yingbo Zhou

They propose a joint model for 1) identifying event keywords in a text, 2) identifying entities, and 3) identifying the connections between these events and entities. They also do this across different sentences, jointly for the whole text.

https://i.imgur.com/ETKZL7V.png

Example of the entity and event annotation that the system is modelling.

The entity detection part is done with a CRF; the structure of an event is learned with a probabilistic graphical model; information is integrated from surrounding sentences using a Stanford coreference system; and these are all tied together across the whole document using Integer Linear Programming.

This paper is about finding naturally looking images for the analysis of machine learning models in computer vision. There are 3 techniques:

* **inversion**: the aim is to reconstruct an image from its representation
* **activation maximization**: search for patterns that maximally stimulate a representation component (deep dream). This does NOT use an initial natural image.
* **caricaturization**: exaggerate the visual patterns that a representation detects in an image

The introduction is nice.


## Code

The paper comes with code: [robots.ox.ac.uk/~vgg/research/invrep](http://www.robots.ox.ac.uk/~vgg/research/invrep/index.html) ([GitHub: aravindhm/deep-goggle](https://github.com/aravindhm/deep-goggle))


## Related

* 2013, Zeiler & Fergus: [Visualizing and Understanding Convolutional Networks ](http://www.shortscience.org/paper?bibtexKey=journals/corr/ZeilerF13#martinthoma)

This paper describes some statistical test very neatly. It also has a nice Figure which classifies statistical questions in machine learning.

* 5x2cv: 5 iterations of 2 fold cross validation
* Quasi-F test
* McNemar's test: Described in detail!

## See also

* [On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach](http://www.shortscience.org/paper?bibtexKey=journals/datamine/Salzberg97)

Investigation of how well LSTMs capture long-distance dependencies. The task is to predict verb agreement (singular or plural) when the subject noun is separated by different numbers of distractors. They find that an LSTM trained explicitly for this task manages to handle even most of the difficult cases, but a regular language model is more prone to being misled by the distractors.

https://i.imgur.com/0kYhawn.png