This paper performs a comparitive study of recent advances in deep learning with human-like learning from a cognitive science point of view. Since natural intelligence is still the best form of intelligence, the authors list a core set of ingredients required to build machines that reason like humans.

- Cognitive capabilities present from childhood in humans.  
    - Intuitive physics; for example, a sense of plausibility of object trajectories, affordances.
    - Intuitive psychology; for example, goals and beliefs.
- Learning as rapid model-building (and not just pattern recognition).
    - Based on compositionality and learning-to-learn.
    - Humans learn by inferring a general schema to describe goals, object types and interactions. This enables learning from few examples. 
    - Humans also learn richer conceptual models.
        - Indicator: variety of functions supported by these models: classification, prediction, explanation, communication, action, imagination and composition.
        - Models should hence have strong inductive biases and domain knowledge built into them; structural sharing of concepts by compositional reuse of primitives.
- Use of both model-free and model-based learning.
    - Model-free, fast selection of actions in simple associative learning and discriminative tasks.
    - Model-based learning when a causal model has been built to plan future actions or maximize rewards.
- Selective attention, augmented working memory, and experience replay are low-level promising trends in deep learning inspired from cognitive psychology.
    - Need for higher-level aforementioned ingredients.

This paper presents an approach to visual question answering by dynamically composing networks of independent neural modules based on the semantic parsing of the question. Main contributions:

- Independent neural modules that can be combined together and jointly trained.
    - Attention: Convolutional layer, with different filters for different instances. For example, attend[dog], attend[cat], etc.
    - Re-attention: FC-ReLU-FC-ReLU, weights are different for different instances. For example, re-attend[above], re-attend[not], etc.
    - Combination: Stacks two attention maps, followed by conv-ReLU to map to a single attention map. For example, combine[and], combine[except], etc.
    - Classification: Combines attention map and image, followed by FC-Softmax to map to answer. For example, classify[colors].
    - Measurement: FC-ReLU-FC-Softmax, takes attention map as input. For example, measure[exists].

- Structured representations are extracted from questions and these are then mapped to network layouts, including the connections between them.
    - All leaves become attend modules, all internal nodes become re-attend or combine modules dependent on their arity, and root nodes become measure modules for yes/no questions and classify modules for all other question types.
    - Networks with the same structure but different instantiations can be processed in the same batch. For example, classify[color]\(attend[cat]\), classify[where]\(attend[truck]\).

- Predictions from the module network are combined with LSTM representations to get the final answer.
    - Syntactic regularities: 'what is flying?' and 'what are flying?' get mapped to the same module network.
    - Semantic regularities: 'green' is an implausible answer for 'what color is the bear?'.

- Experiments are performed on the synthetic SHAPES dataset and VQA dataset.
    - Performance on the SHAPES dataset is better as it is designed to benefit from compositionality.

## Strengths

- This model takes advantage of the inherently compositional property of language, which makes a lot of sense. VQA is an extremely complex task and breaking it up into separate functions/modules is an excellent approach.

## Weaknesses / Notes

- Mapping from syntactic structure to module network is hand-designed. Ideally, the model should learn this too to generalize.

- Due to its compositional nature, this kind of model can possibly be used in the zero-shot learning setting, i.e. generalize to novel question types that the network hasn't seen before.

This paper presents a way to reduce the expected network depth of deep residual networks during training by randomly dropping a subset of residual blocks and bypassing them with identity connections. The 'survival' probability $p\_l$ decreases linearly with depth (from 1.0 to 0.5 at last layer) so as to keep layers that extract low-level features with higher probability. At test time, residual block functions are scaled by the expected number of times it appears during training, i.e. $p\_l$. This model achieves lower test errors than ResNets (with ReLU activations) on CIFAR-10, CIFAR-100 and SVHN.

## Strengths

- Shorter expected depth leads to faster training (>25% speedup).

- Helps reduce the vanishing gradient problem as shown by the mean gradient magnitude v/s epochs plot.

- Linear decay of survival probability works better than uniform survival, which supports the intuition that low-level features need to be reliably present.

- Stochastic depth acts as a regularizer. The 1202-layer stochastic depth residual network shows improvements over the 110-layer network, while the original ResNets paper reports overfitting and higher test error with 1000+ layers.

## Weaknesses / Notes

- Test errors for the updated ResNet architecture (ReLU activation inside residual function) are missing. That should perform better. Also, numbers on ImageNet.

- Stochastic depth can be interpreted as sequential ensembling as compared to parallel ensembles.

- It would be interesting to look at the filters learnt by stochastic depth residual networks, and to understand whether/how these networks learn hierarchical features as compared to the conventional CNN intuitions of compositionality.

This paper builds on top of a bunch of existing ideas for building neural conversational agents so as to control against generic and repetitive responses. 

Their model is the sequence-to-sequence model with attention (Bahdanau et al.), first trained with the usual MLE loss and fine-tuned with policy gradients to optimize for specific conversational properties. Specifically, they define 3 rewards:

1. Ease of answering — Measured as the likelihood of responding to a query with a list of hand-picked dull responses (more negative log likelihood is higher reward). 
2. Information flow — Consecutive responses from the same agent (person) should have different information, measured as negative of log cosine distance (more negative is better).
3. Semantic coherence — Mutual information between source and target (the response should make sense wrt query). $P(a|q) + P(q|a)$ where a is answer, q is question.

The model is pre-trained with the usual supervised objective function, taking source as concatenation of two previous utterances. Then they have two stages of policy gradient training, first with just a mutual information reward and then with a combination of all three. The policy network (sequence-to-sequence model) produces a probability distribution over actions (responses) given state (previous utterances). To estimate the gradient in an iteration, the network is frozen and responses are sampled from the model, the rewards for which are then averaged and gradients are computed for first L tokens of response using MLE and remaining T-L tokens with policy gradients, with L being gradually annealed to zero (moving towards just the long-term reward).

Evaluation is done based on length of dialogue, diversity (distinct unigram, bigrams) and human studies on

1. Which of two outputs has better quality (single turn)
2. Which of two outputs is easier to respond to, and
3. Which of two conversations have better quality (multi turn).

## Strengths

- Interesting results
    - Avoids generic responses
    - 'Ease of responding' reward encourages responses to be question-like
- Adding in hand-engineereed approximate reward functions based on conversational properties and using those to fine-tune a pre-trained network using policy gradients is neat.
- Policy gradient training also encourages two dialogue agents to interact with each other and explore the complete action space (space of responses), which seems desirable to identify modes of the distribution and not converge on a single, high-scoring, generic response.

## Weaknesses / Notes

- Evaluating conversational agents is hard. BLEU / perplexity are intentionally avoided as they don't necessarily reward desirable conversational properties.

This paper introduces Residual Nets (ResNets), which was the
winning submission (152-layer deep) at ILSVRC 2015 and MS-COCO 2015, and achieves
a top-5 error rate of 3.57% (ensemble of two  nets). Main contributions:

- The key idea is that deeper networks face the degradation problem, i.e.
higher training and test error than shallower nets, because they're harder
to optimize for approximating identity mapping by multiple non-linear layers.
    - They mitigate this problem by forcing solvers to learn residual functions
    i.e. $f(x) = H(x) - x$, by adding shortcut connections. If identity mapping is
    the optimal formulation, the learned weights should drive $f(x)$ to 0 (and they
    observe that this is a suitable preconditioning as most residual function responses
    are small).

- Shortcut connections (for identity mapping) don't require additional parameters.
    - Size transformations are done by zero-padding (no parameters) or projections. Projections
    introduce additional parameters and perform slightly better.

- Bottleneck design is used to further reduce computational complexity, i.e. 1x1 convolutional
layers before and after 3x3 convolutions to reduce and increase dimensions.

- For detection and localization tasks, they use ResNets in the Faster-RCNN setting.

## Strengths

- ResNets are significantly deeper and more accurate yet computationally cheaper than VGG.

- A single ResNet outperforms previous state-of-the-art ensembles. Their final winning submission
is an ensemble of two networks.

## Weaknesses / Notes

- The idea of shortcut connections to force blocks to learn residual functions preconditioned
on identity mapping is neat, and more so because it doesn't require additional parameters.

- A lot of results and design decisions merit further investigation and reasoning.
    - Why do shortcuts skip 2 or 3 layers? What happens to performance if we increase the number of layers skipped?
    - How well do shortcut connections work with Inception modules? The statistical principles
    underlying both these architectures seem to be orthogonal, does performance further improve?
    - 152 seems to be an arbitrary number of layers that 'worked'.

- The degradation problem seen when making networks deeper by initializing
layers with identity weight matrices seems to be contradictory to the results
presented in the Net2Net paper.