* The authors start with a standard ResNet architecture (i.e. residual network has suggested in "Identity Mappings in Deep Residual Networks").
    * Their residual block:
      ![Residual block](https://raw.githubusercontent.com/aleju/papers/master/neural-nets/images/Wide_Residual_Networks__residual_block.png?raw=true "Residual block")
    * Several residual blocks of 16 filters per conv-layer, followed by 32 and then 64 filters per conv-layer.
  * They empirically try to answer the following questions:
    * How many residual blocks are optimal? (Depth)
    * How many filters should be used per convolutional layer? (Width)
    * How many convolutional layers should be used per residual block?
    * Does Dropout between the convolutional layers help?

### Results
  * *Layers per block and kernel sizes*:
    * Using 2 convolutional layers per residual block seems to perform best:
      ![Convs per block](https://raw.githubusercontent.com/aleju/papers/master/neural-nets/images/Wide_Residual_Networks__convs_per_block.png?raw=true "Convs per block")
    * Using 3x3 kernel sizes for both layers seems to perform best.
    * However, using 3 layers with kernel sizes 3x3, 1x1, 3x3 and then using less residual blocks performs nearly as good and decreases the required time per batch.
  * *Width and depth*:
    * Increasing the width considerably improves the test error.
    * They achieve the best results (on CIFAR-10) when decreasing the depth to 28 convolutional layers, with each having 10 times their normal width (i.e. 16\*10 filters, 32\*10 and 64\*10):
      ![Depth and width results](https://raw.githubusercontent.com/aleju/papers/master/neural-nets/images/Wide_Residual_Networks__depth_and_width.png?raw=true "Depth and width results")
    * They argue that their results show no evidence that would support the common theory that thin and deep networks somehow regularized better than wide and shallow(er) networks.
  * *Dropout*:
    * They use dropout with p=0.3 (CIFAR) and p=0.4 (SVHN).
    * On CIFAR-10 dropout doesn't seem to consistently improve test error.
    * On CIFAR-100 and SVHN dropout seems to lead to improvements that are either small (wide and shallower net, i.e. depth=28, width multiplier=10) or significant (ResNet-50).
      ![Dropout](https://raw.githubusercontent.com/aleju/papers/master/neural-nets/images/Wide_Residual_Networks__dropout.png?raw=true "Dropout")
    * They also observed oscillations in error (both train and test) during the training. Adding dropout decreased these oscillations.
  * *Computational efficiency*:
    * Applying few big convolutions is much more efficient on GPUs than applying many small ones sequentially.
    * Their network with the best test error is 1.6 times faster than ResNet-1001, despite having about 3 times more parameters.

Assume you have both label and reward (some flexibility in accepting the output) as is the case for hamming distance and negative edit distance.

Assume the output space is discrete, it can be a quantized n-dim space.

Assume that given an input and a set of labels, computing the probability of the labels is $O(L)$, meaning that most of the computation is independent of the label. (not sure about this! You certainly don't want to sample from the network)

Suppose that you want to maximize an approximate objective which is a tradeoff between optimizing the reward or the label and you control this tradeoff by one parameter $\tau$ which you fix beforehand. This actually only directly affects the reward.

Then you can compute the loss, by sampling from each discrete value of reward and then sampling from those labels that have such reward.

Therefore, you can quickly approximate the loss and the gradient through sampling. And the samples are independent of the output of the network, but rather fixed given $\tau$ and ground-truth. This is as opposed to sampling the network and sampling around the predicted label.

#### 2.2 Sampling from the exponentiated payoff distribution

Note that $q$ should be defined such that the entropy is fixed given $\tau$. The paper notes that given the ground-truth label, if the loss is hamming distance or negative edit distance, then samples can be put into buckets of same loss (loss is discrete). Then One possible family of distributions controlled by $\tau$ is given by a normalized count reweighted by $\exp\{e/\tau\}$ for each bucket. By varying $\tau$, you can go from uniform distribution ($\tau=\infty$) to delta ($\tau=0$). Eq. 11 shows the simplified count for each bucket to be used for normalization and further sampling. As noted above the equation, edge cases like repetitions is ignored in this count to make the computation easy.

Given a fixed entropy $q$, then you can compute the RML loss by sampling from $q$ by first selecting an edit distance, and then sampling from the corresponding bucket.

#### Footnotes
I was not satisfied with the results. I didn't quite follow section 3.

Find a topology by reinforcement learning. They use REINFORCE from [Williams 1992](http://www.shortscience.org/paper?bibtexKey=Williams:92).


## Ideas

* Structure and connectivity of a Neural Network can be represented by a variable-length string.
* The RNN controller in Neural Architecture Search is auto-regressive, which means it predicts hyperparameters one a time, conditioned on previous predictions
* policy gradient method to maximize the expected accuracy of the sampled architectures
* In our experiments, the process of generating an architecture stops if the number of layers exceeds
a certain value.

## Evaluation

* Computer Vision - **CIFAR-10**: 3.65% error (State of the art are Dense-Nets with 3.46% error)
* Language - **Penn Treebank**: a test set perplexity of 62.4 (3.6 perplexity better than the previous state-of-the-art)

They had a Control Experiment "Comparison against Random Search" in which they showed that they are much better than a random exploration of the data. However, the paper lacks details how exactly the random search was implemented.

## Related Work

* [Designing Neural Network Architectures using Reinforcement Learning](https://arxiv.org/pdf/1611.02167.pdf) ([summary](http://www.shortscience.org/paper?bibtexKey=journals/corr/1611.02167))

I like the idea proposed in this paper - training on a label-rich domain and transfer the representation to a label-limited domain, but would like to extend it to data more than faces such as transferring the object attributes.

Joint summary from https://mitpress.mit.edu/books/developmental-robotics

Developmental robotics is the interdisciplinary approach to the autonomous design of behavioural and cognitive capabilities in artificial agents (robots) that takes direct inspiration from the developmental principles and mechanisms observed in the natural cognitive systems.  It relies on a highly interdisciplinary effort of empirical developmental sciences such as developmental psychology, neuroscience, and comparative psychology, and computational and engineering disciplines such as robotics and artificial intelligence.  The implementation of these principles and mechanisms into a robot-s control architecture and the testing through experiments where the robot interacts with its physical and social environment simultaneously permits the validation of such principles and the actual design of complex behavioural and mental capabilities in robots. Developmental psychology and developmental robotics mutually benefit from such a combined effort.

Synonym terms of Developmental Robotics include cognitive developmental robotics, autonomous mental development as well as epigenetic robotics \cite{Cangelosi18}. The  later term borrows the term ‘epigenetic’ from Piaget’s Epigenetic Theory of human development, where the child’s cognitive system develops as a result of the interaction between genetic predispositions and the organism’s interaction with the environment.  Therefore, ‘Epigenetic robotics’ was justified by Piaget’s stress on the importance of the role of interaction with the environment (later complemented with the emphasis on social interaction of Zlatev and Balkenius in 2001), and in particular, on the sensorimotor bases of higher-order cognitive capabilities. 
Due to the challenges of robotics domain not only on acquisition of skills in an incremental manner but also in open-ended environments, it is perhaps worth noticing that the state of the art is not as advanced as in other more constrained well defined unique task settings. Because of the importance of embodiment in robotics for open ended learning \cite{Cangelosi18} and because of the scarcity of continual learning strategies on robotics and merely assessed  on perception tasks, we believe both fields methodologies should borrow from each other. The blend could result not only in more effective or robust lifelong learning but also in less forgetful agents (i.e., the main rationale behind CL), which has not yet empirically shown is impossible to reach. We hope this survey helps reach this goal.

Dynamical systems development: In math it is characterized by complex changes over time in the phase state. The complex interaction of nonlinear phenomena results in the production of unpredictable states of the system, often referred to as emergent states. This concept was borrowed by developmental psychologists \cite{Thelen94} to explain child development as the emergent product of the intricate and dynamic interaction of many decentralized and local interactions related to the child growing body and brain and her environment.


Some definitions: 

Novelty, curiosity and surprise: focuses on the autonomy of learning, i.e. on the agents freedom to choose what, when and how it will learn. Intrinsic motivation (IM) is a mechanism to drive autonomous learning, not only in developmental robotics but also more broadly within the field of machine learning \cite{Mirolli13}\cite{Oudeyer07}. IM is task-independent, the agent could be placed in a completely new environment with no prior knowledge  or experience, and through self-directed exploration the robot will potentially learn not only important features of the environment but also the behavioural skills necessary for dealing with the environment. Second, IM promotes the hierarchical learning rather than solving a specific, predefined task. 

The Theory of MInd (ToM): A robot can use its own theory of mind to improve the interaction with human users, for example. It aims at reading the intention of the others and reacting appropriately to the emotional, attentional and cognitive states of the other agents, to anticipate their reactions, and to modify its own behaviour to satisfy these expectation and needs \cite{Scasellati02}.

Cascades: developmental theories refer to the far reach of early developments on later ones in terms of the “developmental cascade”.

Saliency in robotics: 
In robotics vision, a set of feature extraction methods typically can be applied to derive a saliency map \cite{Itti01}, i.e. the identification of the parts of the image that are important for the robot behaviour. It can be created, e.g., by combining a set of feature extraction method for color, motion, orientation and brightness. These features are combined to generate the whole saliency  map used by the model to fixate the objects.