## Ideas

* Find CNN topology with Q-learning and $\varepsilon$-greedy exploration and experience replay


## Evaluation

The authors seem not to know DenseNets

* CIFAR-10: 6.92 % accuracy ([SOTA](https://martin-thoma.com/sota/#image-classification) is 3.46 % - not mentioned in the paper)
* SVHN: 2.06 % accuracy ([SOTA](https://martin-thoma.com/sota/#image-classification) is 1.59% - not mentioned in the paper)
* MNIST: 0.31 % ([SOTA](https://martin-thoma.com/sota/#image-classification) is 0.21 % - not mentioned in the paper)
* CIFAR-100: 27.14 % accuracy ([SOTA](https://martin-thoma.com/sota/#image-classification) is 17.18 % - not mentioned in the paper)


## Related Work

* Google: [Neural Architecture Search with Reinforcement Learning](https://arxiv.org/abs/1611.01578) ([summary](http://www.shortscience.org/paper?bibtexKey=journals/corr/1611.01578#martinthoma))

This paper discusses an approach to integrated task and motion planning. They argue that the primary requirement for the task planner is the ability generate task plans based on the motion planning results. To this end, they employ Satisfiability Modulo Theories solver to solve for a task plan, and dynamically add or remove task constraints based on the motion planning results.

More specifically, their task and motion planner, IDTMP, begins with only initial and goal state constraints, as well as the preconditions and effects at time step 0. It also begins with a time step of 1, which means it will try to find a task within one time step. If it determines that it is unable to produce a task of length 1 that abides by the constraints, it pops the goal constraint and then add additional preconditions and effects at time step h+1, and then add the goal constraint back in. They repeat this until a feasible task plan is found. 

Once the task plan is found, it then tries to find a set of motions plans that is associated with each task operators. They have a special operator, $\lambda_\rho$, which computes the goal configuration for motion planner; however, note that for complex task and motion planning tasks, this function is not easy to define. If a feasible motion plan is not found in any one of the operators, they add the constraint that says you should not try this task operator in this state, and re-plan the task plan using this constraint.

They prove the probabilistic completeness by noting the fact that the incremental SMT solver considers all previous task plans and RRT is probabilistic complete. The experiment is shown for a rather simple task of stacking box objects on top of another; comparison to only one other method is shown.

While this paper has some interesting ideas, such as  incrementally incorporating motion planning results as constraints for finding a task plan, it lacks some motivations as to why they had to use SMT solver. Would simple CSP solver suffice? A random sampling planner? Also, their task is rather too simple, and they get away with the difficulty of sampling continuous parameters for task operators without any explanation. Their comparison with only one benchmark makes this approach not very convincing.

TLDR; The authors build an LSTM Neural Language model, but instead of using word embeddings as inputs, they use the per-word outputs of a character-level CNN, plus a highway layer. This architecture results in state of the art performance and significantly fewer parameters. It also seems to work well on languages with rich morphology.


#### Key Points 

- Small Model: 15-dimensional char embeddings, filter sizes 1-6, tanh, 1-layer highway with ReLU, 2-layer LSTM with 300-dimensional cells. 5M Parameters. Hiearchical Softmax.
- Large Model: 15-dimensional char embeddings, filter sizes 1-7, tanh, 2-layer highway with ReLU, 2-layer LSTM with 670-dimensional cells. 19M Parameters. Hiearchical Softmax.
- Can generalize to out of vocabulary words due to character-level representations. Some datasets already had OOV words replaced with a special token, so the results don't reflect this.
- Highway Layers are key to performance. Susbtituting HW with MLP does not work well. Intuition is that HW layer adaptively combines different local features for higher-level representation.
- Nearest neighbors after Highway layer are more smenatic than before highway layer. Suggests compositional nature.
- Surprisingly combinbing word and char embeddings as LSTM input results in worse performance - Characters alone are sufficient?
- Can apply same architecture to NML or Classification tasks. Highway Layers at the output may also help these tasks.


#### Notes / Questions

- Essentially this is a new way to learn word embeddings comprised of lower-level character embeddings. Given this, what about stacking this architecture and learn sentence representations based on these embeddings?
- It is not 100% clear to me why the MLP at the output layer does so much worse. I understand that the highway layer can adaptively combine feature, but what if you combined MLP and plain representations and add dropout? Shouldn't that result in similar perfomance?
- I wonder if the authors experimented with higher-dimensional character embeddings. What is the intuition behind the very low-dimensional (15) embeddings?

Multilingual embeddings are useful for creating embeddings for low resource languages for things like transfer learning (e.g., learning a POS tagger in a low-resource language using training data from a high resource language). However, they typically require some small amount of supervision in the form of aligned corpora, seed pairs, or dictionaries. This approach attempts to learn a mapping from a source embedding space into a target embedding space without supervision.

The approach uses two networks a la adversarial training. One network (the generator) is parameterized by a projection matrix that attempts to map source words into the target space. The other network (the discriminator) attempts to discriminate true target embeddings from projected source embeddings. Since adversarial training is known to be unstable (a "research frontier" as the authors say), quite a bit of the paper describes tricks and training methods the authors investigated to get training to converge and understand how to select models.

They evaluate on many pairs, including both similar and dissimilar language pairs, and get very nice results. In summary, better than seed-based approaches with 0-100 seeds, competitive with 100-1000 seeds. Much of what would be traditional discussion is instead devoted to details of training regimen, so unfortunately there is little discussion of why this works. Given the difficulty one might encounter attempting to train this, I think it might be a little preliminary to try using this for applications, but continued research in training adversarial networks for NLP and properties of embedding spaces could potentially make this approach reliable enough for real applications.

It’s possible I’m missing something here, but my primary response to reading this paper is just a sense of confusion: that there is an implicit presenting of an approach as novel, when there doesn’t seem to me to be a clear mechanism that is changed from prior work. The premise of this paper is that self-supervised learning techniques (a subcategory of unsupervised learning, where losses are constructed based on reconstruction or perturbation of the original image) should be made into supervised learning by learning on a loss that is a weighted combination of the self-supervised loss and the supervised loss, making the overall method a semi-supervised one. 

The self-supervision techniques that they identify integrating into their semi-supervised framework are: 
- Rotation prediction, where an image is rotated to one of four rotation angles, and then a classifier is applied to guess what angle 
- Exemplar representation invariance, where an imagenet is cropped, mirrored, and color-randomized in order to provide inputs, whose representations are then pushed to be closer to the representation for the unmodified image 

My confusion is due to the fact that the I know that I’ve read several semi-supervised learning papers that do things of this ilk (insofar as combining unsupervised and supervised losses together) and it seems strange to identify it as a novel contribution. That said, this paper does give an interesting overview of self-supervisation techniques, I found it valuable to read for that purpose.