The DeepMind guys present an extension to the Neural Turing Machine architecture.

https://i.imgur.com/4TC6yAp.png

They call it a Differentiable Neural Computer (DNC) and it uses 1) an attention mechanism to access information in a matrix that acts as a memory, 2) an attention mechanism to save information to that memory, and 3) a transition matrix that stores information about the order in which rows in the memory are modified, in order to better handle sequential information. They test on the bAbI question answering dataset, a graph inference task, and on solving a puzzle of arranging blocks.

#### Summary:
The main point of the paper is to show the automatic tranformation process of a java project to run on AWS lambda. For the transformation process a self developed tool named Podilzer is used to perform the tests. Further a comparison of the execution times and cost factors is made, to show if it's valuable to run Java functions on AWS lambda.

#### Good points:
The pipeline process is well described and good understandable. The developed tool called Podilizer implements this pipeline process and is also as open source project available. The experiments are also available on the openscience platform including scripts and code, this grantees repeatability of the conducted tests. 

#### Major comments:
One of the main problems with FaaS in general is preserving the state of an application. This challenge is described well. I wished to get a bit more insight what other problematic functions exist and what the approach of transforming those would be. 
The java projects which were used to conduct the results are available but not described in the paper. Therefore the transformation times are difficult to assess, before a study of the projects itself. Further  the execution performance of the same applications were compared locally and on different cloud offerings. On AWS Lambda the execution performance is significantly higher then on all the other platforms. Probably the times also include network latency, therefore it would be also interesting to see the actual execution times on Lambda itself. 

#### Minor comments:
Generally good grammar, some minor typos. 

#### Recommendations:
Describe the problems of transforming existing functions in more extend. Include also the “real” execution times of the AWS Lambda functions, to have a better comparison between the run times. 
Finally proofread and publish after corrections have been made

The reconstruction of high-fidelity resolution brain MR images is especially challenging because of the highly complex brain structure. Most promising approaches for this task are autoencoders and generative models such as Variational Autoencoders (VAE) or Generative Adversarial Networks (GAN). In Unsupervised Anomaly Detection (UAD), these architectures are only trained with images of healthy brain anatomy and not with images containing anomalies such as lesions. Therefore, processing an anomalous input image $\pmb{x}$ with an architecture trained to reconstruct healthy MR images should produce a high reconstruction error $r$ indicating that the input MR image is likely to exhibit anomalies.

So far developed models for this task are either limited to low resolution reconstructions so that quite a lot of information is lost or to only small image regions. Baur et al. hypothesize that these restrictions are caused by the low dimensionality of the latent space of the models even though a high capacity would be necessary to reconstruct highly complex brain MR images. Thus, Baur et al. introduce skip connections in the autoencoder to enable detailed high resolution reconstructions due to the enhanced gradient flow. In addition, the application of dropout on the skip connections shall prevent the model from learning the identity of the input image since only the corresponding healthy anatomy of the possibly anomalous input image shall be reconstructed. Sampling the networks parameter $\pmb{\theta}$ from a Bernoulli distribution $\pmb{\theta} \sim \mathcal{B}(p)$ with $p$ being the dropout rate  also turns the Skip-Autoencoder into a Bayesian neural network by using a Monte Carlo dropout at test time.
The proposed Skip Autoencoder and the Bayesian Skip-Autoencoder are tested on Mutiple Sclerosis and Glioblastoma datasets and evaluated using the Precision-Recall-Curve (PRC), the AUPRC, and the Dice Score. An investigation of the Bernoulli dropout yields that no input identity is learnt independent on the dropout. A random weight initialization seems to be sufficient. Considering the skip-connections, the best performance is achieved by a single skip-connection close to the bottleneck layer. Applying more skip connections improves the performance on the Glioblastoma test set but reduces the performance on the Multiple Sclerosis test set.

In general, the non-Bayesian Skip-Autoencoder exceeds the performance of the Bayesian Skip-Autoencoder because in the calculation of the reconstruction residual $r = \rvert \pmb{x} - \pmb{\hat{x}} \rvert$ for the Bayesian Skip-Autoencoder the predictive mean of $n$ Monte Carlo samples is applied as reconstructed image $\pmb{\hat{x}}$ which causes a blurriness in the reconstructions  $\pmb{\hat{x}}$. However, the Bayesian Skip-Autoencoder enables to quantify the epistemic uncertainty because the pixels of hyperintense anomalies have a low variance compared to normal tissue.

- *issue:* RL on real systems -> sparse and slow data sampling;
- *solution:* pre-train the agent with the EGAN;
- *performance:* ~20% improvement of training time in the beginning of learning compared to no pre-training; ~5% improvement and smaller variations compared to GAN pre-training.

## Introduction

5G telecom systems -> fufill ultra-low latency, high robustness, quick response to changed capacity needs, and dynamic allocation of functionality.

*Problems:*

1. exploration has an impact on the service quality in real-time service systems;
2. sparse and slow data sampling -> extended training duration.

## Enhanced GAN

**Fomulas**

the training data for RL tasks:

$$x = [x_1, x_2] = [(s_t,a),(s_{t+1},r)]$$

the generated data:

$$G(z) = [G_1(z), G_2(z)] =  [(s'_t,a'),(s'_{t+1},r')] $$

the value function for GAN:

$$V(D,G) = \mathbb{E}_{z \sim p_z(z)}[\log(1-D(G(z)))] + \lambda D_{KL}(P||Q)$$

where the regularization term $D_{KL}$ has the following form:

$$D_{KL}(P||Q) = \sum_i P(i) \log \frac{P(i)}{Q(i)}$$

**EGAN structure**

https://i.imgur.com/FhPxamJ.png

**Algorithm**

https://i.imgur.com/RzOGmNy.png

The enhancer is fed with training data *D\_r(s\_t, a)* and *D\_r(s\_{t+1}, r)*, and trained by supervised learning. After GAN generates synthetic data *D\_t(s\_t, a, s\_{t+1}, r)*, the enhancer could enhance the dependency between *D\_t(s\_t, a)* and *D\_t(s\_{t+1}, r)* and update the weights of GAN.

## Results

two lines of experiments on CartPole environment involved with PG agents: 

1. one for comparing the learning curves of agents with no pre-training, GAN pre-training and EGAN pre-training. => Result: EGAN > GAN > no pre-training

2. one for comparing the learning curves of agents with EGAN pre-training for various episodes (500, 2000, 5000). => Result: 5000 > 2000 ~= 500

During the development of neural circuitry, neurons of different kinds establish specific synaptic connections by selecting appropriate targets from large numbers of alternatives. The range of alternative targets is reduced by well organised patterns of growth, termination, and branching that deliver the terminals of appropriate pre- and postsynaptic partners to restricted volumes of the developing nervous system. We use the axons of embryonic Drosophila sensory neurons as a model system in which to study the way in which growing neurons are guided to terminate in specific volumes of the developing nervous system. The mediolateral positions of sensory arbors are controlled by the response of Robo receptors to a Slit gradient. Here we make a genetic analysis of factors regulating position in the dorso-ventral axis. We find that dorso-ventral layers of neuropile contain different levels and combinations of Semaphorins. We demonstrate the existence of a central to dorsal and central to ventral gradient of Sema 2a, perpendicular to the Slit gradient. We show that a combination of Plexin A (Plex A) and Plexin B (Plex B) receptors specifies the ventral projection of sensory neurons by responding to high concentrations of Semaphorin 1a (Sema 1a) and Semaphorin 2a (Sema 2a). Together our findings support the idea that axons are delivered to particular regions of the neuropile by their responses to systems of positional cues in each dimension.