#### Problem addressed: 
Gradient estimation for stochastic neurons

#### Summary: 
This paper proposed an unbiased estimator of stochastic units so that one can use gradient based learning. In addition, it also proposed a simple, biased estimator called straight through.

#### Novelty:
A new approach for estimating gradient for stochastic units.

#### Drawbacks:
The proposed unbised estimator seems to have large variance, and the biased one seems not performing very well in practice

#### Presenter:
Yingbo Zhou

In recent years, many generative models have been proposed to learn distributed representations automatically from data. One criticism of these models are that they produce representations that are "entangled": no single component of the representation vector has meaning on its own. This paper proposes a novel neural architecture and associated learning algorithm for learning disentangled representations. The paper demonstrates the network learning visual concepts on pairs of frames from Atari Games and rendered faces.

The proposed architecture uses a gating mechanism to select an index to hidden elements that store the "unpredictable" parts of the frame into a single component. The architecture bears some similarity to other "gated" architectures, e.g. relational autoencoders, three-way RBMs, etc. in that it models input-output pairs and encodes transformations. However, these other architectures do not use an explicit mechanism to make the network model "differences". This is novel. The paper claims that the objective function is novel: "given the previous frame $x_{t-1}$ of a video and the current frame x_t, reconstruct the current frame $x_t$. This is essentially the same objective as relational autoencoders (Memisevic) and similar to gated and conditional RBMs which have been used to model pairs of frames. Therefore I would recommend de-emphasizingthe novelty of the objective.

Significance - This paper opens up many possibilities for explicit mechanisms of "relative" encodings to produce symbolic representations. There isn't much detail in the results (it's an extended abstract!) but I think the work is exciting and I'm looking forward to reading a follow up paper.

Pros 
- Attacks a major problem of current generative models (entanglement) 
- Proposes a simple yet novel solution 
- Results show visually that the technique seems to work on two non-trivial datasets

Cons 
- Experiments are really preliminary - no quantitative results 
- Doesn't mention mechanisms like dropout which attempt to prevent co-adaptation of features

#### Problem addressed: 
Strategy for training deep neural networks

#### Summary: 
The input distribution (to every layer) undergoes constant changes while training a deep network. The authors call this internal covariate shift in the input distribution. The authors claim this leads to slow learning of optimal model parameters. In order to overcome this, they introduce the idea of normalizing the input of every layer a part of the optimization strategy. Specifically, they reparameterize the input to each layer so that it is whitened and thus has non-changing distribution at every iteration.
 
They apply 2 approximation in their strategy: 

1. this normalization is done for every mini-batch of training data, 

2. the input dimensions are assumed to be uncorrelated. 

Finally, the output of last layer is mean subtracted and variance normalized (these can be back-propagated while training). Additionally, the authors also introduce 2 learnable scalar parameters $(r,b)$ per dimension such that the final input to a layer is $y=rg(BN(x))+b$ where g is the activation function.
 
 The advantage of BN apart from the intuition mentioned above is that it allows higher learning rate and network behavior remains unaffected by the scale of the parameters W and bias. The authors also empirically show that BN acts as a regularizer since optimization without dropout yields at par performance.

#### Novelty:
Previous work only focused on whitening in 1st layer input. This work extends this idea to all layers and suggests a practical approach for applying this idea to real world data.

#### Datasets:
Imagenet

#### Resources:
presentation video available on cedar server

#### Presenter:
Devansh Arpit

### **Keyword**: RNN, serialized model; non-differentiable backpropogarion; action detection in video 


**Abstract**: This paper uses an end-to-end model which is a recurrent neural network trained by REINFORCE to directly predict the temporal bounds of actions. The intuition is that people will observe moments in video and decide where to look to predict when an action is occurring. After training, Serena et al manage to achieve the state-of-art result by only observing 2% of the video frames.

**Model**: In order to take a long video and output all the instances of given action, they use two parts including an observation network and recurrent network. 
* observation network:  encode the visual representation of video frames.
     * input:  $ln$ -- the normalized location of the frame + frame $v_{ln}$
     * network: fc7 feature of finetuned VGG16 network
     * output: $on$ of 1024 dimension indicate time and frame feature 
     
* recurrent network: sequentially process the visual representations and decide where to watch next and whether to emit detection.
##### for each timestep:
     * input:  $on$ -- the representation of the frame + previous state $h_{n-1}$
     * network: $d_n = f_d(h_n; \theta_d)$. $pn = fp(h_n,\theta_p)$,$fd$ is fc. $fp$ is fc+sigmoid  
     * output: $d_n = (s_n,e_n,c_n )$as the candidate detection, where $s_n$,$e_n$ is the start and end of the detection, $c_n$ is confidence level;  $p_n$ whether $d_n$ is a valid detection. $l_{n+1}$ where to observe next. all the parameter falls in [0,1]
https://i.imgur.com/SeFianV.png


**Training**: in order to learn the supervision annotation in long videos and handle the non-differentiable components, authors use BP to train $d_n$ while use REINFORCE to train $p_n$ and $l_{n+1}$
* for $d_n$: $L(D) = \sum_n L_{cls}(d_n) + \sum_n \sum_m 1[y_{mn} = 1] L_{loc}(d_n,g_m)$
* for $p_n,l_{n+1}$: reward $J(\theta) = \sum_{a\in A} p(a) r(a)$ p(a) is the distribution of action and r(a) is the reward for the action. so training needs to maximize this.


**Summary**:
This paper uses a serialized model which first extract the feature from each frame, then use the frame feature and previous state info to generate the next observation time, detection and detection indicator. Specifically, in order to use previous information, they use RNN to store information and use REINFORCE to train $p_n$ and $l_n$, where the goal is to maximize reward for an action sequence and use Monte-Carlo sampling to numerically calculate the gradient for high dimension function.

**questions**:
1. why  $p_n$ and $l_n$ are non-differentiable components?
2. if $p_n$ and $l_n$ are non-differentiable components indeed, how do we come up with REINFORCE to compute the gradient?
3. why don't we get $p_n$ from $p_n = f_p(h_n, \theta_p)$ directly but rather use fp as the parameter in bernoulli distribution, similar question can be applied to calculation for $l_{n+1}$ in trainning time.

 
  

Edge,  contour or boundary detection in 2D images have been an area of active research,  with a variety of different algorithms. However due to a wide variety of image types and content, developing automatic segmentation algorithms have been challenging, while manual segmentation is tedious and time consuming. Previous algorithms approaching this task have tried to incorporate higher level constraints, energy functional (snakes), global  properties (graph based). However the approaches still do not entirely fulfill the fully automated criteria due to a variety of reasons. 

To this end, Barrett et al. propose a graph-based boundary extraction algorithm called Interactive Live-Wire, which is an extension to the original live-wire  algorithm presented in Mortensen et al. [1]. The algorithm is built upon a reformulation of the segmentation approach into graphs, particularly, an image $I$ is converted to an undirected graph with edges from a pixel $p$ to all it's neighbouring 8-connected pixels. Each pixel or node is assigned a local cost according to a function (described later). The segmentation task then becomes a problem where there needs to be a shortest path from a pixel $p$ (seed) to another pixel $q$ (free goal point), where the cumulative cost of path is minimum. The local cost function is defined as:

$l(p,q) = w_G.f_G(q) + w_Z.f_Z(q) + w_D.f_D(p,q)$

where $w_i, i = {G, Z, D}$ are weight coefficients controlling relative importance of the terms. The three terms that make up the local cost function are gradient magnitude ($f_G(q)$), Laplacian Zero-Crossing feature ($f_Z(q)$), and Gradient Direction feature ($f_D(p,q)$). The first term $f_G(q)$ defines a strong measure of edge strength, and is heavily weighted. The term $f_Z(q)$ provides a second degree measure for edge strength in the form of zero-crossing information. The third term $f_D(p,q)$ adds a smoothness constraint to the live-wire boundary by adding high cost for rapidly changing gradient directions. 
 
 The algorithm also exhibits some other features, namely boundary freezing, on-the-fly learning and data-driven cooling. Boundary freezing proves useful when Live-wire segmentation digresses from the desired object boundary during interactive mode. The boundary can be "frozen" right before the digression point by specifying another seed point, until which the boundary is frozen and not allowed to be changed. On-the-fly learning provides robustness to the method by learning the underlying cost distribution of a known "good" boundary, and using that to guide the live-wire further to follow similar distribution. Data-driven path cooling allows the live-wire to generate new seed points automatically as a function of image data and path properties. Pixels on "stable" paths will cool down and eventually freeze, producing new seed points. 
 The results report that average times taken for  segmenting a region using Live-Wire was roughly 4.6 times less than manual human tracing time. Live-Wire provided same amount of accuracy as manual tracing would in a fraction of time, with high reproducibility. 
 
 However,  the method does not provide a way to ``snap out" of an \textit{automatically} frozen live-wire segmentation. On-the-fly training can fail at instances where the edges of the object change too fast, and not much implementation related information is provided, especially for freezing and training parts.