This paper presents locally linear embedding (LLE) for nonlinear dimensionality reduction. LLE can learn the structure of the underlying low-dimensional manifold of the sampled data in high dimensional space. Therefore, LLE can preserve the distance in the manifold space much better than PCA. Unlike PCA which projects high dimensional space to low dimensional space with a global linear matrix, LLE seeks locally linear projections for locally linear patches formed by neighboring data points. Using many locally linear projections instead of a global linear projection is the key to nonlinear dimensionality reduction.
Technical details 

Fig. 1 shows the problem if nonlinear dimensionality reduction.

![](http://1.bp.blogspot.com/-Vpkju74n9w8/VSX3RGAjKtI/AAAAAAAAA08/NqJA7v1MLuQ/s1600/dimreduct.png)


Fig. 2 summarizes the LLE algorithm. The neighbors of each data point can be computed by K-nearest neighbor or by collecting the data points within a radius. The weights in step 2 reflect intrinsic geometric properties of the data that are invariant to locally linear projections. The third step finds new data points projected by locally linear projections in low-dimensional space.

![](http://3.bp.blogspot.com/-kVz0nYKKSQ0/VSX3_Aio4lI/AAAAAAAAA1E/hrltFFQ7njY/s1600/LLEgraph.png)

![](http://3.bp.blogspot.com/-bfgIDOavP5s/VSX23Jko4tI/AAAAAAAAA0s/IsE4Th0SN1Y/s1600/LLE.png)


$\varepsilon (W) = \displaystyle\sum\_i | \vec{X} = \sum\_j W\_{ij} \vec{X}\_j|^2$

$\Phi (Y) = \displaystyle\sum\_i | \vec{Y} = \sum\_j W\_{ij} \vec{T}\_j|^2$


Results

Figure 4 shows the results of dimensional reduction of images of lips using CFA and LLE.

![](http://3.bp.blogspot.com/-STDK5O9TBX4/VSX4TyurcvI/AAAAAAAAA1M/XUoZ-CmnObQ/s1600/lips.png)

This paper proposes an approach to measure motion similarity between human-human and human-object interaction. The authors claim that human activities are usually defined by the interaction between individual characters, such as a high-five interaction.

As the interaction datasets are not available authors provide multiple small-scale interaction datasets:
https://i.imgur.com/P815TYu.png
where:
- 2C = a Character-Character (2C) database using kick-boxing motions
- CRC = Character-Retargeted Character where the size of characters is adjusted while maintaining the nature of the interaction
https://i.imgur.com/XX1WNpO.png
- HOI = Human-Object Interaction where a Chair is used as an object
https://i.imgur.com/Z6cxd7R.png
- 2PB = 2 People Boxing 
https://i.imgur.com/yUxmpY5.png
- 2PD = 2 People Daily Interaction where people represented as a surface point cloud
https://i.imgur.com/EzyELg3.png


**Methodology**
- *Customized Interaction Mesh Structure*. An interaction mesh is created by generating a volumetric mesh using Delaunay Tetrahedralization. Interaction is therefore represented by a series of interaction meshes. To reduce the bias of unequal number of points per human body part, synthetic points (shown in blue) are derived from available skeleton structure (shown in red):
 https://i.imgur.com/jtlrH49.png

 The edges after Delaunay Tetrahedralization are filtered in a way that all edges connecting to the same character are removed, as they do not contribute to the interaction.
 https://i.imgur.com/nWeUNl1.png
 The temporal sequence of interaction is a series of interaction meshes.

- *Distance between interaction meshes*.
   - Distance between two interaction meshes of two-character interactions:

   $d(e_i, e_j) = (|e_{i1} - e_{j1}| + |e_{i2} - e_{j2}|) \times \frac{1}{2} (1 - cos\theta),$
   https://i.imgur.com/jMHGx3o.png
   where $e_{i1}$ and $e_{i2}$ are two endpoints of an edge.

   - *Earth Mover's Distance*. Earth Mover’s Distance (EMD) is used to find the best correspondence between the input interaction meshes to achieve the comparison of two interactions with different semantic meaning:

    $EMD(E_I^{t_I}, E_J^{tJ}) = \frac{D(E_I^{t_I}, E_J^{tJ})}{\sum_{i=1}^{m} \sum_{j=1}^{n} f_{i,j}^*},$

    where $D(E_I^{t_I}, E_J^{tJ}) = \sum_{i=1}^{m} \sum_{j=1}^{n} d(e_i, e_j) f_{i,j}^*$ represents the minimum distance between two interaction meshes and $f_{i,j}^*$ is the optimal set of flow values returned by the mass transport solver that finds the optimal edge-level correspondence between two interaction meshes. The concept of the mass transport solver is visualized below:
    https://i.imgur.com/wmRhcxP.png


 - *Distance between interactions sequences*.
    - spatial normalization - removing its pelvis translation and its horizontal facing angle in each frame

    - temporal sampling - non-linear sampling strategy based on the frame distance measured by EMD. The sampling algorithm samples fewer in temporal regions with high similarity, which contribute less to the context of the interaction.

    - temporal alignment - keyframes are aligned using Dynamic Time Warping (DTW)

Possible functionality with the proposed method:
- Interaction motion similarity analysis
- Interaction motion retrieval

Notice:
>The pre-process took 1.5 hours, 0.5 hour and 4 hours for the 2C, CRC and HOI databases respectively. Given the meshes, computing the distance between two interactions took 0.2 seconds on average.

Discussion:
- the algorithm focuses on boxing/kickboxing as they have clear rules. Extending the proposed algorithm for measuring motion similarity for daily activities would require careful annotation.
- While the method, in theory, can be applied to single human activities, it quite clear that it would perform worse than other baselines for the task. This implies that it is better not applied for use cases where a comparison between two independent movements having no interaction with each other (dancing, yoga) is required.
- The method works best for close interaction activities as the edges in the interaction mesh would tend to have a similar structure (e.g. edge length) in case of distant interacting objects. 
- Application-wise, the algorithm is not suitable for online real-time use.

#### Goal:

Predict interventions on ICU patients using LSTM and CNN. 

#### Dataset:

MIMIC-III v.1.4 https://mimic.physionet.org/

+ Patients over 15 years of age with intensive care stay between 12h and 240h. (Only the first stay is considered for each patient) - 34148 unique records.
+ 5 static variables.
+ 29 vital signs and test results.
+ Clinical notes of patients (presented as time series).

#### Feature Engineering:

+ Topic Modeling of clinical notes: Vector of topics using Latent Dirichlet Allocation (LDA)
+ Physiological Words: Vital / Laboratory results converted to z-scores - [integer values ​​between -4 and 4] and score is one-hot encoded (each vital / lab is replaced by 9 columns). It is good idea to avoid the imputation of missing values as the physiological word in this case is the all-zero vector.

Feature vector:
+ is the concatenation of the static variables, physiological words for each vital/lab and the  topic vector.
+ 1 feature vector / patient / hour.
+ 6-hour slice used to predict a 4-hour window after a 6-hour gap. All the features values are normalized between 0 and 1. (static variables are replicated).

#### Target Classes:

For some of the procedures to be predicted there are 4 classes:
+ Onset: Y goes from 0 to 1 during the prediction window.
+ Wean: Y goes from 1 to 0 during the prediction window.
+ Stay On: Y stays at 1 throughout prediction window.
+ Stay Off: Y stays at 0 for the entire prediction window.

#### Setup of the Experiments:

+ Dataset Split: 70% training, 10% validation, 20% test.

Long Short Term Memory (LSTM) Networks:
+ Dropout P(keep) = 0.8, L2 regularization.
+ 2 hidden layers: 512 nodes in each.

Convolutional Neural Networks
+ 3 different temporal granularities (3, 4, 5 hours). 64 filters in each.
+ Features are treated as channels. 1D temporal convolution.
+ Dropout between fully connected layers. P (keep) = 0.5.

TensorFlow 1.0.1 - Adam optimizer. Minibatches of size 128. 
Validation set used for early stopping (metric: AUC).

#### Results:

+ Baseline for comparison: L2-regularized Logistic Regression
+ Metrics: 
 + AUC per class.
 + AUC macro = Arithmetic mean of AUC per class.

+ Proposed architectures outperforms baseline.
+ Physiological words improve performance (especially on high class imbalance scenario).

#### Model Interpretability:
+ LSTM: feature occlusion like analysis. The feature is replaced by uniformly distributed noise between 0 and 1 and variation in AUC is computed.
+ CNN: analysis of the maximally activating trajectories.

#### Positive Aspects:
+ Relevant work: In the healthcare domain is very important to anticipate events.
+ Built on top of rich and heterogeneous data: It leverages large amounts of ICU data. 
+ The proposed model is not a complete black-box. Interpretability is crucial if the system is to be adopted in the future. 

#### Caveats:
+ Some of the methodology is not clearly explained:
 + How the split of the dataset was performed? Was it on a patient-level?
 + When testing the logistic regression baseline it is not clear how the feature vector was built. Was it built by simply flattening the 6-hour chunk?
 + For the raw data test, it was not mentioned the way the missing values were treated. 

The Escherichia coli chemotaxis network is a model system for biological signal processing. In E. coli, transmembrane receptors responsible for signal transduction assemble into large clusters containing several thousand proteins. These sensory clusters have been observed at cell poles and future division sites. Despite extensive study, it remains unclear how chemotaxis clusters form, what controls cluster size and density, and how the cellular location of clusters is robustly maintained in growing and dividing cells. Here, we use photoactivated localization microscopy (PALM) to map the cellular locations of three proteins central to bacterial chemotaxis (the Tar receptor, CheY, and CheW) with a precision of 15 nm. We find that cluster sizes are approximately exponentially distributed, with no characteristic cluster size. One-third of Tar receptors are part of smaller lateral clusters and not of the large polar clusters. Analysis of the relative cellular locations of 1.1 million individual proteins (from 326 cells) suggests that clusters form via stochastic self-assembly. The super-resolution PALM maps of E. coli receptors support the notion that stochastic self-assembly can create and maintain approximately periodic structures in biological membranes, without direct cytoskeletal involvement or active transport.

Reward functions are a funny part of modern reinforcement learning: enormously salient from the inside, if you’re coding or working with RL systems, yet not as clearly visible from the outside perspective, where we just see agents playing games in what seem to be human-like ways. Just seeing things from this angle, it can be easy to imagine that the mechanisms being used to learn are human-like as well. And, it’s true that some of the Atari games being examined are cases where there is in fact a clear, explicit reward in the form of points, that human players would also be trying to optimize. But in most cases, the world isn’t really in the habit of producing clear reward signals, and it definitely doesn’t typically do so on time scales that account for most of the learning humans do. 

So, it’s generally hypothesized that in addition to updating on (sparse) environmental rewards, humans also operate according to certain pre-coded, possibly evolutionarily-engineered heuristics, of which one is curiosity. The intuition is: it sure seems like, especially early in life, humans learn by interacting with objects purely driven by curiosity, and we’d love to somehow harness that same drive to allow our learning systems to function in environments lacking dense, informative reward signals. One such environment is the video game Montezuma’s Revenge, which in addition to being amusingly difficult to search for, is a game with sparse, long-range rewards, on which typical reward-based agents have historically performed poorly, and on which this current paper focuses. 

A strong existing tradition of curiosity objectives focuses on incentivizing agents to be able to better predict the next observation, given the current observation and their action within it. Intuitively, by training such a network on historical observations, and giving agents a bonus according to that prediction’s error on a given observation. The theory behind this is that if an agent isn’t able to predict the observation-transition dynamics at a given state, that probably means it hasn’t visited many nearby states, and so we want to incentivize it doing so to gain information. If this sounds familiar to the classic “explore vs exploit” trade-off, it’s very much a similar idea: in cases of clear reward, we should take the reward, but in cases of low or uncertain reward, there’s value to exploration. 

One difficulty of systems like the one described above is that they reward the agent for being in environments where the next observation is difficult to predict from the current one. And while that could describe novel states about which the agent needs to gain information, it can also describe states that are inherently stochastic; the canonical example being random static on a TV screen. The agent has a lot of trouble predicting the next observation because it’s fundamentally non-deterministic to a greater degree than even the random-but-causal dynamics of most games. The proposed alternative of this paper is a little strange, but makes more sense in the context of responding to this stochasticity problem. The authors propose to create a random mapping, in the form of an initialized but untrained neural network, taking in observations and spitting out embedding vectors. Then, they incentivize their agent to go to places that have high prediction error on a network designed to predict these random embeddings. Since the output is just a function mapping, it’s deterministic with respect to observations. The idea here is that if you’ve seen observations similar to your current observation, you’ll be more able to predict the corresponding embedding, even if there’s no meaningful relationship that you’re learning. 

https://i.imgur.com/Ds5gHDE.png

The authors found that this performed well on Montezuma’s Revenge and Private Eye, but only middlingly-well on other environments. I’m a bit torn on this paper overall. On one hand, it seems like a clever idea, and I’m in general interested in seeing more work on curiosity. It does clearly seem to be capturing something that corresponds to novelty-seeking, and the agent trained using it explores a higher number of rooms than alternative options. On the other, I’m a little skeptical of the fact that it only has consistent performance in two environments, and wish there had been more comparisons to simpler forms of observation similarity, since this really does just seem like a metric of “how similar of observation vectors to this have you seen before”. I find myself wondering if some sort of density modeling could even be effective here, especially if (as may be the case, I’m unsure) the input observations are metadata rather than pixels.