* Presents an architecture dubbed ResNeXt
* They use modules built of
    * 1x1 conv
    * 3x3 group conv, keeping the depth constant. It's like a usual conv, but it's not fully connected along the depth axis, but only connected within groups
    * 1x1 conv
    * plus a skip connection coming from the module input

* Advantages:
    * Fewer parameters, since the full connections are only within the groups
    * Allows more feature channels at the cost of more aggressive grouping
    * Better performance when keeping the number of params constant

* Questions/Disadvantages:
    * Instead of keeping the num of params constant, how about aiming at constant memory consumption? Having more feature channels requires more RAM, even if the connections are sparser and hence there are fewer params
    * Not so much improvement over ResNet

## Task
They introduce a dense version of the human pose estimation task: predict body surface coordinates for each pixel in an RGB image.

Body surface is representated on two levels:
- Body part label (24 parts)
    - Head, torso, hands, feet, etc.
    - Each leg split in 4 parts: upper/lower front/back. Same for arms.
- 2 coordinates (u,v) within body part
    - head, hands, feet: based on SMPL model
    - others: determined by Multidimensional Scaling on geodesic distances

## Data
* They annotate COCO for this task
    - annotation tool: draw mask, then click on a 3D rendering for each of up to 14 points sampled from the mask
    - annotator accuracy on synthetic renderings (average geodesic distance)
       - small parts (e.g. feet): ~2 cm
       - large parts (e.g. torso): ~7 cm

## Method

Fully-convolutional baseline
  - ResNet-50/101
  - 25-way body part classification head (cross-entropy loss)
  - Regression head with 24*2 outputs per pixel (Huber loss)

Region-based approach
  - Like Mask-RCNN
  - New branch with same architecture as the keypoint branch
  - ResNet-50-FPN (Feature Pyramid Net) backbone

Enhancements tested:

- Multi-task learning
  - Train keypoint/mask and dense pose task at once
  - Interaction implicit by sharing backbone net

- Multi-task *cross-cascading*
  - Explicit interaction of tasks
  - Introduce second stage that depends on the first-stage-output of all tasks

- Ground truth interpolation (distillation)
  - Train a "teacher" FCN with the pointwise annotations
  - Use its dense predictions as ground truth to train final net
  - (To make the teacher as accurate as possible, they use ground-truth mask to remove background)

## Results

**Single-person results (train and test on single-person crops)**

Pointwise eval measure:
   - Compute geodesic distance between prediction and ground truth at each annotated point
   - For various error thresholds, plot percentage of points with lower error than the threshold
   - Compute Area Under this Curve

Training (non-regional) FCN on new dataset vs. synthetic data improves AUC10 from 0.20 to 0.38

This paper's FCN method vs. model-fitting baseline
- Baseline: Estimate body keypoint locations in 2D (usual "pose estimation" task) + fit 3D model
- AUC10 improves from 0.23 to 0.43
- Speed: 4-25 fps for FCN vs. model-fitting taking 1-3 minutes per frame (!).

**Multi-person results**

- Region-based method outperforms FCN baseline: 0.25 -> 0.32
    - FCN cannot deal well with varying person scales (despite multi-scale testing)
- Training on points vs interpolated ground-truth (distillation) 0.32 -> 0.38
- AUC10 with cross-task cascade: 0.39

Also: Per-instance eval ("Geodesic Point Similarity" - GPS)
   - Compute a Gaussian function on the geodesic distances
   - Average it within each person instance (=> GPS)
   - Compute precision and recall of persons for various thresholds of GPS
   - Compute average precision and recall over thresholds

Comparison of multi-task approaches:
1. Just dense pose branch (single-task) (AP 51)
2. Adding keypoint (AP 53) OR mask branch (multi-task without cross-cascade) (AP 52)
3. Refinement stage without cross-links (AP 52)
4. Multi-task cross-cascade (keypoints: AP 56, masks: AP 53)

This paper proposes a method to obtain a non-vacuous bound on generalization error by optimizing the PAC-Bayes bound directly. The interesting part is that the authors leverage the black magic of neural net itself to bound the neural net. In order to find the optimal Q, the authors' loss function is an empirical err term plus the $KL(Q|P)$, where they choose the prior $P$ to be $N(0, \lambda I)$, and they also provide justification for choosing the right $\lambda$. Overally, this objective is similar to the variational inference in the Bayesian neural net, and the author is able to obtain a test error bound of $17\%$ on MNIST, while the tradition bounds will be mostly meaningless.

[FashionGAN][1] works as follows. Given an input image of a person and a sentence describing an outfit, the model tries to "redress" the person in the image.
The Generator in the model is stacked. 
* The first stage of the generator gets as input a low resolution version of the segmentation of the input image (which is obtained independently) and the design encoding, and generates a **human segmentation map** (not dressed). 
* Then in the second stage, the model renders the generated image using another generator conditioned on the design encoding. It adds region specific texture using the segmentation map and generates the final image.

![FashionGAN Model](https://i.imgur.com/DzwB8xm.png "FasionGAN model")

They added sentence descriptions to a subset of the [DeepFashion dataset][2] (79k examples).

[1]:http://mmlab.ie.cuhk.edu.hk/projects/FashionGAN/
[2]:http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html

Very efficient data augmentation method. Linear-interpolate training set x and y randomly at every epoch.
```python
for (x1, y1), (x2, y2) in zip(loader1, loader2):
    lam = numpy.random.beta(alpha, alpha)
    x = Variable(lam * x1 + (1. - lam) * x2)
    y = Variable(lam * y1 + (1. - lam) * y2)
    optimizer.zero_grad()
    loss(net(x), y).backward()
    optimizer.step()
```
- ERM (Empirical Risk Minimization) is $\alpha = 0$ version of mixup, i.e. not using mixup.
- Reduces the memorization of corrupt labels.
- Increases robustness to adversarial examples.
- Stabilizes the training of GAN.