Brooks' article is a high-level review that attempts to lay out the complicated relationship between science and technology. Although almost impossible broad in scope, the article does a surprisingly good job that conveys both the depth necessary to treat the subject well and effective use of examples that go into enough specifics and examples to convey his points.

He argues that science contributed to technology in six ways:

1. Direct source of new technological ideas where archetypal ideas might be the atomic bomb or X-Rays.
2. A source of engineering design tools and techniques in ways that might be more common in more engineering-focused scientific investigations.
3. Instrumentation, laboratory techniques, and analytical methods which includes techniques and other innovation created in the process of doing science and where scientists act as sorts of lead users creating new technologists in order to investigate questions that were otherwise not possible.
4. Development of human skills through training students in technologies and scientific techniques and methods.
5. Technology assessment that might look at the side effects of technologies like chemical waste and measurement of side effects.
6. Source of development strategy that might help scientists avoid blind alleys.

Additionally, he argues that technology contributes to science in two ways:

1. Source of new challenges as has been the case in material science which are driven by technological research.
2. Instrumentation and measurement techniques where technologists create tools that end up being useful to science more generally and so that scientists don't have to create all their own tools or focus on the parts of tool creation that they are less good at.

Harvey Brooks was the dean of the Harvard Division of Engineering and Applied Sciences for nearly 20 years (1957-1976) before founding the center for Science, Technology and Public Policy at the Kennedy School in 1976. This paper was published more than 10 years after his retirement.

#### Theoretical and practical relevance:

The paper is a "semi-famous" paper and is more of a review article than an empirical piece but plays an important role in framing questions around science policy and has been cited by others exploring the relationship or making policy claims about the promotion of science for public policy reasons.

_Objective:_ Improve GANs convergence to more diverse and visually pleasing images at higher resolution using a novel equilibrium method between the discriminator and the generator that also simplifies training procedures.

_Dataset:_ [LFW](http://vis-www.cs.umass.edu/lfw/)

## Inner workings:

They try to match the distribution of the errors (assumed to be normally distributed) instead of matching the distribution of the samples directly. In order to do this they compute the Wasserstein distance between a pixel-wise autoencoder loss distributions of real and generated samples defined as follow:

1.  Autoencoder loss:

[![screen shot 2017-04-24 at 3 46 32 pm](https://cloud.githubusercontent.com/assets/17261080/25340190/429f9788-2905-11e7-88dc-b44567b9cd34.png)](https://cloud.githubusercontent.com/assets/17261080/25340190/429f9788-2905-11e7-88dc-b44567b9cd34.png)

2.  Wasserstein distance for two normal distributions μ1 = N(m1, C1) and μ2 = N(m2, C2)

[![screen shot 2017-04-24 at 3 46 44 pm](https://cloud.githubusercontent.com/assets/17261080/25340191/42b23474-2905-11e7-9810-58d5326bf886.png)](https://cloud.githubusercontent.com/assets/17261080/25340191/42b23474-2905-11e7-9810-58d5326bf886.png)

They also introduce an equilibrium concept to account for the situation when `G` and `D` are not well balanced and the discriminator `D` wins easily. This is controlled by what they call the diversity ratio that balances between auto-encoding real images and discriminating real from generated images. It is defined as follow:  
[![screen shot 2017-04-24 at 3 56 29 pm](https://cloud.githubusercontent.com/assets/17261080/25340609/992c2188-2906-11e7-8c51-498bbd293119.png)](https://cloud.githubusercontent.com/assets/17261080/25340609/992c2188-2906-11e7-8c51-498bbd293119.png)

To maintain this balance they use a standard SGD but they introduce a variable `kt` initially 0 to control how much emphasis is put on the generator `G`. This removes the need to do `x` steps on `D` followed by `y` steps on `G` or to pretrained one of the two.  
[![screen shot 2017-04-24 at 3 59 57 pm](https://cloud.githubusercontent.com/assets/17261080/25340859/4ee06476-2907-11e7-971f-90421449cb51.png)](https://cloud.githubusercontent.com/assets/17261080/25340859/4ee06476-2907-11e7-971f-90421449cb51.png)

Finally they derive a global convergence measure by using the equilibrium concept that can be used to determine when the network has reached its final state or if the model has collapsed:  
[![screen shot 2017-04-24 at 4 04 12 pm](https://cloud.githubusercontent.com/assets/17261080/25340998/b8bf6ad6-2907-11e7-8afa-294cae32c6af.png)](https://cloud.githubusercontent.com/assets/17261080/25340998/b8bf6ad6-2907-11e7-8afa-294cae32c6af.png)

## Architecture:

They tried to keep the architecture simple to really study the impact of their new equilibrium principle and loss. They don't use batch normalization, dropout, transpose convolutions or exponential growth for convolution filters.

[![screen shot 2017-04-24 at 4 09 29 pm](https://cloud.githubusercontent.com/assets/17261080/25341219/6fb7be28-2908-11e7-8774-287c1b7d7684.png)](https://cloud.githubusercontent.com/assets/17261080/25341219/6fb7be28-2908-11e7-8774-287c1b7d7684.png)

## Results:

They trained on images from 32x32 to 256x256, but at higher resolution images tend to lose sharpness. Nevertheless images are very very good!  
[![screen shot 2017-04-24 at 4 20 30 pm](https://cloud.githubusercontent.com/assets/17261080/25341699/f99b0770-2909-11e7-84a0-3ac0436771e5.png)](https://cloud.githubusercontent.com/assets/17261080/25341699/f99b0770-2909-11e7-84a0-3ac0436771e5.png)

This paper is about data collection for face recognition.

One idea was to use weaker classifiers to rank the data presented to the annotators.

This paper considers a class of structural equation models for times series data.  The models allow nonlinear instantaneous effects and lagged effects. On the other hand, Granger-causality based methods do not allow instantaneous effects and a linear non-Gaussian method TS-LiNGAM (Hyvarinen et al., ICML2008, JMLR2010) assumes linear effects. 

This paper introduces a model and procedure for learning instantaneous and lagged causal relationships among variables in a time series when each causal relationship is either identifiable in the sense of the additive noise model (Hoyer et al. 2009) or exhibits a time structure. The learning procedure finds a causal order by iteratively fitting VAR or GAM models where each variable is a function of all other variables and making the variable with the least dependence the lowest variable in the order. Excess parents are then pruned to produce the summary causal graph (where x->y indicates either an instantaneous or lagged cause up to the order of the VAR or GAM model that is fit). Experiments show that the method outperforms competing methods and returns no results in cases where the model can be identified (rather than wrong results). 

This paper is covered by author in this [talk](https://github.com/udibr/notes/blob/master/Talk%20by%20Sasha%20Rush%20-%20Interpreting%2C%20Training%2C%20and%20Distilling%20Seq2Seq%E2%80%A6.pdf)