## IDS approaches for events
** Misuse-based: **
Detects events that violate system policy. *Snort* is a signature-based open-source IDS system used for misuse detection in this research. 

** Anomaly-based: ** 
Detects events that contain abnormal activity. Uses statistical, heuristic and data mining methods.  *Packet header anomaly detector (PHAD)* [1] and *Network traffic anomaly detector (NETAD)* [2] are used as *Snort* preprocessors for anomaly detection.


## Hybrid architecture
**Preprocessors** (for *Snort)*: Can filter and modify packets before it reaches main detection engine. They can also generate alerts individually.

In section 4, researchers explain *Snort* software details for preprocessor source-code integration and their implementation of the hybrid system.

## Evaluation of hybrid system
*MIT Lincoln Laboratory IDEVAL dataset* used for evaluation of IDS system. A more detailed explanation and simulation environment information about the dataset can be found in section 5, as well in reference articles [3-5].

https://i.imgur.com/SrAvySJ.png

Researchers evaluated *Snort* only, *Snort + PHAD* and *Snort + PHAD + NETAD * (hybrid IDS) variations and compared detection capabilities of each system based on detected intrusion counts. Results can be seen in the figure. Based on results hybrid IDS is the most successful on *IDEVAL dataset*. 

## References

[1] Mahoney MV, Chan PK. PHAD: packet header anomaly detection for identifying hostile network traffic. Florida Institute of Technology Technical Report, CS-2001-04.

[2] Mahoney MV. Network traffic anomaly detection based on packet bytes. In Proceedings of ACM-SAC; 2003.

[3] Lippman R, Haines JW, Fried DJ, Korba J, Das K. Analysis and results of the 1999 DARPA off-line intrusion detection evaluation. In Proceedings of the
third international workshop on recent advances in intrusion detection, Toulouse, France; 2–4 October 2000. p. 162–82.

[4] Haines JW, Lippman R, Fried DJ, Zissman MA, Tran E, Boswell SB. 1999 DARPA intrusion detection evaluation: design and procedures. MIT Lincoln Laboratory Technical Report, TR-1062, Massachusetts, USA; 2001.

[5] Data set, DARPA intrusion detection evaluation data set; 1999. <http://www.ll.mit.edu/IST/ideval/data/1999/1999_data_index.html>
.

Deep Learning has a number of shortcomings.

(1)Requires lot of data: Humans can learn abstract concepts with far less training data compared to current deep learning. E.g. If we are told who an “Adult” is, we can answer questions like how many adults are there in home?, Is he an adult? etc. without much data. Convolution networks can solve translational invariance but requires lot more data to identify other translations or more filters or different architectures. 

(2)Lack of transfer: Most of claims of Deep RL helping in transfer is ambiguous. Consider Deepmind claim of concept learning in Breakout such as digging a tunnel through a wall which was soon proved false by Vicarious experiments that added wall in middle and increased Y coordinate of paddle. Current attempt of transfer is based on correlations between trained sequences and test scenario, which is bound to fail when current scenario is tweaked. 

(3)Hierarchical structure not learnt: Deep learning learns correlations which are non-hierarchical in nature. So sentences like “Salman Khan, who was excellent driver, died in a car accident” can never be represented as major clause(Salman Khan) and minor clause(who was excellent driver) format. Subtleties like these cannot be captured by RNN even though hierarchical RNN tries to capture obvious hierarchies like (letters -> words -> sentences). If hierarchies were captured in Deep RL, transfer would have been easy in Breakout which is not the case. 

(4)Poor inference in language: Sentences that have subtle differences like “John promised Mary to leave” and “John promised to leave Mary” are treated as same by deep learning. This causes major problems during inferencing because questions related to combining various sentences fail. 

(5)Not transparent: Why the neural network made the decision in a certain way can help in debuggability and prove to be beneficial in medical diagnosis systems where it is critical to reason out methodology. 

(6)No priors and commonsense reasoning: Humans function with commonsense reasoning(If A is dad of B, A is elder to B) and priors(physics laws). Deep Learning does not tailor to incorporate this. With heavy interest in end to end learning from raw data, such attempts have been discouraged. 

(7)Deep Learning is correlation not causation: Causality or analogical reasoning or any abstract concepts of left brain is not dealt by deep learning. (8)Lacks generalization outside training distribution: Fails to incorporate scenario in which nature of data is varying. E.g. Stock prediction. (9)Easily fooled: E.g. Parking signs mistaken for refrigerators, turtle mistaken as rifle. 

This can be addressed by:
(1)Unsupervised learning: Build systems that can set their own goals, use abstract knowledge(priors, affordances as objects can be used in any way etc) and solve problem at high level(like symbolic AI). 

(2)Symbolic AI - Deep Learning does what primary sensory cortex does of taking raw inputs and converting it into low level representation. Symbolic AI builds abstract concepts like causal, analogical reasoning which is what prefrontal Cortex does. Humans make decisions based on these abstract concepts.

In object detection the boost in speed and accuracy is mostly gained through network architecture changes.This paper takes a different route towards achieving that goal,They introduce a new loss function called focal loss.

The authors identify class imbalance as the main obstacle toward one stage detectors achieving results which are as good as two stage detectors.

The loss function they introduce is a dynamically scaled cross entropy loss,Where the scaling factor decays to zero as the confidence in the correct class increases.

They add a modulating factor  as shown in the image below to the cross- entropy loss  https://i.imgur.com/N7R3M9J.png
Which ends up looking like this https://i.imgur.com/kxC8NCB.png
in experiments though they add an additional alpha term to it,because it gives them better results.

**Retina Net**

The network consists of a single unified network which is composed of a backbone network and two task specific subnetworks.The backbone network computes the feature maps for the input images.The first sub-network helps in object classification of the backbone networks output and the second sub-network helps in bounding box regression.
The backbone network they use is Feature Pyramid Network,Which they build on top of ResNet.

The paper combines reinforcement learning with active learning to learn when to request labels to improve prediction accuracy. 

- The model can either predict the label at time step $t$ or request it in the next time step, in form of a one-hot vector output of an LSTM with the previous label (if requested) and the current image as an input.
- A reward is issued based on the outcome of requesting labels (-0.05), or correctly (+1) or incorrectly(-1) predicting the label.
- The optimal strategy involves storing class embeddings and their labels in memory and only requesting labels if a unseen class is encountered.
 
The model is evaluated against the *Omniglot* dataset and learns a non-naive strategy to request fewer labels the more data of a class was encountered, using a learned uncertainty measure.
The magnitude of the reward for incorrect labeling decides on the amount of requested labels and can be used to maximize the accuracy during prediction.


## Active Learning
Active Learning is a  special case of semi-supervised learning, which aims to reduce the amount of supervision needed during training. The model typically selects which datapoints to label by applying different metrics like most information content, highest uncertainty or other heuristics.

## Reinforcement Learning
Reinforcement learning agents try to learn an optimal policy $\pi^*(s_t)$ for a state $s_t$ at time $t$ that will maximize future rewards issued by the environment, by choosing an action $a_t$. The policy is represented by a function $Q^*(s_t, a_t)$, which can be approximated and learned in form of a neural network.

The paper starts with the BNN with latent variable and proposes an entropy-based and a variance-based measure of prediction uncertainty. For each uncertainty measure, the authors propose a decomposition of the aleatoric term and epistemic term. A simple regression toy experiment proves this decomposition and its measure of uncertainty. Then the author tries to improve the regression toy experiment performance by using this uncertainty measure into an active learning scheme. For each batch, they would actively sample which data to label. The result shows that using epistemic uncertainty alone outperforms using total certainty, which both outperforms simple gaussian process. The result is understandable since epistemic is directly related to model weight uncertainty, and sampling from high aleatoric uncertain area does help supervised learning.

Then the authors talk about how to extend the model based RL by adding a risk term which consider both aleatoric term and epistemic term, and its related to model-bias and noise aversion. The experiments on Industrial Benchmark shows the method is able prevent overfitting the learned model and better transfer to real world, but the method seems to be pretty sensitive to $\beta$ and $\gamma$.