**Stochastic Depth** aims to shrink the depth of a network during training, while
keeping it unchanged during testing. This is achieved by randomly dropping entire [ResBlocks](https://paperswithcode.com/method/residual-block) during training and bypassing their transformations through skip connections. 

Let $b\_{l} \in$ {$0, 1$} denote a Bernoulli random variable, which indicates whether the $l$th ResBlock is active ($b\_{l} = 1$) or inactive ($b\_{l} = 0$). Further, let us denote the “survival” probability of ResBlock $l$ as $p\_{l} = \text{Pr}\left(b\_{l} = 1\right)$. With this definition we can bypass the $l$th ResBlock by multiplying its function $f\_{l}$ with $b\_{l}$ and we extend the update rule to:

$$ H\_{l} = \text{ReLU}\left(b\_{l}f\_{l}\left(H\_{l-1}\right) + \text{id}\left(H\_{l-1}\right)\right) $$

If $b\_{l} = 1$, this reduces to the original [ResNet](https://paperswithcode.com/method/resnet) update and this ResBlock remains unchanged. If $b\_{l} = 0$, the ResBlock reduces to the identity function, $H\_{l} = \text{id}\left((H\_{l}−1\right)$.

Stochastic Depth

Deep Networks with Stochastic Depth

**MixText** is a semi-supervised learning method for text classification, which uses a new data augmentation method called TMix. TMix creates a large amount of augmented training samples by interpolating text in hidden space. The technique leverages advances in data augmentation to guess low-entropy labels for unlabeled data, making them as easy to use as labeled data.

MixText

MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification

**RetinaNet-RS** is an object detection model produced through a model scaling method based on changing the the input resolution and [ResNet](https://paperswithcode.com/method/resnet) backbone depth. For [RetinaNet](https://paperswithcode.com/method/retinanet), we scale up input resolution from 512 to 768 and the ResNet backbone depth from 50 to 152. As RetinaNet performs dense one-stage object detection, the authors find scaling up input resolution leads to large resolution feature maps hence more anchors to process. This results in a higher capacity dense prediction heads and expensive NMS. Scaling stops at input resolution 768 × 768 for RetinaNet.

RetinaNet-RS

Simple Training Strategies and Model Scaling for Object Detection

Class of methods in Bayesian Statistics where the posterior distribution is approximated over a rejection scheme on simulations because the likelihood function is intractable.

Different parameters get sampled and simulated. Then a distance function is calculated to measure the quality of the simulation compared to data from real observations. Only simulations that fall below a certain threshold get accepted.

Image source: [Kulkarni et al.](https://www.umass.edu/nanofabrics/sites/default/files/PDF_0.pdf)

Accelerating Simulation-based Inference with Emerging AI Hardware

FCANet contains a novel multi-spectral channel attention module. Given an input feature map $X \in \mathbb{R}^{C \times H \times W}$, multi-spectral channel attention first splits $X$ into many parts $x^{i} \in \mathbb{R}^{C' \times H \times W}$. Then it applies a 2D DCT to each part $x^{i}$. Note that a 2D DCT can use pre-processing results to reduce computation. After processing each part,  all results are concatenated into a vector. Finally, fully connected layers, ReLU activation and a sigmoid are used to get the attention vector as in an SE block. This can be formulated as:
\begin{align}
    s = F_\text{fca}(X, \theta) & = \sigma (W_{2} \delta (W_{1}[(\text{DCT}(\text{Group}(X)))]))
\end{align}
\begin{align}
    Y & = s  X
\end{align}
where $\text{Group}(\cdot)$ indicates dividing the input into many groups and $\text{DCT}(\cdot)$ is the 2D discrete cosine transform. 

This work based on information compression and discrete cosine transforms achieves excellent performance on the classification task.

FcaNet

FcaNet: Frequency Channel Attention Networks

**Criss-Cross Network** (**CCNet**) aims to obtain full-image contextual information in an effective and efficient way. Concretely,
for each pixel, a novel criss-cross attention module harvests the contextual information of all the pixels on its criss-cross path. By taking a further recurrent operation, each pixel can finally capture the full-image dependencies. **CCNet** is with the following
merits: **1)** GPU memory friendly. Compared with the [non-local block](https://paperswithcode.com/method/non-local-block), the proposed recurrent criss-cross attention module requires 11× less GPU memory usage. **2)** High computational efficiency. The recurrent criss-cross attention significantly reduces FLOPs by about 85% of the non-local block. **3)** The state-of-the-art performance.

CCNet

CCNet: Criss-Cross Attention for Semantic Segmentation

**KNN and IoU-based Verification** is used to verify detections and choose between multiple detections of the same underlying object. It was originally used within the context of blood cell counting in medical images. To avoid this double counting problem, the KNN algorithm is applied in each platelet to determine its closest platelet and then using the intersection of union (IOU) between two platelets we calculate their extent of overlap. The authors allow 10% of the overlap between platelet and its closest platelet based on empirical observations. If the overlap is larger than that, they ignore that cell as a double count to get rid of spurious counting.

KNN and IOU based verification

Machine learning approach of automatic identification and counting of blood cells

**AmoebaNet** is a convolutional neural network found through regularized evolution architecture search. The search space is NASNet, which specifies a space of image classifiers with a fixed outer structure: a feed-forward stack of [Inception-like modules](https://paperswithcode.com/method/inception-module) called cells. The discovered architecture is shown to the right.

AmoebaNet

Regularized Evolution for Image Classifier Architecture Search

**A3C**, **Asynchronous Advantage Actor Critic**, is a policy gradient algorithm in reinforcement learning that maintains a policy $\pi\left(a\_{t}\mid{s}\_{t}; \theta\right)$ and an estimate of the value
function $V\left(s\_{t}; \theta\_{v}\right)$. It operates in the forward view and uses a mix of $n$-step returns to update both the policy and the value-function. The policy and the value function are updated after every $t\_{\text{max}}$ actions or when a terminal state is reached. The update performed by the algorithm can be seen as $\nabla\_{\theta{'}}\log\pi\left(a\_{t}\mid{s\_{t}}; \theta{'}\right)A\left(s\_{t}, a\_{t}; \theta, \theta\_{v}\right)$ where $A\left(s\_{t}, a\_{t}; \theta, \theta\_{v}\right)$ is an estimate of the advantage function given by:

$$\sum^{k-1}\_{i=0}\gamma^{i}r\_{t+i} + \gamma^{k}V\left(s\_{t+k}; \theta\_{v}\right) - V\left(s\_{t}; \theta\_{v}\right)$$

where $k$ can vary from state to state and is upper-bounded by $t\_{max}$.

The critics in A3C learn the value function while multiple actors are trained in parallel and get synced with global parameters every so often. The gradients are accumulated as part of training for stability - this is like parallelized stochastic gradient descent.

Note that while the parameters $\theta$ of the policy and $\theta\_{v}$ of the value function are shown as being separate for generality, we always share some of the parameters in practice. We typically use a convolutional neural network that has one [softmax](https://paperswithcode.com/method/softmax) output for the policy $\pi\left(a\_{t}\mid{s}\_{t}; \theta\right)$ and one linear output for the value function $V\left(s\_{t}; \theta\_{v}\right)$, with all non-output layers shared.

Asynchronous Methods for Deep Reinforcement Learning

**PREDATOR** is a model for pairwise point-cloud registration with deep attention to the overlap region. Its key novelty is an overlap-attention block for early information exchange between the latent encodings of the two point clouds. In this way the subsequent decoding of the latent representations into per-point features is conditioned on the respective other point cloud, and thus can predict which points are not only salient, but also lie in the overlap region between the two point clouds.

PREDATOR

PREDATOR: Registration of 3D Point Clouds with Low Overlap

**Trust Region Policy Optimization**, or **TRPO**, is a policy gradient method in reinforcement learning that avoids parameter updates that change the policy too much with a KL divergence constraint on the size of the policy update at each iteration.

Take the case of off-policy reinforcement learning, where the policy $\beta$ for collecting trajectories on rollout workers is different from the policy $\pi$ to optimize for. The objective function in an off-policy model measures the total advantage over the state visitation distribution and actions, while the mismatch between the training data distribution and the true policy state distribution is compensated with an importance sampling estimator:

$$ J\left(\theta\right) = \sum\_{s\in{S}}p^{\pi\_{\theta\_{old}}}\sum\_{a\in\mathcal{A}}\left(\pi\_{\theta}\left(a\mid{s}\right)\hat{A}\_{\theta\_{old}}\left(s, a\right)\right) $$

$$ J\left(\theta\right) = \sum\_{s\in{S}}p^{\pi\_{\theta\_{old}}}\sum\_{a\in\mathcal{A}}\left(\beta\left(a\mid{s}\right)\frac{\pi\_{\theta}\left(a\mid{s}\right)}{\beta\left(a\mid{s}\right)}\hat{A}\_{\theta\_{old}}\left(s, a\right)\right) $$

$$ J\left(\theta\right) = \mathbb{E}\_{s\sim{p}^{\pi\_{\theta\_{old}}}, a\sim{\beta}} \left(\frac{\pi\_{\theta}\left(a\mid{s}\right)}{\beta\left(a\mid{s}\right)}\hat{A}\_{\theta\_{old}}\left(s, a\right)\right)$$

When training on policy, theoretically the policy for collecting data is same as the policy that we want to optimize. However, when rollout workers and optimizers are running in parallel asynchronously, the behavior policy can get stale. TRPO considers this subtle difference: It labels the behavior policy as $\pi\_{\theta\_{old}}\left(a\mid{s}\right)$ and thus the objective function becomes:

$$ J\left(\theta\right) = \mathbb{E}\_{s\sim{p}^{\pi\_{\theta\_{old}}}, a\sim{\pi\_{\theta\_{old}}}} \left(\frac{\pi\_{\theta}\left(a\mid{s}\right)}{\pi\_{\theta\_{old}}\left(a\mid{s}\right)}\hat{A}\_{\theta\_{old}}\left(s, a\right)\right)$$

TRPO aims to maximize the objective function $J\left(\theta\right)$ subject to a trust region constraint which enforces the distance between old and new policies measured by KL-divergence to be small enough, within a parameter $\delta$:

$$ \mathbb{E}\_{s\sim{p}^{\pi\_{\theta\_{old}}}} \left[D\_{KL}\left(\pi\_{\theta\_{old}}\left(.\mid{s}\right)\mid\mid\pi\_{\theta}\left(.\mid{s}\right)\right)\right] \leq \delta$$

TRPO

Trust Region Policy Optimization

**RetinaMask** is a one-stage object detection method that improves upon [RetinaNet](https://paperswithcode.com/method/retinanet) by adding the task of instance mask prediction during training, as well as an [adaptive loss](https://paperswithcode.com/method/adaptive-loss) that improves robustness to parameter choice during training, and including more difficult examples in training.

RetinaMask

RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free

Model to adapt:
We use Ultra Fast Structure-aware Deep Lane Detection (UFLD) as baseline and strictly adopt its training scheme and hyperparameters. UFLD treats lane detection as a row-based classification problem and utilizes the row anchors defined by TuSimple.

Unsupervised Domain Adaptation with SGPCS:
SGPCS builds upon PCS  and performs in-domain contrastive learning and crossdomain self-supervised learning via cluster prototypes. 

We reformulate the pseudo label selection mechanism from SGADA. For each lane, we select the highest confidence value from the griding cells of each row anchor. Based on their griding cell position, the confidence values are divided into two cases: absent lane points and present lane points. Thereby, the last griding cell represents absent lane points as in. For each case, we calculate the mean confidence over the corresponding lanes. We then use the thresholds defined by SGADA to decide whether the prediction is treated as a pseudo label.

Our overall objective function comprises the in-domain and cross-domain loss from PCS, the losses defined by UFLD, and our adopted pseudo loss from SGADA. We adjust the momentum for memory bank feature updates to 0.5 and use spherical K-means with K = 2,500 to cluster them into prototypes.

SGPCS

CARLANE: A Lane Detection Benchmark for Unsupervised Domain Adaptation from Simulation to multiple Real-World Domains

An **ECA-Net** is a type of convolutional neural network that utilises an [Efficient Channel Attention](https://paperswithcode.com/method/efficient-channel-attention) module.

ECA-Net

ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

Distributed training has become a pervasive and effective approach for training a large neural network
(NN) model with processing massive data. However, it is very challenging to satisfy requirements
from various NN models, diverse computing resources, and their dynamic changes during a training
job. In this study, we design our distributed training framework in a systematic end-to-end view to
provide the built-in adaptive ability for different scenarios, especially for industrial applications and
production environments, by fully considering resource allocation, model partition, task placement,
and distributed execution. Based on the unified distributed graph and the unified cluster object,
our adaptive framework is equipped with a global cost model and a global planner, which can
enable arbitrary parallelism, resource-aware placement, multi-mode execution, fault-tolerant, and
elastic distributed training. The experiments demonstrate that our framework can satisfy various
requirements from the diversity of applications and the heterogeneity of resources with highly
competitive performance.

E2EAdaptiveDistTraining

End-to-end Adaptive Distributed Training on PaddlePaddle

**Generalized State-Dependent Exploration**, or **gSDE**, is an exploration method for reinforcement learning that uses more general features and re-sampling the noise periodically. 

State-Dependent Exploration (SDE) is an intermediate solution for exploration that consists in adding noise as a function of the state $s\_{t}$, to the deterministic action $\mu\left(\mathbf{s}\_{t}\right)$. At the beginning of an episode, the parameters $\theta\_{\epsilon}$ of that exploration function are drawn from a Gaussian distribution. The resulting action $\mathbf{a}\_{t}$ is as follows:

$$
\mathbf{a}\_{t}=\mu\left(\mathbf{s}\_{t} ; \theta\_{\mu}\right)+\epsilon\left(\mathbf{s}\_{t} ; \theta\_{\epsilon}\right), \quad \theta\_{\epsilon} \sim \mathcal{N}\left(0, \sigma^{2}\right)
$$

This episode-based exploration is smoother and more consistent than the unstructured step-based exploration. Thus, during one episode, instead of oscillating around a mean value, the action a for a given state $s$ will be the same.

In the case of a linear exploration function $\epsilon\left(\mathbf{s} ; \theta\_{\epsilon}\right)=\theta\_{\epsilon} \mathbf{s}$, by operation on Gaussian distributions, Rückstieß et al. show that the action element $\mathbf{a}\_{j}$ is normally distributed:

$$
\pi]_{j}\left(\mathbf{a}\_{j} \mid \mathbf{s}\right) \sim \mathcal{N}\left(\mu\_{j}(\mathbf{s}), \hat{\sigma\_{j}}^{2}\right)
$$

where $\hat{\sigma}$ is a diagonal matrix with elements $\hat{\sigma}\_{j}=\sqrt{\sum\_{i}\left(\sigma\_{i j} \mathbf{s}\_{i}\right)^{2}}$.

Because we know the policy distribution, we can obtain the derivative of the log-likelihood $\log \pi(\mathbf{a} \mid \mathbf{s})$ with respect to the variance $\sigma$ :

$$
\frac{\partial \log \pi(\mathbf{a} \mid \mathbf{s})}{\partial \sigma_{i j}}=\frac{\left(\mathbf{a}\_{j}-\mu\_{j}\right)^{2}-\hat{\sigma\_{j}}^{2}}{\hat{\sigma}\_{j}^{3}} \frac{\mathbf{s}\_{i}^{2} \sigma\_{i j}}{\hat{\sigma_{j}}}
$$

This can be easily plugged into the likelihood ratio gradient estimator, which allows to adapt $\sigma$ during training. SDE is therefore compatible with standard policy gradient methods, while addressing most shortcomings of the unstructured exploration.

For gSDE, two improvements are suggested:

1. We sample the parameters $\theta\_{\epsilon}$ of the exploration function every $n$ steps instead of every episode.
2. Instead of the state s, we can in fact use any features. We chose policy features $\mathbf{z}\_{\mu}\left(\mathbf{s} ; \theta\_{\mathbf{z}\_{\mu}}\right)$ (last layer before the deterministic output $\left.\mu(\mathbf{s})=\theta\_{\mu} \mathbf{z}\_{\mu}\left(\mathbf{s} ; \theta_{\mathbf{z}\_{\mu}}\right)\right)$ as input to the noise function $\epsilon\left(\mathbf{s} ; \theta\_{\epsilon}\right)=\theta\_{\epsilon} \mathbf{z}\_{\mu}(\mathbf{s})$

gSDE

Smooth Exploration for Robotic Reinforcement Learning

**EdgeFlow** is an interactive segmentation architecture that fully utilizes interactive information of user clicks with edge-guided flow. Edge guidance is the idea that interactive segmentation improves segmentation masks progressively with user clicks. Based on user clicks, an edge mask scheme is used, which takes the object edges estimated from the previous iteration as prior information, instead of direct mask estimation (if the previous mask is used as input, poor segmentation results could result).

The architecture consists of a coarse-to-fine network including CoarseNet and FineNet. For CoarseNet, [HRNet](https://paperswithcode.com/method/hrnet)-18+OCR is utilized as the base segmentation model and the edge-guided flow is appended to deal with interactive information. For FineNet, three [atrous convolution](https://paperswithcode.com/method/dilated-convolution) blocks are utilized to refine the coarse masks.

EdgeFlow

EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow

**Side-Aware Boundary Localization (SABL)** is a methodology for precise localization in object detection where each side of the bounding box is respectively localized with a dedicated network branch. Empirically, the authors observe that when they manually annotate a bounding box for an object, it is often much easier to align each side of the box to the object boundary than to move the
box as a whole while tuning the size. Inspired by this observation, in SABL each side of the bounding box is respectively positioned based on its surrounding context. 

As shown in the Figure, the authors devise a bucketing scheme to improve the localization precision. For each side of a bounding box, this scheme divides the target space into multiple buckets, then determines the bounding box via two steps. Specifically, it first searches for the correct bucket, i.e., the one in which the boundary resides. Leveraging the centerline of the selected buckets as a
coarse estimate, fine regression is then performed by predicting the offsets. This scheme allows very precise localization even in the presence of displacements with large variance. Moreover, to preserve precisely localized bounding boxes in the non-maximal suppression procedure, the authors also propose to adjust the classification score based on the bucketing confidences, which leads to further performance gains.

Viet-Anh on Software

AI Glossary

Tech Terminology

Stochastic Depth

MixText

RetinaNet-RS

Approximate Bayesian Computation

Frequency channel attention networks

Criss-Cross Network

KNN and IOU based verification

AmoebaNet

A3C

PREDATOR

Trust Region Policy Optimization

RetinaMask

Self-training Guided Prototypical Cross-domain Self-supervised learning

ECA-Net

End-to-end Adaptive Distributed Training

Generalized State-Dependent Exploration

EdgeFlow

Side-Aware Boundary Localization

Viet-Anh on Software