**In-Place Activated Batch Normalization**, or **InPlace-ABN**, substitutes the conventionally used succession of [BatchNorm](https://paperswithcode.com/method/batch-normalization) + Activation layers with a single plugin layer, hence avoiding invasive framework surgery while providing straightforward applicability for existing deep learning frameworks. It approximately halves the memory requirements during training of modern deep learning models.

CrossTransformers is a Transformer-based neural network architecture which can take a small number of labeled images and an unlabeled query, find coarse spatial correspondence between the query and the labeled images, and then infer class membership by computing distances between spatially-corresponding features.

CrossTransformers

CrossTransformers: spatially-aware few-shot transfer

InPlace-ABN

In-Place Activated BatchNorm for Memory-Optimized Training of DNNs

_**Independent component analysis** (ICA) is a statistical and computational technique for revealing hidden factors that underlie sets of random variables, measurements, or signals._

_ICA defines a generative model for the observed multivariate data, which is typically given as a large database of samples. In the model, the data variables are assumed to be linear mixtures of some unknown latent variables, and the mixing system is also unknown. The latent variables are assumed nongaussian and mutually independent, and they are called the independent components of the observed data. These independent components, also called sources or factors, can be found by ICA._

_ICA is superficially related to principal component analysis and factor analysis. ICA is a much more powerful technique, however, capable of finding the underlying factors or sources when these classic methods fail completely._


Extracted from (https://www.cs.helsinki.fi/u/ahyvarin/whatisica.shtml)

**Source papers**:

[Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture](https://doi.org/10.1016/0165-1684(91)90079-X)

[Independent component analysis, A new concept?](https://doi.org/10.1016/0165-1684(94)90029-9)

[Independent component analysis: algorithms and applications](https://doi.org/10.1016/S0893-6080(00)00026-5)

Source	In-Place Activated BatchNorm for Memory-Optimized Training of DNNs
Year	2000
Data Source	Papers with Code archive (CC BY-SA)