**ZCA Whitening** is an image preprocessing method that leads to a transformation of data such that the covariance matrix $\Sigma$ is the identity matrix, leading to decorrelated features.

Image Source: [Alex Krizhevsky](http://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf)

**Mish** is an activation function for neural networks which can be defined as:

$$ f\left(x\right) = x\cdot\tanh{\text{softplus}\left(x\right)}$$

where

$$\text{softplus}\left(x\right) = \ln\left(1+e^{x}\right)$$

(Compare with functionally similar previously proposed activation functions such as the [GELU](https://paperswithcode.com/method/silu) $x\Phi(x)$ and the [SiLU](https://paperswithcode.com/method/silu) $x\sigma(x)$.)

Mish

Mish: A Self Regularized Non-Monotonic Activation Function

ZCA Whitening

**Shifted Softplus** is an activation function ${\rm ssp}(x) = \ln( 0.5 e^{x} + 0.5 )$, which [SchNet](https://paperswithcode.com/method/schnet) employs as non-linearity throughout the network in order to obtain a smooth potential energy surface. The shifting ensures that ${\rm ssp}(0) = 0$ and improves the convergence of the network. This activation function shows similarity to ELUs, while having infinite order of continuity.

Year	2000
Data Source	CC BY-SA - https://paperswithcode.com

Viet-Anh on Software

What is: ZCA Whitening?

Viet-Anh on Software