FIERCE is an entropic regularization on the **feature** space

**Early Stopping** is a regularization technique for deep neural networks that stops training when parameter updates no longer begin to yield improves on a validation set. In essence, we store and update the current best parameters during training, and when parameter updates no longer yield an improvement (after a set number of iterations) we stop training and use the last best parameters. It works as a regularizer by restricting the optimization procedure to a smaller volume of parameter space.

Image Source: [Ramazan Gençay](https://www.researchgate.net/figure/Early-stopping-based-on-cross-validation_fig1_3302948)

Early Stopping

FIERCE

Preserving Fine-Grain Feature Information in Classification via Entropic Regularization

**Local Patch Interaction**, or **LPI**, is a module used for the [XCiT layer](https://paperswithcode.com/method/xcit-layer) to enable explicit communication across patches. LPI consists of two [depth-wise 3×3 convolutional layers](https://paperswithcode.com/method/depthwise-convolution) with [Batch Normalization](https://paperswithcode.com/method/batch-normalization) and [GELU](https://paperswithcode.com/method/gelu) non-linearity in between. Due to its depth-wise structure, the LPI block has a negligible overhead in terms of parameters, as well as a limited overhead in terms of throughput and memory usage during inference.

Source	Preserving Fine-Grain Feature Information in Classification via Entropic Regularization
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com

Viet-Anh on Software

What is: Feature Information Entropy Regularized Cross Entropy?

Viet-Anh on Software