Compress an ensemble of models into a single one by averaging their weights (under certain pre-conditions).

EncAttAgg introduced two attenders to tackle two problems: 1) We introduce a mutual attender layer to efficiently obtain the entity-pair-specific mention representations. 2) We introduce an integration attender to weight mention pairs of a target entity pair.

EncAttAgg

Soups

Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

A **DenseNet** is a type of convolutional neural network that utilises [dense connections](https://paperswithcode.com/method/dense-connections) between layers, through [Dense Blocks](http://www.paperswithcode.com/method/dense-block), where we connect *all layers* (with matching feature-map sizes) directly with each other. To preserve the feed-forward nature, each layer obtains additional inputs from all preceding layers and passes on its own feature-maps to all subsequent layers.

Source	Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Year	2000
Data Source	Papers with Code archive (CC BY-SA)