Rational Activation Functions, ratio of polynomials as learnable functions

**Domain Adaptive Ensemble Learning**, or **DAEL**, is an architecture for domain adaptation. The model is composed of a CNN feature extractor shared across domains and multiple classifier heads each trained to specialize in a particular source domain. Each such classifier is an expert to its own domain and a non-expert to others. DAEL aims to learn these experts collaboratively so that when forming an ensemble, they can leverage complementary information from each other to be more effective for an unseen target domain. To this end, each source domain is used in turn as a pseudo-target-domain with its own expert providing supervisory signal to the ensemble of non-experts learned from the other sources. For unlabeled target data under the UDA setting where real expert does not exist, DAEL uses pseudo-label to supervise the ensemble learning.

DAEL

Domain Adaptive Ensemble Learning

Rational Activation Function

Padé Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networks

**DLA**, or **Deep Layer Aggregation**,  iteratively and hierarchically merges the feature hierarchy across layers in neural networks to make networks with better accuracy and fewer parameters. 

In iterative deep aggregation (IDA), aggregation begins at the shallowest, smallest scale and then iteratively merges deeper,
larger scales. In this way shallow features are refined as
they are propagated through different stages of aggregation.

In hierarchical deep aggregation (HDA), blocks and stages
in a tree are merged to preserve and combine feature channels. With
HDA shallower and deeper layers are combined to learn
richer combinations that span more of the feature hierarchy.
While IDA effectively combines stages, it is insufficient
for fusing the many blocks of a network, as it is still only
sequential.

Source	Padé Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com

Viet-Anh on Software

What is: Rational Activation Function?

Viet-Anh on Software