**Hard Swish** is a type of activation function based on [Swish](https://paperswithcode.com/method/swish), but replaces the computationally expensive sigmoid with a piecewise linear analogue:

$$\text{h-swish}\left(x\right) = x\frac{\text{ReLU6}\left(x+3\right)}{6} $$

To avoid the problem caused by low-frequent entity-relation pairs, our MBS uses the estimated probabilities from a trained model $\mathbf{\theta}'$ to calculate frequencies for each triplet and query. By using $\mathbf{\theta}'$, the NS loss in KGE with MBS is represented as follows:
\begin{align}
    &\ell_{mbs}(\mathbf{\theta};\mathbf{\theta}') \nonumber \\
=&-\frac{1}{|D|}\sum_{(x,y) \in D} \Bigl[A_{mbs}(\mathbf{\theta}')\log(\sigma(s_{\mathbf{\theta}}(x,y)+\gamma))\nonumber\\
    &+\frac{1}{\nu}sum_{y_{i}\sim p_n(y_{i}|x)}^{\nu}B_{mbs}(\mathbf{\theta}')\log(\sigma(-s_{\mathbf{\theta}}(x,y_i)-\gamma))\Bigr],
\end{align}

Model-based Subsampling for Knowledge Graph Completion

Hard Swish

Searching for MobileNetV3

A **Feedforward Network**, or a **Multilayer Perceptron (MLP)**, is a neural network with solely densely connected layers. This is the classic neural network architecture of the literature. It consists of inputs $x$ passed through units $h$ (of which there can be many layers) to predict a target $y$. Activation functions are generally chosen to be non-linear to allow for flexible functional approximation.

Image Source: Deep Learning, Goodfellow et al

Source	Searching for MobileNetV3
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com

Viet-Anh on Software

What is: Hard Swish?

Viet-Anh on Software