常见激活函数

---

sigmoid: $\sigma (x) = {1 \over 1+ e^{-x}}$

tanh: $tanh(x)$

ReLu: $max(0,x)$

Leaky ReLu: $max(0.1x,x)$

Maxout: $max(w^T_1 x + b1, w^T _2 x + b2)$

ELU: $$