What is: Linear Warmup With Linear Decay?
| Year | 2000 |
| Data Source | Papers with Code archive (CC BY-SA) |
Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for updates and then linearly decay afterwards.