学习率调度器
此页面包含 timm 中包含的学习率调度器的 API 参考文档。
调度器
工厂函数
timm.scheduler.create_scheduler_v2
< 源代码 >( optimizer: 优化器 sched: str = 'cosine' num_epochs: int = 300 decay_epochs: int = 90 decay_milestones: List = (90, 180, 270) cooldown_epochs: int = 0 patience_epochs: int = 10 decay_rate: float = 0.1 min_lr: float = 0 warmup_lr: float = 1e-05 warmup_epochs: int = 0 warmup_prefix: bool = False noise: Union = None noise_pct: float = 0.67 noise_std: float = 1.0 noise_seed: int = 42 cycle_mul: float = 1.0 cycle_decay: float = 0.1 cycle_limit: int = 1 k_decay: float = 1.0 plateau_mode: str = 'max' step_on_epochs: bool = True updates_per_epoch: int = 0 )
调度器类
类 timm.scheduler.CosineLRScheduler
< 源代码 >( optimizer: 优化器 t_initial: 整数 lr_min: 浮点数 = 0.0 cycle_mul: 浮点数 = 1.0 cycle_decay: 浮点数 = 1.0 cycle_limit: 整数 = 1 warmup_t = 0 warmup_lr_init = 0 warmup_prefix = False t_in_epochs = True noise_range_t = None noise_pct = 0.67 noise_std = 1.0 noise_seed = 42 k_decay = 1.0 initialize = True )
带重启的余弦衰减。这在论文https://arxiv.org/abs/1608.03983中进行了描述。
灵感来自https://github.com/allenai/allennlp/blob/master/allennlp/training/learning_rate_schedulers/cosine.py
基于k-decay: A New Method For Learning Rate Schedule 的 k-衰减选项 - https://arxiv.org/abs/2004.05909
类 timm.scheduler.MultiStepLRScheduler
< 源代码 >( optimizer: 优化器 decay_t: 列表 decay_rate: 浮点数 = 1.0 warmup_t = 0 warmup_lr_init = 0 warmup_prefix = True t_in_epochs = True noise_range_t = None noise_pct = 0.67 noise_std = 1.0 noise_seed = 42 initialize = True )
类 timm.scheduler.PlateauLRScheduler
< 源代码 >( optimizer decay_rate = 0.1 patience_t = 10 verbose = True threshold = 0.0001 cooldown_t = 0 warmup_t = 0 warmup_lr_init = 0 lr_min = 0 mode = 'max' noise_range_t = None noise_type = 'normal' noise_pct = 0.67 noise_std = 1.0 noise_seed = None initialize = True )
每次验证损失停滞时，将学习率衰减一个因子。
类 timm.scheduler.PolyLRScheduler
< 源代码 >( optimizer：优化器 t_initial：int power：float = 0.5 lr_min：float = 0.0 cycle_mul：float = 1.0 cycle_decay：float = 1.0 cycle_limit：int = 1 warmup_t = 0 warmup_lr_init = 0 warmup_prefix = False t_in_epochs = True noise_range_t = None noise_pct = 0.67 noise_std = 1.0 noise_seed = 42 k_decay = 1.0 initialize = True )
带预热、噪声和 k-衰减的多项式 LR 调度器
基于k-decay: A New Method For Learning Rate Schedule 的 k-衰减选项 - https://arxiv.org/abs/2004.05909
类 timm.scheduler.StepLRScheduler
< 源代码 >( optimizer：优化器 decay_t：float decay_rate：float = 1.0 warmup_t = 0 warmup_lr_init = 0 warmup_prefix = True t_in_epochs = True noise_range_t = None noise_pct = 0.67 noise_std = 1.0 noise_seed = 42 initialize = True )
类 timm.scheduler.TanhLRScheduler
< 源代码 >( optimizer：优化器 t_initial：int lb：float = -7.0 ub：float = 3.0 lr_min：float = 0.0 cycle_mul：float = 1.0 cycle_decay：float = 1.0 cycle_limit：int = 1 warmup_t = 0 warmup_lr_init = 0 warmup_prefix = False t_in_epochs = True noise_range_t = None noise_pct = 0.67 noise_std = 1.0 noise_seed = 42 initialize = True )
带重启的双曲正切衰减。这在论文 https://arxiv.org/abs/1806.01593 中有描述。