timm 文档
学习率调度器
并获取增强的文档体验
开始使用
学习率调度器
此页面包含 timm 中学习率调度器的 API 参考文档。
调度器
工厂函数
timm.scheduler.create_scheduler_v2
< source >( optimizer: Optimizer sched: str = 'cosine' num_epochs: int = 300 decay_epochs: int = 90 decay_milestones: typing.List[int] = (90, 180, 270) cooldown_epochs: int = 0 patience_epochs: int = 10 decay_rate: float = 0.1 min_lr: float = 0 warmup_lr: float = 1e-05 warmup_epochs: int = 0 warmup_prefix: bool = False noise: typing.Union[float, typing.List[float]] = None noise_pct: float = 0.67 noise_std: float = 1.0 noise_seed: int = 42 cycle_mul: float = 1.0 cycle_decay: float = 0.1 cycle_limit: int = 1 k_decay: float = 1.0 plateau_mode: str = 'max' step_on_epochs: bool = True updates_per_epoch: int = 0 )
调度器类
class timm.scheduler.CosineLRScheduler
< source >( optimizer: Optimizer t_initial: int lr_min: float = 0.0 cycle_mul: float = 1.0 cycle_decay: float = 1.0 cycle_limit: int = 1 warmup_t = 0 warmup_lr_init = 0 warmup_prefix = False t_in_epochs = True noise_range_t = None noise_pct = 0.67 noise_std = 1.0 noise_seed = 42 k_decay = 1.0 initialize = True )
带重启的余弦衰减。这在论文 https://arxiv.org/abs/1608.03983 中进行了描述。
灵感来自 https://github.com/allenai/allennlp/blob/master/allennlp/training/learning_rate_schedulers/cosine.py
k 衰减选项基于 k-decay: A New Method For Learning Rate Schedule - https://arxiv.org/abs/2004.05909
class timm.scheduler.MultiStepLRScheduler
< source >( optimizer: Optimizer decay_t: typing.List[int] decay_rate: float = 1.0 warmup_t = 0 warmup_lr_init = 0 warmup_prefix = True t_in_epochs = True noise_range_t = None noise_pct = 0.67 noise_std = 1.0 noise_seed = 42 initialize = True )
class timm.scheduler.PlateauLRScheduler
< source >( optimizer decay_rate = 0.1 patience_t = 10 verbose = True threshold = 0.0001 cooldown_t = 0 warmup_t = 0 warmup_lr_init = 0 lr_min = 0 mode = 'max' noise_range_t = None noise_type = 'normal' noise_pct = 0.67 noise_std = 1.0 noise_seed = None initialize = True )
每次验证损失停滞时，按因子衰减 LR。
class timm.scheduler.PolyLRScheduler
< source >( optimizer: Optimizer t_initial: int power: float = 0.5 lr_min: float = 0.0 cycle_mul: float = 1.0 cycle_decay: float = 1.0 cycle_limit: int = 1 warmup_t = 0 warmup_lr_init = 0 warmup_prefix = False t_in_epochs = True noise_range_t = None noise_pct = 0.67 noise_std = 1.0 noise_seed = 42 k_decay = 1.0 initialize = True )
带预热、噪声和 k 衰减的多项式 LR 调度器
k 衰减选项基于 k-decay: A New Method For Learning Rate Schedule - https://arxiv.org/abs/2004.05909
class timm.scheduler.StepLRScheduler
< source >( optimizer: Optimizer decay_t: float decay_rate: float = 1.0 warmup_t = 0 warmup_lr_init = 0 warmup_prefix = True t_in_epochs = True noise_range_t = None noise_pct = 0.67 noise_std = 1.0 noise_seed = 42 initialize = True )
class timm.scheduler.TanhLRScheduler
< source >( optimizer: Optimizer t_initial: int lb: float = -7.0 ub: float = 3.0 lr_min: float = 0.0 cycle_mul: float = 1.0 cycle_decay: float = 1.0 cycle_limit: int = 1 warmup_t = 0 warmup_lr_init = 0 warmup_prefix = False t_in_epochs = True noise_range_t = None noise_pct = 0.67 noise_std = 1.0 noise_seed = 42 initialize = True )
双曲正切衰减与重启。 此方法在论文 https://arxiv.org/abs/1806.01593 中进行了描述