# Tracking callbacks


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

------------------------------------------------------------------------

<a
href="https://github.com/fastai/fastai/blob/main/fastai/callback/tracker.py#L14"
target="_blank" style="float:right; font-size:smaller">source</a>

### TerminateOnNaNCallback

``` python

def TerminateOnNaNCallback(
    after_create:NoneType=None, before_fit:NoneType=None, before_epoch:NoneType=None, before_train:NoneType=None,
    before_batch:NoneType=None, after_pred:NoneType=None, after_loss:NoneType=None, before_backward:NoneType=None,
    after_cancel_backward:NoneType=None, after_backward:NoneType=None, before_step:NoneType=None,
    after_cancel_step:NoneType=None, after_step:NoneType=None, after_cancel_batch:NoneType=None,
    after_batch:NoneType=None, after_cancel_train:NoneType=None, after_train:NoneType=None,
    before_validate:NoneType=None, after_cancel_validate:NoneType=None, after_validate:NoneType=None,
    after_cancel_epoch:NoneType=None, after_epoch:NoneType=None, after_cancel_fit:NoneType=None,
    after_fit:NoneType=None
):

```

*A [`Callback`](https://docs.fast.ai/callback.core.html#callback) that
terminates training if loss is NaN.*

``` python
learn = synth_learner()
learn.fit(10, lr=100, cbs=TerminateOnNaNCallback())
```

<table class="dataframe" data-quarto-postprocess="true" data-border="1">
<thead>
<tr style="text-align: left;">
<th data-quarto-table-cell-role="th">epoch</th>
<th data-quarto-table-cell-role="th">train_loss</th>
<th data-quarto-table-cell-role="th">valid_loss</th>
<th data-quarto-table-cell-role="th">time</th>
</tr>
</thead>
<tbody>
</tbody>
</table>

``` python
assert len(learn.recorder.losses) < 10 * len(learn.dls.train)
for l in learn.recorder.losses:
    assert not torch.isinf(l) and not torch.isnan(l)
```

------------------------------------------------------------------------

<a
href="https://github.com/fastai/fastai/blob/main/fastai/callback/tracker.py#L22"
target="_blank" style="float:right; font-size:smaller">source</a>

### TrackerCallback

``` python

def TrackerCallback(
    monitor:str='valid_loss', # value (usually loss or metric) being monitored.
    comp:NoneType=None, # numpy comparison operator; np.less if monitor is loss, np.greater if monitor is metric.
    min_delta:float=0.0, # minimum delta between the last monitor value and the best monitor value.
    reset_on_fit:bool=True, # before model fitting, reset value being monitored to -infinity (if monitor is metric) or +infinity (if monitor is loss).
):

```

*A [`Callback`](https://docs.fast.ai/callback.core.html#callback) that
keeps track of the best value in `monitor`.*

When implementing a
[`Callback`](https://docs.fast.ai/callback.core.html#callback) that has
behavior that depends on the best value of a metric or loss, subclass
this [`Callback`](https://docs.fast.ai/callback.core.html#callback) and
use its `best` (for best value so far) and `new_best` (there was a new
best value this epoch) attributes. If you want to maintain `best` over
subsequent calls to `fit` (e.g.,
[`Learner.fit_one_cycle`](https://docs.fast.ai/callback.schedule.html#learner.fit_one_cycle)),
set `reset_on_fit` = True.

`comp` is the comparison operator used to determine if a value is best
than another (defaults to `np.less` if ‘loss’ is in the name passed in
`monitor`, `np.greater` otherwise) and `min_delta` is an optional float
that requires a new value to go over the current best (depending on
`comp`) by at least that amount.

------------------------------------------------------------------------

<a
href="https://github.com/fastai/fastai/blob/main/fastai/callback/tracker.py#L51"
target="_blank" style="float:right; font-size:smaller">source</a>

### EarlyStoppingCallback

``` python

def EarlyStoppingCallback(
    monitor:str='valid_loss', # value (usually loss or metric) being monitored.
    comp:NoneType=None, # numpy comparison operator; np.less if monitor is loss, np.greater if monitor is metric.
    min_delta:float=0.0, # minimum delta between the last monitor value and the best monitor value.
    patience:int=1, # number of epochs to wait when training has not improved model.
    reset_on_fit:bool=True, # before model fitting, reset value being monitored to -infinity (if monitor is metric) or +infinity (if monitor is loss).
):

```

*A
[`TrackerCallback`](https://docs.fast.ai/callback.tracker.html#trackercallback)
that terminates training when monitored quantity stops improving.*

`comp` is the comparison operator used to determine if a value is best
than another (defaults to `np.less` if ‘loss’ is in the name passed in
`monitor`, `np.greater` otherwise) and `min_delta` is an optional float
that requires a new value to go over the current best (depending on
`comp`) by at least that amount. `patience` is the number of epochs
you’re willing to wait without improvement.

``` python
learn = synth_learner(n_trn=2, metrics=F.mse_loss)
learn.fit(n_epoch=200, lr=1e-7, cbs=EarlyStoppingCallback(monitor='mse_loss', min_delta=0.1, patience=2))
```

<table class="dataframe" data-quarto-postprocess="true" data-border="1">
<thead>
<tr style="text-align: left;">
<th data-quarto-table-cell-role="th">epoch</th>
<th data-quarto-table-cell-role="th">train_loss</th>
<th data-quarto-table-cell-role="th">valid_loss</th>
<th data-quarto-table-cell-role="th">mse_loss</th>
<th data-quarto-table-cell-role="th">time</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>20.437918</td>
<td>26.406773</td>
<td>26.406773</td>
<td>00:00</td>
</tr>
<tr>
<td>1</td>
<td>20.418514</td>
<td>26.406715</td>
<td>26.406715</td>
<td>00:00</td>
</tr>
<tr>
<td>2</td>
<td>20.410892</td>
<td>26.406639</td>
<td>26.406639</td>
<td>00:00</td>
</tr>
</tbody>
</table>

    No improvement since epoch 0: early stopping

``` python
learn.validate()
```

    (#2) [26.406639099121094,26.406639099121094]

``` python
learn = synth_learner(n_trn=2)
learn.fit(n_epoch=200, lr=1e-7, cbs=EarlyStoppingCallback(monitor='valid_loss', min_delta=0.1, patience=2))
```

<table class="dataframe" data-quarto-postprocess="true" data-border="1">
<thead>
<tr style="text-align: left;">
<th data-quarto-table-cell-role="th">epoch</th>
<th data-quarto-table-cell-role="th">train_loss</th>
<th data-quarto-table-cell-role="th">valid_loss</th>
<th data-quarto-table-cell-role="th">time</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>13.408870</td>
<td>19.617222</td>
<td>00:00</td>
</tr>
<tr>
<td>1</td>
<td>13.403553</td>
<td>19.617184</td>
<td>00:00</td>
</tr>
<tr>
<td>2</td>
<td>13.403143</td>
<td>19.617126</td>
<td>00:00</td>
</tr>
</tbody>
</table>

    No improvement since epoch 0: early stopping

------------------------------------------------------------------------

<a
href="https://github.com/fastai/fastai/blob/main/fastai/callback/tracker.py#L76"
target="_blank" style="float:right; font-size:smaller">source</a>

### SaveModelCallback

``` python

def SaveModelCallback(
    monitor:str='valid_loss', # value (usually loss or metric) being monitored.
    comp:NoneType=None, # numpy comparison operator; np.less if monitor is loss, np.greater if monitor is metric.
    min_delta:float=0.0, # minimum delta between the last monitor value and the best monitor value.
    fname:str='model', # model name to be used when saving model.
    every_epoch:bool=False, # if true, save model after every epoch; else save only when model is better than existing best.
    at_end:bool=False, # if true, save model when training ends; else load best model if there is only one saved model.
    with_opt:bool=False, # if true, save optimizer state (if any available) when saving model.
    reset_on_fit:bool=True, # before model fitting, reset value being monitored to -infinity (if monitor is metric) or +infinity (if monitor is loss).
):

```

*A
[`TrackerCallback`](https://docs.fast.ai/callback.tracker.html#trackercallback)
that saves the model’s best during training and loads it at the end.*

`comp` is the comparison operator used to determine if a value is best
than another (defaults to `np.less` if ‘loss’ is in the name passed in
`monitor`, `np.greater` otherwise) and `min_delta` is an optional float
that requires a new value to go over the current best (depending on
`comp`) by at least that amount. Model will be saved in
`learn.path/learn.model_dir/name.pth`, maybe `every_epoch` if `True`,
every nth epoch if an integer is passed to `every_epoch` or at each
improvement of the monitored quantity.

``` python
learn = synth_learner(n_trn=2, path=Path.cwd()/'tmp')
learn.fit(n_epoch=2, cbs=SaveModelCallback())
assert (Path.cwd()/'tmp/models/model.pth').exists()
learn = synth_learner(n_trn=2, path=Path.cwd()/'tmp')
learn.fit(n_epoch=2, cbs=SaveModelCallback(fname='end',at_end=True))
assert (Path.cwd()/'tmp/models/end.pth').exists()
learn.fit(n_epoch=2, cbs=SaveModelCallback(every_epoch=True))
for i in range(2): assert (Path.cwd()/f'tmp/models/model_{i}.pth').exists()
shutil.rmtree(Path.cwd()/'tmp')
learn.fit(n_epoch=4, cbs=SaveModelCallback(every_epoch=2))
for i in range(4): 
    if not i%2: assert (Path.cwd()/f'tmp/models/model_{i}.pth').exists()
    else:       assert not (Path.cwd()/f'tmp/models/model_{i}.pth').exists()
shutil.rmtree(Path.cwd()/'tmp')
```

<table class="dataframe" data-quarto-postprocess="true" data-border="1">
<thead>
<tr style="text-align: left;">
<th data-quarto-table-cell-role="th">epoch</th>
<th data-quarto-table-cell-role="th">train_loss</th>
<th data-quarto-table-cell-role="th">valid_loss</th>
<th data-quarto-table-cell-role="th">time</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>19.453270</td>
<td>12.539286</td>
<td>00:00</td>
</tr>
<tr>
<td>1</td>
<td>19.248507</td>
<td>12.123456</td>
<td>00:00</td>
</tr>
</tbody>
</table>

    Better model found at epoch 0 with valid_loss value: 12.539285659790039.
    Better model found at epoch 1 with valid_loss value: 12.123456001281738.

<table class="dataframe" data-quarto-postprocess="true" data-border="1">
<thead>
<tr style="text-align: left;">
<th data-quarto-table-cell-role="th">epoch</th>
<th data-quarto-table-cell-role="th">train_loss</th>
<th data-quarto-table-cell-role="th">valid_loss</th>
<th data-quarto-table-cell-role="th">time</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>5.197007</td>
<td>5.579152</td>
<td>00:00</td>
</tr>
<tr>
<td>1</td>
<td>5.154862</td>
<td>5.445522</td>
<td>00:00</td>
</tr>
</tbody>
</table>

    Better model found at epoch 0 with valid_loss value: 5.5791521072387695.
    Better model found at epoch 1 with valid_loss value: 5.445522308349609.

<table class="dataframe" data-quarto-postprocess="true" data-border="1">
<thead>
<tr style="text-align: left;">
<th data-quarto-table-cell-role="th">epoch</th>
<th data-quarto-table-cell-role="th">train_loss</th>
<th data-quarto-table-cell-role="th">valid_loss</th>
<th data-quarto-table-cell-role="th">time</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>4.982775</td>
<td>5.264440</td>
<td>00:00</td>
</tr>
<tr>
<td>1</td>
<td>4.887252</td>
<td>5.038480</td>
<td>00:00</td>
</tr>
</tbody>
</table>

<table class="dataframe" data-quarto-postprocess="true" data-border="1">
<thead>
<tr style="text-align: left;">
<th data-quarto-table-cell-role="th">epoch</th>
<th data-quarto-table-cell-role="th">train_loss</th>
<th data-quarto-table-cell-role="th">valid_loss</th>
<th data-quarto-table-cell-role="th">time</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>4.578584</td>
<td>4.781651</td>
<td>00:00</td>
</tr>
<tr>
<td>1</td>
<td>4.454868</td>
<td>4.507101</td>
<td>00:00</td>
</tr>
<tr>
<td>2</td>
<td>4.322047</td>
<td>4.232390</td>
<td>00:00</td>
</tr>
<tr>
<td>3</td>
<td>4.186467</td>
<td>3.957614</td>
<td>00:00</td>
</tr>
</tbody>
</table>

## ReduceLROnPlateau

------------------------------------------------------------------------

<a
href="https://github.com/fastai/fastai/blob/main/fastai/callback/tracker.py#L113"
target="_blank" style="float:right; font-size:smaller">source</a>

### ReduceLROnPlateau

``` python

def ReduceLROnPlateau(
    monitor:str='valid_loss', # value (usually loss or metric) being monitored.
    comp:NoneType=None, # numpy comparison operator; np.less if monitor is loss, np.greater if monitor is metric.
    min_delta:float=0.0, # minimum delta between the last monitor value and the best monitor value.
    patience:int=1, # number of epochs to wait when training has not improved model.
    factor:float=10.0, # the denominator to divide the learning rate by, when reducing the learning rate.
    min_lr:int=0, # the minimum learning rate allowed; learning rate cannot be reduced below this minimum.
    reset_on_fit:bool=True, # before model fitting, reset value being monitored to -infinity (if monitor is metric) or +infinity (if monitor is loss).
):

```

*A
[`TrackerCallback`](https://docs.fast.ai/callback.tracker.html#trackercallback)
that reduces learning rate when a metric has stopped improving.*

``` python
learn = synth_learner(n_trn=2)
learn.fit(n_epoch=4, lr=1e-7, cbs=ReduceLROnPlateau(monitor='valid_loss', min_delta=0.1, patience=2))
```

<table class="dataframe" data-quarto-postprocess="true" data-border="1">
<thead>
<tr style="text-align: left;">
<th data-quarto-table-cell-role="th">epoch</th>
<th data-quarto-table-cell-role="th">train_loss</th>
<th data-quarto-table-cell-role="th">valid_loss</th>
<th data-quarto-table-cell-role="th">time</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>6.122743</td>
<td>7.348515</td>
<td>00:00</td>
</tr>
<tr>
<td>1</td>
<td>6.119377</td>
<td>7.348499</td>
<td>00:00</td>
</tr>
<tr>
<td>2</td>
<td>6.125790</td>
<td>7.348477</td>
<td>00:00</td>
</tr>
<tr>
<td>3</td>
<td>6.131386</td>
<td>7.348475</td>
<td>00:00</td>
</tr>
</tbody>
</table>

    Epoch 2: reducing lr to 1e-08

``` python
learn = synth_learner(n_trn=2)
learn.fit(n_epoch=6, lr=5e-8, cbs=ReduceLROnPlateau(monitor='valid_loss', min_delta=0.1, patience=2, min_lr=1e-8))
```

<table class="dataframe" data-quarto-postprocess="true" data-border="1">
<thead>
<tr style="text-align: left;">
<th data-quarto-table-cell-role="th">epoch</th>
<th data-quarto-table-cell-role="th">train_loss</th>
<th data-quarto-table-cell-role="th">valid_loss</th>
<th data-quarto-table-cell-role="th">time</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>16.747515</td>
<td>15.265999</td>
<td>00:00</td>
</tr>
<tr>
<td>1</td>
<td>16.725756</td>
<td>15.265974</td>
<td>00:00</td>
</tr>
<tr>
<td>2</td>
<td>16.735016</td>
<td>15.265943</td>
<td>00:00</td>
</tr>
<tr>
<td>3</td>
<td>16.733360</td>
<td>15.265934</td>
<td>00:00</td>
</tr>
<tr>
<td>4</td>
<td>16.733513</td>
<td>15.265925</td>
<td>00:00</td>
</tr>
<tr>
<td>5</td>
<td>16.730352</td>
<td>15.265915</td>
<td>00:00</td>
</tr>
</tbody>
</table>

    Epoch 2: reducing lr to 1e-08

Each of these three derived
[`TrackerCallback`](https://docs.fast.ai/callback.tracker.html#trackercallback)s
([`SaveModelCallback`](https://docs.fast.ai/callback.tracker.html#savemodelcallback),
`ReduceLROnPlateu`, and
[`EarlyStoppingCallback`](https://docs.fast.ai/callback.tracker.html#earlystoppingcallback))
all have an adjusted order so they can each run with each other without
interference. That order is as follows:

<div>

> **Note**
>
> in parenthesis is the actual
> [`Callback`](https://docs.fast.ai/callback.core.html#callback) order
> number

</div>

1.  [`TrackerCallback`](https://docs.fast.ai/callback.tracker.html#trackercallback)
    (60)
2.  [`SaveModelCallback`](https://docs.fast.ai/callback.tracker.html#savemodelcallback)
    (61)
3.  `ReduceLrOnPlateu` (62)
4.  [`EarlyStoppingCallback`](https://docs.fast.ai/callback.tracker.html#earlystoppingcallback)
    (63)