Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: bigscience-workshop/Megatron-DeepSpeed
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: bigscience-workshop/Megatron-DeepSpeed
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: scratchpad
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 18 commits
  • 11 files changed
  • 2 contributors

Commits on Jul 10, 2022

  1. Enable loading ckpt for t0 finetuning

    @Muennighoff
    Muennighoff committed Jul 10, 2022
    Configuration menu
    Copy the full SHA
    90b8f46 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2022

  1. Swap decoder_is_inputs & segment_ids

    @Muennighoff
    Muennighoff committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    abdd703 View commit details
    Browse the repository at this point in the history

  2. Add prepend-space arg

    @Muennighoff
    Muennighoff committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    0fcb19c View commit details
    Browse the repository at this point in the history

  3. Update tools/preprocess_data.py 

    Co-authored-by: Thomas Wang <24695242+thomasw21@users.noreply.github.com>
    @Muennighoff @thomasw21
    Muennighoff and thomasw21 authored Jul 11, 2022
    Configuration menu
    Copy the full SHA
    63daa46 View commit details
    Browse the repository at this point in the history

  4. Add helpers & set is_causal to true

    @Muennighoff
    Muennighoff committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    89460c0 View commit details
    Browse the repository at this point in the history

  5. Merge remote

    @Muennighoff
    Muennighoff committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    fb8ecb8 View commit details
    Browse the repository at this point in the history

  6. JSON helper scripts

    @Muennighoff
    Muennighoff committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    a55d2fb View commit details
    Browse the repository at this point in the history

  7. Remove unnec imports

    @Muennighoff
    Muennighoff committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    2dfe5d1 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2022

  1. Remove helper scripts

    @Muennighoff
    Muennighoff committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    ca740f1 View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2022

  1. Avoid loading module when not loading optim

    @Muennighoff
    Muennighoff committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    cb0313b View commit details
    Browse the repository at this point in the history

  2. Allow not using torch distributed

    @Muennighoff
    Muennighoff committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    b62dcaf View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2022

  1. Add prefixlm arg

    @Muennighoff
    Muennighoff committed Jul 16, 2022
    Configuration menu
    Copy the full SHA
    b15ca2d View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2022

  1. Add bos option

    @Muennighoff
    Muennighoff committed Jul 28, 2022
    Configuration menu
    Copy the full SHA
    dc8d0ab View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2022

  1. Merge branch 'main' into t0loading

    @Muennighoff
    Muennighoff committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    0a32459 View commit details
    Browse the repository at this point in the history

  2. Add reset-progress key

    @Muennighoff
    Muennighoff committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    2699721 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2022

  1. Add option to normalize loss per target (#326) 

    * Tmp lossseq

* Efficient loss normalization

* Reuse variable

* Simplify division

* Add norm_target_loss arg

* Clarify loss on targets & remove kwarg

* Loss mask is already float

* Move norm to batch pipe

* Reshape loss mask

* Move view
    @Muennighoff
    Muennighoff authored Nov 3, 2022
    Configuration menu
    Copy the full SHA
    1e77844 View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2022

  1. Add append rndm

    @Muennighoff
    Muennighoff committed Nov 19, 2022
    Configuration menu
    Copy the full SHA
    146a6aa View commit details
    Browse the repository at this point in the history

  2. Fix arg

    @Muennighoff
    Muennighoff committed Nov 19, 2022
    Configuration menu
    Copy the full SHA
    061ff98 View commit details
    Browse the repository at this point in the history
Loading