Merge requests · Administrator / metaseq

BF16 bug fix:

!330 · created Sep 06, 2022 by Administrator cla signed
- MERGED
- 4
updated Sep 07, 2022
properly loading model 0 of 4 checklist items completed

!328 · created Sep 02, 2022 by Administrator cla signed
- 1
updated Feb 12, 2023
[QOL] Better error message for megatron softmax failure

!326 · created Aug 31, 2022 by Administrator cla signed
- MERGED
- 2
updated Sep 06, 2022
[cleanup] Update some f-strings, change checkpoint naming scheme.

!323 · created Aug 30, 2022 by Administrator cla signed
- MERGED
- 5
updated Aug 30, 2022
[checkpoints] Don't validate/save after epochs.

!322 · created Aug 30, 2022 by Administrator cla signed
- MERGED
- 6
updated Aug 30, 2022
[logging] Better dataloader logging

!321 · created Aug 30, 2022 by Administrator cla signed
- MERGED
- 4
updated Aug 30, 2022
[tboard, QOL] Don't log average train metrics.

!320 · created Aug 30, 2022 by Administrator cla signed
- MERGED
- 2
updated Aug 30, 2022
added option to disable elementwise affine in layernorm

!318 · created Aug 23, 2022 by Administrator cla signed
- MERGED
- 3
updated Aug 23, 2022
added code to disable bias

!317 · created Aug 23, 2022 by Administrator cla signed
- MERGED
- 2
updated Aug 23, 2022
Remove train OOM recovery logic

!314 · created Aug 20, 2022 by Administrator cla signed
- MERGED
- 5
updated Aug 22, 2022
Fix inclusion of valid subset in opt-baseline

!313 · created Aug 20, 2022 by Administrator cla signed
- MERGED
- 2
updated Aug 20, 2022
[jsonl] Fix regression in index building.

!312 · created Aug 20, 2022 by Administrator cla signed
- MERGED
- 3
updated Sep 10, 2022
[LM] Remove subshards on validation.

!310 · created Aug 20, 2022 by Administrator cla signed
- MERGED
- 2
updated Aug 20, 2022
[metrics] Stop logging pnorm etc.

!309 · created Aug 20, 2022 by Administrator cla signed
- MERGED
- 3
updated Aug 20, 2022
Repetition Penalties, Factual Nucleus

!306 · created Aug 17, 2022 by Administrator cla signed
- 22
updated Feb 08, 2023
[jsonl] Support other tokenizers.

!305 · created Aug 16, 2022 by Administrator cla signed
- MERGED
- 4
updated Aug 20, 2022
[DO NOT MERGE YET] Ensuring that the API returns None as the first token score in echo mode 0 of 4 checklist items completed

!303 · created Aug 11, 2022 by Administrator cla signed
- 2
updated Aug 19, 2022
Launch api with config 0 of 4 checklist items completed

!302 · created Aug 11, 2022 by Administrator cla signed
updated Feb 12, 2023
Do not save checkpoint in benchmark mode 0 of 4 checklist items completed

!301 · created Aug 09, 2022 by Administrator cla signed
- MERGED
- 4
updated Aug 10, 2022
[not to be merged] bunch of changes that I have been using to train latest models

!300 · created Aug 09, 2022 by Administrator cla signed
- 3
updated Nov 01, 2022