Merged
requested to merge github/fork/bashnick/Unify-transformer_lm_megatron-and-transformer_lm into main
Created by: bashnick
Patch Description Unify transformer_lm_megatron and transformer_lm architectures. Remove *_mp.py files. Solves issue #583 (closed).
Testing steps python -m pytest cpu_tests python -m pytest gpu_tests python -m projects.PROJECT_NAME.sweep_baseline -g 4 -n 1 --rsc --model-size 8m --tokenizer rsc --prefix $RN --local --data /checkpoint/TEAM_NAME/datasets/consolidated/v4.0