Skip to content
GitLab

Add full_megatron_init_scalar flag

Merged Administrator requested to merge susan/megatron_init_scalar into main

Created by: suchenzang

To scale fc2 in FFN and out_proj in MHA with.