Created by: lilisierrayu
Patch Description Adding fix to correctly initialize the non-MP model. And correctly infer inference datatype for model trained with mixed precision.
Testing steps Launch a consolidated model (mp=1) for a model trained with bf16.