Created by: Xirider
This test has been broken since some time (has the same issue as https://github.com/facebookresearch/metaseq/issues/393 tried to fix in I tried to fix it in https://github.com/Xirider/metaseq/tree/fix_test_training_integrity), but was skipped due to the circle machine only having 4 gpus, instead of the required 8.
The new test_checkpoint_saving https://github.com/facebookresearch/metaseq/pull/426 tests training integrity as well, so test_training_integrity is redundant and I removed it here.