Skip to content
GitLab

check model parallel ranks have consistent data

Merged Administrator requested to merge github/fork/zdevito/bigbig into bigbig

Created by: zdevito

  • check parallel ranks have consistent data
  • remove a potential race condition when saving checkpoints that lets sequences_consumed get a head of number of iterations
  • Add code to fixup potentially broken data loaders on a restart