Created by: suchenzang
Tested with small run with nfs:// upload path, confirmed two sets of checkpoints saved without issue.