Created by: zdevito
If you set cloud_upload_path to "nfs:<some_local_directory>" this will save the checkpoints to local disk as normal, but then copy to this directory similar to how we do azcopy, except to a local folder mounted in nfs. To avoid many files in the same directory, it adds a directory for each checkpoint. It uses these directories to discover the most recent checkpoint on restart.
Directories are initially created with a leading _ and only renamed when the checkpoint successfully completes.