md5sum mismatch in resharded OPT-175B
Created by: dpfried
I've consolidated the 992 shards of the OPT-175B model into 8 files, following the instructions here: https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/download_opt175b.md#reshard-the-shards
but the md5sums for all of the shards don't match the ones in that README. Here are mine:
reshard-model_part-0.pt,ca52a4d1a3d1672b5249481169078469
reshard-model_part-1.pt,5c776fbc0f212df52fbf83b0ca4c7a64
reshard-model_part-2.pt,73e90bef8877f96674800712269149ae
reshard-model_part-3.pt,94d9442ec9fe04eab633df27d6e3ab5a
reshard-model_part-4.pt,dac742fe02fba6d500375bdd0a5113ea
reshard-model_part-5.pt,42485e9f21c759cae1f22079c0971a0b
reshard-model_part-6.pt,48ee0e983a8cde23b7544213e329afec
reshard-model_part-7.pt,ac62284b7f93d1f2a28eef6779cabeae
I've verified that the md5sums on all 992 individual shards that I've downloaded match the ones in https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/assets/opt175b_md5sum_shards.csv .
One possible explanation is that I'm using CUDA 11.1 rather than 11.3 as specified on https://github.com/facebookresearch/metaseq/blob/main/docs/setup.md (we have a managed cluster that it's hard to install 11.3 on), but I didn't expect that to affect the model weights. Could it? I've otherwise followed the metaseq install directions exactly.