Created by: ruanslv
Patch Description
Fixes part of https://github.com/facebookresearch/metaseq/issues/383. Model configs without MP don't have "inference" attribute, causing model load to fail with omegaconf.errors.ConfigAttributeError: Key 'inference' is not in struct.
"inference" field seems to be used only in MP for the code to track the current status. However, it is always set inside make_generation_fast_() (e.g. https://fburl.com/4gy1ufjv).
So we don't need this line, we can just rely on make_generation_fast_ to set inference to True for us when the model has MP enabled.
Testing steps Loaded a model with and without MP (1.3B and 350M OPT).