Created by: suchenzang
[ Note: this will be rebased on main after https://github.com/facebookresearch/metaseq/pull/369 is merged in (+ upstream diffs) ]
We currently default to no_encoder_attn=True in our decoder-only paths. Remove this cruft to cleanly separate between decoder and encoders.