Created by: lilisierrayu
Patch Description The prompt_len in current code relies on the batch_generator implementation and can easily introduce error. Remove that dependence in this PR.
Testing steps Describe how you tested your changes
Created by: lilisierrayu
Patch Description The prompt_len in current code relies on the batch_generator implementation and can easily introduce error. Remove that dependence in this PR.
Testing steps Describe how you tested your changes