Created by: davides
Patch Description I noticed the API might be under-batching because we don't correctly detect when a batch might exceed MAX_SEQ_LEN.
Testing steps Ran the API locally + curl
Created by: davides
Patch Description I noticed the API might be under-batching because we don't correctly detect when a batch might exceed MAX_SEQ_LEN.
Testing steps Ran the API locally + curl