Created by: punitkoura
Patch Description
- Ensure that we pad the mask properly when calculating the token distribution.
- Add a function to decode tokens in utils.py - This will be used internally.
Testing steps Ran an API locally to verify generation works.
curl -k <URL> -H "Authorization: Bearer Punit" -H "Content-Type: application/json" \
> -d '{
> "prompt": [2, 44019, 28049, 13770, 6439, 22824, 2306, 14380, 6804, 5457, 21523, 50118, 50118, 713, 16, 10, 569, 9, 10, 11954, 6439, 3704, 81],
> "temperature": 0.0,
> "max_tokens": 300, "min_tokens": 0,
> "top_p": 1.0, "n": 1, "best_of": 1,
> "echo": true, "logprobs": 1, "seed": 1
> }'