questions about inconsistent evaluation result

[coorful]

Hi，i have used deepspeed framework to train gpt-117M model.
when i evaluate model perfomance on wikitext-103, result by using tasks/eval_harness/evaluate.py vs. first convert checkpoint to megatron format and use tasks/main.py , there exists a large performance gap in PPL...
May I ask what is the reason for this phenomenon? @mayank31398

[pal404error]

Noticed bigscience-workshop/Megatron-DeepSpeed and wanted to say the work here is solid. Wanted to share ModelRank in case it helps your HF community: an independent, auditable leaderboard for open-weight LLMs. Free embeddable badges, 5-dimension scoring, zero paid placements. If you want Megatron-DeepSpeed scored and listed, it is a one-line badge in the README (https://rankmodel.github.io). Appreciate you building in the open.