issue with garbled output

[Djerry-h]

Hello, I used LoRA for fine-tuning in the second stage, but the output appeared as garbled text. I'm not sure what caused this issue. Here is my LoRA code.

[RunsenXu]

Hi,

Could you first check wether the loss is going down?