Home /permanent

G-EVAL: NLG Evaluation using GPT-4 with Better Human Alignment

G-Eval is an LLM-as-judge framework that scores generated text with a large model (GPT-4) using a chain-of-thought, form-filling paradigm: it turns the task criteria into evaluation steps, prompts the model to reason through them, then emit a dimension score (Liu et al., 2023). To avoid coarse integer scores it weights the possible scores by the model's output token probabilities, yielding finer-grained continuous ratings that align with human judgment better than BERTScore, MoverScore, or BARTScore, though it inherits LLM-judge weaknesses such as a bias toward text produced by the same model family.

References

Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment. 2023. doi:10.48550/ARXIV.2303.16634.