a new reward modelling dataset:
Avelina/UltraSteer-v0
UltraSteer-V0 is a massive collection of single- and multi-turn dialogue with fine-grained reward labels produced by Nvidia's
nvidia/Llama2-13B-SteerLM-RM
reward model. We have a total of 2.3M labelled sequences taken from high quality datasets with a total of 2.8M labelled turns each containing 9 attributes produced as is from the reward model.
This is still very much an early version of the dataset (but it's fully usable!) and an updated version will be on the way with a full paper.
I would really appreciate if people could take a look at the dataset and suggest any improvements (e.g. more data sources, different cleaning approaches, different label schema, etc) in the community section.
Avelina/UltraSteer-v0
UltraSteer-V0 is a massive collection of single- and multi-turn dialogue with fine-grained reward labels produced by Nvidia's
nvidia/Llama2-13B-SteerLM-RM
reward model. We have a total of 2.3M labelled sequences taken from high quality datasets with a total of 2.8M labelled turns each containing 9 attributes produced as is from the reward model.
This is still very much an early version of the dataset (but it's fully usable!) and an updated version will be on the way with a full paper.
I would really appreciate if people could take a look at the dataset and suggest any improvements (e.g. more data sources, different cleaning approaches, different label schema, etc) in the community section.