If I understood correctly, the OpenAssistant team wants to open-source their community built RLHF dataset.

On the other hand, if you're being cheeky, I bet there's a way to datamine from websites like ShareGPT and profit off shared ChatGPT <> User interactions.