For the 1.3 billion parameter one they do mean train from scratch on your 2 hour... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

UncleEntity on April 13, 2023 | parent | context | favorite | on: DeepSpeed Chat: Easy, fast and affordable RLHF tra...

For the 1.3 billion parameter one they do mean train from scratch on your 2 hour “coffee break”.

Which totally begs the question, Microsoft AI researchers get two hour coffee breaks?

—edit—

The way they word it is confusing but, yeah, fine tune the model on your two hour coffee break.

MasterScrat on April 13, 2023 [–]

No, you can see the time breakdown in the table under the "coffee break" quote: it is the time for the 3-step RLHF process only. Training a 1.3B parameter model from scratch is still a very large undertaking.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact