True, but that bit they got right. What is really interesting to me is that all this knowledge is out there now, we don't have to repeat all these past mistakes and yet somehow we do. If other engineering disciplines would work like that (say, civil engineering) you wouldn't be able to travel at all because every bridge or road would be an adventure in novelty tricks that may or may not work.

A better analogy for LLMs is they out a man on the moon but didn’t learn lessons from civil aviation about having enough redundancy to make the chance of loss of life one in a billion.