On the Difficulty of Extrapolation with NN Scaling

A demo showing the difficulty of applying scaling to predict performance of large models.

Related Keywords

Jascha Sohl Dickstein ,Ethan Dyer ,Igor Mordatch ,Google Research ,Chip Huyen ,