From Tokens to Bytes I: Scaling Hierarchical Networks
- Since Richard Sutton’s influential essay in 2019, the concept of the bitter lesson has guided much of modern progress in AI.
- This concept espouses general methods that leverage computation and learn with scale, as free of human-engineered priors as possible.
- Our interpretation is that, fundamentally, intelligence is about learning from raw data: autonomously finding patterns, building abstractions, and developing meaning from messy low-level inputs, ideally end-to-end and as data-efficiently as possible.
Unverified
- Since Richard Sutton’s influential essay in 2019, the concept of the bitter lesson has guided much of modern progress in AI.
- This concept espouses general methods that leverage computation and learn with scale, as free of human-engineered priors as possible.
- Our interpretation is that, fundamentally, intelligence is about learning from raw data: autonomously finding patterns, building abstractions, and developing meaning from messy low-level inputs, ideally end-to-end and as data-efficiently as possible.
Sources: Cartesia