The AI systems solving real problems aren't the ones with the most parameters but the ones with the best scaffolding.

Language evolution will be interesting. The older stuff will remain in circulation in the training dataset, for significant changes the lack of recent training will be a pain. A broad observation here ...