We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention… Voyager is able to utilize the learned skill library in a new Minecraft world to solve novel tasks from scratch
[…]
Voyager exhibits superior performance in discovering novel items, unlocking the Minecraft tech tree, traversing diverse terrains, and applying its learned skill library to unseen tasks in a newly instantiated world. Voyager serves as a starting point to develop powerful generalist agents without tuning the model parameters.
Last month we saw Sanford researchers create a version of The Sims inhabited by LLM-powered agents. These agents exhibited surprisingly complex social skills.
This new research shows that agents based on a similar architecture can create and explore in novel environments.
As this technology becomes less expensive, we will start to see incredible new virtual experiences that were previously unimaginable.
As capabilities improve further, we will reach the point where we pass some sort of fundamental threshold—like the uncanny valley—where the characters that inhabit our virtual environments become too lifelike.
At its height, people spent a lot of time playing Second Life and it, well, looked like Second Life. We don’t even need hyperrealistic experiences for things to start getting scary, though. Imagine a version of Grand Theft Auto where every NPC has their own unique set of ambitions and relationships. I wouldn’t be surprised if someone could hack that together with the technology available today. Once that happens, we will need to start having some difficult conversations.