Full Article
Skip to main content
Research
Publications
Code & data
People
Microsoft Research blog
Artificial intelligence
Audio & acoustics
Computer vision
Graphics & multimedia
Human-computer interaction
Human language technologies
Search & information retrieval
Data platforms and analytics
Hardware & devices
Programming languages & software engineering
Quantum computing
Security, privacy & cryptography
Systems & networking
Algorithms
Mathematics
Ecology & environment
Economics
Medical, health & genomics
Social sciences
Technology for emerging marketsAcademic programs
Events & academic conferences
Microsoft Research ForumBehind the Tech podcast
Microsoft Research blog
Microsoft Research Forum
Microsoft Research podcast
About Microsoft Research
Careers & internships
People
Emeritus program
News & awards
Microsoft Research newsletter
Africa
AI for Science
AI Frontiers
Asia-Pacific
Cambridge
Health Futures
India
Montreal
New England
New York City
Redmond
Applied Sciences
Mixed Reality & AI - Cambridge
Mixed Reality & AI - Zurich
Register: Research Forum
Microsoft Security
Azure
Dynamics 365
Microsoft 365
Microsoft Teams
Windows 365
Microsoft AI
Azure Space
Mixed reality
Microsoft HoloLens
Microsoft Viva
Quantum computing
Sustainability
Education
Automotive
Financial services
Government
Healthcare
Manufacturing
Retail
Find a partner
Become a partner
Partner Network
Microsoft Marketplace
Software companies
Blog
Microsoft Advertising
Developer Center
Documentation
Events
Licensing
Microsoft Learn
Microsoft Research
View Sitemap
Return to Blog HomeMicrosoft Research BlogAt a glanceSelf-supervised. EvoLib enables large language models to learn from their own experience during inference, without requiring ground-truth labels or external feedback.From experience to knowledge. EvoLib transforms past attempts into reusable skills and reflective insights that can be applied to future tasks.Knowledge that evolves. Useful skills and insights are continually refined, consolidated, and reweighted, turning instance-specific observations into increasingly general knowledge over time.Learning that transfers across tasks. By turning experience into reusable knowledge, EvoLib helps AI models learn from past successes and failures and evolve the knowledge that has the highest potential on improving future performance.Built for today’s AI models. As EvoLib does not require model updates, it can be applied to any black-box language models and AI systems deployed through APIs.Memory has become an important AI agent capability: the ability to store and retrieve past experiences. But memory alone is not learning. A collection of past conversations, reasoning traces, or action histories can quickly grow into a vast archive of experiences, making it difficult to identify the most relevant knowledge for a new task—let alone refine and evolve this knowledge to improve performance over time.Humans learn differently. We do not remember every detail of our past experiences. Instead, we remember what matters: strategies that work, mistakes to avoid, and skills that transfer across situations. Over time, these lessons are refined into increasingly general and reusable knowledge. This ability to transform experience into transferable, evolving knowledge is one of the foundations of human learning.In our recent paper, Test-Time Learning with an Evolving Library , we explore how AI systems can learn from experience in a similar way. We introduce EvoLib , a framework that transforms raw experience into an evolving library of knowledge. Rather than treating memory as a growing archive of past experiences, EvoLib extracts reusable knowledge from those experiences and continually refines it as new experiences arrive. Through the evolution of library, skills become more general, insights become more accurate, and downstream performance gets improved consistently over time. In this way, AI agents can continually learn from accumulating experience without updating the underlying model.How EvoLib WorksSpotlight: Event SeriesMicrosoft Research ForumJoin us for a continuous exchange of ideas about research in the era of general AI. Watch the latest episodes on demand.
Watch on-demand
Opens in a new tabUnlike traditional AI memory systems that store raw experiences as static information, EvoLib is built around the idea of evolving knowledge . In EvoLib, a unit of knowledge can take the form of a reusable skill distilled from a successful solution or a reflective insight learned from mistakes. Rather than simply accumulating more memories over time, EvoLib continually refines, consolidates and reweights existing knowledge as new experiences arrive. Concretely, we design the following mechanisms around knowledge evolution:Consolidation. As new knowledge is extracted from recent experience, EvoLib retrieves similar knowledge from the library and tries to consolidate it with the new knowledge into a more general and reusable one. This allows knowledge to move beyond individual experiences and become applicable across tasks.Weighting mechanism. EvoLib continually updates the importance of each knowledge unit based not only on its immediate utility on the current task, but also on how much it contributes to generating useful knowledge on future tasks. Over time, knowledge with the greatest long-term impact naturally becomes more prominent in the library.Figure 1. EvoLib transforms raw experiences into reusable skills and insights, then continually evolves them through consolidation and dynamic weighting.Key ResultsTo evaluate EvoLib, we tested it across a diverse set of challenging tasks with different types of experiences and demands for learning:Solving mathematical reasoning problemsWriting code to perform the given tasks under efficiency constraintsMaking decisions to explore and interact with an environment to perform long-horizon tasksAcross these tasks, EvoLib consistently outperforms the top retrieval-based memory approaches and other abstract memory mechanisms with more efficient token usage.We also evaluated how effectively EvoLib converts test-time compute into performance gains through continually evolving knowledge. Figure 2 compares EvoLib against both compute scaling methods that perform each task in isolation and strong memory-based learning approaches. Each curve shows how performance improves as the amount of test-time compute increases.Across all three benchmarks, EvoLib achieves higher performance throughout most of the compute range and improves performance more rapidly with increasing compute.These results suggest that the key to better learning may not simply be storing more memories or spending more compute. Instead, the greatest gains come from transforming experience into reusable knowledge that can be continually refined and applied across tasks.Figure 2. Across all tasks, EvoLib converts test-time compute into performance gains more effectively than existing methods.Robustness to random task orderA natural question is whether such learning depends heavily on the order in which tasks are encountered. In the real world, an AI system may face diverse types of tasks in arbitrary order, and a useful learning framework should be robust to the randomness in task order. To evaluate this, we measured the task performance on the same set of heterogeneous tasks but with different task orders. We found that EvoLib consistently improves over existing memory-based learning approaches and maintains stable performance across different orderings. This indicates that EvoLib can continually learn from diverse tasks even when they are interleaved, suggesting its practical advantage in real-world scenarios where an agent must handle and learn from a mixed stream of heterogeneous user requests without relying on a structured curriculum.As AI systems take on longer-running and more complex tasks, learning from experience will become increasingly important. The future of AI may depend not only on larger models and more computation, but also on mechanisms that allow systems to continually accumulate, refine, and reuse knowledge.EvoLib is one step toward that vision. By transforming experience into evolving knowledge, it enables AI systems to continually improve and adapt after deployment. Rather than repeatedly starting from scratch, future AI systems may be able to build upon an evolving library of reusable skills and insights, much like humans do.Code and experiment results are available on GitHub (opens in new tab) to support future research on memory and knowledge evolution in AI systems.Opens in a new tabRelated publicationsTest-Time Learning with an Evolving Library Meet the authorsWeijia XuSenior ResearcherLearn moreAlessandro SordoniSenior Principal Research ManagerLearn moreZelalem GeroSenior ResearcherLearn moreMichel GalleySenior Principal Research ManagerLearn moreEric YuanPrincipal ResearcherLearn moreJianfeng GaoTechnical Fellow & Corporate Vice PresidentLearn moreResearch Areas
Artificial intelligenceRelated labs
Microsoft Research Lab - Redmond
Microsoft Research Lab – Montréal
Microsoft Health FuturesFollow us:
Follow on X
Like on Facebook
Follow on LinkedIn
Subscribe on Youtube
Follow on Instagram
Subscribe to our RSS feedShare this page:
Share on X
Share on Facebook
Share on LinkedIn
Share on Reddit
Surface Pro
Surface Laptop
Surface Laptop Ultra
Surface RTX Spark Dev Box
Copilot for organizations
Copilot for personal use
Explore Microsoft products
Windows 11 apps
Account profile
Download Center
Microsoft Store support
Returns
Order tracking
Certified Refurbished
Microsoft Store Promise
Flexible Payments
Microsoft in education
Devices for education
Microsoft Teams for Education
Microsoft 365 Education
How to buy for your school
Educator training and development
Deals for students and parents
AI for education
Microsoft AI
Microsoft Security
Dynamics 365
Microsoft 365
Microsoft Power Platform
Microsoft Teams
Microsoft 365 Copilot
Small Business
Azure
Microsoft Developer
Microsoft Learn
Support for AI marketplace apps
Microsoft Tech Community
Microsoft Marketplace
Software companies
Visual Studio
Careers
About Microsoft
Company news
Privacy at Microsoft
Investors
Diversity and inclusion
Accessibility
Sustainability
Your Privacy ChoicesConsumer Health Privacy
Sitemap
Contact Microsoft
Privacy
Manage cookies
Terms of use
Trademarks
Safety & eco
Recycling
About our ads