Home | Tyler Burleigh – Tyler Burleigh

About me

Hi! I’m an AI Research Scientist with a multidisciplinary background in experimental psychology and data science. At Khan Academy, I work with Generative AI technologies like Large Language Models (LLMs), engaging in the full lifecycle of AI development – from research and solution architecture to engineering and evaluation. I aim to build high-quality, reliable AI experiences that will transform learning and support the next generation of learners. I’m passionate about using AI to make education more accessible, engaging, and effective for learners worldwide.

My latest blog posts

RAG-Powered LLM Longevity Coach

A simple RAG-powered ‘Longevity Coach’ that uses a vector store and an LLM to deliver targeted health insights informed by user-provided health data (including genetics, lab results, supplements/medications taken, etc.). By using RAG, the app reduces token usage and ensures only the most relevant data is used to respond to user queries.

Feb 2, 2025

Tyler Burleigh

Analysis of the GenAI/LLM job market in January 2025

A data-driven exploration of the GenAI/LLM job market for science and engineering roles in January 2025. I scrape ~1000 job postings from ai-jobs.net, perform data extraction and classification using LLMs, and then analyze the data to identify patterns and insights about the GenAI/LLM job market, including salary ranges, skill requirements, and role distribution.

Jan 24, 2025

Tyler Burleigh

Challenging SAMRE: Comparing multi-round debate-style LLM evaluation to a robust (and much simpler) baseline

In this post, I re-evaluate a method that was recently published in arXiv, critiquing their baseline model and then designing a new baseline model that implements standard best practices for comparison with the new method. I find that the new evaluation method proposed in the paper does not perform better than this robust baseline. This serves to highlight the importance of implementing best practices in baseline models for comparison with new methods, as well as being skeptical of claims in research papers that compare new methods to baseline.

Jan 12, 2025

Tyler Burleigh