How I built Mindscape Explorer

I have listened to Sean Carroll’s Mindscape since it started in 2018, and for most of that time I have had a question about it that no single episode could answer. The show’s premise is breadth: a physicist interviewing biologists, economists, philosophers, novelists, and political scientists, with solo lectures and listener-question episodes in between. I wanted to know what that breadth actually looks like when you treat the whole archive as one body of text. Which subjects does the show keep returning to? Which pairs of subjects have never shared an episode? Does a prepared lecture move through its material differently from a conversation? And could I search all of it by what I meant rather than by the words I happened to remember?

This post is the record of how I went about answering those questions. The result is Mindscape Explorer, and a companion page that checks whether a listener question has already been asked. I want to be upfront that I built both with Claude as a collaborator: the code was written in conversation, run on my own machine, and every number below came off that machine, with the evaluation deciding what shipped.

Getting the transcripts without being a nuisance

Sean publishes a full transcript with every episode, embedded in the WordPress post behind a “show transcript” toggle. That meant I never had to fetch the episode pages themselves. WordPress exposes a JSON API, and paging through it pulls every post in twenty or thirty requests. The scraper checks robots.txt before each request, identifies itself with a user agent that carries my email address, waits three seconds between calls, honors Retry-After on rate limits, and caches every raw response to disk so that no parser change ever causes a second fetch. The first run took about two minutes, most of it deliberate waiting, and it has been run against the live site only a handful of times since; new episodes land on the last cached page, so an update is a few requests.

The parsing was the harder part. Across eight years the transcripts use five different conventions for marking speakers and timestamps, and a survey of the archive was needed to find all five. The rule I held to was that nothing is dropped silently. Of 444 posts, 439 have transcripts and 436 parse into labeled utterances; the 5 without transcripts genuinely have none on the site (a welcome post, two holiday messages, an AMA whose audio was Patreon-only, and a link to an outside interview), and the 3 published without speaker labels carry a flag and null speakers rather than a guess. Any future episode in a sixth convention would surface as a flagged row to extend the parser against, not disappear. The corpus is 7.4 million words of transcript.

A pipeline that knows what changed

The analysis grew to sixteen stages, and early on I lost a day to a chart that had been rendered from a stale intermediate file. So the runner became a small make: each stage writes a manifest recording a fingerprint of the corpus, a hash of its own source, its parameters, and the vintages of its upstream stages, and a stage runs only when one of those has changed. A re-scrape rebuilds everything, an edit to one module rebuilds that stage and its dependents, and a run with nothing changed does nothing. The website is the final stage, so the published page can never lag the analysis. This is unglamorous, and it is the reason every number on the site and in this post agrees with every other.

Deciding how many topics there are

The core representation is a topic model: TF-IDF over one- to three-word terms (the show’s vocabulary is multi-word, “dark matter” and “wave function” being single ideas), factored by non-negative matrix factorization. The question every topic model has to answer is how many topics, and I did not want to pick a number. The pipeline sweeps k and records three curves: topic coherence, stability across random restarts, and reconstruction error. Coherence alone rises with k without limit, so its maximum is an artifact of where the grid ends. The combined score of coherence and stability has a real peak, but the bare maximum turned out to be noise-sensitive: when two episodes were added in September it flipped from 20 to 50 topics. So the selection uses the one-standard-error rule from the tree-pruning and clustering literature. The restarts the sweep already runs give each k a standard error, and the smallest k within one standard error of the best score wins. Candidates within the noise are treated as ties and broken toward the simpler model.

That gives an anchor of 50 topics. The site then works at three resolutions, 27, 50, and 92, and the ratio between them is not a choice either. The Stargazer project this grew out of scales by τ ≈ 1.839, the dominant root of x⁴ − 2x³ + 1, and the resolutions are the anchor divided and multiplied by τ. Whether the coarse topics actually contain the fine ones is inspected rather than assumed: the pipeline writes the similarity matrix between adjacent resolutions and a best-parent table for every fine topic.

Checking the model against the person who made the show

A topic model can always be read into. The check I trusted most was one the model had no part in: Sean assigns each episode a handful of subject tags by hand, and the topics were fit without ever seeing them. If the topics are real, they should be able to predict the tags. One classifier per tag, on each feature set, scored by nested cross-validation so that no score ever touches the data used to tune it, and format tags like “Podcast” and “Solo” excluded from the average.

Reading only the topic loadings, a gradient-boosted model predicts Sean’s 17 subject tags at a macro average precision of 0.649, against a chance level of 0.078. A logistic regression on the full TF-IDF matrix, which sees every word, scores 0.549, so the topics carry more of the tagging signal than the raw words do. The per-tag results are the more interesting reading. Physics, Biology, Astronomy, Economics, and Environment have vocabularies of their own and are recovered at 0.81 to 0.90. Humanity, World, and Psychology barely beat chance, at 0.37 to 0.42, and I think that is the right answer rather than a failure: those tags record an editorial judgment about what an episode is for, which no count of words is going to carry.

The finding I got wrong in advance

I expected a prepared solo lecture to move in a straight line through its subject and a conversation to ramble. To test it, every transcript is cut into tenths, each tenth is projected onto the fixed topic basis by non-negative least squares, and the route an episode walks through topic space is measured as the sum of Hellinger steps between consecutive tenths. Solo lectures have a median route of 3.12 (95% bootstrap interval 2.66 to 3.90, n = 28) and interviews 3.54 (3.38 to 3.74, n = 347). The difference of medians is −0.42 with an interval from −0.90 to +0.44 and a permutation p of 0.22, so the two formats cannot be told apart. The episodes that wander are the AMAs, at 6.44 (6.34 to 6.63, n = 64), which is 2.91 further than an interview (2.66 to 3.16, p < 0.001), because each one answers dozens of unrelated listener questions in a single sitting.

There is a second half to this that I would have missed with only one representation. Measured again in sentence-embedding space (more on that below), the three formats walk similar distances, with medians of 2.14, 2.19, and 2.36, and the rank correlation between the lexical and the semantic routes is 0.04. An AMA changes its words far more than it changes its meaning, which is what you would hope from a host answering physics questions in a physicist’s vocabulary all afternoon.

A recommender scored on more than accuracy

There is no listening data here, so relevance has to mean topical affinity, measured as the Bhattacharyya coefficient between two episodes’ topic distributions and averaged over the three resolutions. Each episode gets three lists. The first is the obvious “more like this”. The second is a serendipity list in the sense of Ge and colleagues: it must clear a relevance floor and it excludes the accuracy top-N, but it is ranked by the affinity carried through the seed episode’s minor topics, with the bridging topic reported, and then spread by maximal marginal relevance. The third is a deliberate “least likely from here”.

The lists are evaluated on the beyond-accuracy suite of Kaminskas and Bridge rather than on a single number, with percentile-bootstrap intervals over the 439 seed episodes. The obvious list is as narrow as you would expect, with an intra-list diversity of 0.49. The serendipity list reaches 0.82, a paired difference of +0.33 (0.32 to 0.34), while still covering 99.8% of the catalogue, and at no measurable cost in novelty (a difference of +0.001, with an interval from −0.007 to +0.008). The “least likely” list reaches only 37% of episodes, with a Gini concentration of 0.91, because the show has a few far corners and every seed points at the same ones. I find that last number reassuring about the other two: the far corners exist, and the serendipity list is not simply sending everyone there.

A map with roads that have not been built

Picture the show as a map on which the well-visited subjects are cities and episodes are the roads between them. The co-presence of two topics is the sum over episodes of the smaller of their two shares, and among the show’s most-visited topics some pairs have almost none. The lowest is cancer and genetics with party politics and elections, followed by cancer and genetics with scalar fields, and the AMA cluster of wave functions and black holes with cancer and genetics. For each such frontier the pipeline names the past guest whose own topic mix sits closest to both sides (Andrew Leigh for the first pair) and the two specialists who could be invited together (James P. Allison with Ezra Klein). I do not imagine Sean books guests by consulting a co-presence matrix, but I did enjoy that the data can make a concrete suggestion rather than a vague one.

A second geometry, to keep the first one honest

Everything so far is built from word counts, and a change of representation could change any of the answers. So the pipeline’s one model-inference stage encodes 33,522 passages of about 220 words, and 49,760 interview turns, with a sentence-embedding model (Qwen3-Embedding-0.6B), and then repeats the main analyses in that space. The two geometries agree without being the same: the Mantel correlation between the lexical and semantic episode distances is 0.644 (p = 0.0005 over 2,000 permutations), and on average 46% of an episode’s ten nearest neighbours are shared between the two. The disagreements are the interesting cases, and the site lists them as hidden bridges (episodes that are close in meaning but share little vocabulary) and vocabulary collisions (the reverse). A few smaller findings replicate across both: guests echo Sean’s words more than he echoes theirs in 85% of interviews, and the concepts whose contexts have drifted most across the show’s three epochs are “emergence”, “intelligence”, and “complexity”, with the semantic and lexical drift rankings agreeing at ρ = 0.75.

Searching by what you meant

The last thing I built is the one I use most. The Search page takes a question in plain words and returns the moments in the archive closest to what was meant, each with its timestamp, its speakers, the part of the passage that carries the query’s words, and a link to the episode. The engine is the relevance floor from Heart of Gold, a retrieval system I built earlier for Claude agents: a dense cosine pool over 20,842 transcript chunks, a lexical channel that adds any chunk containing the query’s rare words so that names and jargon are never lost, and a cross-encoder reranker that reads each candidate against the query and gives a probability that it answers.

The reranker is the expensive part. It scores about 32 chunks a second on my machine, and the candidate pool is 376 dense chunks plus up to 376 lexical additions, so a full floor takes 12 to 24 seconds. Rather than guess at an interactive budget, I measured what each budget keeps, against the fully reranked floor as the reference, over a set of blind queries:

The service now runs at 12 seconds, which returns the full floor on nearly every query, and the page hides the wait by asking twice: the cosine-only floor comes back in about a second and is shown at once with a note, and the reranked list replaces it when it arrives. The whole thing runs on a Mac in my office through a Cloudflare tunnel, which is not how one would host a product, so when that machine is asleep the same search box falls back to a keyword index over the same passages and says so.

What the site does not do

The transcripts are Sean’s, and the site quotes only short passages, each with a link to its episode, and reproduces no transcript. The transcripts were collected from the site’s public API, respectfully and only a few times, and I have told Sean the work exists and offered to change or take down anything he would prefer not to see. That felt like the minimum owed to someone whose eight years of conversations made the whole exercise possible.

If you have a question about the show that the search cannot answer, or you are working on something adjacent, I would be glad to hear from you.