Notes by Lex Toumbourou

“There are only two hard things in Computer Science: cache invalidation and naming things.” - Phil Karlton

I don't think naming things is hard anymore (cache invalidation is about the same).

LLMs, with their default of sampling from the highest-probability tokens, make for painfully boring and mediocre writing, but for many naming tasks, that’s exactly what you want. For example: naming methods for a public API; choosing names for events, actions, states etc; or trying to name an as-yet unnamed but known concept.

It also drastically reduces the time wasted bike-shedding or being stuck in analysis paralysis that often comes with naming tasks.

Simon Willison said something similar in the context of API design:

“A common criticism of these things is that they always come up with the most obvious answer… but when you’re designing an API, that’s exactly what you want.”

I’m sure many others have said it too.

My approach to naming things with LLMs is something like this:

1. Start by coming up with a few names for the thing on my own. I treat these names as a kind of hold-out set. This also helps me clarify exactly what I’m trying to name.
2. Then I ask my LLM of choice to come up with a collection of 5-10 names, with ranking and justification, with enough context. That’s typically:
   • A good description of what the thing is: what the class does, what the idea is, and so on.
   • What the thing is not, especially if there are similarly named concepts that do not fit.
   • The surrounding vocabulary of the codebase, API, paper, or note collection (which may come from the research step in Research, Plan, Implement Workflow)
3. Review the candidate names generated by the model and compare them with my hold-out set.
4. If one of the top candidate names is in the hold-out set, that’s a pretty damn good signal we have a winner. If the model’s candidates and mine are completely different, there’s a good chance I haven’t explained the concept well enough.
5. If I don’t see a clear winner in the candidate set, I’ll do another round. This time, I’ll give it my hold-out set of names and ask it to critique my names against its candidates.

In a short while, I usually find a name that feels obviously right. Sometimes I prefer one of my original names, but at least I have a good idea of why - and why it's better than the other options.

Photo by Compagnons on Unsplash