Noticing You’re Confused

And somewhere in the back of his mind was a small, small note of confusion, a sense of something wrong about that story; and it should have been a part of Harry’s art to notice that tiny note, but he was distracted. For it is a sad rule that whenever you are most in need of your art as a rationalist, that is when you are most likely to forget it. –Harry Potter and the Methods of Rationality, Chapter 3

Noticing You’re Confused is a technique from the rationalist community where you train yourself to recognize even a slight feeling of confusion as a valuable signal that something about your model of the world is wrong. It’s similar to what people sometimes mean by ‘trust your gut’, but where ‘trust your gut’ is usually an end to the conversation, noticing confusion treats the feeling as the starting point for understanding what’s happening. 

I want to tell you a short story about the value of Noticing You’re Confused.

When I was running my company, we searched for a VP of Engineering and it was taking longer than expected, the demand for qualified engineering managers being far, far, greater than the supply.

One day I had a call with a very promising candidate. Let’s call him Sam. In his last two roles, Sam had managed seventy and thirty person engineering teams at well known, respected companies. Our conversation went well and I invited him in to interview.

In my nine years at ZeroCater I probably interviewed around three thousand people. To this day, Sam stands out as one of the best interviews I ever had. Every question was answered thoroughly and thoughtfully, with no unnecessary digressions. I distinctly remember feeling like there were bullet points forming in the air as he talked.

I had the rest of the executive team and several engineers interview Sam. I remember asking one of the engineers how it’d gone after the interview. “Yeah, really impressive guy. You almost expect people to stumble at least a little, but he just knocked down my questions one by one.” The rest of the team unanimously loved him. The next stage was reference checks.

There are two kinds of reference checks: front door (those provided by the candidate), and back channel (those you find yourself). I started with the front door references.

They were good. All of them emphasized his reliability and how little management he needed. The most notable was a VP he’d reported to whose LinkedIn profile showed that she’d gone on to executive roles at multiple Fortune 500 companies. She told me he was one of the best people she’d ever managed and would certainly hire him again given the chance.

Unfortunately, the only backchannel references I’d been able to find were extremely weak. One was a friend of a coworker’s husband who’d never worked with Sam and only knew him casually. The other was the founder of a company I knew where Sam had mentioned having several friends. I reached out to him asking if he knew Sam. Apparently Sam had applied for a Director of Engineering job there a year ago. He hadn’t interviewed him personally and the only note in their system was ‘not a culture fit’, which he admitted was pretty specific to them.

My team was getting impatient. One of our VPs forwarded me a ‘strong yes’ email an engineer had written with the following memes attached.

People wanted to get this done, but I felt a small note of wrongness. It was at this point that I actually said out loud to myself “I notice I’m confused.”

Confusion feels like unresolved surprise. It’s a violation of your expectations that happened somewhere along the way that was never explained. It’s a feeling of things not quite fitting together.
 
I sat down and tried to trace the source of my confusion. Sam didn’t have a LinkedIn profile, and when I asked him why he said he turned it off because he got too much recruiter spam, which was entirely believable. (1) He said he’d turn it back on and send me the link, but hadn’t followed up. It wasn’t much, but it was certainly a bit odd given how reliable he seemed in general. It was enough to reaffirm my desire to find a good back channel reference.

I racked my brain trying to think of people I knew at one of Sam’s previous employers. Suddenly I remembered a company happy hour from two years back. A friend had brought the CTO of the same company where Sam had managed seventy engineers, and we’d had a couple drinks together. I sent him a message on Facebook reintroducing myself, mentioned we were considering their former Director of Engineering, Sam ******, and asked if he had time for a quick call. He called me immediately.

“I’m so sorry, this isn’t the first time this has happened. He wasn’t a Director and he didn’t manage an engineering team of seventy; he managed a quality assurance team of three.”
“What do you mean? I talked to your VP who said he was amazing.”
“Who?”
“Here’s her LinkedIn profile.”
“I’ve never heard of this person and they never worked for us.”
“Then who did I talk to?”
“Probably his wife.”

I was stunned. Not only had he lied about his experience, he’d set up fake identities complete with LinkedIn profiles with hundreds of connections, then gotten people who were complicit in his lie to pretend to be those people on the phone. (2)

Sam was a con man, and in all likelihood, a sociopath. I don’t even want to think about the kind of damage someone like him could have caused in a position of power. The only thing that saved me was noticing that small, small note of confusion.


(You can learn more about practical rationality on the LessWrong Sequences, or if you want the ebook version, Rationality: From AI to Zombies. If that seems boring, Harry Potter and the Methods of Rationality is a fun read and has a lot of the same concepts in fiction form. I also recommend the four day workshops hosted by The Center for Applied Rationality.)

Notes
(1) Some people have asked why Sam would bother setting up multiple fake LinkedIn profiles and not have one himself. My guess is it’s because that LinkedIn would have suggested him as a contact to his former coworkers who would have seen that he was claiming to have done a job he never did. Also, if he happened to keep it up, I would have personally been tempted to contact whoever hired him next.

(2) People have asked me if this means that interviews are worthless. To quote legendary intel CEO Andy Grove:

We know how hard it is to assess the actual past performance of our own subordinates even though we spent much time working closely with them. Here we sit somebody down and try to find out in an hour how well he is likely to perform in an entirely new environment. If performance appraisal is difficult, interviewing is just about impossible. The fact is, we managers have no choice but to perform the interview, no matter how hard it is. But we must realize that the risks of failure are high. 

A test that gives false positives 20% of the time is still a useful test. You just need to combine it with other tests, like references. I will admit that since this experience I’ve often thought that if I had to choose I’d rather have a really solid back channel reference from someone I knew was good who’d worked closely with the person than an interview.

Lichtenberg Shelf

Some time ago I decided to build a shelf to hold my 3D printers. Coincidentally, around the same time our neighbors decided to throw out their microwave oven. Microwaves contain a formidable transformer, so I took it inside and gutted the interesting bits before returning the remainder to the trash.

A word of warning: opening a microwave oven is dangerous: the capacitor can retain enough energy to kill you despite being unplugged, the magnetron can contain beryllium oxide, which if broken into powder and inhaled can kill you, and finally, the transformer puts out enough current to kill you before you hit the ground. In retrospect, this project probably wasn’t worth the micromorts.

Lichtenberg Figures are branching patterns formed by high voltages. If you happen to survive a lightning strike, you might end up with a Lichtenberg figure on your skin. You may note that this would make a pretty badass tattoo.

Golf courses are also known to exhibit Lichtenberg figures after lightning strikes.

This pattern is called a brownian tree and is typically produced by a process called diffusion-limited aggregation. Copper precipitated using electro deposition follows this pattern.


I decided to decorate my shelf with Lichtenberg figures. I started by visiting my favorite hardwood store and buying a piece of African Mahogany.

I also bought some high voltage wire and mounted the transformer on a piece of rubber and a few blocks of wood to make it harder to accidentally short it.

I attached wood screws to the wires to use as electrical probes. The current won’t flow through dry wood, so I used a solution of baking powder and water which I painted on with a brush.

Again, this is dangerous: the transformer puts out around 2200 volts at 0.5 amps. I needed to move the probes many times to make my design, so I created a procedure where I would check to see it was unplugged, then saying “unplugged” to myself out loud before approaching. Before plugging it back in, I’d step back, check that everything was clear and say “powering on” before plugging it back in.

After some trial and error with the electrolyte solution I started making progress. My first attempt was crude with big char marks.

Having proven the concept, it was time to build the shelf. I didn’t have a table saw so I made do with a six foot straight edge clamped in place and a handheld circular saw. This method is is a huge time consuming pain but it works.

Most of the hours in this project came from the fact that the wood I had wasn’t wide enough, so I had to cut and join together two pieces for every part of the shelf. Here are some pieces curing while clamped together.

After joining all the pieces, I gave them a good sanding to make all the edges even. I was proud to note that on some pieces you can’t even tell it’s made of two pieces of wood.

After a several hours of experimenting I discovered how to make the burn patterns I wanted. There’s an art to it: if you just leave the probes in place, the current will either evaporate all the solution and stop conducting, or it’ll find a path between the terminals and your wood will be on fire.

Creating a pattern that resembles a lightning bolt requires moving the probes hundreds of times. I learned to control the size of the burn features by controlling the amount of solution. The more solution, the more current, the thicker the burn mark. To create a lightning pattern, I used a lot of solution to create a thick central burn, then repeatedly moved the probe off to the sides and used less solution to create smaller features, then moved the probes off to the side of those features and used less solution still to make even finer details. You can see a time-lapse of this process below. Note this is spend up around 10x – in reality there is a substantial pause between the current being on and my hand being near the probes.

Each board I did was better and less charred than the last. I decided to make my first attempt (not pictured) the bottom of the bottom shelf where no one would see it. My last and best attempt became the top of the top shelf.

After all the burns were made I used a wire brush to get as much charred wood out of the burns as possible. Then I washed off the soot, gave it another sanding, used a piece of tack cloth to remove any dust, and finally applied Osmo Polyx oil.

The finished piece:

An Opinionated Light

Many years ago I went to an exhibit in the MOMA – a room where everything appeared to be in black and white. It was shades of sepia rather than actual back and white, but the effect felt like being in an old movie. I recently became curious about how the effect was produced and decided I wanted to make my own monochromatic lamp.

The light we’re used to is usually a continuous mix of many different frequencies. For example, here’s sunlight, incandescent bulbs, and LEDs.

Our eyes perceive color by sampling light at different frequencies. If there’s only one frequency to sample, objects can only reflect different intensities of that one color.

The trick then, is to find a light source that puts out light a single frequency of light. For reasons I won’t get into, it’s fundamentally impossible to have light of a single frequency, but we can get light in a narrow band of frequencies.

It turns out that low pressure sodium (SOX) lamps output a very narrow range of frequencies, centered on 589 nanometers.

Here’s a red car and a black car under a SOX lamp. You’ll note they both look black.

So I set out to buy a SOX lamp.

Amazon had plenty of bulbs, but no lamps to put them in. I called six lighting distributors, one of whom told me that in his six years in the business he’d never ordered a SOX lamp. Eventually I got clue I needed: as long as the ballast I used had a matching ANSI code it should work.

I purchased a bulb, a socket, then found a ballast that looked like it might work. The transformer they sent me was a raggedy looking thing but I managed to decipher the attached schematic and wired everything up, and lo’ there was light.

Not wanting to leave high voltage wires sticking out all over the place I now needed a housing, so I modeled out a few options in the CAD mode of Fusion360.

I made an expedition to my favorite hardwood store and ordered an acrylic tube from McMaster Carr.

After a false start where I ruined a lovely piece of lacewood, I settled on cherry, fired up my double bevel sliding compound miter saw and got to work.

Most woods darken from sunlight. I wanted an even tan on my lamp, so after applying some Osmo Polyx oil I set my pieces to sunbathe on the roof.

After sunbathing, I also added four coats of spray lacquer.

Since the transformer was already big I designed the housing to fit fairly closely, which meant I had to Dremel some edges off.

Of course, despite the incredible care I took with measuring and remeasuring the edges of my hexagon did not meet. After a lot more sanding and fighting I eventually glued the sides together.

I now needed to drill a large, precise hole for the acrylic tube. Unfortunately, the cheap hole-saw drill bit I ordered turned out to be more of an oval than a circle so that wasn’t going to work. Finally, a chance to use the giant Shapoko XXL CNC router I’d recently finished assembling.

Watching this machine cut a perfect circle made me realize there must be a reddit community devoted to CNC porn. I was right.

Since the acrylic from McMaster was clear, I took it outside and power sanded it with a fine grit to make it frosted.

With the lamp completed, it was time to test it out.

Using a hand-held spectroscope I first looked at the frequency breakdown of a white LED and saw the typical blur of color you see in rainbows.

Then I looked at the sodium lamp. There was the main band at ~589nm.

It works!

Makes a good party game