What’s Inside the Black Box

A “black box” is an engineering term that means a system whose internal workings are either unknown or irrelevant. Only the inputs and outputs matter. We use it for both design and test purposes. 

Your car is a black box to you, the driver. You don’t need to know how the engine, drivetrain, and electronics work to drive it. Only how to use the steering wheel, pedals, and, of course, how to connect your phone to it. 

However, to the designers and manufacturers of your car, it’s not a black box. It’s a white box. They know the innards. How each piece fits together. What each does and when. 

And most importantly, they know what’s going to happen and why when you turn the wheel or put your foot on the gas. The system is deterministic. If you turn the wheel to the left, the car will always, 100% of the time, go left.

LLMs are black boxes, but not just to the users. Unlike your car, they’re black boxes to their creators. 

AI creators know “sort of” what’s inside, but not exactly. Also, they don’t always know why stuff comes out given what went in. 

From one point of view, that’s a little scary. From another, it’s an opportunity. 

Maybe the trick isn’t trying to crack open the box. It’s learning how to steer it well enough to get where you’re going.