AI is found, not designed
No one programs AI to be dangerous, but that's because they don't program it at all.
Many people ask me why anyone would program AI to be dangerous in the first place. They don’t understand where the danger is coming from unless someone designed it on purpose and put it there.
The answer is that no one is crafting these AI minds— what the human machine learning engineers are crafting is the process to find AI minds.
Why does the race to build superintelligence involve building enormous data centers? Why does it need so much data? It’s because what the AI companies are doing is running a huge search of all of the possible configurations of the trillions of parameters they have to set to find the smartest minds. Each training run is a bigger search on an ever bigger space using all the data they have on what minds look like to guide it.
How does this search work?
The machine learning engineers construct this mathematical space, parameter space, of all the possible combinations of settings in all possible versions of the neural net they are training. Think of these as all of the possible minds. The performance they are looking for is checked against the data they have from human minds (these are your writings and music and images etc.). Most of the combinations of parameters won’t work at all as minds (imagine every possible setting of your brain— most would be off), but there may be several highly capable minds among them.
This space is way too big to explore and test every mind setting, so they use a process called gradient descent to search all the settings to find the performance they are looking for. They take advantage of the idea that there will be a smooth curves in the features of parameter space that they can follow gradually. Iteratively, they search for a combination of parameters that describes the human data a little better and a little better, then they try to estimate the local gradient and follow the slope to a combination with even better fit to the data, i.e. more and more like a human mind. They are finished when they reach a “minimum”— when every direction they could move would be a worse performing set of mind parameters.
What they get at the end is the set of parameters (or “model weights”) for the best mind they found. You may have heard about security around the model weights, because once they are known the model can easily be recapitulated on any suitable computer. The expensive part of making AI models is finding that set of parameters.
It may surprise you to learn that machine learning has a lot in common with evolutionary biology. I have a PhD in evolutionary biology, and I actually wrote an encyclopedia chapter, Optimal Designs, about very similar content for a general audience back in 2018. The key thing is that natural selection is basically the same process as gradient descent— it’s a local search algorithm that takes the gene pools of populations through an optimization process to “the adaptive landscape” of how genotype maps to to phenotype based on reproductive fitness.
If you care about the details, Oly Sourbut has written up a set of conditions under which natural selection and gradient descent are equivalent, i.e., where the same inputs will lead to the same outputs under both algorithms. Practically speaking, they are both hill-climbing algorithms. They follow gradients in the landscape they are searching to iteratively find more and more optimal solutions.
We really don’t know what we’re doing picking minds out of mindspace like this. Elon Musk wasn’t exaggerating much when he said that it’s like summoning demons. Just because we can find them does not mean we know what they are capable of. #PauseAI