Leftover Thoughts

Mathocalypse?

I read this interesting post on Scott Aaronson's blog earlier today. Now that I've had some time to digest it, I must say I'm far less impressed with events than he appears to be. Sometimes it helps to be an expert in the area (which he is) and sometimes it helps to have no expertise at all (which describes me). I'm not sure which is better in this case.

I want to be clear that I'm not one of the people he dismisses at the end of his post. I've been impressed by the power of AI - even if not the subsidies they've gotten in the form of not being prosecuted for stealing intellectual property - for years. AI is obviously a powerful tool and I'm not denying that.

But...and there had to be a "but" coming after a disclaimer like that, didn't there?

The first thing that jumped out at me when reading his post is that he doesn't explain why this is a big deal. There are proofs of some results. Great. That by itself doesn't change anything. The problems that were presumably proven to be true were true ten years ago even if we didn't have the proof.

Have we gained anything of value if we don't understand why those problems are true? I'm not so sure. Yes, it's cool to prove something that the best mathematicians couldn't prove, but being cool and being useful are different things. If all we wanted was true or false, we could have just chosen one by flipping a coin and got on with our lives, because it doesn't matter if we chose the wrong one. There has to be more to it than a binary answer.

The second thing is that this is a form of p-hacking. As he notes, they started with 8000 problems and they identified 372 that were (possibly) low-hanging fruit. OpenAI claims that it didn't take many resources, but that's only true if you ignore the cost of training the model. It also leaves us hanging when it comes to the cost of future proofs. How much will it cost to train the models and then to run them? Reading that was honestly deflationary for me. I was expecting more. I don't care much about the 372 potential proofs. I care about a tool that makes mathematical research more productive.

One way to visualize my feelings after reading his post is to picture yourself going to an expensive restaurant, having an appetizer that's spectacular, and then getting a main course that's no better than you'd get at Applebee's. Oh, it selectively solved the easiest problems? Oh, the cost doesn't include training? Oh, nobody understands the output, and it's potentially not even correct? Definitely not helpful - being dismissive of potential criticisms. I'm not sure from the post if Professor Aaronson understands that there's a middle ground where you accept that this is a big deal but you want to spend time thinking about it critically. Some of us view this as incremental progress rather than a revolution, and it doesn't strengthen your case to dismiss calls for evidence as irrational denialism.

PS: I asked Gemini a question and got an interesting answer. I did not bother checking the accuracy of the information. Nowhere in his post does he reveal any of it.

Is scott aaronson financially connected to any AI companies?

Scott Aaronson has direct financial connections to the AI industry through paid research employment, angel investing, and technical advisory roles: