Kyle Kabasares, a data scientist at the Bay Area Environmental Research Institute in Moffett Field, California, used o1 to replicate some coding from his PhD project that calculated the mass of black holes. “I was just in awe,” he says, noting that it took o1 about an hour to accomplish what took him many months.

Bro it was trained on your thesis

There is definitely a lot of hype.

I’m not being sarcastic when I say I have yet to see a single real world example where the AI does extraordinarily well and lives up to the hype. It’s always the same.

It’s brilliant!*

*When it’s spoonfed in a non real world situation. Your results may vary. Void were prohibited.

OpenAI also tested o1 on a qualifying exam for the International Mathematics Olympiad. Its previous best model, GPT-4o, correctly solved only 13% of the problems, whereas o1 scored 83%.

Ah, I read an article on the Mathematics Olympiad. The NYT agrees!..

Move Over, Mathematicians, Here Comes AlphaProof

A.I. is getting good at math — and might soon make a worthy collaborator for humans.

The problem - as always - is the US media is shit. Comments on that article by randos are better and far more informative than that PR-hype article pretending to be journalism.

Major problem with this article: competition math problems use a standardized collection of solution techniques, it is known in advance that a solution exists, and that the solution can be obtained by a prepared competitor within a few hours.

“Applying known solutions to problems of bounded complexity” is exactly what machines always do and doesn’t compete with the frontier in any discipline.

-–

Note in the caption of the figure that the problem had to be translated into a formalized statement in AlphaGeometry’s own language (presumably by people). This is often the hardest part of solving one of these problems.

AI tech bros keep promising the moon and the stars. But then their AI doesn’t deliver so tech bros lie even more about everything to get more funding. But things don’t pan out again. And the churn continues. Tech bros promise the moon and the stars…

Tried it for python coding involving PDFs, OCR, and text substitution. Did worse than GPT-4o (which also failed).

Gave up and told it so. At least, it was very apologetic.

All of their models have consistently done pretty good on any sort of standard test, and then performed horribly in real use. Which makes sense, because if they can train it specifically to make something that looks like the answers to that test it will probably be good at making the answers to that, but it’s still fundamentally just a language parser and predictor without knowledge or any sort of internal modeling.

Their entire approach is just so fundamentally lazy and grifty, burning massive amounts of energy on what is fundamentally a dumbshit approach to building AI. It’s like trying to make a brain by just making the speech processing lobe bigger and bigger and expecting it’ll eventually get so good at talking that the things it says will be intrinsically right instead of only looking like text.

i feel like it passing the tests, but then also getting 30% incorrect biology responses, might be the most ironic and sad example of the failure of our memory-based education system i can think of

Oh cool, they’re using it in hospitals now

Kyle Kabasares, a data scientist at the Bay Area Environmental Research Institute in Moffett Field, California, used o1 to replicate some coding from his PhD project that calculated the mass of black holes. “I was just in awe,” he says, noting that it took o1 about an hour to accomplish what took him many months.

yeah I’m gonna doubt that, or he didn’t actually compile/run/test that code. like all LLMs it’s amazing until you interact with it a bit and see how incredibly limited it is.

It works 100% of the time 70% of the time now! While this is interesting and chain-of-thought reasoning is a creative way to get better at logic, this is inefficient and expensive to the point where hiring a person is certainly cheaper. I belive the API is only available to those who have already spent 1k on OpenAI subscriptions.

I know people here are very skeptical of AI in general, and…

People here are gonna have to come up with something to say about AI apart from just screeching “I hate it”

When man attains the knowledge of this common essence, he uses it as a guide and proceeds to study various concrete things which have not yet been studied, or studied thoroughly, and to discover the particular essence of each; only thus is he able to supplement, enrich and develop his knowledge of their common essence and prevent such knowledge from withering or petrifying. These are the two processes of cognition: one, from the particular to the general, and the other, from the general to the particular. Thus cognition always moves in cycles and (so long as scientific method is strictly adhered to) each cycle advances human knowledge a step higher and so makes it more and more profound. Where our dogmatists err on this question is that, on the one hand, they do not understand that we have to study the particularity of contradiction and know the particular essence of individual things before we can adequately know the universality of contradiction and the common essence of things, and that, on the other hand, they do not understand that after knowing the common essence of things, we must go further and study the concrete things that have not yet been thoroughly studied or have only just emerged. Our dogmatists are lazy-bones. They refuse to undertake any painstaking study of concrete things, they regard general truths as emerging out of the void, they turn them into purely abstract unfathomable formulas, and thereby completely deny and reverse the normal sequence by which man comes to know truth. Nor do they understand the interconnection of the two processes in cognition— from the particular to the general and then from the general to the particular. They understand nothing of the Marxist theory of knowledge.