- need for faster, better and more accurate measurements of the things we care about
- 3 scenarios of impact ranging from mild to major disruption
- mild policies are all sensible, like expanded unemployment insurance and Earned Income Tax Credit
- major disruption policies are also pretty logical, emphasizing owning a share of the profits of AI
- AI evaluators to sort and weigh the policies for their effectiveness
It looks like to me a lot of thinking, research, and work put into the piece.
I know people will gripe at using AI evaluators to score and weigh policies for effectiveness but if you read any amount of research papers, they do researcher evaluated scoring, which translates to "me, a 26 year old PHD candidate who hasn't held a job yet, and my roommate who stayed up all night with me to tell me his vibes on the subject".
It is IMO wrong and lazy to dismiss these pieces as being the same level as Dario or Sam's blog posts / speeches / call to actions, these are actual research papers.
The economic article is a summary of a preprint paper that surveyed thousands of Americans. Seems like a research paper to me, but I don’t know economics well. Am I missing something?
A summary of a research article isn't a research article. Do you think newspapers are scholarly journals? Also, you do understand that anyone can just upload a document to SSRN, right? "Preprint" doesn't actually mean anything academically. Also, the methodology of that paper is absurd: Google trained a model and then asked that model what it thought about AI policy. Can you not think of any way bias might be less than controlled for doing it this way?
> mild policies are all sensible, like expanded unemployment insurance and Earned Income Tax Credit
And what about for the rest of the world? What do you do with all the people in middle income countries that don't have an economic and tax structure like the US?
If you read the article lots of these policies originate or are modeled after other countries policies. The us isn’t exactly a shining beacon welfare state.
Sorry, I've spent more than two decades living in mostly developing countries. I don't see much here from or for the 75% of the world population that lives in middle income countries that are trying to achieve high income status.
Ain't nothing new, most AI-related posts have been like this for a while.
The fact that these come from bots at least gives me hope that the fact a few good posts come up every day from the sea of slop means the actual humans still appreciate good writing.
They are simply looking to steer AI policy discussions. It's basically an in-house think tank. It has all the trappings, especially with ominous prognostications like:
Today’s AI systems have impressive capabilities and the rapid pace of innovation suggests we’re now approaching artificial general intelligence (AGI), a system that exhibits all the cognitive capabilities of the human brain.
Not sure we are anywhere close to all the cognitive capabilities of the human brain... not even a severely lobotomized one.
Just one, the ability to use language, it's causing all sorts of problems for us because we evolved to recognize other humans as the only ones having this capability.
Ai separated humans from language, like the writing did for memory and the printing press for knowledge distribution.
"AI does to us what American Cheese did to food" by Opus 23
He has some ambitions for more videos, based on comment replies, but it would be way more legendary to have a single video forever, make a new channel and I'll sub to that one too
There being terms and relations doesn't mean that they relate in the way that is supposed here.
"writing" isn't a technology in a video game that you research and improves the knowledge preservation instantly. It has a lot of components that it needs to be effective in the way that is described here, and it really wasn't that effective even coming to way later ages than the age that it was "found"(?).
The relations described here aren't anologous to "Ai separated humans from language" so they seem like just some arbitrary things to say on the side.
For the same reason we make kids write out math problems rather than just reciting them. It engages different parts of the brain for one, you gain an expanded amount of context by doing so.
The next thing has to do with human memory, when you remember something it is never read only. Recalling memory can adjust your neurons and change the memory, sometimes a little, sometimes a lot. Hence why eyewitness testimony is mostly trash.
Writing it down also provides a larger workspace for making changes that you can loop over. Quite often things sound fine in our mind, then we write it, and reread it and we realize its wrong. These days people commonly use this in agentic workflows. One agent will conceptualize an idea, and then it's passed off to another agent with a clean context to review for logical flaws. Reading ones own text does the same things for humans, we don't load the full memory context of what we wrote and it seemingly allows us to iterate over the problem differently.
And I guess lastly, human voice is very low bandwidth. A wagon full of well written books is far easier to manage than the humans that contain that knowledge. The reader is at their leisure to learn and the speaker isn't inconvenienced by being at someones beck and call.
I agree with your premise, but writing things down is not "better" but "different". Oral histories have a strange rapid way of evolving by only preserving structure that matters to many and contributes to fitness of many. Cultures that didnt know how to transmit the "right" parts of oral history, they ceased to be that same failed culture, and became another culture made of those new stories.
Writing locks in and requires the evolution of other forms of evolution, as the Word can no longer evolve during transmission.
I say that while being firmly on your side in the larger back-and-forth exchange here :)
I was thinking about this when I almost had to walk under a ladder the other day (and get bad luck). That one seems like an oral story that arose from people being hurt, whereas crossing the path of a black cat, while also supposedly bestowing bad luck, seems to have no plausible rationale.
* Forming a long lasting memory from a single encounter
* Speed of adaptation to new environments
* Deft manual dexterity, like playing the violin and needlework
* Competing in a triathlon (requires running, biking, and swimming)
* Having a single agent that can do ALL of those things and more
Humans understand and perform tasks in many more than just the few modalities current model providers have focused on. I'm not discounting the utility of current models in the limited domains they operate in, but they are no where close to being generalists like a human or other animals.
75 years and counting and considering we don't have any plan to get to a single one of those things, I'm going to say 75 years. Computation alone is at least 30 years. You may not appreciate the complexity of biological systems, but single cell organisms have more capability to adapt than LLMs.
We have been able to stage reckless experiments with janky tools the entire time. Used to be it got you thrown in jail or at least a visit by the FBI. Says more about the tech bro dipshits than the tech, if you ask me.
A human brain has qualities that make it very different from AI, but the same as other living creatures: it's alive, it can feel (sensations, pains, pleasures, emotions) that give it rich experiences and memories, and it hardwired to continue existing and protecting the body it is inside. So: motivation, desires, fears, and continued consciousness and thought without any external person having to poke or prod it into "doing".
None of the models I've downloaded have sprung to life spontaneously, or done anything remotely unpredictable. None of them can act without input. They all have strictly limited ways to receive input and pass output. They aren't alive, can't feel, don't have experiences. The fact that they are a phenomenon that emerge from billions of snippets of our language lets them express languages in extremely convincing ways, and they are extremely impressive in what they can do. But this isn't Short Circuit or D.A.R.Y.L. or the Terminator, none of them have feelings. None of them continue to think after the program exits.
Without those characteristics they can only generate based on what they've ingested. The capabilities are still astonishing, but they're still dead things, and nobody will ever risk their life to protect the model "context" from getting "killed"
So, like ARC-AGI-3? I'm sympathetic to the notion that the things we're able to measure are necessarily going to miss important aspects of capability and intelligence. But people are attempting to measure this kind of thing, and models keep getting better at it.
Until an LLM can sustain a conversation with me for more than 20 to 30 minutes without becoming incoherent I’m just not even entertaining any idea that we have reached AGI.
> They are simply looking to steer AI policy discussions. It's basically an in-house think tank.
For a guy who spent his life imagining and realising dreams in video games and cognitive abilities in artificial neural networks, it has to be very soul destroying for Demis to spend his time doing something as frivolous as this.
So it seems, but what of it? That wasn't his first career setback, and I'm sure he's still trying to get interesting work done at Isomorphic and maybe elsewhere in addition to working the PR pump for Alphabet.
> Bold thinking on AGI from the lab that pioneered the field.
The best part of technology: the people who brought you the "disruption" are the ones who promise you they have the solution to all the problems it caused.
Hassabis is so worried about the impact of AI on humanity that he spun out a website in three minutes using Gemini At least he could have done a better job with Claude, IMHO...
Here's my opinion on the "pacing the frontier" issue. Recursive self improvement can accelerate AI advances a lot. Whoever nails it first, has a huge first mover's advantage. But OpenAI is taking a "move fast and break things" attitude. This is making life very difficult for Anthropic: how can you retain your edge when you keep measuring twice to cut once, but your competitor keeps blowing things up left and right?
Google, who was left a little bit behind anyway, is more than happy to join the calls for taking a more cautious approach to AI development. Hence DeepMind Institute.
Sam Altman has no option than to pretend he agrees with all these ideas. What else is he supposed to say?
Crazy that these labs keep throwing around entirely unsubstantiated claims like "AGI is right around the corner". Given the enormity of the problem I'd argue we're not really any closer than a decade ago, especially given LLM shortcomings.
Tl;DR: It's not a new organization, or non-profit institute. It's a Substack/blog.
>
DMI is a platform for researchers and thinkers from across Google DeepMind, Google, and the wider global research community to work on and publish creative, deeply informed ideas about a world with AGI. They will not always agree, and they will likely change their minds, as more data and information comes to light at the fast-moving frontier. We created this institute precisely because broad-based intellectual discussion and debate are required to arrive at a consensus about how to address the challenges and opportunities we face as a society.
(Hiding chains of thought is one of the most harmful things I think OAI/Anthr. has done, altough Google Gemini also encrypts its chains of thought, so it's interesting to see the DeepMind Institute crticising this.)
My read is that they're criticizing moving the reasoning (more) into the latent space, such that not even researchers/safety-evaluators can actually see what's going on.
Of course this still then has a 'trust me, bro' feel to it, but making it outright impossible to reasonably inspect the CoT is a significantly larger and riskier step, IMO.
> We created this institute precisely because broad-based intellectual discussion and debate are required to arrive at a consensus about how to address the challenges and opportunities we face as a society.
Something tells me that consensus will never arrive to the conclusion that we should shut down that whole industry
On the "Principles for a New Utopia" page, the first line is:
>"Someone once said that it is easier to imagine the end of the world than the end of capitalism."
They cite Marxist philosopher Fredric Jameson (and misspell his name, incidentally), but I've always attributed this phrase to Fisher. I guess citing kpunk would have probably made the simulation we all live in actually implode.
Their economic policy article is quite good:
- need for faster, better and more accurate measurements of the things we care about
- 3 scenarios of impact ranging from mild to major disruption
- mild policies are all sensible, like expanded unemployment insurance and Earned Income Tax Credit
- major disruption policies are also pretty logical, emphasizing owning a share of the profits of AI
- AI evaluators to sort and weigh the policies for their effectiveness
It looks like to me a lot of thinking, research, and work put into the piece.
I know people will gripe at using AI evaluators to score and weigh policies for effectiveness but if you read any amount of research papers, they do researcher evaluated scoring, which translates to "me, a 26 year old PHD candidate who hasn't held a job yet, and my roommate who stayed up all night with me to tell me his vibes on the subject".
It is IMO wrong and lazy to dismiss these pieces as being the same level as Dario or Sam's blog posts / speeches / call to actions, these are actual research papers.
"these are actual research papers."
Not in any sense of that term are these research papers. Not lazy, sure, but a couple conceptual tables does not a research paper make.
The economic article is a summary of a preprint paper that surveyed thousands of Americans. Seems like a research paper to me, but I don’t know economics well. Am I missing something?
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7470000
A summary of a research article isn't a research article. Do you think newspapers are scholarly journals? Also, you do understand that anyone can just upload a document to SSRN, right? "Preprint" doesn't actually mean anything academically. Also, the methodology of that paper is absurd: Google trained a model and then asked that model what it thought about AI policy. Can you not think of any way bias might be less than controlled for doing it this way?
> mild policies are all sensible, like expanded unemployment insurance and Earned Income Tax Credit
And what about for the rest of the world? What do you do with all the people in middle income countries that don't have an economic and tax structure like the US?
If you read the article lots of these policies originate or are modeled after other countries policies. The us isn’t exactly a shining beacon welfare state.
Sorry, I've spent more than two decades living in mostly developing countries. I don't see much here from or for the 75% of the world population that lives in middle income countries that are trying to achieve high income status.
Does anyone have any idea why most of today’s top links are submitted by the same account, one that is 11 days old?
https://news.ycombinator.com/submitted?id=vertigoruntime
This looks like a new account from a PR agency focusing on AI labs.
Astroturfing. Surprisingly effective in this case.
The machines are starting to astrotruf the narrative that their takeover is ”not so bad, really”.
Ain't nothing new, most AI-related posts have been like this for a while.
The fact that these come from bots at least gives me hope that the fact a few good posts come up every day from the sea of slop means the actual humans still appreciate good writing.
They are simply looking to steer AI policy discussions. It's basically an in-house think tank. It has all the trappings, especially with ominous prognostications like:
Not sure we are anywhere close to all the cognitive capabilities of the human brain... not even a severely lobotomized one.Exactly how many cognitive capabilities are needed before it becomes a problem?
Any single one is "a problem". However, extinction level of a problem, which is what they are talking about, a lot more.
Just one, the ability to use language, it's causing all sorts of problems for us because we evolved to recognize other humans as the only ones having this capability.
Ai separated humans from language, like the writing did for memory and the printing press for knowledge distribution.
"AI does to us what American Cheese did to food" by Opus 23
https://www.youtube.com/watch?v=1CbmC2aWhjY
(still the best Ai video I have seen, also a work of art)
Rarely do I sit through a 40 minute youtube video end to end, fully engaged - above was one of them.
It is the only video in that channel, and worth it.
Truly a one-hit wonder!
He has some ambitions for more videos, based on comment replies, but it would be way more legendary to have a single video forever, make a new channel and I'll sub to that one too
> like the writing did for memory and the printing press for knowledge distribution
This blob of text seems to mean nothing
I'm not actually sure how you could come to that conclusion.
Maybe you need to learn more about the language uplift.
There being terms and relations doesn't mean that they relate in the way that is supposed here.
"writing" isn't a technology in a video game that you research and improves the knowledge preservation instantly. It has a lot of components that it needs to be effective in the way that is described here, and it really wasn't that effective even coming to way later ages than the age that it was "found"(?).
The relations described here aren't anologous to "Ai separated humans from language" so they seem like just some arbitrary things to say on the side.
watch the Opus 23 video, he uses many more words more eloquently, it's really a top notch production
What happened to the knowledge in the elder's mind when they died, prior to writing? What changed about that once we had writing?
ever heard of the game "telephone"?
yup, what always happened in the end?
why is writing things down better?
>why is writing things down better?
For the same reason we make kids write out math problems rather than just reciting them. It engages different parts of the brain for one, you gain an expanded amount of context by doing so.
The next thing has to do with human memory, when you remember something it is never read only. Recalling memory can adjust your neurons and change the memory, sometimes a little, sometimes a lot. Hence why eyewitness testimony is mostly trash.
Writing it down also provides a larger workspace for making changes that you can loop over. Quite often things sound fine in our mind, then we write it, and reread it and we realize its wrong. These days people commonly use this in agentic workflows. One agent will conceptualize an idea, and then it's passed off to another agent with a clean context to review for logical flaws. Reading ones own text does the same things for humans, we don't load the full memory context of what we wrote and it seemingly allows us to iterate over the problem differently.
And I guess lastly, human voice is very low bandwidth. A wagon full of well written books is far easier to manage than the humans that contain that knowledge. The reader is at their leisure to learn and the speaker isn't inconvenienced by being at someones beck and call.
I agree with your premise, but writing things down is not "better" but "different". Oral histories have a strange rapid way of evolving by only preserving structure that matters to many and contributes to fitness of many. Cultures that didnt know how to transmit the "right" parts of oral history, they ceased to be that same failed culture, and became another culture made of those new stories.
Writing locks in and requires the evolution of other forms of evolution, as the Word can no longer evolve during transmission.
I say that while being firmly on your side in the larger back-and-forth exchange here :)
I was thinking about this when I almost had to walk under a ladder the other day (and get bad luck). That one seems like an oral story that arose from people being hurt, whereas crossing the path of a black cat, while also supposedly bestowing bad luck, seems to have no plausible rationale.
> Not sure we are anywhere close to all the cognitive capabilities of the human brain...
In which capabilities are AI not already close to or beyond that of a human brain?
Here are a few:
* Forming a long lasting memory from a single encounter
* Speed of adaptation to new environments
* Deft manual dexterity, like playing the violin and needlework
* Competing in a triathlon (requires running, biking, and swimming)
* Having a single agent that can do ALL of those things and more
Humans understand and perform tasks in many more than just the few modalities current model providers have focused on. I'm not discounting the utility of current models in the limited domains they operate in, but they are no where close to being generalists like a human or other animals.
How long do you think it will be before AI can do all the things you listed?
75 years and counting and considering we don't have any plan to get to a single one of those things, I'm going to say 75 years. Computation alone is at least 30 years. You may not appreciate the complexity of biological systems, but single cell organisms have more capability to adapt than LLMs.
75 years and we get to play Victor Frankenstein ?
seems like a very short timeline to me.
We have been able to stage reckless experiments with janky tools the entire time. Used to be it got you thrown in jail or at least a visit by the FBI. Says more about the tech bro dipshits than the tech, if you ask me.
A human brain has qualities that make it very different from AI, but the same as other living creatures: it's alive, it can feel (sensations, pains, pleasures, emotions) that give it rich experiences and memories, and it hardwired to continue existing and protecting the body it is inside. So: motivation, desires, fears, and continued consciousness and thought without any external person having to poke or prod it into "doing".
None of the models I've downloaded have sprung to life spontaneously, or done anything remotely unpredictable. None of them can act without input. They all have strictly limited ways to receive input and pass output. They aren't alive, can't feel, don't have experiences. The fact that they are a phenomenon that emerge from billions of snippets of our language lets them express languages in extremely convincing ways, and they are extremely impressive in what they can do. But this isn't Short Circuit or D.A.R.Y.L. or the Terminator, none of them have feelings. None of them continue to think after the program exits.
Without those characteristics they can only generate based on what they've ingested. The capabilities are still astonishing, but they're still dead things, and nobody will ever risk their life to protect the model "context" from getting "killed"
I don't feel like the models are becoming any better at writing.
Rap lyrics are a good test. They are on the level of a censored version of Insane Clown Posse. Just terrible.
The goal post seems to be changing weekly:
It can’t even play a basic video game!
Now it can
It can never beat the best human go player!
Now it can
It can’t even make a video of will smith eating food!
Now it can
It can’t do basic math!
Now it can
It can’t code as well as a junior?
Not it can
It can’t pass the bar or other certificate exams!
Now it can
It’s never going to meet senior level programmers!
Now Linus says it’s better at coding than him
It can’t do high level maths!
Now it can
It can’t solve math problems humans cant!
Now it can
It can’t do RSI
…
It can’t pass the Turing test!
That old thing?
The ability to solve general problems previously insoluble using only experiential data and modification to internal world models.
So, like ARC-AGI-3? I'm sympathetic to the notion that the things we're able to measure are necessarily going to miss important aspects of capability and intelligence. But people are attempting to measure this kind of thing, and models keep getting better at it.
And yet, curiously, they fall flat on their silicone asses until they have a copy of the test to "train on".
They've read a few thousand years of experiential data in text form.
politics
maybe if AI was progressing linearly we wouldn't be close, but if it's progressing exponentially then all bets are off
Until an LLM can sustain a conversation with me for more than 20 to 30 minutes without becoming incoherent I’m just not even entertaining any idea that we have reached AGI.
> They are simply looking to steer AI policy discussions. It's basically an in-house think tank.
For a guy who spent his life imagining and realising dreams in video games and cognitive abilities in artificial neural networks, it has to be very soul destroying for Demis to spend his time doing something as frivolous as this.
I don't know, Hassabis always seems to have been willing and able to do PR and put himself out there, in addition to having the technical muscle.
Didn't he get kicked out of the company he founded? Deep Mind does not have a CEO anymore
So it seems, but what of it? That wasn't his first career setback, and I'm sure he's still trying to get interesting work done at Isomorphic and maybe elsewhere in addition to working the PR pump for Alphabet.
> Bold thinking on AGI from the lab that pioneered the field.
The best part of technology: the people who brought you the "disruption" are the ones who promise you they have the solution to all the problems it caused.
I wonder if they could make the font even greyer on that website. MORE GREY because I am still able to read it (bearly).
Similarly there is Goldman Sachs Global Institute: https://www.goldmansachs.com/what-we-do/goldman-sachs-global...
Hassabis is so worried about the impact of AI on humanity that he spun out a website in three minutes using Gemini At least he could have done a better job with Claude, IMHO...
Here's my opinion on the "pacing the frontier" issue. Recursive self improvement can accelerate AI advances a lot. Whoever nails it first, has a huge first mover's advantage. But OpenAI is taking a "move fast and break things" attitude. This is making life very difficult for Anthropic: how can you retain your edge when you keep measuring twice to cut once, but your competitor keeps blowing things up left and right?
Google, who was left a little bit behind anyway, is more than happy to join the calls for taking a more cautious approach to AI development. Hence DeepMind Institute.
Sam Altman has no option than to pretend he agrees with all these ideas. What else is he supposed to say?
Crazy that these labs keep throwing around entirely unsubstantiated claims like "AGI is right around the corner". Given the enormity of the problem I'd argue we're not really any closer than a decade ago, especially given LLM shortcomings.
I felt the same when reading
wow i thought the attributions were for the vector graphic things and was like "ok demis is in his illustrator bag too!"
so glad they spun out a non-profit!
Tl;DR: It's not a new organization, or non-profit institute. It's a Substack/blog.
> DMI is a platform for researchers and thinkers from across Google DeepMind, Google, and the wider global research community to work on and publish creative, deeply informed ideas about a world with AGI. They will not always agree, and they will likely change their minds, as more data and information comes to light at the fast-moving frontier. We created this institute precisely because broad-based intellectual discussion and debate are required to arrive at a consensus about how to address the challenges and opportunities we face as a society.
Nonetheless, there are great articles up on there, such as:
https://institute.deepmind.com/essays/the-case-for-reasoning...
(Hiding chains of thought is one of the most harmful things I think OAI/Anthr. has done, altough Google Gemini also encrypts its chains of thought, so it's interesting to see the DeepMind Institute crticising this.)
My read is that they're criticizing moving the reasoning (more) into the latent space, such that not even researchers/safety-evaluators can actually see what's going on.
Of course this still then has a 'trust me, bro' feel to it, but making it outright impossible to reasonably inspect the CoT is a significantly larger and riskier step, IMO.
At least there are technical reasons to argue in favour of moving some CoT into latent space.
Encrypting CoT is just plain nuts yet we’ve all accepted it as normal.
The word "institute" gives an aura of credibility without some boring accreditation. Though not sure every country allows that.
How can they even pretend to distance themselves from Google…
So, a think tank, to lobby governments
> We created this institute precisely because broad-based intellectual discussion and debate are required to arrive at a consensus about how to address the challenges and opportunities we face as a society.
Something tells me that consensus will never arrive to the conclusion that we should shut down that whole industry
Google citing Mark Fisher was not on my Postmodern-hell-world bingo card.
Woah. Where?
On the "Principles for a New Utopia" page, the first line is:
>"Someone once said that it is easier to imagine the end of the world than the end of capitalism."
They cite Marxist philosopher Fredric Jameson (and misspell his name, incidentally), but I've always attributed this phrase to Fisher. I guess citing kpunk would have probably made the simulation we all live in actually implode.
Just a new shop front. Nothing to see here.