Overtly, OP is trying to shun Cremieux for failing to meet an assumed-widely-agreed “standard of care and compassion”. This is obviously based on OP’s belief that Cremieux’s conduct is unacceptably outside the Overton window, even if they used the word “standard” instead of the word “Overton window”. The only point of deploying phrases like “Hoo boy” and “do better” is to appeal to a social consensus. OP isn’t being sneaky here or anything, you’re just misinterpreting their dialect.
David Hornbein
Karma: 1,320
Selection bias isn’t the whole story. The median paper in almost every field is notably worse than it was in, say, 1985. Academia is less selective than it used to be—in the U.S., there are more PhDs per capita, and the average IQ/test scores/whatever metric has dropped for every level of educational attainment.
Grab a journal that’s been around for a long time, read a few old papers and a few new papers at random, and you’ll notice the difference.
OKcupid is certainly a better product for hundreds of thousands, or possibly millions, of unusually literate people, including ~all potential developers and most people in their social circles. It’s not a small niche.
What does your company do, specifically? I found the brief description at HealthcareAgents.com vague and unclear. Can you walk me through an example case of what you do for a patient, or something?
Hm, this timing suggests the change could be a consequence of Karnofsky stepping away from the organization.
Which makes sense, now that I think about it. He’s by far the most politically strategic leader Open Philanthropy has had, so with him gone, it’s not shocking they might revert towards standard risk-averse optionality-maxxing foundation behavior.
“Open Phil higher-ups being friends with Anthropic higher-ups” is an understatement. An Open Philanthropy cofounder (Holden Karnofsky) is married to an Anthropic cofounder (Daniela Amodei). It’s a big deal!
I don’t recall Eliezer claiming that dath ilani characters never give in to threats. *Dath ilani characters* claim they never give in to threats. My interpretation is that the characters *say* “We don’t give in to threats”, and *believe* it, but it’s not *true*. Rather it’s something between a self-fulfilling prophecy, a noble lie-told-to-children, and an aspiration.
There are few threats in dath ilan, partly because the conceit of dath ilan is that it’s mostly composed of people who are cooperative-libertarianish by nature and don’t want to threaten each other very much, but partly because it’s a political structure where it’s much harder to get threats to actually *work*. One component of that political structure is how people are educated to defy threats by reflex, and to expect their own threats to fail, by learning am idealized system of game theory in which threats are always defied.
However, humans don’t actually follow ideal game theory when circumstances get sufficiently extreme, even dath ilani humans. Peranza can in fact be “shattered in Hell beyond all hope of repair” in the bad timeline, for all that she might rationally “decide not to break”. Similarly when the Head Keeper commits suicide to make a point: “So if anybody did deliberately destroy their own brain in attempt to increase their credibility—then obviously, the only sensible response would be to ignore that, so as not create hideous system incentives. Any sensible person would reason out that sensible response, expect it, and not try the true-suicide tactic.” But despite all that the government sets aside the obvious and sensible policy because, come on, the Head Keeper just blew up her own brain, stop fucking around and get serious. And the Head Keeper, who knows truths about psychology which the members of government do not, *accurately predicted they would respond that way*.
So dath ilani are educated to believe that giving in to threats is irrational, and to believe that people don’t give in to threats. This plus their legal system means that there are few threats, and the threats usually fail, so their belief is usually correct, and the average dath ilani never sees it falsified. Those who think carefully about the subject will realize that threats can sometimes work, in circumstances which are rare in dath ilan, but they’ll also realize that it’s antisocial to go around telling everyone about the limits of their threat-resistance and keep it quiet. The viewpoint characters start believing the dath ilani propaganda but update pretty quickly when removed from dath ilan. Keltham has little trouble understanding the Golarian equilibrium of force and threats once he gets oriented. Thellim presumably pays taxes off camera once she settles in to Earth.
>Are the near-term prospects of AGI making long-term prospects like suspension less attractive?
No. Everyone I know who was signed up for cryonics in 2014 is still signed up now. You’re hearing about it less because Yudkowsky is now doing other things with his time instead of promoting cryonics, and those discussions around here were a direct result of his efforts to constantly explain and remind people.
I agree with your argument here, especially your penultimate paragraph, but I’ll nitpick that framing your disagreements with Groves as him being “less of a value add” seems wrong. The value that Groves added was building the bomb, not setting diplomatic policy.
What is the mechanism, specifically, by which going slower will yield more “care”? What is the mechanism by which “care” will yield a better outcome? I see this model asserted pretty often, but no one ever spells out the details.
I’ve studied the history of technological development in some depth, and I haven’t seen anything to convince me that there’s a tradeoff between development speed on the one hand, and good outcomes on the other.
I’m coming to this late, but this seems weird. Do I understand correctly that many people were saying that Anthropic, the AI research company, had committed never to advance the state of the art of AI research, and they believed Anthropic would follow this commitment? That is just… really implausible.
This is the sort of commitment which very few individuals are psychologically capable of keeping, and which ~zero commercial organizations of more than three or four people are institutionally capable of keeping, assuming they actually do have the ability to advance the state of the art. I don’t know whether Anthropic leadership ever said they would do this, and if they said it then I don’t know whether they meant it earnestly. But even imagining they said it and meant it earnestly there is just no plausible world in which a company with hundreds of staff and billions of dollars of commercial investment would keep this commitment for very long. That is not the sort of thing you see from commercial research companies in hot fields.
If anyone here did believe that Anthropic would voluntarily refrain from advancing the state of the art in all cases, you might want to check if there are other things that people have told you about themselves, which you would really like to be true of them, but you have no evidence for other than their assertions, and would be very unusual if they were true.
Ben is working on a response, and given that I think it’s clearly the right call to wait a week or two until we have another round of counter-evidence before jumping to conclusions. If in a week or two people still think the section of “Avoidable, Unambiguous falsehoods” does indeed contain such things, then I think an analysis like this makes sense
This was three months ago. I have not seen the anticipated response. Setting aside the internal validity of your argument above, the promised counterevidence did not arrive in anything like a reasonable time.
TracingWoodgrains clearly made the right call in publishing, rather than waiting for you.
Yes, obviously, but they use different strategies. Male sociopaths rarely paint themselves as helpless victims because it is not an effective tactic for men. One does notice that, while the LW community is mostly male, ~every successful callout post against a LW community organization has been built on claims of harm to vulnerable female victims.
When you say “it’s clearly the right call to wait a week or two until we have another round of counter-evidence before jumping to conclusions”, is this a deliberate or accidental echo of the similar request from Nonlinear which you denied?
Like, on the deliberate way of reading this, the subtext is “While Lightcone did not wait a week or two for counter-evidence and still defends this decision, you should have waited in your case because that’s the standard you describe in your article.” Which would be a hell of a thing to say without explicitly acknowledging that you’re asking for different standards. (And would also misunderstand TracingWoodgrains’s actual standard, which is about the algorithm used and not how much clock time is elapsed, as described in their reply to your parent comment.) Or on the accidental way of reading this, the subtext is “I was oblivious to how being publicly accused of wrongdoing feels from the inside, and I request grace now that the shoe is on the other foot.” Either of these seems kind of incredible but I can’t easily think of another plausible way of reading this. I suppose your paragraph on wanting to take the time to make a comprehensive response (which I agree with) updates my guess towards “oblivious”.
On Pace’s original post I wrote:
“think about how bad you expect the information would be if I selected for the worst, credible info I could share”
Alright. Knowing nothing about Nonlinear or about Ben, but based on the rationalist milieu, then for an org that’s weird but basically fine I’d expect to see stuff like ex-employees alleging a nebulously “abusive” environment based on their own legitimately bad experiences and painting a gestalt picture that suggests unpleasant practices but without any smoking-gun allegations of really egregious concrete behavior (as distinct from very bad effects on the accusers); allegations of nepotism based on social connections between the org’s leadership and their funders or staff; accusations of shoddy or motivated research which require hours to evaluate; sources staying anonymous for fear of “retaliation” but without being able to point to any legible instances of retaliation or concrete threats to justify this; and/or thirdhand reports of lying or misdirection around complicated social situations.
[reads post]
This sure has a lot more allegations of very specific and egregious behavior than that, yeah.
Having looked at the evidence and documentation which Nonlinear provides, it seems like the smoking-gun allegations of really egregious concrete behavior are probably just false. I have edited my earlier comment accordingly.
This is a bit of a tangent, but is there a biological meaning to the term “longevity drug”? For a layman like me, my first guess is that it’d mean something like “A drug that mitigates the effects of aging and makes you live longer even if you don’t actively have a disease to treat.” But then I’d imagine that e.g. statins would be a “longevity drug” for middle-aged men with a strong family history of heart disease, in that it makes the relevant population less susceptible to an aging-related disease and thereby increases longevity, yet the posts talk about the prospect of creating the “first longevity drug” so clearly it’s being used in a way that doesn’t include statins. Is there a specific definition I’m ignorant of, or is it more of a loose marketing term for a particular subculture of researchers and funders, or what?
We can certainly debate whether liability ought to work this way. Personally I disagree, for reasons others have laid out here, but it’s fun to think through.
Still, it’s worth saying explicitly that as regards the motivating problem of AI governance, this is not currently how liability works. Any liability-based strategy for AI regulation must either work within the existing liability framework, or (much less practically) overhaul the liability framework as its first step.
Cars are net positive, and also cause lots of harm. Car companies are sometimes held liable for the harm caused by cars, e.g. if they fail to conform to legal safety standards or if they sell cars with defects. More frequently the liability falls on e.g. a negligent driver or is just ascribed to accident. The solution is not just “car companies should pay out for every harm that involves a car”, partly because the car companies also don’t capture all or even most of the benefits of cars, but mostly because that’s an absurd overreach which ignores people’s agency in using the products they purchase. Making cars (or ladders or knives or printing presses or...) “robust to misuse”, as you put it, is not the manufacturer’s job.
Liability for current AI systems could be a good idea, but it’d be much less sweeping than what you’re talking about here, and would depend a lot on setting safety standards which properly distinguish cases analogous to “Alice died when the car battery caught fire because of poor quality controls” from cases analogous to “Bob died when he got drunk and slammed into a tree at 70mph”.
It’s fun to come through and look for interesting threads to pull on. I skim past most stuff but there’s plenty of good and relevant writing to keep me coming back. Yeah sure it doesn’t do a super great job of living up to the grandiose ideals expressed in the Sequences but I don’t really mind, I don’t feel invested in ~the community~ that way so I’ll gladly take this site for what it is. This is a good discussion forum and I’m glad it’s here.
A good summary, but it’s worth noting that while the death penalty for failing to fight was on the books, Byng’s execution was the only time it was ever actually carried out. It’s a bit similar to how the US military legally has the authority to execute deserters, but in the past century has only ever exercised this once out of tens of thousand of sentences (Eddie Slovak during WWII).
From reading the autobiography of Lord Cochrane, an insanely aggressive and insanely successful captain during the Napoleonic Wars, my impression is that the Royal Navy was very concerned with the allocation of credit, glory, and prestige, as well as money prizes. He describes a system where officers are writing to the Admiralty headquarters to describe the meritorious or shameful actions of their peers and subordinates after every major action, and sometimes convoked courts martial to resolve disputes, as in the case of Lord Cochrane’s rival Lord Gambier. Lord Cochrane also argues that the prize system is critical to maintaining the Royal Navy’s morale and effectiveness.