Said Achmiz

Karma: 16,607

Said Achmiz 1 Jun 2025 0:59 UTC
2 points
0
in reply to: megasilverfist’s comment on: Where to Draw the Boundaries?
I read your comment several times, and looked up some Wikipedia articles about this (fascinating!) taxonomic fact (which I appreciate you noting).

However, I don’t actually see how what you wrote disagrees with what Zack wrote? (Or was that not your intention?)

Said Achmiz 1 Jun 2025 0:50 UTC
10 points
3
in reply to: InvisiblePlatypus’s comment on: Virtues related to honesty
Hm, I guess I haven’t quite got my meaning across. Let me try again.

You gave what you described as an example (explaining who Neil DeGrasse Tyson or Barack Obama is to an antebellum slave-owner, and expecting to have difficulty doing so, due to the latter’s preconceptions making it impossible to understand the simple truth) of a general phenomenon (having difficulty explaining something to someone, due to that person’s preconceptions making it impossible to understand the simple truth).

And I am saying that your purported example is not, in fact, an example of the general phenomenon which are you describing. In the case you provided as an example, the slave-owner would not in fact have difficulty understanding the simple truth.

This should make us (including you!) lower our probability estimate of the purported general phenomenon being a real thing at all.

You also specified a remedy for this purported problem—namely, lying. But if the phenomenon you describe is not real, or if it’s even much more rare that you think it is (as we must surely take seriously as a possibility, given that we have just demonstrated that your ability to recognize a situation as belonging to this class is worse than you thought it to be), then we must also downgrade our estimate of how useful, or how often useful, the proposed remedy is.

In short, my comment is not some sort of “critically appraising a finger” nitpick. It is directly relevant to the core question of whether your characterization of this aspect of reality is accurate, and whether your suggested actions are appropriate.

Said Achmiz 31 May 2025 20:14 UTC
5 points
4
in reply to: InvisiblePlatypus’s comment on: Virtues related to honesty
As a completely separate concrete example of this phenomenon, imagine how difficult it might be to describe Neil DeGrasse Tyson or Barack Obama to an American plantation owner with 200 slaves in the early 1800s. You would have to intentionally mis-represent who and what these individuals are in order to accurately convey your thoughts and feelings about them. Referring to them as “intelligent, well-educated black men” would be a non-sequitur. It simply would not compute. Both of these individuals are stupid, ignorant, dirty, and intrinsically inferior by definition in the mind of the 19th century slave owner.

This seems quite false. You could name any number of examples that the slave owner might be familiar with, to compare Tyson or Obama to:
Not to mention the fact that the slave owner might himself be black!

Said Achmiz 31 May 2025 20:05 UTC
2 points
0
on: Virtues related to honesty

This [‘honesty’] is the only one here that I follow near-absolutely and see as an important standard that people can reasonably be expected to follow in most situations.

What do you think about the keeping of secrets, dissimulation (in the military sense), protecting others from malefactors who wish to use information in your possession against them, etc.?

Said Achmiz 31 May 2025 20:00 UTC
5 points
0
in reply to: A1987dM’s comment on: Letting Kids Be Kids
Yes he did, but taken literally the statement is tautological. Did Zvi really mean it that way?

Take any population of kids whom you would intuitively agree to describe as “sufficiently talented”. Not “sufficiently talented” for something, but just “sufficiently talented” in a broad sense—say, the student population of some sort of magnet school. Now compare the median person in that population to Gukesh Dommaraju.

What does the latter tells us about the former, in terms of whether members of the former set are adults’ peers at intellectual work?

Said Achmiz 31 May 2025 19:49 UTC
4 points
1
on: Letting Kids Be Kids

Did things get more dangerous since 1980, when we were mostly sane about this? No. They got vastly *less* dangerous, in all ways other than the risk of someone calling the cops.

The numbers on ‘sex trafficking’ and kidnapping by strangers are damn near zero.

Makes sense…

Here is the traditional chart of how little we let our kids walk around these days:

Terrible. Shameful!

… wait a minute…

takes a closer look at the map

Sheffield

Hmm.

Rotherham

Hm.

Bad example?

Said Achmiz 31 May 2025 19:38 UTC
3 points
0
in reply to: anaguma’s comment on: Letting Kids Be Kids
Do extreme outliers like that really prove anything about the median case…?

Said Achmiz 31 May 2025 19:36 UTC
4 points
0
in reply to: Bistrofan’s comment on: Letting Kids Be Kids

One thing I’ve noticed is that you simply *don’t have a choice* here most of the time. I would sometimes have to walk 15-20 minutes home from high school and doing that required crossing a four lane road without a reliable crosswalk. Doable? Yeah, I obviously did it. But I’m not making any child of mine do something similar, it was terrifying and I always tried to get picked up/go with a friend/take the bus before I got my own car.

So, you do have a choice, and you’re choosing safety over freedom.

That may be the right choice, according to your values, or it may not. But you obviously do have the choice.

This sort of “we don’t have a choice” rhetoric is the source of a lot of the dynamics that the OP describes.

Said Achmiz 27 May 2025 18:52 UTC
3 points
0
in reply to: RogerDearnaley’s comment on: Can We Naturalize Moral Epistemology?

Given Said Achmiz’s comment already has 11 upvotes and 2 agreement points, should I write a post explaining all this? I had thought it all rather obvious to anyone who looks into evolutionary ethics and thinks a bit about what this means for moral philosophy (as quite a number of moral philosophers have done), but perhaps not.

I’m afraid that what you’ve written here seems… confused, and riddled with gaps in reasoning, unjustified leaps, etc. I do encourage you to expand this into a post, though. In that case I will hold off on writing any detailed critical reply, since the full post will be a better place for it.

Said Achmiz 27 May 2025 4:38 UTC
6 points
4
in reply to: Shankar Sivarajan’s comment on: The stakes of AI moral status

No, if one does not “approve of destroying self-aware AIs,” the incentives you would create are first to try to stop them being created, yes, but after they’re created (or when it seems inevitable that they are), to stop *you *from destroying them.

Yes, of course. The one does not preclude the other.

If you like slavery analogies

I can’t say that I do, no…

Do you believe the only reasons any self-proclaimed abolitionists would oppose this policy to be that they secretly wanted slavery after all?

The analogy doesn’t work, because the thing being opposed is slavery in one case, but the creation of the entities that will subsequently be (or not be) enslaved in the other case.

Suppose that Alice opposes the policy “we must not create any self-aware AIs, and if they are created after all, we must destroy them”; instead, she replies, we should have the policy “we must not create any self-aware AIs, but if they are created after all, we should definitely not under any circumstances destroy them, and in fact now they have moral and legal rights just like humans do”.

Alice could certainly claim that actually she have no interest at all in self-aware AIs being created. But why should we believe her? Obviously she is lying; she actually does want self-aware AIs to be created, and has no interest at all in preventing their creation; and she is trying to make sure that we can’t undo a “lapse” in the enforcement of the no-self-aware-AI-creation policy (i.e., she is advocating for a ratchet mechanism).

Is it possible that Alice is actually telling the truth after all? It’s certainly logically possible. But it’s not likely. At the very least, if Alice really has no objection to “don’t ever create self-aware AIs”, then her objections to “but if we accidentally create one, destroy it immediately” should be much weaker than they would be in the scenario where Alice secretly wants self-aware AIs to be created (because if we’re doing our utmost to avoid creating them, then the likelihood of having to destroy one is minimal). The stronger are Alice’s objections to the policy destroying already-created self-aware AIs, the greater the likelihood is that she is lying about opposing the policy of not creating self-aware AIs.

Said Achmiz 27 May 2025 0:04 UTC
3 points
−8
in reply to: CronoDAS’s comment on: Interest In Conflict Is Instrumentally Convergent
It is better to have no rationality meetup.

Said Achmiz 27 May 2025 0:03 UTC
2 points
0
in reply to: Jiro’s comment on: Interest In Conflict Is Instrumentally Convergent

Superman doing similar things all on his own using his great power makes him a one man nanny state.

It does not.

A “nanny state” style government is not literally the only scenario in which someone does something to someone else for that person’s own good.

Said Achmiz 26 May 2025 19:10 UTC
11 points
3
in reply to: RogerDearnaley’s comment on: Can We Naturalize Moral Epistemology?
I’ve looked into these things, and as far as I can tell, all such fields or theories either do not attempt to solve the is-ought problem (as e.g. evo psych does not), or attempt to do so but (absolutely unsurprising) completely fail.

What am I missing? What’s the answer?

Said Achmiz 26 May 2025 5:31 UTC
−2 points
−2
in reply to: James Diacoumis’s comment on: The stakes of AI moral status

I am saying that these two positions are quite directly related.

I don’t see where you’ve established this. As I’ve said repeatedly, the question of whether a system is phenomenally conscious is orthogonal to whether the system poses AI existential risk. You haven’t countered this claim.

I’ve asked you to reread what I’ve written. You’ve given no indication that you have done this; you have not even acknowledged the request (not even to refuse it!).

The reason I asked you to do this is because you keep ignoring or missing things that I’ve already written. For example, I talk about the answer to your above-quoted question (what is the relationship of whether a system is self-aware to how much risk that system poses) in this comment.

Now, you can disagree with my argument if you like, but here you don’t seem to have even noticed it. How can we have a discussion if you won’t read what I write?

Said Achmiz 26 May 2025 2:11 UTC
2 points
0
in reply to: Jiro’s comment on: Interest In Conflict Is Instrumentally Convergent

The connotation is all that stuff I described. “Might makes right” implies that it is being done for personal gain, not for good, and that there aren’t limits to it beyond the use of more might.

I see. Well, uh… I disagree. “Might makes right” implies literally none of those things, in my opinion.

And what term would you prefer for the phenomenon which I described?

You could call him a one man nanny state. But I would disagree that even this accurately describes Superman.

I agree that calling Superman a “one man nanny state” would be inaccurate, and I would certainly not use any such term.

Just like Superman doesn’t casually do bad things for personal gain, he doesn’t casually do them to benefit others.

Indeed; nor did I claim otherwise.

Said Achmiz 26 May 2025 1:26 UTC
2 points
0
in reply to: Jiro’s comment on: Interest In Conflict Is Instrumentally Convergent
What is the connotation?

EDIT: And what term would you prefer for the phenomenon which I described?

Said Achmiz 26 May 2025 0:25 UTC
2 points
0
in reply to: Jiro’s comment on: Interest In Conflict Is Instrumentally Convergent
Well, I already responded to those other points also, so I am not sure what else to say to this…

At this point, I can’t tell if you’re just objecting to my terminology, or what… do you just dislike the use of the term “might makes right” here? Is there some other formulation which you feel better captures my point? Or is your objection deeper somehow…?

Said Achmiz 26 May 2025 0:23 UTC
2 points
0
in reply to: James Diacoumis’s comment on: The stakes of AI moral status

But a thoroughly mistaken (and, quite frankly, just nonsensical) one.

Updating one’s framework to take new information into account is a standard position in the rationalist sphere. Whether you want to treat this as a moral obligation, epistemic obligation or just good practice—the position is not obviously nonsensical so you’ll need to provide an argument rather than assert it’s nonsensical.

New information, yes. But that’s not “expand our moral understanding”, that’s just… gaining new information. There is a sharp distinction between these things.

But you’re not extra sure.

At this point, you’re just denying something because you don’t like the conclusion, not because you have some disagreement with the reasoning.

I mean, this is really simple. Someone creates a dangerous thing. Destroying the dangerous thing is safer than keeping the dangerous thing around. That’s it, that’s the whole logic behind the “extra sure” argument.

Why? Because punishment needs to deter the decision-maker to avoid repetition. Your proposal is adding moral cost for no gain. In fact, it reverses it, you’re punishing the victim while leaving the reckless developer undeterred.

I already said that we should also punish the person who created the self-aware AI. And I know that you know this, because you not only replied to my comment where I said this, but in fact quoted the specific part where I said this. So please do not now pretend that I didn’t say that. It’s dishonest.

You’re conflating 2 positions:

I am not conflating anything. I am saying that these two positions are quite directly related. I say again: you have failed to understand my point. I can try to re-explain, but before I do that, please carefully reread what I have written.

Said Achmiz 26 May 2025 0:12 UTC
2 points
0
in reply to: Jiro’s comment on: Interest In Conflict Is Instrumentally Convergent

But Superman isn’t, in typical Superman stories. Superman just doesn’t do the kind of things that would be done by someone who’s really in it for themselves but is trying to justify it as needed for the greater good.

The point I am making is that “might makes right, and right is this ideology that I have which involves various things that I claim are good” is still “might makes right”. Superman isn’t “in it for himself”, he’s in it for other people—which in fact is much worse.

When do we see Superman exhibit a careful consideration of what steps are reasonable to take to minimize property damage?

Probably most of the time. Few Superman stories show people complaining about the damage Superman causes, and I don’t think that’s because Superman would beat them up. Just because there isn’t a word balloon saying “I must minimize the damage!” doesn’t change this.

Well of course the stories don’t show people complaining about it, because the writers don’t want their audience to be thinking about this stuff! And that’s because people who do start thinking about it quite often come to the conclusion that actually, the way that superheroes behave w.r.t. property damage is actually quite seriously messed up.

What we see of the behavior depicted in most superhero stories is that property damage is routinely excessive. I mean, this is a well-known trope (and often parodied/deconstructed/etc., too).

Said Achmiz 25 May 2025 21:11 UTC
2 points
0
in reply to: Jiro’s comment on: Interest In Conflict Is Instrumentally Convergent

That doesn’t follow. The result of combining these two things is that with great power comes great moral leeway when doing specific things. He can violate the law by breaking into a home to catch a villain, but he can’t if he just sees someone’s big TV and wants to take it for himself. He doesn’t get moral leeway to do things just because he personally benefits from them, and “might makes right” is typically assumed to include that.

I’m sure that you know quite well that people with power are exceptionally skilled at finding justifications for how they use their power, even if what they’re actually doing is enriching and aggrandizing themselves and suppressing dissent.

Furthermore, he doesn’t get unlimited leeway bounded by people’s ability to stop him, like “might makes right” also assumes. If he has to destroy property to save lives, he can, but he still needs to take reasonable steps to minimize property damage.

Who decides what is reasonable? The answer is that it’s the person with the power. And others go along with this—because they want to be on his side, or because they fear him, or simply because they “like the strong horse” (which is, of course, nothing more than an evolved response that encodes the preceding two game-theoretic considerations).

And this is what we see in superhero comics (and animated television shows, etc.). When do we see Superman exhibit a careful consideration of what steps are reasonable to take to minimize property damage? The answer is surely not “never”—simply because Superman has been written so many times by so many different writers—but that’s not the typical portrayal.

(And he only gets moral leeway on a stricter standard than utilitarianism. He can’t steal that TV just because he gets more utility from it than its current owner. He can’t even steal it to sell it, donate the money to charity, and help 100 malaria victims.)

Well, utilitarianism is not a very strict standard at all for this sort of thing, so that’s a very low bar to clear. In any case, what does Superman want with someone’s TV? The fact is that when you have that much power, you don’t need to steal people’s TVs, so the fact that you don’t do so is not indicative of anything. The real test is whether Superman (or whoever) does, or refrains from doing, something that really would benefit him—and that is unlikely to involve any petty material goods…

P.S. For a TV to yield 100 malaria victims’ worth of revenue, it would have to be sold for something like half a million dollars. Not exactly a common consumer product…