Epistemic status: Intuition pump/map for navigating ethics
Introduction
In this essay, I argue that John Rawls’ veil of ignorance is not merely a philosophical thought experiment, but a useful and mostly accurate description of reality when viewed through certain theories of personal identity. This framing leads to sentientism, that all sentient beings are morally relevant . This helps clarify the is/ought problem, and has implications for how we should act both individually and as a species.
The Veil of Ignorance and Its Traditional Use
John Rawls’s “original position,” part of his social contract theory of justice, introduces the thought experiment of the “veil of ignorance.” Behind this veil, individuals are tasked with designing a just society without knowing their own future place within it. They are aware of general facts about human society, but critically, they are ignorant of their personal attributes, social status, or even their specific talents and disadvantages. This means they could emerge as the most privileged or the most disadvantaged, or anywhere in between. This epistemic limitation, Rawls argues, compels individuals to choose principles of justice that are fair to all, as they would not want to risk being subjected to an unjust system themselves. From this perspective, Rawls posits two core principles: equal basic liberties for all, and that any social and economic inequalities must benefit the least advantaged members of society.
While the veil of ignorance offers a powerful framework for conceptualizing justice within human societies, its true potential, I argue, emerges when we extend its scope. Broadening the range of potential identities to encompass all sentient beings, and by further integrating the complexities of personal identity, I believe, offers a more robust and inclusive approach to ethical and societal design.
Sentience and the Scope of Moral Relevance
By ‘sentience,’ I mean the capacity to experience positive and negative subjective states, particularly suffering and pleasure (i.e. valence). This includes human and non-human animals, and potentially future artificial minds or aliens. I’m using consciousness interchangeably with sentience to generally point to the experience of experiences and feelings, especially valenced ones (like pleasure and pain).
Personal Identity and the Ontology of Consciousness
Viewing it from the three main categories of personal identity I’m aware of, Closed, Empty, and Open Individualism, I think the original position accurately describes an aspect of reality for Closed and Empty Individualism, and that there are large ethical implications of it. Open Individualism just has massive ethical implications in it’s own right and in a sense my argument is that even Closed and Empty Individualists end up at a similar ethical place.
I’ll be summarizing the forms of Individualism as described in Ontological Qualia: The Future of Personal Identity, and it has this warning which may apply to the rest of this essay as well: “WARNING If you are not psychologically robust, this may be a memetic hazard. It talks about ideas that may affect hedonic tone in people susceptible to bad philosophical experiences.” The diagrams are also from the same blog post.
Closed Individualism
Closed Individualism is the common, standard, intuitive-to-most-people view of identity. “You begin to exist when you are born and you stop existing when you die.” This can be modified slightly with beliefs such as souls and an afterlife, but generally speaking “The main conditions for a view to be classified as Closed Individualism is that (1) there is at most one instance of you at any given point in time, and (2) you continue to exist moment after moment.”
If you use the lens of Closed Individualism, before you are born, you could be born as any possible sentient being. In other words, the set of all conscious beings born in a given moment is in a sense all the beings exiting the veil of ignorance. Behind the veil we’d rather be born into the best life possible, and certain lives filled with suffering would be especially undesirable, but we wouldn’t know which one we’d get—we could be born as any species or entity that’s sentient, which means that from behind the veil, we’d want those outside the veil to reduce our odds of being born as a suffering sentient being before it happens.
Empty Individualism
Empty Individualism is the view that “you” are a time slice of consciousness. Although it feels like you’re a continuous being as described in Closed Individualism, under Empty Individualism that’s an illusion caused by memory being part of the information within each slice of consciousness. So, at least in “normal” states of consciousness, each time slice feels like it’s the same being as the one that formed the memories that give rise to it’s own experience. The moment of consciousness includes awareness of things experienced by previous moments of consciousness.
From the lens of Empty Individualism, the time-slice of consciousness “you” find “yourself” in could be the slice of any sentient being. All time-slices of consciousness at a given moment are essentially exiting the veil of ignorance. Again, before you find out which time-slice you experience, you’d want all the time-slices to coordinate to make sure your slice is not one of suffering but ideally is one of pleasure instead. Some time slices choose to sacrifice their own well-being in the short run for the well-being of other slices in the long run, which may be admirable, but ideally we’d find ways to do both—improve the experience of other time slices without creating suffering in our own.
Open Individualism
Open Individualism is the view that all consciousness is a part of one larger “whole”, so although there may be many separations and distinctions between different sentient beings, at the root of it all, they are all one. One way of looking at this is to consider that there may be parts of your subconscious that are actually conscious, but you don’t have access to those experiences any more than you have access to the experiences of “another” person. Open Individualism would suggest that it’s the same thing for the whole universe.
From the lens of Open Individualism, you already are every conscious being. Ethical concern for others becomes indistinguishable from self-interest. This doesn’t fit as cleanly into a veil of ignorance analogy, but I think if you hold this view it should be even more clear that reducing suffering is very important, even (in fact, especially) “selfishly” speaking.
How the Veil of Ignorance Describes Reality under Closed and Empty Individualism
Both Closed and Empty Individualism posit that consciousness comes into being at some point, either when they are born or at the beginning of a time-slice of consciousness. As soon as a consciousness begins to exist it can be viewed as someone leaving the veil of ignorance and finding out what kind of experience they’ll have.
From behind the veil of ignorance, we’d want the people in society to be making sure we don’t enter into bad experiences when we leave the veil of ignorance. Now we are the ones outside the veil of ignorance and can choose to either uphold and enact the values we would have endorsed while behind the veil, or ignore them and act selfishly. Of course, you could have a position behind the veil of “no matter who I’m born as, I will put my own subjective experience above all others”, but I think many people being honest with themselves would have pretty high-minded ideals from behind the true veil of ignorance—the one that includes all sentient beings—while facing the odds that their experience is full of suffering. Now our task is to live up to those as best we can.
Sentientism and the Is/Ought Problem
This view, taking into account the veil of ignorance, the ontology of personal identity, and the resulting sentientist perspective allows us to resolve the is/ought problem in the same way that it’s resolved when you touch a hot stove—what “is” is the experience of pain and heat, and the “ought” is removing your hand. Sentientism takes this logic and generalizes: any conscious experience of suffering gives rise to a moral imperative to alleviate it.
Of course, taking into account the entire moral landscape makes things a lot trickier. We are still individuals without direct access to the experiences of others (and there are a lot of relevant “others”), so as usual, good epistemology (“evidence and reason”) is vital to guiding our actions. Thus, the tagline for sentientism: “Evidence, reason and compassion for all sentient beings”. Evidence and reason enables us to succeed at our goals without making things worse, and either the veil of ignorance or Open Individualism gets us compassion for all sentient beings.
This is not a literal description of reality
To be clear, I’m not saying that there are literally conscious entities (what people might call “souls”) hanging around behind some literal cosmic boundary that separates them from our world, which then get transferred from there and connected to the brains of sentient beings here. I think conscious entities get created when their brain/nervous system starts generating consciousness, and that this is analogous to leaving the veil of ignorance, in such a way that it makes the veil of ignorance perspective a useful map of how we should do ethics.
Conclusion
In this essay, I argued that Rawls’ veil of ignorance is not merely a philosophical thought experiment, but a useful and mostly accurate description of reality when viewed through certain theories of personal identity. This framing leads to sentientism, “evidence, reason, and compassion for all sentient beings”. This helps clarify the is/ought problem, and has implications for how we should act both individually and collectively. I think it provides a selfish-ish backing for caring about the suffering of others, if pondering The Horror Of Unfathomable Pain doesn’t do it for you.
To me, the ideas I’ve put forward here seem almost tautological, but it took me a long time to understand things in this way, and I see barely anyone talking about the veil of ignorance. I’m planning to start writing about the importance of AI moral alignment and why we need AIs to be sentiententists rather than only humanists, but first I think it’s important to establish sentientism as more than just another ungrounded philosophy. I hope you found it useful, enjoyable, and interesting! Also, any notes on my writing are welcome, I’m just getting into writing like this :)
It would be good if, in a post like this, you acknowledged the very serious criticisms of Rawls’s “veil of ignorance” idea. Of course, that would seriously undermine the thesis of this post as well…
I’m curious – what are the “very serious criticisms” you refer to? Your comment would more helpful and productive if you pointed to specific disagreements you have with the post rather than abstractly mentioning the existence of them.
More importantly, what are these criticisms directed at? The stated thesis of this post is that the veil of ignorance is a “useful and mostly accurate description of reality when viewed through certain theories of personal identity.”
This list of notable criticisms of the Veil of Ignorance doesn’t seem to include any disagreement with this claim in particular. In fact, the last criticism mentioned seems to support it – albeit with different reasoning.
Likewise, the first LessWrong post that comes up when searching for “Veil of Ignorance” argues that the thought experiment doesn’t necessarily support Rawl’s ideal society – not that it doesn’t accurately describe reality.
To go through the ones listed in wikipedia:
This is criticizing Rawls’s proposed next steps after he saw the map laid out by the veil. I’m just pointing to the map and saying “this is a helpful tool for planning next steps, which will probably be different than steps proposed by Rawls”. I’d point out that this criticism would hold up better if everyone started with equal claim to resources, but that’s an entirely separate conversation.
Well yeah of course it’s impossible to do it perfectly. It’s impossible for any of us to be ideal-reasoning agents, I guess rationalism is doomed. Sorry guys, pack up and go home.
Makes sense, “evidence and reason” is critical to planning specific next steps even if you have a high level map.
Sure. Again, I’m not arguing for specific interpretations of the map, just saying it’s there and it’s helpful even if you don’t come to the same conclusions as others looking at a similar one. The help principle seems reasonable, as do other strategies like giving 10% of your income rather than selling all you have to give to the poor.
😏😏😏
Seconding @neo’s question, which criticisms are you referring to? I’ll grant you that I didn’t look for criticisms of it beyond searching for it on lesswrong and looking at the wikipedia page, but ultimately, my thesis is based on the general thrust of the veil of ignorance/original position and not the specifics of Rawls’s work.
Yeah… the problem in this case, as in many other such, is that the criticisms listed on Wikipedia are pretty weak. To find real, substantive criticism of a philosophical argument or framework, you need to go much further. The relevant SEP entry should be considered one starting point (but by no means the last instance).
In this case, though, the most serious criticism—the one which is absolutely fatal to the entire argument—is also the one which might be the most obvious. It’s basically what @jchan outlines in his sibling comment, but I’ll go ahead and offer my own formulation of it, why not:
The point of a moral/ethical framework of any sort—the point of ethics, generally—is to provide you with an answer to the question “what is the right thing for me to do”.
The “veil of ignorance” argument asks us to imagine that we are disembodied, impersonal spirits, existing in a void, having no identity, no desires, no interests, no personality, and no history (who are, nevertheless, somehow having thoughts, etc.).
The problem is that this is not a thing that ever happens. We aren’t disembodied, impersonal spirits, existing in a void, having no identity, no desires, no interests, no personality, and no history. Not only does this never happen, but in fact it won’t ever happen; and it can’t ever happen; and the entire description of this scenario is completely incoherent.
We exist as physical beings in a specific place, time, social and historical context, material conditions, etc. (And how could it be otherwise?) Our thoughts (beliefs, desires, preferences, personality, etc.) are the products of physical processes. We are the products of physical processes. That includes our preferences, and our moral intuitions, and our beliefs about morality. These things don’t come from nowhere! They are the products of our specific neural makeup, which itself is the product of specific evolutionary circumstances, specific cultural circumstances, etc.
“Imagine that you are a disembodied, impersonal spirit, existing in a void, having no identity, no desires, no interests, no personality, and no history (but you can think somehow)” is basically gibberish. It’s a well-formed sentence and it seems to be saying something, but if you actually try to imagine this scenario, and follow the implications of what you’ve just been told, you run directly into several brick walls simultaneously. The whole thought experiment, and the argument the follows from it, is just the direst nonsense.
So ethics can’t have anything to say about what you should do if you find yourself in this hypothetical situation of being behind the veil of ignorance. You won’t and can’t find yourself there, because even the notion of “you” “finding yourself” “behind the veil of ignorance” is completely incoherent.
Now, you say:
But as @jchan notes, you don’t follow the implications of this admission. Instead you make a swerve:
But this is a non sequitur. Of course it’s not “analogous to leaving a veil of ignorance”. There isn’t any pre-existing thinking entity which gets embodied. There is just an entirely de novo generation, with absolutely nothing preceding it, in that mind’s history. Behind the veil of ignorance is nothing.
Another way to put it is that you are asking us (by extending what Rawls is asking us) to perform a mental operation that is something like “imagine that you could have been a chicken instead of a human”. When you ask a question like this, who are you talking to? It is obviously impossible for me—Said Achmiz, the specific person that I am, right now—to have turned out to be a chicken (or, indeed, anyone other than who I am). So you can’t be talking to me (Said Achmiz).
The only entity that “could have turned out to be” something else is the hypothetical disembodied spirit that hypothetically existed in a hypothetical void behind the veil of ignorance, which was then instantiated into the human who became Said Achmiz, but could just as easily have been instantiated into a chicken, or into whatever or whoever else. But: (a) there ain’t no such entity; and (b) even if there were, that entity is not here right now, and you can’t talk to it. So your argument is addressed at beings who don’t exist, can’t exist, and couldn’t converse with you even if they could and did exist.
Then, of course, there’s the problem of cultural dependence of moral intuitions. Suppose that you ask me how society should be arranged, from behind the veil of ignorance, and I (taking your question seriously) answer that, obviously, the master should command the slave, and the slave should obey the master; the husband should command the wife, and the wife should obey the husband; the ruler should rule his subjects justly, and the subjects should venerate and obey the ruler; and that whatever position in life I were to find myself occupying, I should behave in accordance with that position, because that is the right ordering of the universe.
Now, the obvious question to ask is whether there’s anything you can do to convince me that the reasoning from behind the veil of ignorance should proceed as you say, and not as I (in this hypothetical scenario) say; or, is there nothing more you can say to me? And if the latter—what, if anything, does this prove about the original position argument? (At the very least, it would seem to highlight the fact that Rawls’s reasoning is shaky even granting the coherence of his hypothetical!)
But the trickier and more interesting question is this:
How exactly do you know that the disembodied spirits will have the moral intuitions that you have, and not the moral intuitions that hypothetical “right ordering of the universe” guy has?
It sure seems awfully convenient that when you posit these totally impersonal disembodied spirits, they turn out to have the moral beliefs of modern Westerners. Why should that be the case? Our own moral intuitions, again, don’t come from nowhere. What if we find ourselves behind the veil of ignorance, and all the disembodied spirits are like, “yeah, the slave should obey the master, etc.”?
(Of course this is silly, but why is it silly? Why is it any sillier than the disembodied spirits having our moral beliefs?)
And what about aliens?
As Eliezer comments in his writings about metaethics, human morality is just that—human. It’s the product of our evolutionary history, and it’s instantiated in our neural makeup. It doesn’t exist outside of human brains.
So what about the Pebblesorters? Do they get disembodied spirits in the empty void, too? Do they draw from the same “pool” of disembodied spirits? How do you construct a scenario where a disembodied spirit might get instantiated in a human, or a chicken, or a Pebblesorter—given the fact that the Pebblesorters are never going to agree with humans about what the disembodied spirits should decide from behind the veil of ignorance (they will claim that the disembodied spirits should vote on such an arrangement of the universe that maximizes the right sort of heaps of pebbles, not anything about “fairness” or “justice” etc.—those are human concepts!).
So do you have a scenario where the disembodied spirits care about fairness and justice, but then once they get instantiated into material bodies, they might find themselves still caring about fairness and justice (if their instantiation turns out to be human… or maybe not—see the “cultural dependence” commentary above), or suddenly caring about heaps of pebbles instead (if their instantiation turns out to be a Pebblesorter), or not having any thoughts whatsoever about any of this (if their instantiation turns out to be a chicken)?
Or do Pebblesorters get a requisite number of disembodied spirits who care about heaps of pebbles? If so, do they nevertheless get to vote, along with the disembodied spirits “allocated” to humans (and chickens)? What if there are more of them?
This is getting increasingly absurd now, but these are the questions that arise straightforwardly out of the “original position” / “veil of ignorance” scenario! The only thing that makes these questions vaguely absurd is that… the scenario, and the argument, are incoherent nonsense to begin with.
The bottom line is that Rawls’s argument is an intuition pump, in Dennett’s derogatory sense of the term. It is designed to lead you to a conclusion, while obscuring the implications of the argument, and discouraging you from thinking about the details. Once looked at, those implications and details show clearly: the argument simply does not work.
The way I interpret it: in the thought experiment, you are not literally imagining yourself as a mind without properties, and then asking “what would I want?” You are imagining that you can become any of the sentient minds entering existence at any given moment, and that you will inherit their specific circumstances and subjective preferences.
The goal of the thought experiment, then, is to construct a desirable world that encompasses the specific desires and preferences of all sentient beings.[1] This seems obviously relevant to the alignment problem.
With no constraints, your answer might look like “give every sentient being a personal utopia.” Our individual ability to change the real world is heavily constrained, though, so some realistic takeaways can be:
Oppose factory farming, because it affects trillions of sentient beings in a very negative way.
Try to stop AI from killing everyone, because a lot of sentient beings could exist in the future, and they could live very desirable lives. Also, a lot of the ones that currently exist would prefer to continue living so that their lives can become more desirable with technological progress.
This would not be a faithful execution of the thought experiment, as you would obviously be ignoring the possibility of existing as a slave (which, presumably, does not prefer to be enslaved).
In a sense, the veil of ignorance is an exercise in rationality – you are recognizing that your mind (and every other mind) is shaped by its specific circumstances, so your personal conception of what’s moral and good doesn’t necessarily extend to all other sentient beings. If I’m not mistaken, this seems to be in line with Eliezers position on meta-ethics.
I agree that the notion of “you” existing before your entering into existence seems incoherent – I prefer to describe it as “your mind emerged from physical matter, and developed all of its properties based on that matter, so when you develop a moral/ethical framework that includes any minds outside of your own, it should logically extend to all minds that exist/will ever exist.” In other words, caring about anyone else means you should probably care about all sentient beings.
The thought experiment is supposed to be somewhat practical, so I don’t think you need to consider aliens, or the entire set of all possible minds that can ever exist.
But of course I can’t do that either. Nor can you, nor can anyone else. And this is just as nonsensical, in multiple ways, as the “disembodied spirits” scenario; for example, if “you” “become” a different sentient mind that is entering existence at some moment, then in what sense are “you” still the “you” that did the “becoming”? This scenario is just as incoherent as the other one!
No, the goal of the thought experiment is to argue that you should want to do this. If you start out already wanting to do this, then the thought experiment is redundant and unmotivated.
It is a strange “somewhat practical” thought experiment, that posits a scenario that is so thoroughly impossible and incoherent!
These suggestions, of course, assume that the thought experiment has already persuaded you… which, again, would make it redundant.
Uh… I think you have not understood my point in offering that hypothetical. I will ask that you please reread what I wrote. (If you still don’t see what I was saying after rereading, I will attempt to re-explain.)
But the Rawlsian argument is not a metaethical argument. It is an object-level ethical argument.
You’re quite right: Eliezer’s position on metaethics (with which I wholly agree) is that my personal conception (and, broadly construed, the human conception) of what’s moral and good not only does not necessarily extend, but definitely does not extend, to all other sapient[1] beings. Another way of putting it (and this is closer to Eliezer’s own formulation) is that “morality” synonymous with “human morality”. There isn’t any such thing as “alien morality”; what the aliens have is something else, not morality.
But, again, that is Eliezer’s position on metaethics—not on ethics. His position on ethics is just various object-level claims like “more life is better than less life” and “you shouldn’t lie to people to coerce them into doing what you want” and “stealing candy from babies is wrong” and so on.
So, yes, as a point of metaethics, we recognize that aliens won’t share our morality, etc. But this has zero effect on our ethics. It’s simply irrelevant to ethical questions—a non sequitur.
None of this has any resemblance to Rawlsian reasoning, which, again, is an ethical argument, not a metaethical one.
“Logically”? Do please elaborate! What exactly is the logical chain of reasoning here? As far as I can tell, this is a logical leap, not a valid inference.
But… why?
Is there something incoherent about caring about only some people/things/entities and not others? Surely there isn’t. I seem to be able to care about only a subset of existing entities. Nothing appears to prevent me from doing so. What exactly does it mean that I “should” care about things that I do not, in fact, care about?
To say this of merely sentient beings—in the sense of “beings who can experience sensations”—is also true, but trivial; out of all sentient beings known to us, humans are the only ones that are sapient (a.k.a. self-aware); cows, chickens, chimpanzees, etc., are not capable of having any “conception of what’s moral and good”, so the fact that they don’t have our conception of what’s moral and good, specifically, is wholly uninteresting.
To refine this discussion, I’ll only be responding to what I think are major points of disagreement.
No, it is not.
Imagining yourself as a “disembodied spirit,” or a mind with no properties, is completely incoherent.
To imagine being someone else is something people (including you, I assume) do all the time. Maybe it’s difficult to do, and certainly your imagination will never be perfectly accurate to their experience, but it is not incoherent.
I do see your point: you can’t become someone else because you would then no longer be you. What you are arguing is that through your view of personal identity (which, based on your comments, I presume is in line with Closed Individualism), the Veil of Ignorance is not an accurate description of reality. Sure, I don’t disagree there.
I think it would be helpful to first specify your preferred theory of identity, rather than dismissing the VOI as nonsense altogether. That way, if you think Closed Individualism is obviously true, you could have a productive conversation with someone who disagrees with you on that.
If you accept sentientism, I think it is still very useful to consider what changes to our current world would bring us closer to an ideal world under that framework. This is how I personally answer “what is the right thing for me to do?”
I don’t see why a meta-ethical theory cannot inform your ethics. Believing that my specific moral preferences do not extend to everyone else has certainly helped me answer “what is the right thing for me to do?”
I believe that humans (and superintelligences) should treat all sentient beings with compassion. If the Veil of Ignorance encourages people to consider the perspective of other beings and reflect on the specific circumstances that have cultivated their personal moral beliefs, I consider it useful and think that endorsing it is the right thing for me to do.
Of course, this should have no direct bearing on your beliefs, and you are perfectly free to do whatever maximizes your internal moral reward function.
I think we agree here.
What I meant was: caring about another sentient being experiencing pain/pleasure, primarily because you can imagine what they are experiencing and how un/desirable it is, indicates that you care about experienced positive and subjective states, and so this care applies to all beings capable of experiencing such states.
I generally accept Eliezers meta-ethical theory and so I don’t assume this necessarily applies to anybody else.
But the scenario isn’t about imagining anything, it’s…
… exactly.
I’ve avoided engaging with the whole “views of personal identity” stuff, because “empty individualism” and “open individualism” are so obviously absurd if taken anything like literally. People sometimes bring up these things in a way that seems like positing weird hypotheticals for convoluted rhetorical reasons, but if someone really believes either of these things, I generally assume that person either is bizarrely credulous when presented with superficially coherent-seeming arguments, or has done too much psychedelic drugs.
In any case it seems a moot point. OP is very clear that his argument is perfectly meaningful, and works, under “closed individualism”:
Now, as I’ve said, this is complete nonsense. But it’s definitely a claim that was made in the OP’s post, not some additional assumption that I am making. So as far as I can tell, there is no need, for the purposes of this discussion that we’ve been having, to talk about which “view of personal identity” is correct.
Whether to accept “sentientism” is the thing that you (or the OP, at least) are supposedly arguing for in the first place… if you already accept the thing being argued for, great, but then any reasoning that takes that as a baseline is obviously irrelevant to criticisms of the argument for accepting that thing!
I didn’t make any such claim. I was talking about this specific metaethical claim, and its effects (or lack thereof) on our ethical views.
Now, I do want to be sure that we’re clear here: if by “my specific moral preferences do not extend to everyone else” you are referring to other humans, then Eliezer’s metaethical view definitely does not agree with you on this!
Once again I would like to point out that this cannot possibly be relevant to any argument about whether Rawlsian reasoning is correct (or coherent) or not, since what you say here is based on already having accepted the conclusion of said reasoning.
In essence you are saying: “I believe X. If argument Y convinces people to believe X, then it’s good to convince people that argument Y is valid.”
This is a total non sequitur in response to the question “is argument Y valid?”.
In other words, your bottom line is already written, and you are treating the “veil of ignorance” argument as a tool for inducing agreement with your views. Now, even setting aside the fact that this is inherently manipulative behavior, the fact is that none of this can possibly help us figure out if the “veil of ignorance” argument is actually valid or not.
This would seem to be circular reasoning. What you’re saying here boils down to “if you care about any experienced pain/pleasure, then you care about any experienced pain/pleasure”. But your claims were:
Now, this might be true if the reason why your moral/ethical framework includes any minds outside of your own—the reason why you care about anyone else—is in fact that you “care about experienced positive and subjective [did you mean ‘negative’ here, by the way?] states”, in a generic and abstract way. But that need not be the case (and indeed is untrue for most people), and thus it need not be reason why you care about other people.
I care about my mother, for example. The reason why I care about my mother is not that I started with an abstract principle of caring about “experienced pain/pleasure” with no further specificity, and then reasoned that my mother experiences pain and pleasure, therefore my mother is a member of a class of entities that I care about, therefore I care about my mother. Of course not! Approximately 0.00% of all humans think like this. And yet many more than 0.00% of all humans care about other people—because most of them care about other people for reasons that look very different from this sort of “start with the most general possible ideas about valence of experience and then reason from that down to ‘don’t kick puppies’” stuff.
Thanks for such an in depth response! I’ll just jump right in. I haven’t deeply proofread this, so please take it with a grain of salt
I’m not trying to frame the veil of ignorance (VOI) as a moral or ethical framework that answers that question. I’m arguing for the VOI as a meta-meta-ethical framework, which grounds the meta-ethical framework of sentientism, which can ground many different object-level frameworks that answer “what is the right thing for me to do”, as long as those object-level frameworks consider all sentient beings as morally relevant.
100% agree with you here.
I agree that nobody can do literally that. I do think that doing your best at that will allow you to be a lot more impartial. Minor nitpick, the imagined disembodied spirit should have desires and interests in the thought experiment, at the very least, the desire to not experience suffering when they’re born.
I agree, in the post I even point out that from behind the veil you could endorse other positions for outside the veil, such as being personally selfish even at others expense. The point of the thought experiment is that thinking about it can help you refine your views on how you think you should act. The point is not to tell you what to do if you find yourself behind the veil of ignorance, which as you say is incoherent.
I’m not following how this rules it out from being an analogy. My understanding of analogies is that they don’t need to be exactly the same for the relevant similarity to help transfer the understanding.
Well, yeah, that is almost exactly what I’m doing! Except generalized to all sentient beings :) I don’t see why you would take so much issue with a question like that? There are many things we don’t (and likely can’t) know about chickens internal experiences, but there’s a lot of very important and useful ground that can be covered from asking that question because there is a lot we can know to a high degree of confidence. If I were asked that, I would look at our understanding of chicken neurology, and how they respond to different drugs like painkillers and pleasurable ones, and our understanding of evolutionary psychology and what kinds of mental patterns would lead to chickens behaving in the ways that they do. I could not give an exact answer, but if I was a chicken I’m almost certain I’d experience positive valence eating corn and fruit and bugs and negative valence if I got hit or I broke a bone or whatever, and that’s just what I’m highly confident on. With enough time and thought I’m sure I could discuss a wide range of experiences with a wide range of how confident I am at how I’d experience them as a chicken. Even though it would be impossible for me writing this to ever actually experience those things, it’s still easy to take my understanding of the world and apply it in a thought experiment.
This I resonate much more with. If someone would genuinely be happy with a coin flip deciding whether they’re master or slave, I don’t think there’s anything I could say to convince them against slavery.
I don’t think they ought to have the moral beliefs of modern westerners. I think I’m probably wrong or confused or misguided about a lot of moral questions, I think probably everyone is, modern westerner or not. The slave question is sillier on the assumption that they don’t want to be slaves, if they’re equally fine with being a slave or master it wouldn’t be very silly of them.
Once again, absolutely agree.
On the pebblesorters question, my interpretation of that story was that we humans do a mind numbing amount of things equally as silly as the pebblesorters. To take just one example, music is just arranging patterns of sound waves in the air in the “right” way, which is no more or less silly than the “right” pebble stacks. Behind the human/chicken/pebblesorter/etc veil, I would argue that all of us look extremely silly! Behind the veil likely wouldn’t care all that much about fairness/justice from behind the veil, beyond how it might impact valence.
I do in fact want to lead towards sentientism. Is it fair to use it in the derogatory way if I’m quite clear and explicit about that? I already described this whole post as an intuition pump before the introduction, I just think that transparent intuition pumps are not just fine, but they can be quite useful and good.
I think you’ve got your meta-levels mixed up. For one thing, there isn’t any such thing as “meta-meta-ethics”; there’s just metaethics, and anything “more meta” than that is still metaethics. For another thing, “sentientism” is definitely object-level ethics; metaethics is about how to reason about ethics—definitionally it cannot include any ethical principles, which “sentientism” clearly is. This really seems like an attempt to sneak in object-level claims by labeling them as “meta-level” considerations.
There is no such thing as “doing your best at” imagining an incoherent scenario. Or, to put it another way, to do your best at doing this is to not do it. That is the best. Any attempt to imagine a scenario which we have already established is incoherent is less than the best. To attempt this is nothing more than to confuse yourself.
This really is a very common sort of mistake, I find.[1] “Doing X is impossible, because X is not coherently defined in the first place.” “But what if we do our best to do X as well as we can?” If you say this, then you have not understood the point: we cannot “do our best at” something which cannot be done at all, because it isn’t “a thing” in the first place. You have words, but those words do not refer to anything.
But this, too, is nonsense. “They” (the disembodied spirits) will not experience anything at all when “they” are born—because “they” will cease to exist as soon as they’re embodied. There is no posited continuity of consciousness, experience, memory, or anything between the disembodied spirit and the incarnated being. (The whole thought experiment really is fractally nonsensical.)
Yes, that claim is what makes it an intuition pump—but as I said, it doesn’t work, because the hypothetical scenario in the thought experiment has no bearing on any situation we could ever encounter in the real world, has no resemblance to any situation we could ever encounter in the real world, etc.
But this isn’t just a case of “not exactly the same”. Nothing approximately like, or even remotely resembling, the hypothetical scenario, actually takes place.
Uh… I’m afraid that nothing in this paragraph is even slightly responsive to the part of my comment that you’re responding to. I’m honestly not sure how it’s even supposed to be, or how it could be. Basically all of those things seem like non sequiturs.
Like the rest of this paragraph, this is non-responsive to my comment, but I am curious: do you have a principled disagreement with all of the arguments for why nothing remotely like this is possible even in principle, or… are you not familiar with them? (Thomas Nagel’s being the most famous one, of course.)
This seems hard to square with the positions you take on all the stuff in your post…
I think there’s some very deep confusion here… are you familiar with Eliezer’s writing on metaethics? (I don’t know whether that would necessarily resolve any of the relevant confusions or disagreements here, but it’s the first thing that comes to mind as a jumping-off point for untangling this.)
One sees it in discussions of utilitarianism, for instance. “Interpersonal comparisons of decision-theoretic utility are incoherent as a concept.” “Can’t we sort of do our best to approximately compare utilities across agents, though?” No, we can’t, because there isn’t anything to approximate. There is no ground truth that we can approach with an estimate. The comparison does not mean anything.
On Yudkowsky, keep in mind he’s written at least two (that I know of) detailed fictions imagining and exploring impossible/incoherent scenarios—HPMOR, and planecrash. I’ve read the former and am partway through reading the latter. If someone says “imagining yourself in a world with magic can help tune your rationality skills,” you certainly could dismiss that by saying that’s an impossible situation so the best you can do is not imagine it, and maybe your rationality skills are already at a level where the exercise would not provide any value. But at least for me, prompts like that and the veil of ignorance are useful for sharpening my thinking on rationality and ethics, respectively
Er… what?
HPMOR is impossible, of course; it’s got outright magic, etc. No argument there. But “incoherent”? How so…?
(As for Planecrash, I do think it’s kind of incoherent in places, but only in a boring literary sense, not in the sense we’re discussing here. But I didn’t read it to the end, so mostly my response to that one is “no comment”.)
If someone says “conclusions about morality reached from considering scenarios in a world with magic hold in actual real-world morality, even if you don’t validate them with reasoning about non-impossible situations”, then I will definitely dismiss that and I will be right to do so. Again: ethics is our attempt to answer the question “what is the right thing for me to do”. Reasoning about situations which are fundamentally impossible (for very strong reasons of outright incoherence, not mere violations of physical laws) cannot constitute a part of that answer.
(Also, yes, I am highly skeptical of “imagining yourself in a world with magic can help tune your rationality skills”. Rationality skills are mostly tuned by doing things. Thinking about things that you’ve done, or things that you have concrete plans to do, or things that other people you know have done, etc., is also useful. Thinking about things that other people you don’t know have done is less useful but might still be useful. Thinking about things that nobody has done or will ever do is very low on the totem pole of “activities that can assist you in honing your rationality skills”.)
Ah I see, yes I did have them mixed up. Thanks for the correction.
On the incoherence of the thought experiment, @neo’s comment explains it pretty well I thought. I will say that I think the thought experiment still works with imaginary minds, like the pebblesorters. If the pebblesorters actually exist and are sentient, then they are morally relevant.
What? In the thought experiment and the real world, a great deal of beings are born into a world that gives rise to a variety of valenced experiences. In the thought experiment, you are tasked with determining whether you would be ok with being the one finding themself in any given one of those lives/experiences.
You said that it is impossible for you to have turned out to be a chicken, and so I can’t be talking to you if I say “imagine that you could have been a chicken instead of a human”. I demonstrated how to imagine that very thing, implying that I could indeed be talking to you when I ask that. I agree that it is impossible for you to turn into a chicken, or for you to have been born a chicken instead of you. I disagree that it is impossible to imagine and make educated guesses on the internal mental states of a chicken.
I’m not following, sorry. Can you give an example of a position I take in the post that’s inconsistent with what I said there?
Maybe? I’ve read the sequences twice, one of those times poring over ~5 posts at a time as part of a book club, but maybe his writing on metaethics isn’t in there. I think we are likely talking past each other, but I’m not sure exactly where the crux is. @neo described what I’m trying to get at pretty well, and I don’t know how to do better, so maybe that can highlight a new avenue of discussion? I do appreciate you taking the time to go into this with me though!
See my response to that comment.
“Works” how, exactly? For example, what are your actual answers to the specific questions I asked about that variant of the scenario?
In the real world, you only ever find yourself being the person who you turned out to be. There is never, ever, under any circumstances whatsoever, any choice you can make in this matter. You come into existence already being a specific person. There is nothing in reality which is even slightly analogous to there being any kind of reasoning entities that exist behind some sort of “veil of ignorance” prior to somehow becoming real.
Ok… it seems that you totally ignored the question that I asked, in favor of restating a summary of your argument. I guess I appreciate the summary, but it wasn’t actually necessary. The question was not rhetorical; I would like to see your answer to it.
I can, but this really seems like a tangent, since it concerns questions like “what beliefs would the disembodied spirits have and why”, which really seems like “how many angels can dance on the head of a pin”, given that the whole “disembodied spirits” concept is so thoroughly nonsensical in the first place. That part of your argument (and of Rawlsian reasoning generally) is an amusing incongruity, but on the whole it’s more of a distraction from the key points than anything.
The Metaethics Sequence (which contains a few posts that didn’t make it into R:AZ a.k.a. “The Sequences” as the term usually meant today) is what you’ll want to check out.
The thought experiment as I execute it requires me to construct a model of other minds, human or not, that is more detailed than what I would normally think about, and emotionally weight that understanding in order to get a deeper understanding of how important that is. To give an example, it’s possible for me to think about torture and be very decoupled with it and shrug and think “that sucks for the people getting tortured”, but if I think about it more carefully, and imagine my own mental state if I was about to be tortured, then the weight of how extremely fucked up it is becomes very crisp and clear.
Perhaps it was a mistake to use Rawl’s VOI if it also implies other things that I didn’t realize I was invoking, but the way I think of it, every sentient being is actually feeling the valence of everything they’re feeling, and from an impartial perspective the true weight of that is not different from ones own valenced experiences. And if you know that some beings experience extreme negative valence, one strategy to get a deeper understanding of how important that is, is to think about it as if you were going to experience that level of negative valence. No incoherent beings of perfect emptiness required, just the ability to model other minds based on limited evidence, imagine how you would personally react to states across the spectrum of valence, and the ability to scale that according to the distribution of sentient beings in the real world.
And this works on pebblesorters too, although it’s more difficult since we can’t build a concrete model of them beyond what’s given in the story + maybe some assumptions if their neurobiology is at all similar to ours. If an “incorrect” pebble stack gives them negative valence of around the same level that the sound of nails on a chalkboard does for me, then that gives me a rough idea of how important it is to them (in the fictional world). If pebblesorters existed and that was the amount of negative valence caused by an “incorrect” stack, I wouldn’t mess up their stacks any more than I go around scratching chalkboards at people (while wearing earplugs so it doesn’t bother me).
To go back to the master/slave example, if the master truly thought he was about to become a slave, and everything that entails, I’m not convinced he would stick to his guns on how it’s the right order of the universe. I’m sure some people would genuinely be fine with it, but I’m guessing if you actually had a mercenary trying to kidnap and enslave him, he’d start making excuses and trying to get out of it, in a similar way as the one claiming the invisible dragon in their garage will have justifications for why you can’t actually confirm it exists.
In other words, I’m trying to describe a way of making moral views pay rent about the acceptable levels of negative valence in the world. Neither my views, nor the thought experiment I thought I was talking about, depends on disembodied spirits.
I only see two questions in this line of conversation?
I’m not familiar with the specific arguments you’re referring to, but I don’t think it’s actually possible for disembodied minds to exist at all, in the first place. So no I don’t have principled disagreements for those arguments, I have tentative agreement with them.
(bold added to highlight your question, which I’m answering) When I ask a question like that, I’m talking to you (or whoever else I’m talking to at the time).
I’ll check it out! and yeah that’s where I read the sequences
The “this” in “all of the arguments for why nothing remotely like this is possible even in principle” was referring not to the “disembodied spirits” stuff, but rather to:
And I mentioned Nagel because of this essay (which was by no means the only argument for a position like Nagel’s, just the most famous one).
So it sounds like you’re not familiar with this part of the literature. If that’s so, then I think you’ll find it interesting to delve into it.
Can’t speak for Said Achmiz, but I guess for me the main stumbling block is the unreality of the hypothetical, which you acknowledge in the section “This is not a literal description of reality” but don’t go into further. How is it possible for me to imagine what “I” would want in a world where by construction “I” don’t exist? Created Already in Motion and No Universally Compelling Arguments are gesturing at a similar problem, that there is no “ideal mind of perfect emptiness” whose reasoning can be separated from its contingent properties. Now, I don’t go that far—I’ll grant at least that logic and mathematics are universally true even if some particular person doesn’t accept them. But the veil-of-ignorance scenario is specifically inquiring into subjectivity (preferences and values), and so it doesn’t seem coherent to do so while at the same time imagining a world without the contingent properties that constitute that subjectivity.
Are you able to imagine things you will want in the future? But assuming the universe isn’t just a big 4d-block, that version of you doesn’t exist, so wouldn’t imagining that be incoherent? Why wouldn’t the unreality of that be a stumbling block?
This is indeed neither a universally compelling argument nor is it possible to be an “ideal mind of perfect emptiness”. Think of this post as more along the lines of asking “if I was much more impartial and viewed all sentient beings as morally relevant, how would I want the world to look, what values would I have?”. Some people would answer “I don’t care about group X, if I was one of them I’d hope I get treated poorly like they do” and if they were being honest, this could not change their mind
Strong upvoted. Wrote roughly similar thing, but I think that this position leads to total hedonistic utilitarianism, and that the “retrospective probability of finding oneself as another point” in the block universe of empty individualism could even force a superintelligence to change the utility function to the simplest. I’m interested in your opinion on this matter, since you seem to share some of my premises and you don’t seem to be a total hedonistic utilitarian, so you can probably prove me wrong. I won’t make you read the whole post if you don’t want to, but in short—I think that you cannot strive for complex values, because the “joy of scientific discovery” or something among these lines will not be experienced by you-point and you could find yourself with different values, so it only makes sense to maximize pure pleasure.
Hmm good question. Coordinating with other time slices of your body is a very tough problem if you take empty individualism seriously (imo it is the closest to the truth of the three, but I’m not certain by any means). From the perspective of a given time slice, any experience besides the one they got is not going to be experienced by them, so why would they use their short time to get a spike in pleasure in a future time slice of the body they’re in, rather than a smaller but more stable increase in pleasure for any other time slice, same body or not? If the duration of a time slice is measured in seconds, even walking to the fridge to get a candy bar is essentially “altruism” for future time slices to enjoy it.
In terms of coordination for other goals, you can use current time slices to cultivate mental patterns in themselves that future ones are then more likely to practice such as equanimity, accepting “good-enough” experiences, recognizing that your future slices aren’t so different from others and using that as motivation for altruism, and even making bids with future time slices. If this time slice can’t rely on future ones to enact it’s goals, future ones can’t rely on even further future ones either, vastly limiting what’s possible (if no time slice is willing to get off the couch for the benefit of other slices, that being will stay on the couch until it’s unbearable not to). Check out George Ainslie’s Breakdown of Will for a similar discussion on coordinating between time slices like that https://www.semanticscholar.org/paper/Pr%C3%A9cis-of-Breakdown-of-Will-Ainslie/79af8cd50b5bd35e90769a23d9a231641400dce6
Other strategies that are way less helpful for aligning ai are more just making use of the fact that we evolved to not feel like time slices, probably because it makes it easier to coordinate between them. So there’s a lot of mental infrastructure already in place for the task
On the fear of different values, I think you need to figure out which values you actually care if future slices hold and make sure they are all well grounded and can be easily re-derived, and the ones that aren’t that important you just need to hold loosely and accept that your future selves might value something else and hope that their new values are well founded. That’s where cultivating mental patterns of strong epistemology comes in, you actually want your values to change for the better, but not for the worse
I’ve added your post to my reading list! So far it’s a pretty reliable way for me to get future time slices to read something :)