One problem with the FAQ: The standard metaethics around here, at least EY’s metaethics, is not utilitarianism. Utilitarianism says maximize aggregate utility, with “aggregate” defined in some suitable way. EY’s metaethics says maximize your own utility (with the caveat that you only have partial information of your utility function), and that all humans have sufficiently similar utility functions.
You get something pretty similar to utilitarianism from that last condition (if everyone has the same utility function and you’re maximizing your own utility function, then you’re also maximizing aggregate utility in many senses of the term “aggregate”). But note that the interpersonal comparison of utility problem vanishes: you’re maximizing your own utility function. Maximization of the aggregate (under some methods of comparisons) is merely a consequence of this, nothing more. Also note that if we relax that last condition and let humans have differing utility function, there is no intrinsic problem for EY’s theory. If someone has a legitimate preference for killing people, the utilitarian has to take that into account as positive utility (or add some ad hoc assumptions about which preferences matter). On EY’s theory, sans that last condition, you only take into account someone’s preference for murder if your utility function tells you to. You may value other humans satisfying their preferences, but that doesn’t mean you have to value every single preference every single human has. You could, but it really just depends on what your utility function says.
One problem with the FAQ: The standard metaethics around here, at least EY’s metaethics, is not utilitarianism. Utilitarianism says maximize aggregate utility, with “aggregate” defined in some suitable way. EY’s metaethics says maximize your own utility (with the caveat that you only have partial information of your utility function), and that all humans have sufficiently similar utility functions.
Utilitarianism isn’t a metaethic in the first place; it’s a family of ethical systems. Metaethical systems and ethical systems aren’t comparable objects. “Maximize your utility function” says nothing, for the reasons given by benelliott, and isn’t a metaethical claim (nor a correct summary of EY’s metaethic); metaethics deals with questions like:
What does moral language mean? Do moral facts exist? If so, what are they like, and are they reducible to natural facts? How can we know whether moral judgments are true or false? Is there a connection between making a moral judgment and being motivated to abide by it? Are moral judgments objective or subjective, relative or absolute? Does it make sense to talk about moral progress?
EY’s metaethic approaches those questions as an unpacking of “should” and other moral symbols. While it does give examples of some of the major object-level values we’d expect to find in ethical systems, it doesn’t generate a brand of utilitarianism or a specific utility function.
(And “utility” as in what an agent with a (VNM) utility function maximizes (in expectation), and “utility” as in what a utilitarian tries to maximize in aggregate over some set of beings, aren’t comparable objects either, and they should be kept cognitively separate.)
Utilitarianism isn’t a metaethic in the first place; it’s a family of ethical systems.
Good point. Here’s the intuition behind my comment. Classical utilitarianism starts with “maximize aggregate utility” and jumps off from there (Mill calls it obvious, then gives a proof that he admits to be flawed). This opens them up to a slew of standard criticisms (e.g. utility monsters). I’m not very well versed on more modern versions of utilitarianism, but the impression I get is that they do something similar. But, as you point out, all the utilitarian is saying is which utility function you should be maximizing (answer: the aggregate of the utility functions of all suitable agents).
EY’s metaethics, on the other hand, eventually says something like “maximize this specific utility function that we don’t know perfectly. Oh yeah, it’s your utility function, and most everyone else’s.” With a suitable utility function, EY’s metaethics seems completely compatable with utilitarianism, I admit, but that seems unlikely. The utilitarian has to take into account the murderer’s preference for murder, should that preference actually exist (and not be a confusion). It seems highly unlikely to me that I and most of my fellow humans (which is where the utility function in question exists) care about someone’s preference for murder. Even assuming that I/we thought faster, more rationally, etc.
Oh, and a note on the “maximize your own utility function” language that I used. I tend to think about ethics in the first person: what should I do. Well, maximize my own utility function/preferences, whatever they are. I only start worrying about your preferences when I find out that they are information about my own preferences (or if I specifically care about your preferences in my own.) This is an explanation of how I’m thinking, but I should know better than to use this language on LW where most people haven’t seen it before and so will be confused.
all the utilitarian is saying is which utility function you should be maximizing (answer: the aggregate of the utility functions of all suitable agents)
The answer is the aggregate of some function for all suitable agents, but that function needn’t itself be a decision-theoretic utility function. It can be something else, like pleasure minus pain or even pleasure-not-derived-from-murder minus pain.
Ah, I was equating preference utilitarianism with utilitarianism.
I still think that calling yourself a utilitarian can be dangerous if only because it instantly calls to mind a list of stock objects (in some interlocutors) that just don’t apply given EY’s metaethics. It may be worth sticking to the terminology despite the cost though.
One problem with the FAQ: The standard metaethics around here, at least EY’s metaethics, is not utilitarianism...EY’s metaethics says maximize your own utility...
Be careful not to confuse ethics and metaethics. You’re talking about ethical theories here, rather than metaethical theories. (EY’s metaethics is a form of cognitivism).
I’m glad you brought that up, since it’s something I’ve mentally been circling around but never heard verbalized clearly before.
Both the classical and the Yudkowskian system seem to run into some problems that the other avoids, and right now I’m classifying the difference as “too advanced to be relevant to the FAQ”. Right now my own opinions are leaning toward believing that under reflective equilibrium my utility function should reference the aggregate utility function and possible be the same as it.
It is literally impossible to maximise anything other than your own utility function, because your utility function is defined as ‘that which you maximise’. In that sense EY’s meta-ethics is a tautology, the important part is about not knowing what your utility function is.
People can intentionally maximize anything, including the number of paperclips in the universe. Suppose there was a religion or school of philosophy that taught that maximizing paperclips is deontologically the right thing to do - not because it’s good for anyone, or because Divine Clippy would smite them for not doing it, just that morality demands that they do it. And so they choose to do it, even if they hate it.
In that case, I would say their true utility function was “follow the deontological rules” or “avoid being smited by divine clippy”, and that maximising paperclips is an instrumental subgoal.
In many other cases, I would be happy to say that the person involved was simply not utilitarian, if their actions did not seem to maximise anything at all.
If you define “utility function” as “what agents maximize” then your above statement is true but tautological. If you define “utility function” as “an agent’s relation between states of the world and that agent’s hedons” then it’s not true that you can only maximize your utility function.
I certainly do not define it the second way. Most people care about something other than their own happiness, and some people may care about their own happiness very little, not at all, or negatively, I really don’t see why a ‘happiness function’ would be even slightly interesting to decision theorists.
I think I’d want to define a utility function as “what an agent wants to maximise” but I’m not entirely clear how to unpack the word ‘want’ in that sentence, I will admit I’m somewhat confused.
However, I’m not particularly concerned about my statements being tautological, they were meant to be, since they are arguing against statements that are tautologically false.
Unless I’m misunderstanding you, your definition leaves no room for moral error. Surely it’s possible to come up with some utility function under which your actions are maximizing. So everyone has a utility function under which the actions they took were maximizing.
One problem with the FAQ: The standard metaethics around here, at least EY’s metaethics, is not utilitarianism. Utilitarianism says maximize aggregate utility, with “aggregate” defined in some suitable way. EY’s metaethics says maximize your own utility (with the caveat that you only have partial information of your utility function), and that all humans have sufficiently similar utility functions.
You get something pretty similar to utilitarianism from that last condition (if everyone has the same utility function and you’re maximizing your own utility function, then you’re also maximizing aggregate utility in many senses of the term “aggregate”). But note that the interpersonal comparison of utility problem vanishes: you’re maximizing your own utility function. Maximization of the aggregate (under some methods of comparisons) is merely a consequence of this, nothing more. Also note that if we relax that last condition and let humans have differing utility function, there is no intrinsic problem for EY’s theory. If someone has a legitimate preference for killing people, the utilitarian has to take that into account as positive utility (or add some ad hoc assumptions about which preferences matter). On EY’s theory, sans that last condition, you only take into account someone’s preference for murder if your utility function tells you to. You may value other humans satisfying their preferences, but that doesn’t mean you have to value every single preference every single human has. You could, but it really just depends on what your utility function says.
Utilitarianism isn’t a metaethic in the first place; it’s a family of ethical systems. Metaethical systems and ethical systems aren’t comparable objects. “Maximize your utility function” says nothing, for the reasons given by benelliott, and isn’t a metaethical claim (nor a correct summary of EY’s metaethic); metaethics deals with questions like:
EY’s metaethic approaches those questions as an unpacking of “should” and other moral symbols. While it does give examples of some of the major object-level values we’d expect to find in ethical systems, it doesn’t generate a brand of utilitarianism or a specific utility function.
(And “utility” as in what an agent with a (VNM) utility function maximizes (in expectation), and “utility” as in what a utilitarian tries to maximize in aggregate over some set of beings, aren’t comparable objects either, and they should be kept cognitively separate.)
Good point. Here’s the intuition behind my comment. Classical utilitarianism starts with “maximize aggregate utility” and jumps off from there (Mill calls it obvious, then gives a proof that he admits to be flawed). This opens them up to a slew of standard criticisms (e.g. utility monsters). I’m not very well versed on more modern versions of utilitarianism, but the impression I get is that they do something similar. But, as you point out, all the utilitarian is saying is which utility function you should be maximizing (answer: the aggregate of the utility functions of all suitable agents).
EY’s metaethics, on the other hand, eventually says something like “maximize this specific utility function that we don’t know perfectly. Oh yeah, it’s your utility function, and most everyone else’s.” With a suitable utility function, EY’s metaethics seems completely compatable with utilitarianism, I admit, but that seems unlikely. The utilitarian has to take into account the murderer’s preference for murder, should that preference actually exist (and not be a confusion). It seems highly unlikely to me that I and most of my fellow humans (which is where the utility function in question exists) care about someone’s preference for murder. Even assuming that I/we thought faster, more rationally, etc.
Oh, and a note on the “maximize your own utility function” language that I used. I tend to think about ethics in the first person: what should I do. Well, maximize my own utility function/preferences, whatever they are. I only start worrying about your preferences when I find out that they are information about my own preferences (or if I specifically care about your preferences in my own.) This is an explanation of how I’m thinking, but I should know better than to use this language on LW where most people haven’t seen it before and so will be confused.
The answer is the aggregate of some function for all suitable agents, but that function needn’t itself be a decision-theoretic utility function. It can be something else, like pleasure minus pain or even pleasure-not-derived-from-murder minus pain.
Ah, I was equating preference utilitarianism with utilitarianism.
I still think that calling yourself a utilitarian can be dangerous if only because it instantly calls to mind a list of stock objects (in some interlocutors) that just don’t apply given EY’s metaethics. It may be worth sticking to the terminology despite the cost though.
Be careful not to confuse ethics and metaethics. You’re talking about ethical theories here, rather than metaethical theories. (EY’s metaethics is a form of cognitivism).
I’m glad you brought that up, since it’s something I’ve mentally been circling around but never heard verbalized clearly before.
Both the classical and the Yudkowskian system seem to run into some problems that the other avoids, and right now I’m classifying the difference as “too advanced to be relevant to the FAQ”. Right now my own opinions are leaning toward believing that under reflective equilibrium my utility function should reference the aggregate utility function and possible be the same as it.
It is literally impossible to maximise anything other than your own utility function, because your utility function is defined as ‘that which you maximise’. In that sense EY’s meta-ethics is a tautology, the important part is about not knowing what your utility function is.
No. People can be stupid. They can even be wrong about what their utility function is.
It is “that which you would maximise if you weren’t a dumbass and knew what you wanted’.
Good point.
Perhaps I should have said “its impossible to intentionally maximise anything other than your utility function”.
People can intentionally maximize anything, including the number of paperclips in the universe. Suppose there was a religion or school of philosophy that taught that maximizing paperclips is deontologically the right thing to do - not because it’s good for anyone, or because Divine Clippy would smite them for not doing it, just that morality demands that they do it. And so they choose to do it, even if they hate it.
In that case, I would say their true utility function was “follow the deontological rules” or “avoid being smited by divine clippy”, and that maximising paperclips is an instrumental subgoal.
In many other cases, I would be happy to say that the person involved was simply not utilitarian, if their actions did not seem to maximise anything at all.
If you define “utility function” as “what agents maximize” then your above statement is true but tautological. If you define “utility function” as “an agent’s relation between states of the world and that agent’s hedons” then it’s not true that you can only maximize your utility function.
I certainly do not define it the second way. Most people care about something other than their own happiness, and some people may care about their own happiness very little, not at all, or negatively, I really don’t see why a ‘happiness function’ would be even slightly interesting to decision theorists.
I think I’d want to define a utility function as “what an agent wants to maximise” but I’m not entirely clear how to unpack the word ‘want’ in that sentence, I will admit I’m somewhat confused.
However, I’m not particularly concerned about my statements being tautological, they were meant to be, since they are arguing against statements that are tautologically false.
Unless I’m misunderstanding you, your definition leaves no room for moral error. Surely it’s possible to come up with some utility function under which your actions are maximizing. So everyone has a utility function under which the actions they took were maximizing.
I’m not quite sure what you mean.