As someone who seems to care more about terminology than most (and as a result probably gets into more terminological debates on LW than anyone else (see 1234)), I don’t really understand what you’re suggesting here. Do you think this advice is applicable to any of the above examples of naming / drawing boundaries? If so, what are its implications in those cases? If not, can you give a concrete example that might come up on LW or otherwise have some relevance to us?
Hi, Wei—thanks for commenting! (And sorry for the arguably somewhat delayed reply; it’s been a really tough week for me.)
can you give a concrete example that might come up on LW or otherwise have some relevance to us?
Is Slate Star Codex close enough? In his “Anti-Reactionary FAQ”, Scott Alexander writes—
Why use this made-up word [“demotism”] so often?
Suppose I wanted to argue that mice were larger than grizzly bears. I note that both mice and elephants are “eargreyish”, meaning grey animals with large ears. We note that eargreyish animals such as elephants are known to be extremely large. Therefore, eargreyish animals are larger than noneargreyish animals and mice are larger than grizzly bears.
As long as we can group two unlike things together using a made-up word that traps non-essential characteristics of each, we can prove any old thing.
This post is mostly just a longer, more detailed version (with some trivial math) of the point Scott is making in these three paragraphs: mice and elephants form a cluster if you project into the subspace spanned by “color” and “relative ear size”, but using a word to point to a cluster in such a “thin”, impoverished subspace is a dishonest rhetorical move when your interlocutors are trying to use language to mostly talk about the many other features of animals which don’t covary much with color and relative-ear-size. This is obvious in the case of mice and elephants, but Scott is arguing that a similar mistake is being made by reactionaries who classify Nazi Germany and the Soviet Union as “demotist”, and then argue that liberal democracies suffer from the same flaws on account of being “demotist.” Scott had previously dubbed this kind of argument the “noncentral fallacy” and analyzed how it motivates people to argue over category boundaries like “murder” or “theft.”
My interest in terminological debates is usually not to discover new ideas but to try to prevent confusion (when readers are likely to infer something wrong from a name, e.g., because of different previous usage or because a compound term is defined to mean something that’s different from what one would reasonably infer from the combination of individual terms).
I agree that preventing confusion is the main reason to care about terminology; it only takes a moderate amount of good faith and philosophical sophistication for interlocutors to negotiate their way past terminology clashes (“I wouldn’t use that word because I think it conflates these-and-such things, but for the purposes of this conversation …” &c.) and make progress discussing actual ideas. But I wanted to have this post explaining in detail a particular thing that can go wrong when philosophical sophistication is lacking or applied selectively, which was mostly covered by Eliezer’s “A Human’s Guide to Words”, but of which I hadn’t seen the “which subspace to pay attention to / do clustering on” problem treated anywhere in such terms.
Thanks, I think I have a better idea of what you’re proposing now, but I’m still not sure I understand it correctly, or if it makes sense.
mice and elephants form a cluster if you project into the subspace spanned by “color” and “relative ear size”, but using a word to point to a cluster in such a “thin”, impoverished subspace is a dishonest rhetorical move when your interlocutors are trying to use language to mostly talk about the many other features of animals which don’t covary much with color and relative-ear-size.
But there are times when it’s not a dishonest rhetorical move to do this, right? For example suppose an invasive predator species has moved into some new area, and I have an hypothesis that animals with grey skin and big ears might be the only ones in that area who can escape being hunted to extinction (because I think the predator has trouble seeing grey and big ears are useful for hearing the predator and only this combination of traits offers enough advantage for a prey species to survive). While I’m formulating this hypothesis, discussing how plausible it is, applying for funding, doing field research, etc., it seems useful to create a new term like “eargreyish” so I don’t have to keep repeating “grey animals with relatively large ears”.
Since it doesn’t seem to make sense to never use a word to point to a cluster in a “thin” subspace, what is your advice for when it’s ok to do this or accept others doing this?
(I continue to regret my slow reply turnaround time.)
But there are times when it’s not a dishonest rhetorical move to do this, right?
Right. In Scott’s example, the problem was using the “eargrayish” concept to imply (bad) inferences about size, but your example isn’t guilty of this.
However, it’s also worth emphasizing that the inferential work done by words and categories is often spread across many variables, including things that aren’t as easy to observe as the features that were used to perform the categorization. You can infer that “mice” have very similar genomes, even if you never actually sequence their DNA. Or if you lived before DNA had been discovered, you might guess that there exists some sort of molecular mechanism of heredity determining the similarities between members of a “species”, and you’d be right (whereas similar such guesses based on concepts like “eargrayishness” would probably be wrong).
Since it doesn’t seem to make sense to never use a word to point to a cluster in a “thin” subspace, what is your advice for when it’s ok to do this or accept others doing this?
Um, watch out for cases where the data clusters in the “thin” subspace, but doesn’t cluster in other dimensions that are actually relevant in the context that you’re using the word? (I wish I had a rigorous reduction of what “relevant in the context” means, but I don’t.)
As long as we’re talking about animal taxonomy (dolphins, mice, elephants, &c.), a concrete example of a mechanism that systematically produces this kind of distribution might be Batesian or Müllerian mimicry (or convergent evolution more generally, as with dolphins’ likeness to fish). If you’re working as a wildlife photographer and just want some cool snake photos, then a concept of “red-‘n’-yellow stripey snake” that you formed from observation (abstractly: you noticed a cluster in the subspace spanned by “snake colors” and “snake stripedness”) might be completely adequate for your purposes: as a photographer, you just don’t care whether or not there’s more structure to the distribution of snakes than what looks good in your pictures. On the other hand, if you actually have to handle the snakes, suddenly the difference between the harmless scarlet kingsnake and the poisonous coral snake (“red on yellow, kill a fellow; red on black, venom lack”) is very relevant and you want to be modeling them as separate species!
Sometimes people redraw boundaries for reasons of local expediency. For instance, the category of AGI seems to have been expanded implicitly in some contexts to include what might previously have just been called a really good machine learning library that can do many things humans can do. This allows AGI alignment to be a bigger-tent cause, and raise more money, than it would in the counterfactual where the old definitions were preserved.
This article seems to me to be outlining a principled case that such category redefinitions can be systematically distinguished from purely epistemic category redefinitions, with the implication that there’s a legitimate interest in tracking which is which, and sometimes in resisting politicized recategorizations in order to defend the enterprise of shared mapmaking.
I don’t see how this article argues against a wider AGI definition. The wider definition is still a correlational cluster.
The article doesn’t say that it’s worthwhile to keep historical meaning of a term like AGI. It also doesn’t say that it’s good to draw the boundaries in a way that a person can guess where the boundary is based on understanding the words artificial, general and intelligence.
It’s not a thinner boundary so that “38. Your definition draws a boundary around a cluster in an inappropriately ‘thin’ subspace of Thingspace that excludes relevant variables, resulting in fallacies of compression.” might be violated.
The article starts by speaking about ” It is what people should be trying to do ”, say in it’s middle “This leaves aspiring instructors of rationality in something of a predicament: in order to teach people how categories can be more or (ahem) less wrong,” and ends with speaking about what people must do.
That does appear to me like an article that intends to make a case that people should prefer certain definition over other definitions.
If your case is rather that the value of the article is about classification of how boundaries are drawn to distinct ways those boundaries are drawn, it seems to me surprising that you read out of the article that certain claims should be classified as redrawing boundaries for reasons of local expediency that seems odd to me given that the article neither speaks about redrawing boundaries nor redefining boundaries nor about classifying anything under the suggested category of “local expediency”.
Rationality discourse is necessarily about specific contexts and purposes. I don’t think the Sequences imply that a spy should always reveal themselves, or that actors in a play should refuse to perform the same errors with the same predictable bad consequences two nights in a row. Discourse about how to speak the truth efficiently, on a site literally called “Less Wrong,” shouldn’t have to explicitly disclaim that it’s meant as advice within that context every time, even if it’s often helpful to examine what that means and when and how it is useful to prioritize over other desiderata.
I’m not sure what your position happens to be. Is it “This post isn’t advice. It’s wrong for you (ChristianKl) to expect that the author explicitely disclaims giving advice when he doesn’t intent to give advice.”?
If that’s the case, it seems strange to me. This post contains explicit statemensts about what people should/must do. It contains those in the beginning and in the end, which are usually the places where an essay states it’s purpose.
It’s bad to be too vague to be wrong.
Postmodern writing about how to speak truth efficiently that’s to vague to be wrong is problematic and I don’t think having a bunch of LW signaling and cheers for rationalists make it better.
The article seems indirectly relevant to example 4, in which an epistemic dispute about how to divide up categories is getting mixed with a prudential dispute on which things to prioritize. Once a category is clearly designated as “that which is to be prioritized,” it becomes more expensive to improve the expressive power of your vocabulary by redrawing the conceptual boundaries, since this might cause your prioritization to deteriorate.
Possibly the right way to proceed in that case would be to work out a definition of the original category which more explicitly refers to the reasons you think it’s the right category to prioritize, perhaps assigning this a new name, so that these discussions can be separated.
My interest in terminological debates is usually not to discover new ideas but to try to prevent confusion (when readers are likely to infer something wrong from a name, e.g., because of different previous usage or because a compound term is defined to mean something that’s different from what one would reasonably infer from the combination of individual terms). But sometimes terminological debates can uncover hidden assumptions and lead to substantive debates about them. See here for an example.
Whether to call something dephlogisticated air or oxygen was a very important terminological debate in chemisty even when the correlational cluster was the same. It matters if you conceptualize it as absense of something or as positive existence.
In medicine the recent debate about renaming chronic fatigue syndrome (CFS) into systemic exertion intolerance disease (SEID) is a quite interesting one.
With CFS it’s a quite unclear where to draw the boundary. With SEID you can let someone exercise and then observe how long their body needs to recover and when they take much longer to recover from the exertion you can put the SEID diagnosis on them.
CFS and SEID are both cases where certain states correlate with each other Zacks post doesn’t help us at all to reason about whether we should prefer CFS or SEID as a term.
CFS and SEID are both cases where certain states correlate with each other Zacks post doesn’t help us at all to reason about whether we should prefer CFS or SEID as a term.
I’m definitely not claiming to have the “correct” answer to all terminological disputes. (As the post says, “Of course, there isn’t going to be a unique way to encode the knowledge into natural language.”)
Suppose, hypothetically, that it were discovered that there are actually two or more distinct etiologies causing cases that had historically been classified as “chronic fatigue syndrome”, and cases with different etiologies responded better to different treatments. In this hypothetical scenario, medical professionals would want to split what they had previously called “chronic fatigue syndrome” into two or more categories to reflect their new knowledge. I think someone who insisted that “chronic fatigue syndrome” was still a good category given the new discovery of separate etiologies would be making a mistake (with respect to the goals doctors have when they talk about diseases), even if the separate etiologies had similar symptoms (which is what motivated the CFS label in the first place).
In terms of the configuration space visual metaphor, we would say that while “chronic fatigue syndrome” is a single cluster in the “symptoms” subspace of Diseasespace, more variables than just symptoms are decision-relevant to doctors, and the CFS cluster doesn’t help them reason about those other variables.
As someone who seems to care more about terminology than most (and as a result probably gets into more terminological debates on LW than anyone else (see 1 2 3 4)), I don’t really understand what you’re suggesting here. Do you think this advice is applicable to any of the above examples of naming / drawing boundaries? If so, what are its implications in those cases? If not, can you give a concrete example that might come up on LW or otherwise have some relevance to us?
Hi, Wei—thanks for commenting! (And sorry for the arguably somewhat delayed reply; it’s been a really tough week for me.)
Is Slate Star Codex close enough? In his “Anti-Reactionary FAQ”, Scott Alexander writes—
This post is mostly just a longer, more detailed version (with some trivial math) of the point Scott is making in these three paragraphs: mice and elephants form a cluster if you project into the subspace spanned by “color” and “relative ear size”, but using a word to point to a cluster in such a “thin”, impoverished subspace is a dishonest rhetorical move when your interlocutors are trying to use language to mostly talk about the many other features of animals which don’t covary much with color and relative-ear-size. This is obvious in the case of mice and elephants, but Scott is arguing that a similar mistake is being made by reactionaries who classify Nazi Germany and the Soviet Union as “demotist”, and then argue that liberal democracies suffer from the same flaws on account of being “demotist.” Scott had previously dubbed this kind of argument the “noncentral fallacy” and analyzed how it motivates people to argue over category boundaries like “murder” or “theft.”
Downthread, you wrote—
I agree that preventing confusion is the main reason to care about terminology; it only takes a moderate amount of good faith and philosophical sophistication for interlocutors to negotiate their way past terminology clashes (“I wouldn’t use that word because I think it conflates these-and-such things, but for the purposes of this conversation …” &c.) and make progress discussing actual ideas. But I wanted to have this post explaining in detail a particular thing that can go wrong when philosophical sophistication is lacking or applied selectively, which was mostly covered by Eliezer’s “A Human’s Guide to Words”, but of which I hadn’t seen the “which subspace to pay attention to / do clustering on” problem treated anywhere in such terms.
Thanks, I think I have a better idea of what you’re proposing now, but I’m still not sure I understand it correctly, or if it makes sense.
But there are times when it’s not a dishonest rhetorical move to do this, right? For example suppose an invasive predator species has moved into some new area, and I have an hypothesis that animals with grey skin and big ears might be the only ones in that area who can escape being hunted to extinction (because I think the predator has trouble seeing grey and big ears are useful for hearing the predator and only this combination of traits offers enough advantage for a prey species to survive). While I’m formulating this hypothesis, discussing how plausible it is, applying for funding, doing field research, etc., it seems useful to create a new term like “eargreyish” so I don’t have to keep repeating “grey animals with relatively large ears”.
Since it doesn’t seem to make sense to never use a word to point to a cluster in a “thin” subspace, what is your advice for when it’s ok to do this or accept others doing this?
(I continue to regret my slow reply turnaround time.)
Right. In Scott’s example, the problem was using the “eargrayish” concept to imply (bad) inferences about size, but your example isn’t guilty of this.
However, it’s also worth emphasizing that the inferential work done by words and categories is often spread across many variables, including things that aren’t as easy to observe as the features that were used to perform the categorization. You can infer that “mice” have very similar genomes, even if you never actually sequence their DNA. Or if you lived before DNA had been discovered, you might guess that there exists some sort of molecular mechanism of heredity determining the similarities between members of a “species”, and you’d be right (whereas similar such guesses based on concepts like “eargrayishness” would probably be wrong).
(As it is written: “Having a word for a thing, rather than just listing its properties, is a more compact code precisely in those cases where we can infer some of those properties from the other properties.”)
Um, watch out for cases where the data clusters in the “thin” subspace, but doesn’t cluster in other dimensions that are actually relevant in the context that you’re using the word? (I wish I had a rigorous reduction of what “relevant in the context” means, but I don’t.)
As long as we’re talking about animal taxonomy (dolphins, mice, elephants, &c.), a concrete example of a mechanism that systematically produces this kind of distribution might be Batesian or Müllerian mimicry (or convergent evolution more generally, as with dolphins’ likeness to fish). If you’re working as a wildlife photographer and just want some cool snake photos, then a concept of “red-‘n’-yellow stripey snake” that you formed from observation (abstractly: you noticed a cluster in the subspace spanned by “snake colors” and “snake stripedness”) might be completely adequate for your purposes: as a photographer, you just don’t care whether or not there’s more structure to the distribution of snakes than what looks good in your pictures. On the other hand, if you actually have to handle the snakes, suddenly the difference between the harmless scarlet kingsnake and the poisonous coral snake (“red on yellow, kill a fellow; red on black, venom lack”) is very relevant and you want to be modeling them as separate species!
Sometimes people redraw boundaries for reasons of local expediency. For instance, the category of AGI seems to have been expanded implicitly in some contexts to include what might previously have just been called a really good machine learning library that can do many things humans can do. This allows AGI alignment to be a bigger-tent cause, and raise more money, than it would in the counterfactual where the old definitions were preserved.
This article seems to me to be outlining a principled case that such category redefinitions can be systematically distinguished from purely epistemic category redefinitions, with the implication that there’s a legitimate interest in tracking which is which, and sometimes in resisting politicized recategorizations in order to defend the enterprise of shared mapmaking.
I don’t see how this article argues against a wider AGI definition. The wider definition is still a correlational cluster.
The article doesn’t say that it’s worthwhile to keep historical meaning of a term like AGI. It also doesn’t say that it’s good to draw the boundaries in a way that a person can guess where the boundary is based on understanding the words artificial, general and intelligence.
It’s not a thinner boundary so that “38. Your definition draws a boundary around a cluster in an inappropriately ‘thin’ subspace of Thingspace that excludes relevant variables, resulting in fallacies of compression.” might be violated.
The article didn’t “argue against” a wider AGI definition. It implied a more specific claim than “for” or “against.”
The article starts by speaking about ” It is what people should be trying to do ”, say in it’s middle “This leaves aspiring instructors of rationality in something of a predicament: in order to teach people how categories can be more or (ahem) less wrong,” and ends with speaking about what people must do.
That does appear to me like an article that intends to make a case that people should prefer certain definition over other definitions.
If your case is rather that the value of the article is about classification of how boundaries are drawn to distinct ways those boundaries are drawn, it seems to me surprising that you read out of the article that certain claims should be classified as redrawing boundaries for reasons of local expediency that seems odd to me given that the article neither speaks about redrawing boundaries nor redefining boundaries nor about classifying anything under the suggested category of “local expediency”.
Rationality discourse is necessarily about specific contexts and purposes. I don’t think the Sequences imply that a spy should always reveal themselves, or that actors in a play should refuse to perform the same errors with the same predictable bad consequences two nights in a row. Discourse about how to speak the truth efficiently, on a site literally called “Less Wrong,” shouldn’t have to explicitly disclaim that it’s meant as advice within that context every time, even if it’s often helpful to examine what that means and when and how it is useful to prioritize over other desiderata.
I’m not sure what your position happens to be. Is it “This post isn’t advice. It’s wrong for you (ChristianKl) to expect that the author explicitely disclaims giving advice when he doesn’t intent to give advice.”?
If that’s the case, it seems strange to me. This post contains explicit statemensts about what people should/must do. It contains those in the beginning and in the end, which are usually the places where an essay states it’s purpose.
It’s bad to be too vague to be wrong.
Postmodern writing about how to speak truth efficiently that’s to vague to be wrong is problematic and I don’t think having a bunch of LW signaling and cheers for rationalists make it better.
The article seems indirectly relevant to example 4, in which an epistemic dispute about how to divide up categories is getting mixed with a prudential dispute on which things to prioritize. Once a category is clearly designated as “that which is to be prioritized,” it becomes more expensive to improve the expressive power of your vocabulary by redrawing the conceptual boundaries, since this might cause your prioritization to deteriorate.
Possibly the right way to proceed in that case would be to work out a definition of the original category which more explicitly refers to the reasons you think it’s the right category to prioritize, perhaps assigning this a new name, so that these discussions can be separated.
This makes me curious—have you found that terminological debates often lead to interesting ideas? Can you give an example?
My interest in terminological debates is usually not to discover new ideas but to try to prevent confusion (when readers are likely to infer something wrong from a name, e.g., because of different previous usage or because a compound term is defined to mean something that’s different from what one would reasonably infer from the combination of individual terms). But sometimes terminological debates can uncover hidden assumptions and lead to substantive debates about them. See here for an example.
Whether to call something dephlogisticated air or oxygen was a very important terminological debate in chemisty even when the correlational cluster was the same. It matters if you conceptualize it as absense of something or as positive existence.
In medicine the recent debate about renaming chronic fatigue syndrome (CFS) into systemic exertion intolerance disease (SEID) is a quite interesting one.
With CFS it’s a quite unclear where to draw the boundary. With SEID you can let someone exercise and then observe how long their body needs to recover and when they take much longer to recover from the exertion you can put the SEID diagnosis on them.
CFS and SEID are both cases where certain states correlate with each other Zacks post doesn’t help us at all to reason about whether we should prefer CFS or SEID as a term.
I’m definitely not claiming to have the “correct” answer to all terminological disputes. (As the post says, “Of course, there isn’t going to be a unique way to encode the knowledge into natural language.”)
Suppose, hypothetically, that it were discovered that there are actually two or more distinct etiologies causing cases that had historically been classified as “chronic fatigue syndrome”, and cases with different etiologies responded better to different treatments. In this hypothetical scenario, medical professionals would want to split what they had previously called “chronic fatigue syndrome” into two or more categories to reflect their new knowledge. I think someone who insisted that “chronic fatigue syndrome” was still a good category given the new discovery of separate etiologies would be making a mistake (with respect to the goals doctors have when they talk about diseases), even if the separate etiologies had similar symptoms (which is what motivated the CFS label in the first place).
In terms of the configuration space visual metaphor, we would say that while “chronic fatigue syndrome” is a single cluster in the “symptoms” subspace of Diseasespace, more variables than just symptoms are decision-relevant to doctors, and the CFS cluster doesn’t help them reason about those other variables.