I would consider that you cannot weight these things along a single metric. Say evolution → human values really is only 4% of your value alignment, if that 4% is the fundamental core then it’s not part of the sum of all values, but a coefficient or a base where the other stuff is the exponent. It’s the hardware the software has to be loaded on, but not totally tabula rasa either.
Correct me if I’m wrong, but this would assume that if you could somehow make a human level intelligence snake and raise it in human society (let’s pretend nobody considers it weird that there’s a snake taking Chemistry class with them), then that snake would be 96% aligned with humanity?
My intuition would be along the lines of the parable of the scorpion and the frog.
The “4%” wasn’t addressing the question of “where do humans get their values from?” It was addressing “When trying to make predictions about AGI outcomes, how much weight should we assign these various sources of evidence?”
Correct me if I’m wrong, but this would assume that if you could somehow make a human level intelligence snake and raise it in human society (let’s pretend nobody considers it weird that there’s a snake taking Chemistry class with them), then that snake would be 96% aligned with humanity?
My perspective isn’t blank slatism. The genome has various leavers by which it can influence the sorts of values that a human forms. E.g., the snake wouldn’t have human-like reward circuitry, so it would probably learn to value very different things than a human which went through the same experiences. For more on this, see: “Learning from scratch” in the brain.
E.g., the snake wouldn’t have human-like reward circuitry, so it would probably learn to value very different things than a human which went through the same experiences.
So in this case I think we then agree. But it seems a bit at odds with the 4% weighting of genetic roots. If we agree the snake would exhibit very different values despite experiencing the ‘human learning’ part then shouldn’t this adjust the 60% weight you grant that? Seems the evolutionary roots made all the difference for the snake. Which is the whole point about initial AGI alignment having to be exactly right.
Otherwise I understand your post to be ‘for humans, how much of human value is derived from evolution vs learning’. But that’s using humans as evidence who are human to begin with.
I would consider that you cannot weight these things along a single metric. Say evolution → human values really is only 4% of your value alignment, if that 4% is the fundamental core then it’s not part of the sum of all values, but a coefficient or a base where the other stuff is the exponent. It’s the hardware the software has to be loaded on, but not totally tabula rasa either.
Correct me if I’m wrong, but this would assume that if you could somehow make a human level intelligence snake and raise it in human society (let’s pretend nobody considers it weird that there’s a snake taking Chemistry class with them), then that snake would be 96% aligned with humanity?
My intuition would be along the lines of the parable of the scorpion and the frog.
The “4%” wasn’t addressing the question of “where do humans get their values from?” It was addressing “When trying to make predictions about AGI outcomes, how much weight should we assign these various sources of evidence?”
My perspective isn’t blank slatism. The genome has various leavers by which it can influence the sorts of values that a human forms. E.g., the snake wouldn’t have human-like reward circuitry, so it would probably learn to value very different things than a human which went through the same experiences. For more on this, see: “Learning from scratch” in the brain.
So in this case I think we then agree. But it seems a bit at odds with the 4% weighting of genetic roots. If we agree the snake would exhibit very different values despite experiencing the ‘human learning’ part then shouldn’t this adjust the 60% weight you grant that? Seems the evolutionary roots made all the difference for the snake. Which is the whole point about initial AGI alignment having to be exactly right.
Otherwise I understand your post to be ‘for humans, how much of human value is derived from evolution vs learning’. But that’s using humans as evidence who are human to begin with.