Are you reinventing Asimov’s Three Laws of Robotics?
Lumifer
Karma: 19,795
I suspect the solution is this.
tomorrow
That’s not conventionally considered to be “in the long run”.
We don’t have any theory that would stop AI from doing that
The primary reason is that we don’t have any theory about what a post-singularity AI might or might not do. Doing some pretty basic decision theory focused on the corner cases is not “progress”.
It seems weird that you’d deterministically two-box against such an Omega
Even in the case when the random noise dominates and the signal is imperceptibly small?
So the source-code of your brain just needs to decide whether it’ll be a source-code that will be one-boxing or not.
First, in the classic Newcomb when you meet Omega that’s a surprise to you. You don’t get to precommit to deciding one way or the other because you had no idea such a situation will arise: you just get to decide now.
You can decide however whether you’re the sort of person who accepts their decisions can be deterministically predicted in advance with sufficient certainty, or whether you’ll be claiming that other people predicting your choice must be a violation of causality (it’s not).
Why would you make such a decision if you don’t expect to meet Omega and don’t care much about philosophical head-scratchers?
And, by the way, predicting your choice is not a violation of causality, but believing that your choice (of the boxes, not of the source code) affects what’s in the boxes is.
Second, you are assuming that the brain is free to reconfigure and rewrite its software which is clearly not true for humans and all existing agents.
Old and tired, maybe, but clearly there is not much consensus yet (even if, ahem, some people consider it to be as clear as day).
Note that who makes the decision is a matter of control and has nothing to do with freedom. A calculator controls its display and so the “decision” to output 4 in response to 2+2 it its own, in a way. But applying decision theory to a calculator is nonsensical and there is no free choice involved.
LW is kinda dead (not entirely, there is still some shambling around happening, but the brains are in short supply) and is supposed to be replaced by a shinier reincarnated version which has been referred to as LW 2.0 and which is now in open beta at www.lesserwrong.com
LW 1.0 is still here, but if you’re looking for active discussion, LW 2.0 might be a better bet.
Re qualia, I suggest that you start with trying to set up hard definitions for terms “qualia” and “exists”. Once you do, you may find the problem disappears—see e.g. this.
Re simulation, let me point out that the simulation hypothesis is conventionally known as “creationism”. As to the probability not being calculable, I agree.
The truth that curi and myself are trying to get across to people here is… it is the unvarnished truth… know far more about epistemology than you. That again is an unvarnished truth
In which way all these statements are different from claiming that Jesus is Life Everlasting and that Jesus dying for our sins is an unvarnished truth?
Lots of people claim to have access to Truth—what makes you special?
LOL. You keep insisting that people have to play by your rules but really, they don’t.
You can keep inventing your own games and declaring yourself winner by your own rules, but it doesn’t look like a very useful activity to me.
genetic algorithms often write and later read data, just like e.g. video game enemies
Huh? First, the expression “genetic algorithms” doesn’t mean what you think it means. Second, I don’t understand the writing and reading data part. Write which data to what substrate?
your examples are irrelevant b/c you aren’t addressing the key intellectual issues
I like dealing with reality. You like dealing with abstractions in your head. We talked about this—we disagree. You know that.
But if you are uninterested in empirical evidence, why bother discussing it at all?
you won’t want to learn or seriously discuss
Yes, I’m not going to do what you want me to do. You know that as well.
you will be hostile to the idea that you need a framework in which to interpret the evidence
I will be hostile to the idea that I need your framework to interpret the evidence, yes. You know that, too.
The problem is that very very few orcas do that—only two pods in the world, as far as we know. Orcas which live elsewhere (e.g. the Pacific Northwest orcas which are very well-observed) do not do anything like this. Moreover, there is evidence that the technique is taught by adults to juvenile orcas. See e.g .here or here.
If you want to debate that you need an epistemology which says what “knowledge” is. References to where you have that with full details to rival Critical Rationalism?
Oh, get stuffed. I tried debating you and the results were… discouraging.
Yes, I obviously think that CR is deluded.
This sentence from the OP:
Like the algorithms in a dog’s brain, AlphaGo is a remarkable algorithm, but it cannot create knowledge in even a subset of contexts.
A bit more generally, the claim that humans are UKCs and nothing else can create knowledge which is defined as a way to solve a problem.
the AI risks starting these triggers when it starts to think first thoughts about existing of the triggers
So basically you have a trap which kills you the moment you become aware of it. The first-order effect will be a lot of random deaths from just blundering into such a trap while walking around.
I suspect that the second-order effect will be the rise of, basically, superstitions and some forms of magical thinking which will be able to provide incentives to not go “there” without actually naming “there”. I am not sure this is a desirable outcome.
It’s also rank nonsense—this bit in particular:
dog genes contain behavioural algorithms pre-programmed by evolution
Some orcas hunt seal pups by temporarily stranding themselves on the beaches in order to reach their prey. Is that behaviour programmed in their genes? The genes of all orcas?
Show results in 3 separate domains.
Chess
Go
Shogi
Unreason is accepting the claims of a paper at face value, appealing to its authority
Which particular claim that the paper makes I accepted at face value and which you think is false? Be specific.
I was aware of AlphaGo Zero before I posted—check out my link
AlphaGo Zero and AlphaZero are different things—check out my link.
In any case, are you making the claim that if a neural net were able to figure out the rules of the game by examining a few million games, you would accept that it’s a universal knowledge creator?
You sound less and less reasonable with every comment.
It doesn’t look like you conversion attempts are working well. Why do you think this is so?
AlphaGo is a remarkable algorithm, but it cannot create knowledge
Funny you should mention that. AlphaGo has a successor, AlphaZero. Let me quote:
The game of chess is the most widely-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. In contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.
Note: “given no domain knowledge except the game rules”
There seems to be a complexity limit to what humans can build. A full GAI is likely to be somewhere beyond that limit.
The usual solution to that problem—see the EY’s fooming scenario—is to make the process recursive: let a mediocre AI improve itself, and as it gets better it can improve itself more rapidly. Exponential growth can go fast and far.
This, of course, gives rise to another problem: you have no idea what the end product is going to look like. If you’re looking at the gazillionth iteration, your compiler flags were probably lost around the thousandth iteration and your chained monitor system mutated into a cute puppy around the millionth iteration...
Probabilistic safety systems are indeed more tractable, but that’s not the question. The question is whether they are good enough.