But a further case reveals just how far-reaching a threat RTE, taken as a fully general principle, would pose to our ability to confirm hypotheses in the light of our evidence. Consider the following example:
Fish. You know there are some large fish and some small fish in a lake, but you don’t know the proportions: either there are mostly large fish (HL) or mostly small fish (HS). You know that you will catch one fish each time you stick in your net. You stick in your net and catch a large fish. Let’s describe your evidence in the following way:
EG. You caught some large fish.
EG would seem to give you evidence in favor of HL over HS, since you are more likely to catch a large fish if the majority of the fish are large than if the majority of the fish are small: P(EG|HL) > P(EG|HS). Reasoning in this way seems like a paradigm example of empirically confirming a hypothesis, and, as such, it seems beyond reproach.
But wait! The fish you caught, in addition to being large, is a particular fish (as fish tend to be). You name her Asha (ostensively, without need of any clever rigid designation). So you also have the evidence:
ES. You caught Asha.
We can now ask whether ES is more likely given HL than given HS: Were you more likely to catch Asha if there were mostly large fish, rather than mostly small fish? It seems clear that you were equally likely to catch Asha on either hypothesis: whatever the size of the other fish, the likelihood that your net would happen upon the specific large fish Asha was exactly the same. Asha was swimming along, and happened to be in the wrong place at the wrong time. The probability of that event is presumably sensitive to such factors as how frequently Asha swims in that area of the lake. But one thing that seems completely irrelevant is the size of the other fish around her. So, once again, the specific evidence fails to confirm the hypothesis: P(ES|HL) = P(ES|HS). Thus, if we follow RTE, we will be led to the false conclusion that we get no confirmation of HL over HS.
The above cases show that we should not always follow RTE in evaluating the likelihood of competing hypotheses. There are cases in which proper reasoning requires ignoring a more-specific statement of our evidence in favor of a less-specific version.
To briefly paraphrase, Epstein is saying that the chance of catching Asha is the same regardless of whether HL (mostly large fish) or HS (mostly small fish) is true. For example, suppose in either case there are 100 fish, then he would say the chance of catching Asha is 1% in either case, and therefore the detailed knowledge of catching specifically Asha doesn't favor HL over HS. By contrast, the general knowledge of catching some big fish clearly does favor HL - we are more likely to pull out a big fish if there are more of them around! So the specific information, that it is Asha, is not just irrelevant as one might have thought, it's outright harmful: the correct conclusion should be that we must now prefer HL, but using our total, detailed knowledge (we caught big Asha) leads to no preference for HL. And so, Epstein argues, RTE, which says using our total knowledge trumps using partial knowledge, is false. I will give you my take in a second, but what do you think of his example so far?
RTE survives because there is a critical problem with Epstein's analysis. The problem is that he fails to give an adequate justification to his "Master Equation", which is the cornerstone of his argument:
HS0. All the fish are small.
Now run through the story again: we catch a big fish, rightly conclude that HS0 is false, then name the fish Asha. Are we now going to get in trouble with RTE because of M, i.e. P(ES|HL) = P(ES|HS0)?
Of course not, M is not just unjustified, it's clearly not true! The left-hand side of that equation is non-zero, but the right-hand side is clearly zero (there would be no chance of catching Asha, the big girl that she is, if the lake had only small fish!).
So maybe, you might say, the Master Equation is false for the case of HS0, but true for any other possible HS. It's not, but, importantly, we don't even need to prove that. Remember, it's Epstein who is arguing that the Fish example contradicts RTE, and his argument has no teeth until and unless he proves the Master Equation. All we need to do for a successful objection is to show that he hasn't done that, that it only seems like he has proven it.
Critical thinking PSA. This is of course a general point - to show that an argument is bad we don't need to show that any premises or conclusion are false. The argument can be terrible even if all premises and the conclusion are true. For instance: (1) my black cat gave birth to different color kittens; (2) therefore the theory of evolution is true. To show that is a silly argument we don't need to disprove (1) or (2), both could easily be true. We just need to show that one of the steps lacks justification, in this case going from (1) to (2).
If we clearly understand the exact reason Epstein's justification for the Master Equation fails for the special case of HS0, we will know why it fails for any HS. To do that, let's look at his justification while paying attention to which part goes wrong for HS0:
Were you more likely to catch Asha if there were mostly large fish, rather than mostly small fish? It seems clear that you were equally likely to catch Asha on either hypothesis: whatever the size of the other fish, the likelihood that your net would happen upon the specific large fish Asha was exactly the same. Asha was swimming along, and happened to be in the wrong place at the wrong time.
Let's deconstruct this, he seems to be arguing:
P1. The chance of catching a fish swimming in the lake is independent of the other fish being large or small. And therefore
M. P(ES|HL) = P(ES|HS), which says we're equally likely to catch Asha if HL or HS.
P1 is certainly true in the story, Epstein basically assumes that if there are say 100 fish the chance to catch any one of them is 1%, which is fine. And M might seem like it's saying the same thing as P1, so where is the problem with the logic? The problem is that, despite appearances, M is very different from P1, and does not automatically follow from P1 at all. To show that they are different, we can look at a case where the difference is especially stark.
As you might have guessed that case is HS0, with for instance 100 total fish in either the HL or HS0 scenario. P1 here is true, for any fish in the lake the chance of being caught is 1%, independent of the sizes of the other fish. But, as we mentioned before, M is false, since the right-hand side of the equation is just zero. And therefore the logical link between P1 and M is broken, and Epstein couldn't claim that simply stating P1 establishes M. Even if he were to say M is true for other versions of HS, he would still need to provide the missing link from P1 to M. To summarize then, here is a compact form of my objection:
Objection to Fish. There is a fundamental difference between P1 and M, and the step of going from one to the other lacks justification.
We are strictly speaking done responding to Epstein's specific argument, but it might be helpful to go one step further. We can see how RTE can be applied correctly. We have already set up most of the pieces, so this won't be too hard.
Sample scenario.
To get a nice numerical result, let's consider this scenario:
- HS = 100 fish, 5 big
- HL = 100 fish, 90 big
- EG = the fish we caught is big (general evidence)
- ES = the fish we caught is big and is Asha (specific evidence)
- Bayes factor = P(ES|HL) / P(ES|HS)
To make sure we understand the connection to Epstein's claim, he was saying that the above Bayes factor, calculated based on the total evidence ES, is 1 (that's exactly what M says). We will do the calculation more carefully and see that it's actually greater than 1, thus supporting HL over HS as it should. We can do the calculation the easy way or the hard way.
RTE calculation the easy way.
This way is suggested by Alex Popescu, and is by far the simplest. We note that, on either hypothesis, the probability of ES (catching Asha) is equal to the probability of EG (catching some big fish) times the probability that the random big fish we pull out wuld turn out to be Asha:
P(ES|HL) = P(EG|HL) * P(ES|EG&HL) = 90/100 * blah1
P(ES|HS) = P(EG|HS) * P(ES|EG&HS) = 5/100 * blah2,
where blah1 represents the unknown probability that if we pull out a big fish (in the HL scenario) that big fish would be Asha, and blah2 is the same in the HS scenario. But the story assumes there is nothing special about Asha that would give us some additional information about the hypotheses based on any details about her apart from her being large. This means that the chance the big fish we caught would turn out to be Asha, while being unknown, is unaffected by whether HL or HS is true, which means
blah1 = blah2,
and they cancel out when we take the ratio of the above two equations:
Bayes factor = (90/100) / (5/100) = 90/5 = 18,
the exact same correct result we would get if we ignored ES and only used EG.
RTE calculation a la Epstein.
The final result must of course be independent of how we set up the calculation, but it is instructive to see what happens if we set it up along Epstein's lines of starting with P1. We note that, on either hypothesis, the probability of ES (catching Asha) is equal to the probability of 1% to be caught if she is in the lake times the probability that she would be in the lake at all:
P(ES|HL) = 1% * P(AL|HL)
P(ES|HS) = 1% * P(AL|HS), where
P(AL|HS) = probability Asha would be in the lake in the first place in the HS case, and the same for HL.
That last factor is responsible for why the probability is zero in the case of HS0. I hope this gives you the intuition that this factor is important in other versions of HS as well. Indeed, assuming again that we don't think Asha is special, we should think that more big fish in the lake means more chance that one of them would be specifically Asha. Then, taking the ratio of the above two equations, we get:
Bayes factor = P(AL|HL) / P(AL|HS) = 90 / 5 = 18,
same as before, and RTE is happy.
Objection 1. But we already know that Asha was definitely in the lake, because we caught her! So P(AL|HL) = P(AL|HS) = 100%.
A. This would commit the somewhat common error of thinking that the probability of an event is 100% if we know the event happened. The same error would be committed if we said: EG (the general statement that a large fish was caught) is already known to be true, therefore P(EG|HL) = P(EG|HS) = 100%.
Objection 2. This comes from Epstein's paper, on page 16 he writes:
The above line of thought [that more big fish means higher chance one of them would turn out to be Asha] relies on the claim that there is a probabilistic link between HL and Asha’s being in the lake: according to the argument just rehearsed, Asha is more likely to be in the lake if the lake contains mostly large fish. But there is simply no reason to believe in such a link. Since Asha’s essence is, presumably, tied to her specific genetic origins, it’s not as if she could have been born to different large-fish parents. So the presence of any large fish not in her direct lineage does not raise the probability of her being in the lake.
A. This is a surprisingly self-defeating objection. It presents a perfectly kosher account of what it means to be Asha in the first place (basically, having a specific DNA). But then it completely undermines itself - it admits that having more big fish unrelated to Asha would not affect the chance Asha would come to be in the lake, but having fish in her direct lineage would. If the lake has only few big fish, what are the chances two of them would have the right DNA to be Asha's parents? The bigger the big fish population in the lake is, the higher the chances two of them would happen to have the right DNA, and the more children they have (affecting the overall number of big fish) the higher the chance one of them will have just the right DNA to be Asha.
Here is a helpful analogy. Instead of fish think of Rubik's cubes, and instead of DNA think of the precise way each cube is scrambled. Asha's DNA corresponds to some specific configuration A of a cube. Now if all you know is that lake L has 90 randomly scrambled cubes in it, and lake S has only 5 cubes, clearly lake L would have a higher chance of having a cube with configuration A.
54 Comments - Go to bottom
Hi Dmitriy,
ReplyDeleteI don't think I agree with your analysis. Once you catch Asha, HS0 is not on the table. It is true that catching Asha disconfirms HS0, simply because it is incompatible with it. But I don't see that the same is true for HS1 or any other version of HS.
So I still think we can conclude
P(ES|HS) = P(ES|HL) = 1%
Meanwhile, I don't think "P(A|HS) = probability Asha would be in the lake in the first place in the HS case." is well defined at all, for reasons discussed in the conclusion of the paper. Asha is not some pre-existing fish we can pick out from a space of possible fish. If Asha is not in the lake, then Asha doesn't exist in the first place.
It's open to you to stick to your guns and insist on these points, as I have done. But I wonder do you at least acknowledge that your argument here is less convincing than in the case of Fred?
> P(ES|HS) = P(ES|HL) = 1%
DeleteBut that would imply that the hypotheses in question are (wlog suppose the two possibilities have exactly 2 fish).
HS. Asha and one small fish
HL. Asha and another large fish
Is that the scenario you are considering? If so, Epstein is in an even bigger trouble :)
I don't follow. I don't understand either where you're getting these scenarios from or why they would be trouble for Epstein.
DeleteIn Epstein's scenario, we don't know how many fish are in the lake. You can take HS to be <50% of the fish in the lake are "large". You can take HL to be >50% of the fish in the lake are "large".
To make the math and logic easier, we can, without loss of generality, consider a simple case. You said:
ReplyDelete"So I still think we can conclude
P(ES|HS) = P(ES|HL) = 1%"
So you are considering a scenario in which the lake has exactly 100 fish and, in either HL or HS, one of the fish is guaranteed to be Asha, correct? If not, then the equation you gave doesn't hold.
Well, P(ES|HS) = P(ES|HL).
DeleteI grant that we don't actually know it is 1% unless we know the number of fish in the lake, which we don't. I'm not sure if that matters. If you want to assume we know there are 100 fish in the lake, where does that take us?
I also don't think it makes sense to assume that one of the fish is guaranteed to be Asha. ES is just the evidence of whichever specific fish we catch. We only come to know Asha after Asha is caught.
To clarify, we can say ES means we caught Asha retrospectively. Not prospectively. Once we come to know that Asha has been caught, of course Asha is guaranteed to have been one of the fish in the lake.
DeleteWe can carefully do the math, just like with the Fred atom, and the simpler we make the setup the easier and more intuitive it will be. So do you think, for example, that his argument applies to the case:
ReplyDeleteHS. 100 fish, 5 are big
HL. 100 fish, 90 are big
It's not the same as his setup (where the number of fish are not known), and there is the risk that as you make it simpler and easier to analyse you'll be introducing assumptions the paper's original argument doesn't depend on and which are required to defeat the argument.
DeleteBut in any case it's not immediately obvious to me that the original Fish argument is inapplicable to these assumptions. It seems you can proceed. I reserve the right to backtrack if it becomes clear that knowing the number of fish makes a difference.
I'm also unclear why you're restricting it to 5 being big or 90 big. What if 50 are big? The original argument is either the fish are mostly big or they're not. It would be more analogous if you went < 50 are big or >= 50 are big.
But also I just want to say that while it seems like it makes sense that you ask me to accept some premises before you lay out your argument, there is a cost, in that it makes me paranoid and I start worrying about every little thing that could be wrong with the premises. It's easier to see whether I should accept the premises when I see what you're going to do with them. That may seem backward, but often it's the conclusions that tease apart subtleties that may at first be hidden. For me, for instance, TER at first seems reasonable, but arguments like Fish make it clear that I shouldn't accept it. So typically my answer to "Do you accept these premises?" wil be "I don't see anything obviously wrong with them but I don't know yet".
DeleteI look at it this way: I want to question his justification for the premise
ReplyDelete(*) P(ES|HS) = P(ES|HL)
The justification he gives is
"It seems clear that you were equally likely to catch Asha on either hypothesis: whatever the size of the other fish, the likelihood that your net would happen upon the specific large fish Asha was exactly the same. Asha was swimming along, and happened to be in the wrong place at the wrong time. "
It seems to me that if this justification is sound, it is in particular sound for the case of 100 fish, and either 90 or 5 being big. So I want to show why it is not actually sound for this case. Once we understand what is wrong with the justification here, I think we will understand the problem in the general case.
Let L(x) = fish x is among the 100 fish in the lake,
E(x) = fish x is caught
a = Asha
So ES = E(a)
What Epstein says, to justify his premise, is that (since Asha is not special)
(1) for any x P(E(x) | L(x) & HL) = P(E(x) | L(x) & HS)
I don't have a problem with (1). For example, we can assume that the chance of catching any fish in the lake is 1 / [number of fish] = 1 / 100 = 1%.
But now,
P(E(x) | HL) = P(E(x) | L(x) & HL) * P(L(x) | HL) = 1% * P(L(x) | HL),
and the same for HS. Then Epstein's premise (*) is true iff
(**) P(L(a) | HL) = P(L(a) | HS)
But unless we think there is something special about Asha (which Epstein obviously doesn't assume in his story), that equation is not justified. Why? Because if Asha is not special, then we presumably believe that no matter what big fish x we caught the same would be true about it:
(***) P(L(x) | HL) = P(L(x) | HS)
This last equation is false. Let B be all the possible configurations of particles that would qualify as a big fish that we could catch. Then the average number of big fish in the HL scenario is given by
90 = = [sum or integral over all x in B] P(L(x) | HL),
and the same for HS:
5 = = [sum or integral over all x in B] P(L(x) | HS)
But that means that if we sum over x in B in equation (***) we will get
90=5
So that disproves (***), and therefore (**), and therefore (*).
It's a clever argument but I think it gets dubious as soon as you start talking about L(x) and B. Neither L(x) nor B are very well defined. For instance it's not clear whether two very nearly identical configurations of particles should count as different, or whether the same configuration of particles instantiated twice should count as the same fish twice or two different fish.
DeleteEpstein discusses these problems in the conclusion, where he says there may be more to the fish being Asha than being in some set of possible fish. Part of what makes that fish Asha is its history, which includes being in the lake. It doesn't make sense to talk of the probability that it might have been in the lake or not in the lake. We don't have a concept of Asha at all until we catch Asha, and you're pretending that we do.
I'll concede that the line you are taking is defensible, but I hardly think it a knockdown argument and I ultimately don't find it persuasive, personally. I'm curious as to how strongly you feel that I'm obliged to accept it. Do you cede that there are grounds for disagreement or do you think I'm obviously wrong?
I agree that the vagueness of Asha's condition prevents us from asking what is the likelihood of her having been in the lake in the first place. We lack the ability to tell whether she was more likely to have appeared in HL or HS, for perhaps she actually prefers lakes that are HS for example. But notice this just reduces to a case involving irrelevant evidence; a case we know TER can handle.
ReplyDeleteThis objection seems to be in a similar style to others, which suppose that our adopting TER must somehow mean that we have to reason only on A (Asha) and discard B (some big fish). But of course if A entails B, we get B too, and so we can see that reasoning on the total evidence doesn't modify the Bayes Factor.
P(B|HL) * P(A| B & HL)= High * Irrelevant
P(B|HS) * P(A|B * HS) = Low * Irrelevant
We still remain with High/Low.
One might object that Asha being vague means we don't know her reference class; so she could potentially be of a reference class which would modify the Bayes Factor. But I think we have pretty strong grounds to think that no matter how Asha is possibly defined, the Bayes factor must remain unchanged, unless her particular information was somehow relevant. For example, if we found out that Asha *only swam in HS, then of course that would change things, but the point is that we would be right to modify our answer on the basis of that information!
Dmitriy showed that TER is upheld in cases like Fred where the reference class was constant between the two hypothesis (one N), and where it wasn't (two N's: N1 & N2). And if we purposefully want Asha to be not well-defined, as the author seems to prefer by writing that we are supposed to rely on ostensive definition (as opposed to a rigid designation); then we see that Asha is just irrelevant.
I feel this is no different from the Fred case "blocking" our inference because Fred supposedly makes M=S; similarly here, Asha supposedly makes L=S.
"P(B|HL) * P(A| B & HL)= High * Irrelevant
DeleteP(B|HS) * P(A|B * HS) = Low * Irrelevant"
That's another great way to understand the problem with Epstein's claim. If there is nothing special about Asha in relation to HL or HS, then the second factors are the same. The chance that some big fish we are presented with would be Asha has nothing to do with HL or HS, precisely by assumption that her specific identity is irrelevant.
And if that chance were somehow dependent on the scenario, then learning her identity absolutely would and should modify our credences beyond the conclusion we would come to from just the fact that we caught some big fish!
In this second case, TER would not only not fail, it would be absolutely critical to getting the right answer.
I would say you are not obliged to accept the exact account with B etc, but you're I think obliged to accept this:
ReplyDeleteEpstein claims that his example demonstrates that TER is false. But to demonstrate his conclusion he would have to demonstrate the crucial premise (*). He hasn't provided any proof of (*). All he does is basically just state (1). But until there is a clear demonstration of how (*) follows from (1), or from something else, he hasn't made his case.
I am sure you agree that for an objection to an argument to be successful, it's not necessary to prove one of the premises false, it's enough to show that no adequate justification for it has been provided. So the objection was actually complete by just pointing out:
"Yes, it's true that the chance of catching any fish in the lake doesn't depend on whether the other fish are big or small, but that's not enough for the equation! Why not? Because, as is evident in HS0, that true fact doesn't justify concluding that we had an equal chance of catching specifically Asha in the HL and HS scenarios. "
The rest, actively disproving the equation, is just icing on the cake.
Hi Alex & Dmitriy,
ReplyDelete> But of course if A entails B, we get B too, and so we can see that reasoning on the total evidence doesn't modify the Bayes Factor.
But that is exactly what I was doing in my Bayesian argument. I observed this universe, decided that the specific universe didn't matter, and proceeded with "some universe", because "this universe" entails "some universe". So we appear to be disagreeing on whether the specific identity of this universe matters even while we agree that the specific identity of the fish does not.
> The chance that some big fish we are presented with would be Asha has nothing to do with HL or HS, precisely by assumption that her specific identity is irrelevant.
And I would say that the chance that some universe we would be presented with would be our specific universe has nothing to do with M or S, precisely by the assumption that our specific identity is irrelevant.
First, if you accept that statement about the fish then Alex's explanation goes through and RTE is safe.
DeleteMore importantly, about your last paragraph, let's look carefully.
B = some big fish is presented to us
A = big Asha is presented to us
P(A | B & M) = P(A | B & S)
For the universe,
B = some life-permitting universe is presented to us
A = our specific life-permitting universe is presented to us
Here the exact same is true as in the fish case:
P(A | B & M) = P(A | B & S),
because yes, our specific identity is irrelevant in this sense.
What is not true however is this other statement, which you expressed before:
P(A | O & M) = P(A | O & S), where
O = there is at least one life-permitting universe
The parallel with the fish case holds, because there the above equation is equally untrue, where
O = there is at least one big fish
Hi Dmitriy,
ReplyDelete> But to demonstrate his conclusion he would have to demonstrate the crucial premise (*). He hasn't provided any proof of (*). All he does is basically just state (1).
He offers the following justification of (*):
> But one thing that seems completely irrelevant is the size of the other fish around her.
And I agree: to me it seems completely irrelevant. I'm more inclined to grant this assumption than I am to accept TER.
There's no real proof of TER either. In both cases, the acceptance of these ideas is dependent on intuition.
As a practical matter, it seems sensible not to interpret TER as insisting that we should always reason from specific evidence rather than general evidence where the specific evidence doesn't seem all that relevant. While I respect that you think you can make the specific evidence consistent with the conclusions form the generic evidence, (by bringing in notions of the set of all possible fish etc), it's certainly a lot simpler to just use the generic evidence in this case. So we all ought to agree that a lot of the time it makes sense to reason from generic evidence, TER notwithstanding. We only disagree about whether the multiverse case is one of these.
My goal here is just to establish a basis for seeing this question as contentious. I think reasonable people can disagree. If we can agree on that then I think that's enough for now. The reason I want to make this point is I think your ensemble argument oversimplifies the issues. But there's more to come on that point.
Hi DM,
Delete******
> But to demonstrate his conclusion he would have to demonstrate the crucial premise (*). He hasn't provided any proof of (*). All he does is basically just state (1).
He offers the following justification of (*):
> But one thing that seems completely irrelevant is the size of the other fish around her.
And I agree: to me it seems completely irrelevant.
******
I think you are missing the point of my charge, because I also agree! But as I said, that's just to state (1), which everybody agrees with. He hasn't justified going from (1) to (*), simply stating (1) doesn't accomplish that task.
Moreover, there clearly are cases where (1) is true but (*) is unambiguously false, such as HS0. So, even if one wants to exclude HS0 from consideration, it should be clear that we can't just presuppose that in other cases (1) immediately implies (*), we are owed a clear justification for that step.
Hi Dmitriy,
ReplyDelete> First, if you accept that statement about the fish then Alex's explanation goes through and RTE is safe.
I don't see how, if you mean that Alex is respecting TER when he uses B (generic) instead of A (specific) just because A entails B. Because that seems to me like just what I did in the multiverse case, and I was accused of not respecting TER for so doing.
> A = our specific life-permitting universe is presented to us
> O = there is at least one life-permitting universe
We basically disagree here on which is the relevant question. A entails O after all, so if Alex can use B just because A entails B then I can use O just because A entails O.
I think that A is not the relevant question, because it presupposes the observed outcome (that we exist), thereby making itself post hoc. It seems as irrelevant to me as that the fish was Asha and in exactly the same way. The paper discusses this in the section "A Second Reference Class Problem", where he says the question is "whether I should classify my evidence as falling into the reference class observation of life-sustaining universe by me or rather into the broader class observation of life-sustaining universe by someone"
> But as I said, that's just to state (1), which everybody agrees with. He hasn't justified going from (1) to (*)
I don't see that as stating (1). I have a problem with (1) because it relies on the assumption that there is a defined probability for a fish to be in the lake, L(x). Rather, if the other fish in the lake are irrelevant, then to me that means that (*) holds.
>Moreover, there clearly are cases where (1) is true but (*) is unambiguously false, such as HS0
But HS0 is not the same as HS. I don't think you can prove a point about HS by proving a point about HS0.
But again, it seems to me that we just have clashing intuitions on this, so we're not likely to agree or get any further with it. I'm only trying to establish that it's not clear cut and there is at least space for rational disagreement. Can we agree on that and then perhaps leave it there, or do you think there is more that's worth exploring?
DM,
Delete"I don't see how, if you mean that Alex is respecting TER when he uses B (generic) instead of A (specific) just because A entails B. Because that seems to me like just what I did in the multiverse case, and I was accused of not respecting TER for so doing.
> A = our specific life-permitting universe is presented to us
> O = there is at least one life-permitting universe
We basically disagree here on which is the relevant question. A entails O after all, so if Alex can use B just because A entails B then I can use O just because A entails O."
This is a misunderstanding of what I'm actually doing. In the above analysis it can be seen that I'm reasoning on the basis of both A and B; that's what it means to respect TER. I'm not reasoning on B *instead of A. I specifically incorporated A into the analysis, found that it introduces a Bayes factor of 1/1 (exactly as the author said it should), and hence deemed it (correctly) irrelevant to the ultimate conclusion.
In the multiverse case, no one is claiming that you are violating TER because you use O, we are saying that you are disrespecting TER because you don't *also use A. From before, I did the exact same analysis as the above and found that reasoning on both A and O is the same as just reasoning on A (which makes sense of course because A entails O).
Hi guys,
ReplyDeleteI did a major overhaul of the article, I hope it is clearer and more complete now, and it answers an objection from Epstein.
Hey Dmitriy,
DeleteI like it. Also, you give me far too much credit; I was simply repeating the exact same type of reasoning you showed me first (a habit I’ve picked up more than once!).
:)
When put that way, so that the Bayes factor is determined by the total evidence; it seems obvious that Asha (and Fred and all the other examples) can’t make a difference.To be fair (and I also fell for this), expressing the nature of the specific evidence in English makes it seem as if we are required to adopt that extra condition *instead of the conditions required by the generic piece of evidence.
The trick is to realize that taking into account the total evidence is to ask multiple questions (“what is the likelihood that Asha would have happened” along with “what is the likelihood of a big fish” etc...). But that’s not easy to see; I think it is helpful to realize that reasoning just on the evidence that is Asha (which entails big fish) is not the same as asking just “what is the likelihood of Asha being the case”. Even though they sound very similar.
Hi Dmitriy,
DeleteTook some time to go through this.
On the easy calculation, I think there's a mistake.
blah1 and blah2 are not unknown.
blah1 is P(ES|EG&HL) or the "probability that if we pull out a big fish (in the HL scenario) that big fish would be Asha".
You're supposing that there are 90 big fish, and you're also supposing that we've pulled out one of these big fish at random (as EG is a given). Under these conditions, the chance that the fish is Asha is 1/90.
Similarly, blah2 is 1/5.
Therefore:
P(ES|HL) = P(EG|HL) * P(ES|EG&HL) = 90/100 * 1/90 = 1/100
P(ES|HS) = P(EG|HS) * P(ES|EG&HS) = 5/100 * 1/5 = 1/00
And the Bayes Factor is one, which is what Epstein would want to show.
DM,
Delete“ blah1 is P(ES|EG&HL) or the "probability that if we pull out a big fish (in the HL scenario) that big fish would be Asha".”
That’s not ES; ES is the probability that a big fish will have been caught AND that Asha will have been caught. It’s not just the latter conjunct.
The probability that Asha will be caught knowing just that a big fish has been caught and that there are a certain number of big fish in the lake (HL or HS), is not determinable.
That’s because we have no idea how many Asha’s there could be in such a lake. Maybe every fish is named Asha, or maybe only 1 in a million fish are named Asha. Unless we have a reason for thinking that there are more big fish than small fish named Asha; we can’t prefer HL or HS on the basis of such evidence.
Oops, seems I misread the definitions (I was relying on my earlier memories). Sorry about that! In any case, the probability of Asha, or ES, given EG and HL is not 1/90, because EG and HL are insufficient knowledge to tell us how many fish named Asha there are (or whether there is a numerical discrepancy between big fish and small fish named Asha).
DeleteOr another way to put this (my apologies for any silly mistakes thus far; I’m tired); our knowing that the big fish is named Asha isn’t increased on HL over HS. For all we know, Asha could prefer to swim in only HL. In which case HS would entail a 0 probability of Asha.
DeleteIt’s that preference between HS and HL (being unknown) which is the relevant factor, and not whether there could be multiple fish named Asha. Provided we are asking what is the probability of this Asha being caught, and not just any generic Asha.
The only way it would be 1/5 in HS for Asha is if we knew ahead of time that Asha had to be in HS (and the same applies for HL).
On the more difficult calculation, I would say that P(AL|HS) or P(AL|HL) is undefined and therefore illegitimate. We only know of Asha's existence after the fact. I don't buy that we can talk of the probability of her being in the lake before.
ReplyDeleteAlso, you've misunderstood one point. Epstein never refers to Asha's DNA. Epstein mentions her genetic origins. He's not talking here about DNA, he's using an older meaning of the word and talking about where she comes from as in meaning 2 from the OED: https://www.lexico.com/definition/genetic
> Relating to origin, or arising from a common origin.
> ‘the genetic relations between languages’
What he means is that it is implausible to talk about the probability of Asha being in the lake as if she were plucked from a Platonic realm of possible fish. Each fish is in the lake because of a particular history of events. That might include each fish having the right genes, but there's more to it than that. Arguably, a fish in another possible world with the exact same genes as Asha but different parents would not be Asha. Just as two humans with the same genes (identical twins) are not the same human.
Hi Alex,
ReplyDeleteDisregarding your first comment as you seem to think you're mistaken. Apologies if there was anything in there you think merits a response.
The rest of your comments concern the idea that we're trying to work out the probabilities of fish being named Asha. I think this is a misunderstanding. There are no fish named Asha before we catch a fish. We catch a fish and give it a name Asha only after we catch it. Asha is used to refer to whichever fish we catch and only that fish. This is not some fact we learn after we catch the fish, it's a matter of definition.
> For all we know, Asha could prefer to swim in only HL
This seems to run counter to the supposition that there is nothing special about Asha. Quoting the article: "Indeed, assuming again that we don't think Asha is special". So we're assuming that Asha is just some big fish chosen randomly from the population of fish.
It also assumes that Asha has some choice about being in the lake or not, which is like saying that we have some choice about being in this universe or not.
> The only way it would be 1/5 in HS for Asha is if we knew ahead of time that Asha had to be in HS
Ahead of time, we have not named Asha. Ahead of time, we can only refer to "the fish we will catch". We know ahead of time that the fish we will catch is in the lake. Given that we catch a large fish, and given that there are five large fish in the lake, the chances that we catch any specific one of these, including Asha is 1/5.
You can disagree with this by talking about the chances that Asha was in the lake in the first place, as Dmitriy does in the second calculation. But as I mention I don't buy it because I don't think we can meaningfully talk about the probability that Asha is in the lake before we know of Asha's existence.
Hey DM,
ReplyDeleteAbout your first objection:
"Similarly, blah2 is 1/5."
You are confusing two different conditional probabilities:
1. Given HS, what's the probability to catch Asha?
2. Given HS and AL, what's the probability to catch her?
It's the latter one that's equal to 1/5. By claiming that they are the same, you are basically asserting that P(AL | HS) = 1, which is answered by objection 1 in the article.
Hi Dmitriy,
DeleteCheck again.
blah 1 is not "Given HS, what's the probability to catch Asha?"
blah1 is "Given HS and EG, what's the probability to catch Asha?"
EG is critical here.
Asha just designates whichever fish we catch. I claim (in my discussion of the follow up point) that it is meaningless to talk about AL, the probability that Asha is in the lake.
So blah1, according to me, is "Given there are 5 large fish in the lake, and given that we catch a large fish, the probability that we catch the specific one of these large fish which we catch is 1/5", because we already know that the specific fish we catch must be in the lake or we couldn't catch it.
DM,
ReplyDeleteWhat Dmitriy said :)
Also if
“ Asha just designates whichever fish we catch”
Then the probability of Asha is going to be 1/1 no matter what. If Asha just designates any big fish we catch on the other hand, then it’s still 1/1 because we have EG.
You are right, I made a mistake in specifying the two conditional probabilities. But otherwise my response is exactly the same, so I will just put the correct version below this comment.
ReplyDeleteAbout your first objection:
ReplyDelete"Similarly, blah2 is 1/5."
You are confusing two different conditional probabilities:
1. Given HS and EG, what's the probability to catch Asha?
2. Given HS and EG and AL, what's the probability to catch her?
It's the latter one that's equal to 1/5. By claiming that they are the same, you are basically asserting that P(AL | HS) = 1 (or that P(AL | HS&EG) = 1), which is answered by objection 1 in the article.
It seems like DM is trying to say that we are given a high/certain AL because we know ahead of time that, if HL or HS, then Asha must be in (HL or HS), and we know this because we were planning on designating any fish we caught ‘Asha’.
DeleteHowever, this now means that our probability of catching Asha (given that any caught big fish is named Asha) is 1 on both HS and HL, and so the Bayes factor is unmodified.
Hi Dmitriy,
DeleteI basically reject talk of AL. Unlike the probability of EG, which you bring up by analogy in your discussion of Objection 1, I say it isn't meaningful to talk about P(AL) in advance of naming Asha. P(AL) is 1 because we can only talk of AL after we have named Asha.
Hi Alex,
> However, this now means that our probability of catching Asha (given that any caught big fish is named Asha) is 1 on both HS and HL
No, because ES and AL are not the same thing. ES is the prior probability of catching whichever fish we catch. AL is the probabilty that Asha was in the lake, and is necessarily a posterior probabilty because Asha is only defined after we catch the fish. AL is 1 and ES is 1/N where N is the number of fish in the lake.
DM,
ReplyDeleteIf there is no legitimate prior probability for Asha being caught then P(ES | EG & (HL v HS)) is just blah like we said it was. That’s because the question is specifically asking, what is the chance we would see Asha given that we caught a big fish if (HS or HL)? You’re saying that’s a meaningless question to ask prior to our catching Asha; therefore it can’t be 1/5 or 1/90.
I think it doesn't enter into it prior to our catching Asha.
DeleteP(ES | EG & HL) is not meaningless.
It means the probability of catching some specific large fish given that we catch one large fish and there are 90 large fish. It's the same as the probability of picking a winning raffle ticket from a hat containing 90 possible raffle tickets where there is just one winner.
P(ES | EG & HL & AL) is meaningless. AL here would be the probability that a specific raffle ticket would be in the hat in the first place. I don't think that's a normal sort of question to ask.
DM,
Delete“ It means the probability of catching some specific large fish given that we catch one large fish and there are 90 large fish.”
This only follows if we designate our specific fish (or our winning ticket in the raffle) prior to catching our fish. The whole reason that one ticket is the winning ticket is because we designated it prior to picking our ticket from the raffle.
It would be meaningless to ask what is the probability of our picking the winning ticket if we only designate the winning ticket post facto (because every ticket is potentially designated as winning).
You can’t simultaneously believe that “ Asha just designates whichever fish we catch. I claim (in my discussion of the follow up point) that it is meaningless to talk about AL”
And also that the P(ES) is determined by the probability that we will catch a specifically designated fish! Either the fish is designated beforehand, in which it isn’t meaningless to talk about AL, or it is designated afterwards, in which case P(ES) can’t be the probability of a specific fish in HL or HS. Because the fish hasn’t been designated yet, we can’t possibly know that it must be in either!
Without our designating a specific fish beforehand, there is no specific fish to look for in ES. Therefore ES just reduces to any possible specific big fish, which is just equal to EG.
DeleteHey DM,
ReplyDeleteOne possibly helpful question would be asking yourself, “what is the function of the adjective specific”?
Normally of course such an adjective is used to designate a unique fish, but you claim that we are not supposed to designate Asha beforehand. Therefore it seems that ‘specific’ is a superfluous adjective, that functions in a manner similar to something like this: “will be designated after being caught”.
But this applies to any big fish! So we would just end up with a 90/90 or a 5/5 probability.
“what is the function of the adjective specific”?
DeleteAs opposed to generic.
What is the chance of picking a specific raffle ticket from N raffle tickets? 1/N
What is the chance of picking a generic raffle ticket from N raffle tickets? 1
What is the chance of a specific raffle ticket being in a set of N raffle tickets in the first place? Meaningless.
When we're asking what is the probability of ES, we're asking something like "What would be the probability of picking a specific fish from the set of fish in the lake if we knew in advance what fish were in the lake". Naturally, AL would be 1 if we knew what fish were in the lake, but that doesn't mean that ES would be 1.
I think that framing it this way is fair game, because Dmitriy is making assumptions about the fish in the lake, e.g. there are 100 fish in the lake and the number of large fish is either 90 or 5. If we can make these assumptions then why not assume Asha is in the lake?
Before we catch a fish, we know nothing about the fish in the lake. Let's say there are N fish and B of them are big. But we know we are going to catch a fish, and we know that whatever fish we catch must be in the lake. So we know that the probability of catching a generic fish is N/N=1, the probability of catching a generic big fish is B/N, the probability of picking any specific fish is 1/N, and any specific big fish is 1/B given that we catch a big fish.
But the question "What are the chances that the specific fish we are going to catch would have been in the lake in the first place" is as meaningless as "how long is a piece of string?"
Or at least I say so. I think reasonable people could disagree. I just doubt that it's a sensible question as the probability is undefined.
DM,
DeleteI’m still not sure how you envision the method by which the word ‘specific’ quantifies over the set of fish.
Let’s assume we have three tickets for simplicity (A & B & C). The P(A) is 1/3 and so are P(B) & P(C). But are all tickets specific under your definition? If A and B and C all get to qualify as ‘specific tickets’ then the chance that some specific ticket will be picked P(ES) is 1. If say, only A was our specific ticket, then the chance is 1/3, or equivalent to our 1/N case. But then of course that’s exactly the same as asking about the probability of Asha being the case, knowing that Asha must be in the lake in question.
So it can’t both be true that:
“ So we know that... the probability of picking any specific fish is 1/N”
and that
Asking about the probability of Asha being in the lake is meaningless.
Either “some specific fish will be caught” refers to an individual fish x (in which case knowing the probability of Asha or fish x being in the lake is not meaningless) or it means any possible ‘specific’ (but really generic) fish, in which case the probabilities are not 1/N.
So since you agree that the probabilities of ES are 1/N, do you then concede that ES must refer to a single individual fish? In other words, P(ES) is just the probability that this ‘specific fish x’ in question was caught.
DeleteHi DM,
ReplyDelete"
No, because ES and AL are not the same thing. ES is the prior probability of catching whichever fish we catch. AL is the probabilty that Asha was in the lake, and is necessarily a posterior probabilty because Asha is only defined after we catch the fish. AL is 1 and ES is 1/N where N is the number of fish in the lake.
"
There seem to be some misconceptions revealed by this passage, which I think are partially responsible for the disagreement you have with the analysis. ES is not a prior probability, and AL is not a posterior probability. They are both events, or statements. We need to not confuse events and probabilities/credences of those events. The words prior and posterior refer to probabilities, not to events. So,
ES = "Asha is caught"
P(ES) = probability of Asha being caught
P(ES|EG & HL) = probability of Asha being caught, if the statements EG and HL are true,
etc.
Similarly,
AL = "Asha is in the lake" (ES implies AL, but EG doesn't)
P(AL|EG & HL) = probability of Asha being in the lake, if the statements EG and HL are true (this probability is not 1 because EG doesn't imply AL)
P(AL|ES & HL) = probability of Asha being in the lake, if the statements ES and HL are true (this probability is 1 because ES implies AL)
The fact remains that we can only define the event AL after meeting Asha. We cannot even express AL before meeting Asha, so talk of a prior probability for P(AL) is nonsense.
DeleteI agree that Epstein did not really talk about his assumptions regarding Asha being in the lake, but to clarify, I think that P(ES) should be interpreted as the probability of whichever fish we end up catching being selected randomly from whatever population of fish happen to be in the lake. Note that I didn't say "having been caught". To understand what I mean, you can imagine trying to define the probability of throwing the fish back in the lake and trying to catch it again on the next attempt, on the assumption that the probability of catching the same fish a second time is a good proxy for the intrinsic probability it would have had on the first occasion.
On this framing, the probability of Asha being in the lake in the first place doesn't enter into it.
DM,
ReplyDeleteSince you write
“ probability of ES, we're asking something like "What would be the probability of picking a specific fish from the set of fish in the lake if we knew in advance what fish were in the lake". Naturally, AL would be 1 if we knew what fish were in the lake, but that doesn't mean that ES would be 1.”
Do you now concede that your results of 1/5 are based on asking
P(ES | EG & HS & AL)? If so, how do you meet objection 1?
I'm not very happy to concede that, since I think all talk of AL is misguided. AL can only be defined once we know the fish is in the lake. If not true by definition, it's true-by-definition-adjacent, not something that can be true or false.
DeleteI already answered objection 1.
I said:
"Unlike the probability of EG, which you bring up by analogy in your discussion of Objection 1, I say it isn't meaningful to talk about P(AL) in advance of naming Asha. P(AL) is 1 because we can only talk of AL after we have named Asha."
So Dmitriy's response to objection one is to make an analogy to EG, and I pointed out the disanalogy.
To elaborate, I would say in response to:
"This would commit the somewhat common error of thinking that the probability of an event is 100% if we know the event happened."
that this is not what I'm doing. It's not just that we know the even happened.
The probability of AL is 100% just because we can't even express or define AL unless AL is true. Asha has to be in the lake because the only way we can define Asha is if we find Asha in the lake.
Hi DM,
ReplyDeleteyour response to my answer to objection 1 is:
""Unlike the probability of EG, which you bring up by analogy in your discussion of Objection 1, I say it isn't meaningful to talk about P(AL) in advance of naming Asha. P(AL) is 1 because we can only talk of AL after we have named Asha.""
I think that's a misunderstanding of the whole framework of probability theory. Perhaps the raffle example you brought up can help.
Suppose I put 100 raffle tickets in a bowl, each with a unique number written on it. Call a number big if it starts with a 9. Suppose you know I chose between
HL = there are 90 big tickets in the bowl out of 100 total
HS = there are 5 big tickets
You reach out and grab one at random, it's 9270. You call
Asha = 9270
Define
ES= E(9270) = you got 9270
EG = you got 9***
B(x) = ticket with number x was put in the bowl
AB = B(Asha)
So would you now say the same thing about AB as you did about AL:
"" I say it isn't meaningful to talk about P(AB) in advance of naming Asha. P(AB) is 1 because we can only talk of AB after we have named Asha."?
I hope you see what I am getting at. You wouldn't say it's meaningless to talk about the event B(x) for x=8562 or 9270, or that the probability of any of those events, conditioned on HS, is one, or anything like that. These are all perfectly legitimte events, B(x) is as legitimate as E(x).
DeleteIf you can talk about B(x) then you can talk about AB, since it's just defined as B(9270). It's probability is of course unclear, but it's most certainly not 1.
Hi Dmitriy,
DeleteIf you intend to specify that:
* all raffle tickets have 4 digits
* and are drawn from 10,000 integers from 0000 - 9999
* and there are no duplicate raffle tickets
* and are selected at random for inclusion in the bowl
Then and only then B(9270) is well defined, because you've specified a well-defined probability distribution for tickets and included 9270 as a possible result from a set of finite results.
I don't accept that this is the case for fish.
Hey DM,
ReplyDeleteI can answer that, but note that this is a separate worry from
"I say it isn't meaningful to talk about P(AL) in advance of naming Asha. P(AL) is 1 because we can only talk of AL after we have named Asha.""
My AB example demonstrated that it is not a problem to talk about the P of an event (AB) in advance of naming it
I don't see it as a separate worry.
DeleteIf we specify the possible tickets in advance, we can pick out 9270 in advance and talk about the possibility of 9270 being in the bowl. These are the circumstances under which we can talk about P(9270) in advance of its being picked. It's already effectively been "named" as a possibility by implicit inclusion in a well-defined set of possible tickets. It has a sort of abstract existence independently of whether it is actually in the bowl or not.
But Asha does not. There is no well-defined finite set of possible fish. There is no sense in which Asha has been named as a possibility before Asha is caught. This is why we can't talk about P(AL) before catching Asha.
Hi Dmitriy,
ReplyDeleteTo sum up where I am on this...
Your argument using AL does not seem crazy to me, but neither does it seem particularly compelling. The fact that it relies on undefined probabilities makes it unattractive to me, especially since we can get the right answer by just using EG and ignoring ES.
So I am convinced that for many pragmatic purposes, we can use generic evidence and ignore specific evidence when it seems like the specific evidence has no relevance. We are not required to bring in all sorts of ideas about the space of possible fish or the probability of a possible fish being in the lake in the first place to justify ignoring ES, we can just ignore it without worrying about it.
This means that at least from a pragmatic point of view, TER isn't quite true for many cases (even if it is possible to jump through all sorts of hoops to make TER fit even those case).
This is enough to make me skeptical about adopting TER for cases like anthropic reasoning.
But I want to clarify that I'm much less certain that rejecting TER is the correct approach than I am on my position in the other thread.
My credences on some positions (roughly -- it's possible that some rigorous analysis could show that these precise values are incompatible):
1. TER is wrong: 60%
2. The mere fact that we exist at all doesn't give us reason to increase our credence in a multiverse: 70%
3. Fine tuning gives us reason to increase our credence in a multiverse: 90%
4. The ensemble analysis as presented on the other article is inapplicable to these questions: 99.99999%
One thing that may be confusing is that it would seem that point 1 entails points 2 and 3 and vice versa, so they should have the same credences. But this assumes that there is no way to account for the specific evidence and use without changing our credences from that of the generic evidence, as you are trying to do here in the case of Asha. I'm open to the idea that there is some way of doing that, so the credences are not the same.
Given that I don't think your analysis of Asha is either obviously wrong or compelling, is there more to say on this topic? Do you think you can make it more compelling than you already have?
Hi DM,
ReplyDelete***
Your argument using AL does not seem crazy to me, but neither does it seem particularly compelling. The fact that it relies on undefined probabilities makes it unattractive to me, especially since we can get the right answer by just using EG and ignoring ES.
So I am convinced that for many pragmatic purposes, we can use generic evidence and ignore specific evidence when it seems like the specific evidence has no relevance
***
I think the discussion in the other thread about the validity of the ensemble analysis is the more important one now, especially given that you are 99.999 percent sure I am making a mistake there, but let me just very quickly make a few points about the worries you expressed here. I won't expand too much since we probably want to focus on the other discussion. There actually seem to be 3 separate worries.
The first one concerns the fact that we attach the name to an event, Asha, and the question is: can we talk about the probability of Asha prior to us naming her. My example with AL addressed that specific concern.
The second worry is about the fact that this specific event we want to talk about doesn't belong to a well defined finite set of events. Of course the fact that it's not finite should definitely not be a problem, but what about the fact that it's not well defined? That's not a problem too, in fact we have such a situation pretty much always in real life, when we are not discussing some very exact mathematical formulation. The only thing that means is that it might be hard to calculate the probability, but that's completely normal in real life. It doesn't mean it's meaningless to say something like: this was extremely unlikely to happen. Not being able to calculate something is very different from that something being meaningless, or from it not actually having some objective value.
Finally, the third worry might be that the event in question is not precisely specified. You might say: the ticket having the number 9270 is a precise specification, but the event of catching this particular fish is not a precise specification. But actually there's no fundamental difference when it comes to events in the real world, as opposed to in the math land. In either case we imagine that there is some sort of set of possibilities that are compatible with the event in question. For example, a ticket with the number 9270 and a smudge in one corner is compatible. What about a torn ticket that used to be 9270, but now it's ripped in half so you can only see the first two digits? When we do calculations, in other words when we translate real world events into some sort of mathematical representation, we don't actually ever specify all the conditions. Basically what we do is we just imagine that there is some sort of set of possibilities that are compatible without the need to actually 100% specify this set.
Anyway, all of this would probably require a much longer discussion, which we probably don't want to have at this point. Note however that the second and third worries don't seem to be worries for Epstein, because he uses ES, which is subject to those two worries.
Finally, the part I quoted above the seems to indicate some misunderstanding of RTE, or at least my definition of it in this article. My definition is perfectly happy with ignoring specific evidence if the specific evidence has no relevance. It only says that if specific evidence gives a different answer than generic evidence, then the answer based on strictly more information trumps that based on less information.
Post a Comment