Against "in theory"
Jun. 2nd, 2012 09:34 pm"In theory" is an often worthless and sometimes misleading phrase. There are situations where it makes sense, as I'll discuss in a moment. But the way I've seen it often used suggests not really understanding what theories are for.
The colloquial division into "practice" and "theory" suggests "theory" as some freestanding thing, coming from... well, I don't know where. But the right way to think of it is, there is reality, and there are models of reality. Obviously, the models are not always correct, but if they are wrong too often or by too much, we abandon them, because corresponding to reality is what we make them for. But note that there is not just one model, because models are typically limited in scope.
Let's focus on this last point first -- "in theory" often means "according to this model I'm using". Which is fine, except, well, which model are you using? State it, so we know what you're talking about! Simply labeling it "in theory" gives the misleading impression that there is just this one "theory" all predictions come from.
There are cases where this isn't a problem, because the model is clear from context. E.g., particle physics is all done with reference to the Standard Model, which, after all, is called that for a reason. Thus, if you make the statement "in theory, cosmic rays from sufficiently far away should have energies below the GZK limit", it's pretty clear that you mean that the Standard Model predicts this. So there's not really any problem with this -- although possibly you'd be encouraging the erroneous division that leads to the bad uses of "in theory".
But some uses go beyond mere inclarity as to which model is meant, into apparently forgetting what models are for. For instance, take "Why Three Prongs?", currently on the top of Hacker News. Great article, I definitely recommend reading it. But the use of of "in theory" right at the beginning is frustrating. What's meant is, "according to this naïve model I just cooked up myself". But the author then goes on to immediately show the flaws in this naïve model. Which is good -- this is how we work; we start from naïve models and then refine. The problem is he didn't say "according to the naïve model", he said "in theory". And nowhere later did he suggest that these later corrections are refinements to this theory; no, the only mention of "theory" is at the beginning, as if this is the one and only theory, and the improvements he discusses should not be incorporated into the theory. (Notice the wording: "The real world is a bit more complex than the theory". Not "The real world is a bit more complex than this theory", or "than the naïve theory", but "the theory".)
So many uses seem to fail to understand that if your model is wrong, you should fix it. Now maybe that's not feasible because you don't have enough information (as e.g. with the Standard Model), or that would complicate the model beyond the point of usefulness, but this comes back to the point of "you should specify which model you're talking about", not just act like it's immutable "theory".
Or you get silly statements like "Yeah, Communism works... in theory." (More of a sneer than a statment, I guess.) What does this even mean? Let's not for now tackle the question of what's meant by "Communism works", since that's not the point here; the point is it's a statement we can generally agree is false. If you don't agree with that, then substitute something analogous.
Does the speaker mean it's true according to well-accepted models? I certainly hope it's not, because any such model is evidently drastically wrong; it should be thrown out. Models don't have to always be right to be useful, but this one sounds like it's truly broken, unless it just really wasn't built for predicting the outcomes of economic systems. Maybe the statement sadly is true, in which case it is a relevant strike against such models... but the speaker don't seem to be phrasing it like that, doesn't seem to be specifying which models it is a strike against. They make it sound like a reason to distrust "theory", not particular models. Perhaps the whole field is pathological, filled with people making worthless theories they never test. When it comes to economics, many would even say that's the the case. But again, that's a strike against their particular models and methodologies, and the people using these, not against the general endeavour of model-building and studying economies, even if it turns out to be the case that literally all the work on the subject so far has to be thrown out.
Does the speaker mean it's true according to models that have since been discarded? Well, then why are they even making this statement? What a worthless thing to say. Do they mean it's true according to some other model they had in mind? Then they should specify which one! I think most likely what was meant was "according to its proponents' theory". That actually sounds like it was a legitimate sneer. But note that this is really just saying "according to its proponents' claims"; not only have they failed to specify which model they are talking about, but the issue is not even about the model, but about the credibility of the people claiming the model. Saying "in theory" only confuses the issue, once again.
Maybe they mean it works under conditions that don't actually hold in the real world, or that some model predicts that it does. That too actually seems like a legitimate sneer -- "Yeah, it works... just not under conditions that ever actually hold." But just labelling it "in theory" confuses the issue of reality vs. model, with the issue of reality vs. counterfactual. Now counterfactuals are often used as models; but if they're being used as models they should be judged as models, and one of the paragraphs above applies. If it's not being used as a model, and is just a counterfactual, then, well, it's not being used as a model and so the label of "in theory" doesn't really make sense.
Finally there's the use of "in theory" as an answer to a question about what could happen -- "In theory, it could happen that..." when the event in question is very unlikely to happen. But this isn't a question of reality vs. models, this is a question of possible vs. worth preparing for. Most of the time, when people ask, "Could X happen?", what's meant is "Should I prepare for the possibility of X?". So in fact, when people say "in practice, X won't happen", what's actually meant is, it might be the case that in reality X could happen, but even so it's not worth preparing for. So people appear to be talking about an epistemic issue, but really they're talking about an instrumental issue. And so this usage is very confused.
In short, the phrase "in theory" manages to conflate a number of different things and suggest a bad way of thinking about things. I would therefore strongly discourage using it outside of contexts where there is a "standard model", and perhaps you shouldn't use it even then.
Actually, I suppose simply using the word "heuristically" without specifying the heuristic could potentially also be problematic; and yet, in my experience, these sorts of problems have not cropped up with that. Perhaps because the people using it actually understand what heuristics are for, understand there really is more than one, and understand that they should be refined when possible. (Not "in theory it could be problematic but in practice it has not been", but "going by the above reasoning it could be problematic but in practice it has not been"! See, that's explicitly stating the model. Of course, I didn't explicitly state the model outside the parenthetical, and I think that made sense, because it seemed clear from context. But even so, labelling it "in theory" would not have been helpful.)
-Harry
The colloquial division into "practice" and "theory" suggests "theory" as some freestanding thing, coming from... well, I don't know where. But the right way to think of it is, there is reality, and there are models of reality. Obviously, the models are not always correct, but if they are wrong too often or by too much, we abandon them, because corresponding to reality is what we make them for. But note that there is not just one model, because models are typically limited in scope.
Let's focus on this last point first -- "in theory" often means "according to this model I'm using". Which is fine, except, well, which model are you using? State it, so we know what you're talking about! Simply labeling it "in theory" gives the misleading impression that there is just this one "theory" all predictions come from.
There are cases where this isn't a problem, because the model is clear from context. E.g., particle physics is all done with reference to the Standard Model, which, after all, is called that for a reason. Thus, if you make the statement "in theory, cosmic rays from sufficiently far away should have energies below the GZK limit", it's pretty clear that you mean that the Standard Model predicts this. So there's not really any problem with this -- although possibly you'd be encouraging the erroneous division that leads to the bad uses of "in theory".
But some uses go beyond mere inclarity as to which model is meant, into apparently forgetting what models are for. For instance, take "Why Three Prongs?", currently on the top of Hacker News. Great article, I definitely recommend reading it. But the use of of "in theory" right at the beginning is frustrating. What's meant is, "according to this naïve model I just cooked up myself". But the author then goes on to immediately show the flaws in this naïve model. Which is good -- this is how we work; we start from naïve models and then refine. The problem is he didn't say "according to the naïve model", he said "in theory". And nowhere later did he suggest that these later corrections are refinements to this theory; no, the only mention of "theory" is at the beginning, as if this is the one and only theory, and the improvements he discusses should not be incorporated into the theory. (Notice the wording: "The real world is a bit more complex than the theory". Not "The real world is a bit more complex than this theory", or "than the naïve theory", but "the theory".)
So many uses seem to fail to understand that if your model is wrong, you should fix it. Now maybe that's not feasible because you don't have enough information (as e.g. with the Standard Model), or that would complicate the model beyond the point of usefulness, but this comes back to the point of "you should specify which model you're talking about", not just act like it's immutable "theory".
Or you get silly statements like "Yeah, Communism works... in theory." (More of a sneer than a statment, I guess.) What does this even mean? Let's not for now tackle the question of what's meant by "Communism works", since that's not the point here; the point is it's a statement we can generally agree is false. If you don't agree with that, then substitute something analogous.
Does the speaker mean it's true according to well-accepted models? I certainly hope it's not, because any such model is evidently drastically wrong; it should be thrown out. Models don't have to always be right to be useful, but this one sounds like it's truly broken, unless it just really wasn't built for predicting the outcomes of economic systems. Maybe the statement sadly is true, in which case it is a relevant strike against such models... but the speaker don't seem to be phrasing it like that, doesn't seem to be specifying which models it is a strike against. They make it sound like a reason to distrust "theory", not particular models. Perhaps the whole field is pathological, filled with people making worthless theories they never test. When it comes to economics, many would even say that's the the case. But again, that's a strike against their particular models and methodologies, and the people using these, not against the general endeavour of model-building and studying economies, even if it turns out to be the case that literally all the work on the subject so far has to be thrown out.
Does the speaker mean it's true according to models that have since been discarded? Well, then why are they even making this statement? What a worthless thing to say. Do they mean it's true according to some other model they had in mind? Then they should specify which one! I think most likely what was meant was "according to its proponents' theory". That actually sounds like it was a legitimate sneer. But note that this is really just saying "according to its proponents' claims"; not only have they failed to specify which model they are talking about, but the issue is not even about the model, but about the credibility of the people claiming the model. Saying "in theory" only confuses the issue, once again.
Maybe they mean it works under conditions that don't actually hold in the real world, or that some model predicts that it does. That too actually seems like a legitimate sneer -- "Yeah, it works... just not under conditions that ever actually hold." But just labelling it "in theory" confuses the issue of reality vs. model, with the issue of reality vs. counterfactual. Now counterfactuals are often used as models; but if they're being used as models they should be judged as models, and one of the paragraphs above applies. If it's not being used as a model, and is just a counterfactual, then, well, it's not being used as a model and so the label of "in theory" doesn't really make sense.
Finally there's the use of "in theory" as an answer to a question about what could happen -- "In theory, it could happen that..." when the event in question is very unlikely to happen. But this isn't a question of reality vs. models, this is a question of possible vs. worth preparing for. Most of the time, when people ask, "Could X happen?", what's meant is "Should I prepare for the possibility of X?". So in fact, when people say "in practice, X won't happen", what's actually meant is, it might be the case that in reality X could happen, but even so it's not worth preparing for. So people appear to be talking about an epistemic issue, but really they're talking about an instrumental issue. And so this usage is very confused.
In short, the phrase "in theory" manages to conflate a number of different things and suggest a bad way of thinking about things. I would therefore strongly discourage using it outside of contexts where there is a "standard model", and perhaps you shouldn't use it even then.
Actually, I suppose simply using the word "heuristically" without specifying the heuristic could potentially also be problematic; and yet, in my experience, these sorts of problems have not cropped up with that. Perhaps because the people using it actually understand what heuristics are for, understand there really is more than one, and understand that they should be refined when possible. (Not "in theory it could be problematic but in practice it has not been", but "going by the above reasoning it could be problematic but in practice it has not been"! See, that's explicitly stating the model. Of course, I didn't explicitly state the model outside the parenthetical, and I think that made sense, because it seemed clear from context. But even so, labelling it "in theory" would not have been helpful.)
-Harry
