Science
1. Gathers data
2. Finds correlations
3. Deduces reasons

The question here is the relation between points 2 and 3. It seems wrong to say that finding correlations through induction involves building models, as that is a deductive task. So-called "statistical models" don't count, as they do nothing more than identify trends in data and have no deductive content.

The risk implied by the Wired paper is that the scientific focus on deduction rather than induction can lead scientists to overlook important new findings emerging from induction.

I think I would replace "Deduces reasons" with "Explains correlations using existing principles or organizes the observational data and creates new principles which explain the observations." The formation of fundamentally new principles, for instance Einstein's special and general relativity is not a deductive process, but rather a process involving insight and inspiration. Deduction is involved only after the principles have been formulated. For instance, the deepest part of special relativity is the formulation of the two assumptions that the laws of physics are the same in all inertial reference frames and that the speed of light is the same in all inertial reference frames. Only after those principles are stated does deduction come into play to derive the Lorentz transformation. But the hard part of the creation of the theory was the formulation of those two principles, which did not come about through deduction.

Even in mathematics, which is not really a science, the important aspect of the research is the inspiration and intuition that results in good guesses. Generally the process is that one first makes a good guess and then shows through deduction that the guess was right. This is often not apparent to people not directly involved in mathematical research because all that is published are the results and deductive proof of the validity of the guess. Along the way it is common to work on specific examples, somewhat equivalent to gathering data and finding correlations. Sometimes this is published and sometimes not, depending on how novel and interesting the examples are.

If this is meant to imply that scientists would be so busy working solely with existing theoretical models that they ignore insights from new experimental data or findings implied by that data, then I think I must disagree entirely. I cannot imagine anything that is farther from the real processes of scientific research or the scientific method.

I

Einstein once said: "It is theory which decides what we can observe." If we accept the important role of theory in deciding what observations should be made (as a means of testing the theory) then I don't think it is unreasonable to suggest that theory also decides what observations individual scientists will find "interesting" or "useful" and thus just by human nature certain experimental data will be ignored simply because it will not be "interesting".

__________________
"The scientist who asks the right question reconnoiters a new patch of the unknown, and may, with luck, bring it within the constricted but expanding boundaries of the known."

~Timothy Ferris (The Red Limit) 1982

While an individual scientist might have some bias that would result in some data being ignored, I don't think that he would get away with ignoring important and relevant data for very long. Someone else would evaluate that data and make any significant findings known. The field is too competitive for good data to remain ingnored.

If there is "data" or "theories" that are ignored by the community at large then there is probably a reason for that. ATM theories that are ignored are ignored for good reason, usually because they have been evaluated and found to be severly wanting once, so revisiting them over again is pointless.

Um, the steps above are how science sometimes progresses, but are not the scientific method. The scientific method is to create a hypothesis (by whatever means, including crunching of raw data in a computer) and then testing it.

Eventually tested observations can be used to create a theory that can be used for extrapolation, and those extrapolations tested. This theory can be mechanical, mathematical, purely statistical, etc. Theory can change and improve, i.e. at one point we can say

"fire burns me"
then
"Fire is hot, therefore it burns me"
and then on to the biochemical effects, etc.

The first is what you can get from crunching of data alone. It leads to a valid scientific conclusion. In short, this shows that what all this petabyte crunching is doing is doing science.

Which is a far cry from "science is dead".

Absolutely!

The most exciting data one can get is the stuff that DOES NOT agree with the theory. From such (after more testing) one proceeds to NEW THEORY.

Confirmation is good, new results are better.

I think Einstein meant that theory shows what is observable at all. This is not "deciding" what we should observe, but rather saying that we can observe some things, and not observe others, ever.

As had been said before, it's not the shout of "Eureka!" but the quiet "Hmm, that's odd" which introduced the true breakthough.

__________________
‘To those who regard “crime fiction” as some sacred icon which must follow a rigid formula, I will always be the man who writes 18-syllable haiku.’Trying to make sense of computers, The Error Log.

How true. In 26 years at Bell Labs in research, the "whoo hoo" brought smiles and applause. The "now THAT's interesting" would draw a crowd in 10 seconds.

Perhaps I melded two separate thoughts. Yes, I agree that what is consistent with Einstein's statement is the idea that any given theory will allow us to predict things that can and things that cannot be observed if the theory is a reasonably correct description whatever aspect of nature it is attempting to explain.

What I was suggesting is that scientists just as a matter of human nature - will tend to find results in the research literature that are consistent with the theory much more interesting than those that may be at odds with the theory in some way. There will also be more of a tendency not to look for things happening that are not supposed to happen according to the theory. Why would one waste time looking for something that can't happen when they can invest time trying to confirm the things that are expected to happen?

In general I would agree with this. However in some instances an ATM idea may be ignored not because observations have definitively ruled it out, but for the practical reason that definitive tests would require more investment in resources (eg telescope time) than is likely to be rewarded with useful results.

__________________
"The scientist who asks the right question reconnoiters a new patch of the unknown, and may, with luck, bring it within the constricted but expanding boundaries of the known."

~Timothy Ferris (The Red Limit) 1982

Of course, in doing observations, we get to test the theory that 'x' can not be observed, or 'y' can.

So, we do have a check on both the theory and whatever else is going on, as well.

This is again the basis of science, testing and confirmation (or sometimes not confirmation, too).

My own interpretation of what Einstein meant might be a bit different. I think he was commenting on the fact that if you do an experiment, you have to decide what is a "control", what is a "variable", and what is a "result". Theory tells you what to do with these concepts-- without some kind of model or theory to organize your thinking, how is any experiment even possible? Everyday life is like experiments without the support of theory-- and that's basically why we say that we just never learn. In the absence of theory, observations feel just like repeating the same mistakes.

The point from Einstein is valid - the theories of evolution and geology 'decide' that we cannot observe Permian fossils in Jurassic strata, and the theory of general relativity 'decides' that we cannot observe events inconsistent with it. However, the point about what scientists find interesting illustrates how existing deductive models can entrain thinking, leaving little space for considering inductive patterns which don't spring from the existing theoretical model. This is where the petabyte revolution is breaking ground with new methods of discovery - as we are seeing in genetics.

Aye, and if mah grandmother had wheels, she'd be a wagon.

It's an expansion of a very useful tool. I don't think anyone is arguing that. But it's not a "revolution", it's not a new method, and it doesn't mean that science is obsolete, which was the premise of the Wired article.

I had forgotten about this little thread, so apologies for the slow response

And how does such a correlation get tested? By the scientific method.

In cases where there are large data bases this can be useful. But not all sciences have large data bases. Not all data is readily reducible to searchable databases. Not all data in data bases are of equal quality. And all knowledge is not contained in data bases. So the technique has severe limits.

Not at all. We simply have one extra techique that can be applied to some problems. A technique is not the sum of the scientific method.

Jon

You have it completely backward. Theories of evolution and geology do not "decide" that you cannot observe Permian fossils in Jurassic strata. It is a basic observation that particular strata contain one set of fossils (e.g. Permian) and other, overlying strata contain others (e.g. Jurassic). Theories of geology and evolution (and they are quite different things BTW) are developed to explain the observationss

Jon

Yes - Robert you've in the bold above on the point I was trying to make.

__________________
"The scientist who asks the right question reconnoiters a new patch of the unknown, and may, with luck, bring it within the constricted but expanding boundaries of the known."

~Timothy Ferris (The Red Limit) 1982

Access to immense data sources does have a revolutionary potential to find highly unexpected correlations. The challenge is how well science and society will adapt to new findings. Climate science in particular is leading the way in presenting new and unexpected correlations, so of course the petabyte method is at the heart of the scientific hunger for data.

Thanks Jon, what I was trying to say arises from the view that science claims that its theories are true knowledge because they are powerfully confirmed by data, correlation and law. Theories in evolution and geology and astronomy have a status as natural law, so in a sense the theory does decide what can and cannot be observed. When theory has status as knowledge, it is not possible to observe things that do not conform with the theory. This recognition of the nature of scientific knowledge relates to the Wired contention that correlation has superseded causation through its demonstration that science only achieves this decisive power to rule possible obervations in and out when its theories have status as natural law. Einstein held that statements about general relativity have such a status as scientific knowledge. Correlation alone cannot show the laws of nature, as finding laws is a scientific task which requires synthetic analysis through deductive reasoning.

I would put it the other way round. The hunger for data is at the heart of the petabyte method.

If this were true then no theory would be overturned by contrary data. However many theories have been overturned by data contrary to theory. And it is possible to find Permian fossils in Jurassic strata. However to date this has been because of reworking, not because biostratigraphy does not work.

Jon

It is about what is classed as knowledge. There are claims which are so well proven that the theory supporting them cannot possibly be overturned by data. Claims which are vulnerable to being overturned by data cannot be classed as knowledge. This is about science being more confident to demarcate the distinction between knowledge and belief, with knowledge confined to facts and theories that are totally corroborated. I am not sure where this demarcation line sits, but knowledge has to include the large quantity of definite facts about the universe established by scientific method, as well as those laws and principles which explain observations with complete accuracy. If a theory can be overturned by data then it was not scientific knowledge in the first place, but rather a belief or hypothesis postulated in the absence of good data.

When a pamphlet was published entitled 100 Authors Against Einstein, Einstein retorted "If I were wrong, one would be enough." Einstein was not surprised that the Mercury perihelion measurement corroborated the theory of relativity because the logic of the theory was compelling. Explaining relativity was a great advance in scientific knowledge of the nature of the universe. The theory is decisive for what is observable.

And how is this relevant to not seeing Permian fossils in Jurassic strata?

Jon

Permian fossils are not ordinarily found in Jurassic strata. The truth of this statement is not merely my opinion, but is forced by the data, in a sense it is decided for us. Similar law-bound truth is found in astronomy. The relevance here is that data mining alone cannot explain the law which may underpin a correlation. In the case of geology, the theory of evolution has provided a coherent and elegant explanation which forces many hypotheses out of consideration. The core of the theory is not going to be overturned by data.

The potential is there, if compelling contrary evidence were found. It is unlikely such evidence exists, but nothing about the theory, or any theory, disallows the possibility. Regarding Einstein, we're still testing that. For example, Gravity Probe B.

__________________
I say there is an invisible elf in my backyard. How do you prove that I am wrong?

Disclaimer: Avatar is not an official NASA image and does not imply any specific interplanetary or interstellar capability.

The Leif Ericson Cruiser

Evolution was not relevant to the recognition that Permian fossils do not occur in the Jurassic. The Permian was defined in 1841, the Jurassic in 1795, The Origin was published in 1859.


Jon

Okay, but the issue is about current understanding, not the history of discovery. Permian life did not appear in the Jurassic because life had evolved. The theory of evolution, as a main explanatory connector for the fossil record, explains why it is impossible. Pre-Darwin opinion on fossils was still largely gathering data without a clear story to explain it. Not wanting to traduce Charles Lyell here, he could not see a mechanism for fossil change that could rebut the vigorous unscientific critics. Evolution functions as a decisive theory ruling in and out what can be found in each period.

"Knowledge" and "belief," however are not sacred statuses or properties of scientific theories. They are our informal assessments of them. We use the terms too loosely. Fortunately, we lose nothing because of this. We put scientific theories into practice not because they have the holy aura of "knowledge" around them, but because our lives improve as a result of doing so. We still use classical mechanics even though it has been surpassed by more general theories.

And that's what matters: we are better off with the theory than with something else.

Does evolution itself, then, require an underpinning, or can it stand on its own without a separate underpinning? My point here is that I notice a tendency to disqualify something because it does not have an underpinning (such as the claim that Permian fossils are not found in Jurassic strata), but the underpinnings, when supplied, seem not to be held to the same standard.

An relevant note is from a talk I just attended at the 2008 SDSS meeting on the ISW effect. At the end of the talk, the speaker noted that ISW is a very good probe of deviations from Lambda-CDM, but other tests are much better at specifying the piarameters of lambda-CDM. The conclusion is that ISW is vital, specifically because it can rule-out LCDM very strongly.

Note that measuring ISW is only possible with the large databases and fancy statistics that started this whole thread...

That strikes me as a very relevant example of the use of databases and statistical analysis as a tool in research. It is an excellent use of that tool. It is not an example of a displacement of the classic scientific method by data miining, or of replacement of deep physical theories by simple correlations.

Be careful and skeptical when you see the fancy statistics. Fancy statistics can be based on some pretty fancy assumptions.

Is it? The history of discovery shows clearly that the fossils succession was recongised well before the theory of evolution.

Actually we don't see Permian forms in the Jurassic because they were extinct by then.

There was a lot more to pre-Darwinian palaeonotlogy than simple data collection. There was a great deal of what might be called foresenic palaeontology, working out fossil behaviour, functional anatomy, taphonomy, and palaeoecology. As to explaining the fossil succession, while there was no one explanation there were a number of possibilities, organic evolution being but of one them.

What has Lyell got to do with it? Which unscientific theories was Lyell trying tyo rebut? You are aware that Lyell was deeply hostile to the idea of evolution for most of his life?

No it does not, at least in the way you see to imply. Evolution does not rule out Permian fossils in the Jurassic, extinction does. Because of evolution we do not expect to find Jurassic fossils in Permian strata, but that is a different story.

Furthermore it is observation, not theory, which plays the key role in saying what does, and does not occur in particular strata. Of course we might well find Jurassic fossils in the Permian and Permian fossils in the Jurassic, in some specific instances. The reasons we can find (and recongise them) is rarely anything to do with evolution. In these few cases where it might be it would be evolutionary theory that would have to be modified.

Jon