I took a look at the author instructions for the Journal of Scientific Exploration and this passage about the review process is key. I've bolded what I think is a critical component of keeping the review process fair:

The idea is that if the analysis is scientifically sound, but there are some differences of opinion, the referee would have the option to contribute a small response in which he/she responds to the points or interpretations that he/she disagrees with.

I've really only looked at a couple of articles in this journal. I think David Pratt had an article about plate tectonics on their list of free example articles. Actually you might find it interesting ExpErdMan because I believe he was challenging Plate tectonics. But there doesn't seem to be a lot of astronomy related work at that journal.

Note added: One of the founding members of the Journal for Scientific Exploration is Y. Terzian who has about 268 hits on an ADS abstract search and appears to publish very mainstream work.

I recall that the now defunct journal "Speculations in Science and Technology" had a similar procedure. I think they even went further sometimes and let the author also reply to the critical reviewer! We sorely need more of this kind of give-and-take in the process.

It was too bad when Spec Sci Tech went down. They used to have lots of interesting articles on diverse topics. The editors explained that they ceased because they were just too tired dealing with all the papers opposed to Special Relativity that they were receiving, and that the original aims of the journal were being served elsewhere (like where, I wonder!) But now one does not see pointed critiques of SR very often, and SR looks more and more like untouchable terrain. That does not help science much.

I think the Pratt article is on the web somewhere, as well as a letter by Arp in response to it. Will check it again.

Special relativity has a LARGE amount of evidence behind it. Plus if one is going to overthrow SR, they better have a firm grasp of electrodynamics and its experimental verification. It one thing to express an opinion that scientists hold SR on a pedestal but it would take a large number of contrary experiments to knock it off that pedestal. I imagine that the publishers just got tried of trying to explain that declaring your theory better than Einstein's without any evidence or rigid theory to back it up is a wasted effort.

It is always a good idea to have a FIRM grip on the prevalent theory before you start to overturn it. Without that you not only fail to know it strengths but also its weakness.

Sure SR has evidence going for it, but it also has major weaknesses too. And there are a number of variations of relativity out there, such as Lorentz's version (involving a preferred frame), which can explain things at least as well as SR. (Some of these other versions are being discussed in other threads at the moment). The fact remains that mainstream journals have closed the door on this, and this is certainly not in the best interests of science.

Editors receiving papers about new theories of relativity may naturally tend to cringe. Almost always the papers have nothing to them. The trick is to try to have an open mind for that one special paper, and the courage to face down your colleagues.

Whoa, I knew this was going to happen. I go away for half a day and find myself woefully behind. 29+ replies!! All full of great stuff! Thanks for all these replies! This thread needs to be peer reviewed, so I don?t drown in information!

Thanks to Nukular for pointing out the big discoveries in nuclear science in the past fifty years - lacking a knowledge of history I have no way of knowing how peer review might have facilitated or impeded adoption of the new models, but I thought nukular?s argument was that without peer review they might never have come to light. The review of great discoveries does not in and of itself illuminate the effect of peer review specifically... or maybe I?m not seeing something.

Actually, since these great discoveries might have been lost in the clutter (the clutter that peer review clears away), perhaps in this way peer review does indeed facilitate the introduction of (meritorious) new thought (some of the time). Thanks to Nukular for pointing this out also.

I also appreciate Tim Thompson?s reply that whatever deleterious effects peer review might have in suppressing needed fundamental change, they are very likely time delimited. That certainly rings very true... My only answer to this is the possibility that the more fundamental the change needed, the more it will be resisted and the longer it will take. Old scientists might die, but they have younger proteges to carry on (the house of cards).

As for Tim Thompson?s comment that researchers jobs are not on the line, I beg to differ. If you?ve been doing work for years and years on cosmic deuterium abundances, and how nucleogenesis or inflation explains this or that aspect of it, and someone comes along and proves the big bang never happened... I think you?d suddenly find a need for business elsewhere. Wouldn?t you?!

I suppose there would be a transition period, in which you might try to fight the new theory with your own work... sort of like Ptolemy continuing to perfect his epicircles in an attempt to account for the precession in Mercury?s orbit (Uh, I guess I?m skipping paradigms there, but just to make a point! And Ptolemy was hardly a contemporary of Einstein... or even of Copernicus! Maybe he didn?t even come up with epicircles!!).

In the meantime, I did some of my own reading on peer review, stimulated by the comments here. I remember there being a recent paper analysing (rigorously) the benefits vs. possible demerits of the current system, but I am moving my office currently and the pertinent journal where I read this is in a box somewhere. My search on the web for the this particular paper did not bear fruit, but I did find this:

from: http://www.wikipedia.org/wiki/Peer_review

Sociologists of science have analysed the way in which the system of peer review works in practice and have advanced the criticism that it is vulnerable to capture by an elite. Experiments have been performed which suggest that peer reviewers tend to report, or abstain from reporting, logically identical errors in papers submitted for peer review accordingly as the paper's conclusions contradict, or substantiate, the conclusions of the reviewer's own research.

Unfortunately this page did not offer a reference!

This same page notes that Physics has been a field relatively tolerant to new thought - where reviewers enjoy seeing research fail or succeed in the ?free market? as it where, rather than judge (and possibly reject) papers with apparently ridiculous results or conclusions.

To recap what I?ve learned from you all so far: you reinforce in me the sense that peer review has definite value (and I believed that to be true basically from the outset). But it is at the margins where it becomes problematic - and I?m glad to hear some assenting voices here. Perhaps I should whittle down my thesis a bit.

On the assumption that the greatest and most fundamental changes in science are found ?on the margins? (beyond which we find, by definition, bad science or pseudo-science), to what extent is peer review delaying needed change? If peer review is a filter, to weed out bad science, how can it distinguish between bad science and simply science that is ahead of its time?

I don?t think it can, by its very nature. Can any other institution?

I acknowledge there may not be a good answer, and there may be no way to find a good answer.

Boris Starosta


ps sorry about these stupid typos, i can't stand typing in my browser, but the characters I am generating in a text editor are not coming over into the browser properly? Where on this BB is there an explanation on how to use the various buttons in text entry (particularly an easy way to incorporate quotations of other posters. I can't seem to make that quote button work very well). :-?

I'm not surprised that research on the matter finds this. In the process of those 4 reviews my paper went through I had a referee criticize one aspect of the paper because of calibrator sample size. I had split the traditional calibrators into 3 subsamples (actually calculated 3 different zero points for the relation) in order to account for a systematic effect that is normally ignored. Turns out that very referee had only several years earlier published a paper with a calibrator sample size even less than my smallest subsample and only 1/3 the size of my largest subsample. :roll: To make it more ironic even those calibrators in the referee's paper indicated the presence of the systematic effect I was trying to correct for!

Another referee criticized one way I presented my results which was a method the referee of my earlier paper had said would be the best way to present it! #-o

Boris,

If you can read Russian, try reading this piece from 40 years ago - http://www.univer.omsk.su/omsk/Sci/Bartini/s2.htm . Then ask yourself if it could ever pass a peer review, mind also that the author was not a professional physisist...
(The paper was literally forced into print by the author's high-ranked friends without any peer-reviewing and later had a profound influence on physical science.)

I know it is enormously dangerous to generalise, but I would say that the most important thing to remember about peer reviewing is that it’s ever so much like REAL LIFE – a mixture of the careful and diligent, but more often messy, erratic, slovenly, self-interested, lazy, rushed, and chaotic.

Now when you see that big book on astronomy at the bookstore, you know that it was not actually written by the famous name astronomer whose name appears on the cover. He was just paid $50,000 to write the foreword and attend a couple of meetings of an editorial board. So what makes you think that the editor of a journal is really the editor? He may be, of course. But he may not actually put in any more hours than necessary. Someone else may be at least half running the show for him – especially if the editor is a really big name.

And what about the reviewers. These are most likely drawn from the editorial or advisory board members, international representatives, etc, whose names appear on the inside cover of the journal, thus indicating that they are “somebodies”. They crave this recognition, but they pay for it by being called upon to serve as reviewers. They do not enjoy reading papers and writing reviews. It is a chore they do not at all relish, but must sometimes submit to. They find reasons to excuse themselves. They explain that the paper is not quite in their line. Their PA says they are abroad. They bury manuscripts in a draw and forget about them.

So how are reviewers for a particular paper selected? One could read the manuscript and decide who could do the best job. But the quickest method is to read the list of references, and to cross-check the names against the reviewer list. If you find a match, that’s the poor sod who is going to get landed with the job.

So remember that when you are writing a paper! If Samuelson is on the editorial board, and you reference a couple of his papers, Samuelson may be the guy providing the review. Well, now! Instead of “Samuelson (1993a) claims that …” you could try something more flattering. “This point is addressed very succinctly in Samuelson (1993a) …”

Sometimes, extracting reviews reaches hair-tearing levels. Something is needed fast! In this case, the choice of reviewer may come down to someone who owes you a favor. Then again, there are various sorts of bribes than can be offered – an invitation to that conference in the Seychelles, perhaps. Or there’s old so-and-so who is retired now, and not quite on the ball, but who has time on his hands.

Just remember that behind the smart letterheads and the formalities, REAL LIFE rules – in its customary chaotic and unreliable fashion.

(Caveat: None of the above should for a moment be assumed to arise from my own experiences.)

__________________
"True skepticism encompasses not dismissing evidence because it seems to defy rational explanation."

Sorry i don't read russian. I do read German, though... technical German can be a fun challenge, too.

Now I just noticed a new topic on faster than light gravity (Tom van Flandern's Meta Model / c-gravitons). I must rush over there and see what's going on!

Boris

PS agorabasta (edit -> mutineer) great review of Real Life. Indeed.

Seems that your comment should go to mutineer

Having just edited a paper after I got my peer-review from to referees I guess i should write a little here. I think that the peer-review as it is at the moment works pretty good. Indeed, sometimes there is the personal "vendetta" in which some authors get a nasty report (my ex-boss got one of those), but taken over the whole, all my experiences have been good. I work in space physics, by the way.

What mutineer wrote is a little black I think and specially about the referees that are chosen does not agree with my experience.

I know from the journals that I submit to, I need to give a list of 3 to 5 possible names of people who could be qualified to referee the paper. Naturally I could scrap a co-author in a nice deal that (s)he will write something nice about me and we will do the reverse the next time. But that does not help me. I rather have a very critical review of my paper than one that says, "very nice, A-, no comments" (even though that would be easier).

Another is that I know a lot of editors of journals do put in much work in getting the journal up to standards. Another ex-boss of mine (yeah I have had a few) was editor when the first wave of "small comets" papers appeared. He put a lot of effort in it, because he was not sure if it was right or not, and the referees were not either. Therefore, he put in work, setting up a discussion in the journal and made sure that all sides were viewed.

Next to that there is always the possibility that if you really don't like the review of your paper, you can ask the editor for another referee. Most of the time, if you can base your discontent clearly, this request will be granted.

Ah well, that is about all I wanted to say, I wish you all the best.

Martin

__________________
20 minutes into the future
This message is bra-bra-brought to you by z-z-z-zik zak
And-And-And I'm going to be back with you - on Network 23 after these real-real-real-really exciting messages
(Max Headroom)

Any comments in red are to be considered in ModeratorMode.