I am not a biologist nor a chemist, and maybe neither are you, but suppose we were. Now, if I described a procedure, in “standard” detail, to produce a result XYZ, you would (based on your reasonably expected expertise in the field), follow the steps you believe were described and either reproduce XYZ or, if I was mistaken, not be able to reproduce XYZ. This is called scientific reproducibility. It is cructial to what I believe is one of the fundamental principles of science, namely Popper’s Falsifiability Criterion.
More and more people are arguing, correctly in my opinion, that in the computational realm, in particular in mathematical research which uses computational experiments, that much higher standards are needed. The Ars Technica article linked above suggests that “it’s time for a major revision of the scientific method.” Also, Victoria Stodden argues one must “facilitate reproducibility. Without this data may be open, but will remain de facto in the ivory tower.” The argument basically is that to reproduce computational mathematical experiments is unreasonably difficult, requiring more that a reasonable expertise. These days, it may in fact (unfortunately) require purchasing very expensive software, or possessing of very sophisticated programming skills, accessibility to special hardware, or (worse) guessing parameters and programming procedures only hinted at by the researcher.
Hopefully, Sage can play the role of a standard bearer for such computational reproducibility. Sage is free, open source and there is a publically available server it runs on (sagenb.org).
What government agencies should require such reproducibility? In my opinion, all scientific funding agencies (NSF, etc) should follow these higher standards of computational accountability.
Here is a collection of some favorite quotes from scientists and writers. For more, see this post.
There are some things which cannot be learned quickly,
and time, which is all we have,
must be paid heavily for their acquiring.
They are the very simplest things,
and because it takes a man’s life to know them
the little new that each man gets from life
is very costly and the only heritage he has to leave.
(From A. E. Hotchner, Papa Hemingway, Random House, NY, 1966)
I believe that a scientist looking at nonscientific problems is just as dumb as the next guy.
The best thing for being sad is to learn something. That is the only thing that never fails. You may grow old and trembling in your anatomies, you may lie awake at night listening to the disorder of your veins, you may miss your only love, you may see the world about you devastated by evil lunatics, or know your honor trampled in the sewers of baser minds. There is only one thing for it then to learn. Learn why the world wags and what wags it. That is the only thing which the mind can never exhaust, never alienate, never be tortured by, never fear or distrust, and never dream of regretting.
T. H. White in The Once and Future King
Truth, like gold, is to be obtained not by its growth, but by washing away from it all that is not gold. Leo Tolstoy
Education is what survives when what has been learnt has been forgotten.
B. F. Skinner
The advantage is that mathematics is a field in which one’s blunders tend to show very clearly and can be corrected or erased with a stroke of the pencil. It is a field which has often been compared with chess, but differs from the latter in that it is only one’s best moments that count and not one’s worst. A single inattention may lose a chess game, whereas a single successful approach to a problem, among many which have been relegated to the wastebasket, will make a mathematician’s reputation.
Norbert Wiener, in Ex-Prodigy: My Childhood and Youth
Science is a differential equation. Religion is a boundary condition.
Everything is vague to a degree you do not realize till you have tried to make it precise.
For every complicated problem there is a solution that is simple, direct, understandable, and wrong.
H. L. Mencken
If people do not believe that mathematics is simple, it is only because they do not realize how complicated life is.
John Louis von Neumann
To be what we are, and to become what we are capable of becoming, is the only end in life.