The data I posted that showed a sudden improvement in my arithmetic ability is among the most interesting data I’ve ever collected. Not because it revealed something wildly new — I was already sure flaxseed oil helped — but because it revealed something intriguing and new (the time course of the improvement is puzzling).
I collected the data in an unusual way — watchful waiting. I didn’t do an experiment, the way experimental psychology data is usually collected. I didn’t do a survey, the way epidemiological data is collected. In the emphasis on one person it resembles a case report in medical journals — but I didn’t have a problem to be solved and the data is far more numerical and systematic than the data in a case report.
And this rarely-used scientific method paid off. Hmm. I think the scientific methods currently taught have a big weakness: They focus almost entirely on idea testing, whereas idea generation is just as important. Tools that work well for idea testing work poorly for idea generation. The effect of this imbalance — a kind of nutritional deficiency in intellectual diet — is that scientists don’t do a good job of coming up with new ideas.
What should scientists be doing? I would like to find out. My watchful-waiting data collection is/was part of trying to find out. That it paid off pretty quickly is a good sign. It’s the third step in a long process. Step 1. When I was a grad student, my acne self-experimentat led me to realize that one of my prescribed medicines didn’t work — a surprising and useful new idea. Step 2: Later self-experiments had the same effect: Generated surprising and useful ideas. At a much higher rate than my conventional experiments. Why? Perhaps because it involves cheap frequent tests of something important. Step 3: Arrange such a situation — cheap frequent tests of something important — and see what happens.