Looking for the best information about new products? Test it on your own farm
Is that shiny new product likely to work on your farm? To separate the heifer dust from the cream of the crop, farmers need to take a hard look at the research behind the marketing claims.
John Heard, soil fertility specialist with Manitoba Agriculture, says the best way to learn about good research “is to bruise your knuckles by becoming an on-farm tester.” Here are seven tips for conducting those tests and evaluating ag research.
1. Don't go it alone
Heard advises farmers to find an on-farm testing coach rather than trying to do it on their own.
“It’s really hard to have the discipline to do it on your own and to collect the needed information to make it research, not just a demonstration,” he says. That includes helping with in-season measurements, collecting data at harvest, and analyzing statistics.
Another plus is the coach can work with different growers, organize the same test at different sites, and pool the results. Pooling data from different environments bumps it to the next level of research, Heard says.
Farmers should look for a coach with a track record, Heard says. Ideally the coach should “understand the discipline of research. But also the intricacies of field-scale farm equipment.”
2. Know what you're working with
Troy Basaraba is a market development specialist with Bayer CropScience. This summer he ran field-scale fungicide trials testing early, ideal, and late applications of Folicur and Prosaro on AAC Brandon.
Basaraba’s trial includes seven treatments and an untreated check. Each treatment, along with the check, is duplicated within the same field. Bayer has been running field-scale fungicide trials, with cooperating farmers, for nine years.
Basaraba says farmers who want to run their own trials should learn about the product first so they fully understand it.
Heard compared getting a product to the on-farm testing stage to getting to the Olympics. First it has to go through the Olympic trials. “In the research world, you’d call that doing a literature search.”
Before testing something, it’s better to see if someone else has researched it, and whether “there’s a shadow of a chance” that it will work, Heard says. “I’ll save my on-farm tests for some legit things.”
A good source for research papers on agricultural products is http://extension.agron.iastate.edu/compendium/index.aspx.
3. Keep things simple and avoid distorting results
“The thing about field trials is that they can be very complex or they can be really simple,” says Basaraba. It’s much easier to reap reputable, consistent data from trials with fewer variables, he adds.
It’s also important to make sure every aspect is treated fairly in these trials, Basaraba says. “If we do this treatment at 10 gallons and another at 15, obviously we know there will be varied results.”
Field topography also comes into play. “So when you are in some of these areas that have rolling topography, then you have to have a second look with regards to where to place the trial in the field in order to minimize some of that variability,” says Basaraba.
Basaraba also looks for consistency in plant population, plant staging, and growth. “Obviously, it is not going to be perfect because we are in a commercial field and you will see some variations.”
Heard agrees that farmers should keep it simple. Creating complex field trials, with several treatments, for example, risks distorting results. Do that, and you’ll learn nothing from the trial, Heard says.
“Yes, we want to know more things that just what we’re testing. But that doesn’t mean that farmer needs to bear the burden of all of the wheat sector’s questions,” says Heard.
Small plot trials are the place to test several treatments. For example, Dr. Don Flaten with the University of Manitoba has been running small plot trials with 30 or more treatments on wheat, Heard says.
Researchers then cherry-picked a few of the most promising treatments for on-farm trials. One treatment included a base rate of nitrogen plus 30 lbs., and another included the base rate plus 60 lbs. Those two extra base rates require 12 strips per field, Heard says.
4. Replicate, replicate, replicate
A grower who does a test on one field in one year “should expect to be scoffed at the same way that a researcher is scoffed at” for submitting a paper with only one site, one year, says Heard. Researchers strive to have a minimum of two sites in two years, he added.
The more comparisons and strips, the better, says Basaraba. “I don’t think there is a need to go over four replications but if you can do two to four in the same field, that would be perfect.”
Basaraba explains that duplicating the strips within the same field helps make sure the land isn’t influencing results. “If the two untreated strips are consistent, we know that there is no land gradient or anything like that.”
Replication helps agronomists identify consistency, Heard says. If a product seems to boost yield by six bushels an acre on one farm, Heard wants to know the probability of it bumping yield across 10 or 20 farms, he says.
“There are lots of products out there that will work one out of four times, one out of five times,” says Heard.
Multiyear trials also reveal trends, Basaraba says, and the data becomes more predictable.
“As I have summarized the data for the past three years, it is actually a little eerie how the data from 2015 and 2014, very much line up with our long term numbers,” he says.
Heard says replicating treatments and analyzing stats can be very frustrating for farmers new to on-farm research. They might see, for example, a four bushel gain with a treatment, only to have the statistical analysis report it’s due to chance.
“After you’ve done a few of those, you become pretty ruthless with people if they stand up at meetings and talk about two, three bushel differences being real,” says Heard.
When looking at other research, Heard advised farmers to ask to see the stats. One trick to watch for is people cherry-picking their best sites. Look at the scale of the experiment and whether the differences are small, Heard says.
Basaraba also suggests farmers ask questions. “Look to see if there has been replication in the same field. That is the biggest thing to show the true consistency of a product.”
5. Report the failures
Successes are great, but it’s also important to report the times a product doesn’t work, says Heard.
One recent example is the work the Manitoba Pulse & Soybean Growers’ inoculant work. They evaluated whether there was an economic return in using in-furrow inoculants, in addition to seed-applied, in soybeans grown on ground that had seen at least two previous soybean crops. Results from 2013 and 2014 showed an economic return to the double inoculation only three out 17 times. The grower group is still analyzing 2015 results, according to the Manitoba Pulse website.
“They’ve taken heat over that to no end,” says Heard.
There’s an advantage to having a grower group leading some of this to find out “the true chances of things working or not,” Heard adds.
Part of the fun of research is figuring out how products might work under different conditions, Heard says. Right now Heard is looking at how to influence yield and protein levels in wheat through nitrogen application. He and his colleagues are also taking flag leaf samples for nitrogen, and using sensors, to flag fields that might benefit from a nitrogen application. That type of scouting could help farmers make better decisions, he says.
6. Watch for weasel words
When reviewing marketing claims, Heard advises farmers to watch for “weasel words.” One example is “tested by Agriculture Canada,” he says. It implies endorsement, but that doesn’t mean Agriculture Canada found any benefit to the product.
It’s also important to note the federal government doesn’t require efficacy testing anymore. That shifts the onus of proof onto the product manufacturer, Heard says.
7. Follow through
“The more that you can go back, take a walk through the field and look for visual difference the better,” says Basaraba. “Then of course it is best to take the trial all the way to yield.”
Basaraba says scales are better for measuring yield differences than yield monitors. And Heard agrees, saying weigh wagons are still the standard for on-farm research.
Heard hopes to be able to calibrate yield monitors so they’re reliable enough for research one day. In fact, this summer he and his colleagues looked at in-field calibration of yield monitors in their wheat studies. Once that calibration is done, they’ll do direct comparisons of the strips through weigh wagons and yield monitors to test the calibrated yield monitors.
As for Basaraba, he’s pleased with the on-farm fungicide trials have progressed over the years.
“The level of pride I have in regards to our program is unbelievable. It is all done field scale. It is all replicated. I think that it is about as close to real life as we can get.”