The recent headline in the tech magazine Wired cut straight to the point.
“Data is the new oil of the digital economy,” it proclaimed.
Data is everywhere, it said, an immensely valuable and untapped resource that will drive the digital economy forward just as oil fuelled the industrial economy.
Vast fortunes are available for the taking for those smart and fast enough to exploit it, the article concluded.
For oil that would have meant wildcatters sinking wells. Today, that means mining enormous amounts of data that could potentially be turned into information. Today data is used to maximize profits, ensure smooth delivery of goods and services and to tease out hidden patterns that can reveal opportunities. If something can be reliably measured, goes the thinking, it can surely be improved upon.
By now we’ve all heard plenty about the promises of Big Data. But for most of us, it’s just a catchy buzz phrase we’ve heard thrown around.
But make no mistake, the trend is already affecting our lives.
Have you ever been on a trip and suddenly found you need to call your credit card company to assure them your card hasn’t been stolen? That’s an annoying but very real application of Big Data in the real world. Finance companies are using Big Data techniques to monitor transactions in real time and flag suspicious activity. So if you’ve never been to New York City before, your card might get flagged if it’s suddenly being used there.
Most of the popular social media such as Facebook are really just a disguised form of data gathering. The old saying in the tech industry is that if you’re not paying for a product “you are the product.” Mark Zuckerberg and company have become extremely wealthy by convincing us we should give them our information and allow them to track our every online move. Other tech titans like Google are similar.
Zuckerberg once made an unvarnished comment to a colleague that later became public. The colleague was asking how Zuckerberg and Facebook had managed to amass such an enormous catalogue of personal information. “People just submitted it. I don’t know why. They ‘trust me,’” he wrote, adding an off-colour remark about the intelligence of his users.
A lack of information infrastructure has muted the effect of this trend back on the farm. But it’s certainly coming if the hype is to be believed. Large agriculture companies have begun making investments in the data business, with an eye to getting in on the ground floor. They’ve no guarantee of success. But their interest is a sign of what may be to come.
One of the key problems they all seem to be grappling with is assembling the required data into something large enough to be meaningful, because for Big Data to work there needs to be, well, big data. In this context big is truly enormous. Taking the example of fraud detection a bit further, it relies on sophisticated algorithms that are applied to millions of transactions a day.
Compare that to the average farm and its data stream. The individual data is far from worthless, but it must be combined with more data from other farmers for it to become valuable in the sense Wired is talking about.
This leads to the million-dollar question no one has yet answered — who are you going to trust with your data? And if you do, what will you get out of it?
One facet locally is the Enterprise Machine Intelligence and Learning Initiative (EMILI) which earlier this year got a boost in the form of some federal funding. Machine learning is a subfield of computer science that gives the computers the ability to learn without being specifically programmed. It’s a type of artificial intelligence that builds computer programs that can change when exposed to new data.
This initiative promises to directly target the agriculture sector and aims to make the province into a world leader in the field. It’s a lofty goal, but worth pursuing — and sooner or later it’s going to need data to crunch.
Another intriguing model is found south of the international boundary, in the form of the Midwest Big Data Hub, at the University of Illinois, but also involving the University of Michigan, Iowa State, Indiana University and the University of North Dakota. It’s a data-collection effort that aims to centralize data on a regional basis for better decision-making in a number of fields, including agriculture.
An example that’s closer to home is the data collected from farmers by the crop insurance system that allows farmers to see how crops, management and inputs affect performance locally.
Just what form Big Data in agriculture will take in the future is as yet unknown — but soon a lot of folks are going to be interested in seeing your data.
The challenge facing farmers is figuring out how to manage their collective data in such a way that they capture its value.