For the past 20 years, open data has been rapidly changing the face of earth observation applications and innovations.
Twenty years ago (yikes!) Luther Vandross and Janet Jackson crooned ‘the best things in life are free’ and it seems that Barack Obama agreed. By the end of 2008, the entire archive of Landsat data became freely available to users.
In another milestone for the Landsat legacy, last week the USGS welcomed Landsat 9 data into its open data archive. So let’s check out what that legacy started in open data innovations and the geospatial industry more broadly.
That was then…
When I was an undergraduate student studying remote sensing in the late 1990’s, data were expensive. So just like our clothes in the days before fast fashion, we used and reused many times.
I remember every student in my class learning image processing by working on the same Landsat image subset. And in subsequent years when I became a tutor for the class, many more students also processed that same scene. No choice, no variety, that’s just all we had.
I soon learned that it cost $1500 to purchase one quarter of a Landsat scene. We always hoped that any new field site didn’t fall across multiple scenes!
And this is now…
Today the story is quite different. My students can select imagery from any time and place in the world. They begin learning the fundamentals of remote sensing in an ecosystem and location they are passionate about. They can even learn how their location has changed over the past 50 years thanks to the open Landsat archive.
We don’t need to place an order for a scene and then wait to receive it on a CD in the post. There are a variety of open data repositories at our fingertips where we can search for, discover, and immediately download the data. Or we can stream the data directly to local or cloud processing systems.
To use a covid-19 cliche, companies that previously on-sold Landsat data had to pivot. As they lost their income stream from selling raw data, they began to create more value-added products and services. The new trend of open data forced them to innovate to survive.
In 2017, the USGS valued the economic benefits of these innovations at $3.45 billion. So just because something is ‘free’ doesn’t mean that it can’t lead to an incredibly successful business model!
In reality, ‘free’ doesn’t even exist. Someone always has to pay. But that doesn’t mean it’s not worth it!
What is open data?
The terms ‘freely available’ and ‘open access’ are often used interchangeably. At the most basic level, they mean that the data are available to be used, reused, and modified by anyone, including for commercial purposes.
Those who create open data often reasonably request attribution. Beyond that, they are not remunerated for the data itself. However, there may be a cost involved in accessing APIs or value added retrieval services.
What are the benefits of open data?
Both suppliers and consumers gain benefits from open data. From a supplier perspective, open data can cut organizational costs by empowering the users to search, discover, and extract data for themselves. This helps to streamline their services and increase efficiencies. This type of ‘self serving’ ability is also great for users to interact with the process whenever and wherever they want.
Many forms of data have been captured for a specific purpose. But that doesn’t mean they can’t have a life beyond that purpose. Open data encourages innovation by allowing users to repurpose data and discover new applications. This is one of the most commonly cited benefits of open data.
Ever wondered if the outcomes of a study are data driven and legitimate? Open data forces accountability! Peer reviewed journals are increasingly encouraging scientists to also publish their data in open repositories. The public is also able to hold governments to account by independently analysing their data.
This is just getting started. With an estimated market size of €184 billion in Europe in 2019, open data is no small player in our economy.
Why do some people object to open data?
‘I haven’t finished with it yet, I’m still waiting for my Nature paper…’
Realistically, you know more about your data and its context than anyone else. So back yourself with the knowledge that you’ll find the gold nuggets before others ever will. Your insights are the most valuable component! Further, if the taxpayer has paid for your data, don’t you think they deserve to be able to access it?
‘It cost me a lot of money to collect, I don’t want to give it away for free’
But have you extracted all the value that you can from it? If so, why not let others have a try? I often think of this as being similar to making a decision about whether to sell a clothing item second hand, or donating it to Vinnies. If my preloved items no longer give me joy, then I hope they will bring something to someone else.
‘Someone else captured it for me so I don’t actually own the data’
There’s not much I can say about this one! Except well done for reading the license agreement 🙂
‘I don’t have the infrastructure or place to put the data’
You don’t actually have to manage your own infrastructure to make your data open. Many people take advantage of publishing through Github for example. And certainly if you’re capturing drone mapping data, most definitely check out GeoNadir. Neither of these options will cost you anything other than your time.
‘My data are sensitive or there are privacy concerns’
Is it possible to aggregate your data so that it is de-identified? Or perhaps consider an embargo time where the data can be released after a certain period of time.
Where can I find open earth observation data?
Let’s circle back to Landsat. It’s openly available from its source at the USGS, but that’s not the only place to find it. There are some seriously cool platforms where you can find Landsat and other earth observation data. Which one you choose will depend on your level of expertise and what you hope to achieve with it. Here are some great options:
- USGS Earth Explorer – awesome for downloading a whole range of data including Landsat, AVHRR, some commercial imagery, elevation data, and there’s even some USGS drone data there! Once you’ve downloaded the data, you’ll need to have your own software for processing it.
- Open Data Cube – platform to manage and analyse earth observation data including Landsat, Sentinel, MODIS. Get ready to work with Python and Jupyter notebooks
- Google Earth Engine – cloud based platform streaming a variety of earth observation data including Landsat, Sentinel, MODIS as well as derived products such as surface temperature, elevation, landcover. Super fast processing as long as you are comfortable with javascript
- European Space Agency’s Copernicus Open Access Hub – check out the interactive GUI to download Sentinel data or the API hub for cloud streaming services
- GeoNadir – If you’re into drone data, then this is the place to be. Data are contributed from people around the world and then built into orthomosaics on the platform.The repository is growing on a daily basis and you can interact as both a contributor as well as a user.
What’s in the future for open data?
So it’s true – there’s no such thing as free. Open data always has a cost associated with it, but in many cases, that cost has already been borne by us as taxpayers. When the US Government realised that in the mid to late 2000’s, the open Landsat archive opened a world of new opportunities. Now it’s not just governments contributing to the open data movement, but industry, private enterprise, and community groups all play a valuable role in the open data ecosystem. With distributed collection and decentralised control on the supplier side of the equation, we are sure to see many more innovations in open data and analysis. Maintaining collection standards and – most importantly – appropriate metadata will be key to ensuring we don’t go down the lines of ‘garbage in = garbage out’.
We use and benefit from open data on a daily basis. Whether that’s from looking at weather forecasts, determining the best travel route on google maps, or tracking covid infection rates, open data are everywhere. So if you appreciate using open data, perhaps also now consider how you can actively contribute to it.
Because of course we are always passively contributing to creating open data, but that’s another story!
