A standard image processing sequence has driven remote sensing and Earth observation applications for decades. This sequence is a framework of stages through which to progress and applies equally to drones as it does to more traditional satellite mapping. It takes us from defining the problem to communicating results and all the steps in between, and is particularly helpful for guiding newcomers in the industry.
Here we dive into the image processing sequence and what it means from a drone mapping perspective.
Contents
What are the stages of image processing?
While image processing is often an iterative process, there are eight key stages in the sequence as below. This is derived from the original remote sensing text by John Jensen.
Why defining the problem makes all the difference
In the army, I was taught ‘prior planning prevents poor performance’. And there’s no better place to employ that mindset than when seeking to solve environmental challenges.
So whether the challenge at hand is to determine the extent of coral mortality on the Great Barrier Reef, the size of a stockpile on a construction site, or the impact of a natural disaster we always need to start by clearly defining the problem. Here are some questions to answer as part of this process:
- How much detail will we need in our data?
- What area will we need to cover?
- How frequently?
- Is RGB enough? What are the other options?
- What accuracy is needed?
When you understand the fundamentals of the problem and keep these at the forefront of your mind, you will have the greatest chance of success in progressing through the image processing sequence.
At the end of the problem definition phase, you will also have matched your information requirements (e.g. detail, area to cover) with your ideal data characteristics.
The following articles might help guide you further:
- Required detail and area to cover
- Mapping with thermal or multispectral data
- Using ground control points (GCPs)
When you’re ready to capture data, check out this blog for the latest advice!
Pre-processing
We often say that processing drone data means to create the orthomosaic, digital surface model (DSM), and digital terrain model (DTM). However strictly speaking, this is pre-processing. When we take the original overlapping images and use photogrammetry to create these products, this only takes us to the third stage of the image processing sequence.
After the pre-processing phase, we might now consider that we have ‘analysis ready data’, and we are ready to progress. Note however that there may be other radiometric and geometric corrections to apply at this stage, particularly with thermal and multispectral data.
The image sequence below shows the outcome of pre-processing a dataset to orthomosaic, DSM, and DTM, thanks to data from David Rogers.
Relevant pre-processing tools on GeoNadir include: upload, GCPs, crop to polygon.
Image enhancement
Our data aren’t always easy to visually interpret, so that’s where contrast enhancements come in handy. For more about how enhancing contrast can help us see the unseen, and for some tips about how to perform it, read on here.
The below imagery from Micahelmas Cay by Johanna Karam (Queensland Department of Environment, Science, and Innovation) is over exposed and washed out. After contrast enhancement, the colours look more natural, helping us to detect features with more confidence.
While the contrast enhancements don’t alter pixel values in our data, we can perform other numeric enhancements that do. The Normalized Difference Vegetation Index is the most commonly used spectral index, and can be applied to multispectral drone data that have both red and near infrared bands (learn more here).
If you don’t have a multispectral drone, don’t worry! We can still apply other enhancements to your data based on the red, green, and blue (RGB) information. The ‘greenness index’ is a great example of this. It calculates how green individual pixels are, and helps distinguish variations in vegetation. And yes, you can enhance the contrast of greenness index images too!
These images show an RGB orthomosaic with the greenness index applied.
You can learn more about the greenness index from this blog about mapping and monitoring mangroves.
Relevant enhancement tools on GeoNadir include: contrast enhancement, Greenness Index.
Extracting information comes in all shapes and sizes
The type of information to extract from your data will depend on the questions you are asking, and of course the data you have on hand. For example, drone mapping data is particularly suited to deriving information based on 3D surfaces, as we create a DSM and DTM in the pre-processing stage. This means we can answer questions like:
- What is the volume of a pile of gravel or sand?
- How steep is the profile of sand on a beach? Is it gradual or cut away from erosion?
Given the high levels of detail in drone imagery, it also lends itself to detecting, counting, and mapping specific features. For example:
- What is the size of the area covered by mangroves?
- How many houses have solar panels?
- How many plants in an agricultural plot
Sometimes it is enough to manually count features and make these measurements. However, depending on how frequently you need to extract a particular type of information, you may like to look at more automated approaches.
Relevant information extraction tools on GeoNadir include: calculate volume, calculate height, terrain profile, inspect, manual digitising (points, lines, polygons).
Is image processing accurate?
A wise colleague once said to me that you can tell a true expert when their answer to your question is ‘it depends’. Not that I claim to be a true expert, but I will certainly say that the results of your information extraction phase will always depend on many factors. So this stage of the image processing sequence calls for accuracy assessment.
The first thing that we can check is the horizontal (x,y) and vertical (z) accuracy of the data. This helps us understand if the data products are in the right location and if the elevation is correct. Using RTK GNSS and GCPs will significantly increase the accuracy of your data.
It’s also important to assess the accuracy of any downstream information products that you create. For example, if you have annotated your drone mapping data with points, lines, or polygons to represent specific features, is your interpretation correct? Using secondary data such as ground survey can help you make this comparison.
Relevant accuracy assessment tools on GeoNadir include: compare, adjust transparency, inspect, import vector.
Multi-temporal analysis
Beyond getting information at a single point in time, drone mapping is particularly powerful for documenting changes in our environment. Whether that be assessing the building phases on a construction site, or tracking coral reef mortality, multi-temporal analysis allows us to quantify dynamics.
At the most basic level, image processing can allow us to detect the change between two dates/times. For example, to detect the amount of change along a beach after a storm, we might choose to subtract the elevation values (DSM) of one date from that of another.
Similarly, if we were interested in changes in vegetation vitality, we could subtract the greenness index values in one image from another.
We can also analyse the spatial coverage of a feature (e.g. bleached coral) and compare that with a previous or future date to calculate the percentage or area cover change.
Relevant multi-temporal analysis tools on GeoNadir include: compare, adjust transparency, inspect, digitise.
Communicate information
If you’ve made it this far, it’s time to tell the world about it (or at least your clients / stakeholders!). Although this is set out as a linear image processing sequence, I believe that communicating information is important throughout your mapping task.
From the point of pre-processing your data, you can invite collaborators into your GeoNadir projects with view, comment, or edit permissions. This will allow you to get real time feedback as you progress.
Then you need to consider the format of your final products. Is it hardcopy or softcopy? A report? Summarized data? Graphs? Online mapping products? Scientific manuscript? Webinar? Regardless of your desired format, chances are this stage will take you longer than you expect if you are going to do a good job of it! I’ve shared a video below about our coral bleaching project and the range of communication avenues we chose.
Relevant information communication tools on GeoNadir include: share (view, comment, edit), copy map, save map as image, download, export.
Final word
The image processing sequence is a great framework for structuring drone mapping projects. But like framework, it doesn’t need to be applied rigidly to be useful. Perhaps you skip some steps and reiterate through others. Find the flow that works for you, and hopefully this helps you along the way!
