Concepts, Terms, and Outcomes

On this page, you will find general help for the ProRaster product family including links to documentation, instructional videos, and training videos.
Previous Topic: Limitations
Next Topic: The Multispectral Scene Database
Back to: ProRaster Scientific Help
Back to: ProRaster Help
Back to: ProRaster
Go to: ProRaster Essential help page, ProRaster Premium help page, ProRaster Scientific help page.
The ProRaster User Guide is available for download as a PDF.

Radiance

When a satellite with a multispectral instrument observes the Earth, it measures the radiance of the Earth in discrete spectral bands that cover a certain range of electromagnetic wavelengths. For example, it measures radiance in the red, green, and blue bands, and from these bands we can build a “Natural Color” representation of the imagery. Typically, the spectral bands will extend beyond the visible light wavelengths, particularly to longer infrared wavelengths. Landsat includes a thermal camera that measures even longer wavelength thermal bands.

Spatial Resolution

These measured radiance values are supplied to us as rasters, and rasters have a cell size that we refer to as the spatial resolution of the data. This cell size may reflect the natural spatial resolution of the instrument that acquired the data. The resolution tends to vary by instrument and by spectral band.

ProRaster links the “base resolution” of a scene to the “Blue” spectral band. For example, a Landsat 9 scene has a base cell size of 30×30 metres, and a Sentinel 2 scene has a base cell size of 10×10 metres. The panchromatic band, if supplied, will be higher resolution (15 metres in Landsat 9). Longer wavelength spectral bands will be lower resolution, and thermal bands tend to be lower still.

ProRaster will happily work with spectral bands that are different resolutions, but the same is not true for other software systems. For this reason, data providers often resample their spectral bands to a common resolution.

Spectral Resolution

In addition to a spatial resolution, the data will also have a spectral resolution. This reflects how many levels of grey the instrument can differentiate between, generally expressed as a number of bits. 8-bit imagery, for example, has 256 levels of grey.

Panchromatic band

Some platforms include a panchromatic spectral band, which covers a wide range of wavelengths. This pan band has a higher spatial resolution than the other spectral bands. For example, Landsat 7, 8, 9 return a 15×15 metre pan band in addition to the 30×30 metre resolution spectral bands.

Digital Numbers

All data providers supply the spectral data as Digital Numbers. Typically, this is a representation of the data scaled into a 16-bit integer. These numbers have no units and, as a rule, should not be used just as they are. The scene typically provides metadata that enables the user to convert the DN values to radiance via a relatively simple mathematical expression. This expression will require constant values, which may vary by scene, that are specified in the metadata.

Radiance vs Reflectance

Radiance, the light the instrument sees, is ultimately converted into reflectance, which is purely a property of the material being observed. From a scientific standpoint, reflectance values are the numbers we want to work with. They typically range between 0 and 1.

Top of Atmosphere (TOA)

From radiance observed in space, the first major correction is to convert the digital numbers to top of atmosphere radiance, reflectance, and brightness temperature (in Kelvin for thermal bands). The reflectance correction includes a correction for the sun angle. This may be provided per pixel as a raster. ProRaster will perform this correction if required.

Bottom of Atmosphere (BOA) / Surface Reflectance (SR)

This secondary correction takes TOA data and attempts to remove atmospheric effects such as aerosol scattering and thin clouds. There are multiple inputs to this processing, and the inputs can vary by methodology. Inputs can include topographic DEM, MODIS derived water vapor and ozone models, air temperature, thermal bands, and sun angle.

BOA is considered the best data for scientific analysis, but it may not be available for all scenes. ProRaster does not compute BOA. You will need to source this data directly from your provider.

All scenes are provided with quality assurance information, both for the scene as a whole and for each pixel in the scene. For example, the metadata may record the percentage of the scene that is covered by cloud. In addition, raster QA data may be provided that identifies cloud in each pixel. Other parameters like radiometric saturation, water, snow, and defective pixels may also be recorded. This information is generally quite difficult to tweeze out of the scene rasters. In ProRaster, it is used to mask pixels to prevent them from being rendered or used in data analysis and statistics.

Tile vs Scene

Some satellite data is distributed as scenes, and some is distributed as tiles. A scene contains the full spatial extent of the raster data that the satellite collected in each imaging operation. Typically, a scene will have a rhombus shape, reflecting the north-south orbit of the bird and the rotation of the Earth. In contrast, a tile is a rectangular sub-section of a scene. It may be in the same coordinate system as the scene, but frequently it is in a different coordinate system. Only data from a single scene is distributed in a tile, even though the tile may overlap the footprint of multiple scenes. Consequently, the coverage in a tile varies depending on which scene is contributing data to the tile.

Scene vs Mosaic

A satellite will acquire data in a swath that is called a scene. However, some providers do not ship data as complete scenes. Instead, they split the data up into tiles which are in fixed locations. Consequently, the coverage in a tile may vary depending on where the satellite was in its orbit. Regardless of whether it is a tile or a scene, when you import it into ProRaster we consider it a scene from that point forward.

A mosaic is a collection of adjacent scenes. Scenes will generally overlap adjacent scenes a certain amount, so there are no gaps in a mosaic. Depending on how many scenes you gather, your mosaic may cover a region, a continent, or the whole planet. It is important to remember that adjacent scenes may not be contemporaneous. They may have been acquired on a different orbit of the bird. As a rule, the larger your mosaic, the broader the range of times at which the scenes were acquired. Remember that the birds travel in the NE direction, so scenes may be approximately contemporaneous from south to north, but from east to west this will not be possible.

Also, a mosaic may combine scenes that are supplied with different coordinate systems. In this scenario, it may be necessary to reproject each scene to a suitable common coordinate system. ProRaster will perform this reprojection on the fly.

Collated Scenes

When you mask a scene to remove low-quality pixels – whether they be saturated, invalid, or cloud affected, or whether they represent surfaces that you do not want to include, like water and snow – you will almost always end up with a scene that has pixel gaps in it. You can fill these gaps with data acquired nearly contemporaneously. This may be an earlier or later revisit of the bird to that scene, or it may be a visit to that scene from a sister bird from the same family. So, we can collate multiple instances of a scene together, hopefully as contemporaneous as possible, to fill the gaps in the desired scene with quality data from other visitations of that scene. This is a collated scene. At the end of this process, we have an improved dataset that may be more suitable for rendering an analysis.

The virtual raster that represents the collated scene will take multiple input sample points per pixel, select one based on your rules, and will output a single data point per pixel.

Sequences

A sequence is a collection of revisits to the same scene recorded as separate temporal events. You can think of each event as a frame in a movie, except that the frame stores a dataset rather than an image. As a raster can contain multiple events, a sequence is still considered a single dataset and, as such, can be used in any processing operation.

By combining these concepts – scene, mosaic, collated, and sequence – we can build a set of possible datasets that can be created as a multispectral product.

Scene: A virtual raster representing a satellite scene or tile containing multiple fields, with multiple bands per field, containing corrected and uncorrected spectral data, QA data, ancillary data and imagery, and mask bands.

Collated Scene: A virtual raster for a scene that merges two or more masked revisits of the scene together, as contemporaneously as possible, preserving high-quality pixels and rejecting low-quality pixels to populate as many pixels with high-quality data as possible.

Scene Sequence: A virtual raster for a scene that contains two or more independent temporal events, where each event represents the scene surveyed at a particular revisit time.

Collated Scene Sequence: A virtual raster for a scene that contains two or more independent temporal events, where each event is a collated scene that merges two or more nearly contemporaneous revisits of the scene together and represents the scene surveyed at a designated revisit time.

Mosaic: Two or more adjacent scenes, as contemporaneous as possible, merged into a single dataset.

Collated Mosaic: A mosaic where one or more contributing scenes is a collated scene that merges two or more nearly contemporaneous revisits of the scene together.

Mosaic Sequence: A virtual raster for a mosaic that contains two or more independent temporal events, where each event represents the mosaic surveyed at a particular designated revisit time.

Collated Mosaic Sequence: A virtual raster for a mosaic that contains two or more independent temporal events, where each event is a collated mosaic, and represents the mosaic surveyed at a designated revisit time.

Spectral Dataset

A spectral dataset is a virtual raster that is derived from one of the eight dataset descriptions above, that preserves the spectral bands and other bands from the original dataset. Furthermore, the original scenes that were used to create the dataset are all spectrally compatible.

We can argue about what the definition of spectral compatibility is. Are the spectral bands collected by Landsat 7 compatible with the spectral bands collected by Landsat 9? What about the spectral bands collected by Sentinel 2 vs Landsat? In ProRaster, I take a very strict definition of spectral compatibility. This is out of necessity, because if two scenes are combined in a dataset then it is a requirement that the virtual rasters have the exact same field and band structure. The consequence of this is that I consider scenes to be spectrally compatible if they come from the same satellite family, and they use the same processing, and they are the same product. So, for example, you can mix and match Sentinel 2A and 2B scenes if they are both TOA or BOA. You can mix and match Landsat 8 and 9 scenes if they use the same Collection, Processing Level, and Product.

If scenes are not spectrally compatible, you cannot combine them into a single dataset. However, you can still treat them individually and then combine downstream products from those scenes, like spectral index computations or rendering algorithms. In that instance, you need to be aware of the possible incompatibilities.

Spectral Band Combinations

If we have red, green, and blue spectral bands then we can use a RGB layer to display the data and connect the appropriate spectral bands to the respective components. This is referred to as a “Natural Color” band combination. By using different spectral band combinations, we can generate imagery that highlights certain features of the earth, soil, vegetation, and water. ProRaster allows you to choose from about 20 commonly used band combinations, or to define your own. Not all these combinations will be available for all scenes – it depends on what spectral bands are present in the scene. For example, you will have a much richer choice of combinations in a Landsat 9 scene compared to a Landsat 5 scene.

Index Computations

Index computations, drawing data from one or more spectral bands, use a mathematical expression to compute a spectral index. Scientists have published hundreds of these formulas, but there are just a couple of dozen that are in common use. For example, the Normalised Difference Vegetation Index (NDVI) is used to compute and compare the amount of green vegetation. The formula to compute the index is:

NDVI = (NIR-Red)/(NIR+Red)

ProRaster contains scores of these index computations as generic formulas. If the scene or dataset you are processing contains the required spectral bands, then ProRaster will allow you to select and compute the index. Note that some index formulas assume that the inputs are scaled to between 0 and 1 (i.e., reflectance).

Difference Computations

If you have data acquired at two or more times, then you can compute the difference between the data acquired at those times. Generally, the data will be an Index or some other kind of computed value. With a sequence data product, you can compute the difference between consecutive time points (events) and create a new sequence data product that quantifies how the data changes over time. You can also compute the difference between a key time (event) and each other time.

DN vs TOA vs BOA

What spectral target should you use? ProRaster will give you spectral data as digital numbers, top of atmosphere corrected radiance and reflectance and brightness temperature, and bottom of atmosphere surface reflectance and surface temperature (if available). What data should you choose for rendering and analysis?

In other software packages, the desire to do as little work as possible influences the decision. So, you can technically use DN values to compute an index like NDVI, and it is easier to do so as no pre-processing is required. However, in ProRaster it is just as easy to use BOA as TOA or DN. My advice is to use the most processed data you have available. So, use reflectance and temperature rather than radiance. Use BOA rather than TOA, if you have it. Use DN values only if you have no other option.

It follows that I would recommend you download Level 2 (BOA) scene data from your provided rather than Level 1 (TOA) scene data. Unfortunately, for both Sentinel 2 and Landsat, your choice can force you to make a compromise.

In the case of Sentinel 2, the BOA data is superior as it contains a QA band and so you can mask out cloud and other unwanted pixels. But band 10 (Cirrus) is not included in the BOA product! Even so, my recommendation is to download BOA (Level 2A) Sentinel 2 tiles.

In the case of Landsat 8 and 9, the BOA product does not contain any additional critical information. Unfortunately, it does not contain bands 8 (Pan), 9 (Cirrus), and 11 (TIRS2)! The most grievous loss, in my view, is the Pan band which prevents you from employing pan-sharpening. I find it difficult to make a recommendation for Landsat. It depends on how much you need or would benefit from, BOA data over TOA data.

Rendering

ProRaster is designed to render raster data, and rendering satellite multispectral data is just what it likes to do.

You will see options to “Display Scene” or “Render Scene” (or whatever data product you are working with). In the first case, the virtual raster will be opened in the main ProRaster interface using a default Look-up Color layer. This will allow you to explore the fields, bands, and events of the MVR and render each band individually.

In the second case, the virtual raster will be opened in the main ProRaster interface using an RGB color layer. A dialog will be presented asking you to choose from a list of valid band combinations, or you can define your own combination. You will also be able to enable pan-sharpening, if available. You will be able to enable masking, if available.

Pan-sharpening in ProRaster is only available when rendering. It is not a processing operation and cannot be used as any kind of input to a processing operation.

Masking in ProRaster is available in any layer and can be used to prevent rendering of pixels that overlap invalid pixels in the mask raster. Although it can be used with any kinds of rasters, its main purpose is to mask out compromised pixels in multispectral data.

Clipping and Masking

When analysing data, bad pixels in means bad statistics out, and a compromised result. Therefore, you will want to mask out bad pixels aggressively and you will want to use polygons to clip the dataset to only that area that you wish to study. In ProRaster, this is achieved by “Clipping to raster” (masking) or “Clipping to polygon”.

To help mask out bad pixels, ProRaster builds a Mask field that contains some standard bands. It populates these bands by interrogating the QA band data associated with the scene. For example, you will often see these bands – Pixels (clear), Land (clear), Water (clear), Snow (clear). If you only want pixels that are designated Land, and you want to reject all compromised pixels, choose “Land (clear)”. If you just want to reject band pixels, choose “Pixels (clear)”.

This data is supplied by the data provider and is of mixed quality. As a rule, older processing is not as good as later processing. You might find that an old Landsat scene in Precollection format may have much poorer QA data than the same scene reprocessed to Collection 2 format. Even so, often the Land – Water – Snow differentiation is poor. In my view, the pixel labeling is rarely aggressive enough and many pixels that are designated as clear of cloud may be partially cloud affected.

Statistics

For any raster dataset, you can compute statistics. This will compute summary statistics – like the minimum, maximum, mean – as well as distribution statistics which include a distribution histogram and other measures like the standard deviation, median, mode, and quartiles. These statistics can be used to form the basis of your data analysis. You can track how the statistics change over time and quantify the changes.

Export

ProRaster Scientific creates MVR virtual rasters that cannot be loaded, displayed, or used in other products. To get your raster data out of ProRaster and into other software, you can use the export capability. This will write the raster data to a new raster file in a transportable format.

The MRR format is best suited for exporting MVR files. An MRR can contain multiple fields, bands, and events, and can store all the data in an MVR in a single raster file. However, MRR is difficult to use outside of ProRaster and MapInfo Pro.

ProRaster supports exporting to transportable raster formats like GeoTIFF and Band Interleaved (BIL). These formats are not able to support the complicated field, band, and event structure on an MVR raster. Consequently, when you export to a transportable format every data band (by field and event) is written to a separate raster file. Even so, concepts like Classified fields are not properly supported by other raster formats or software.

MVR Chains

An MVR virtual raster is connected to one or more sources of input raster data. It will usually modulate that data in some way – by applying a calculator expression for example – and then emit it again as output. An MVR can be the source raster to another MVR. This allows ProRaster to build chains of MVR rasters that implement multiphase processing operations.

Creating chains of MVR rasters does not invoke any processing. However, when you consume the MVR at the end of the chain – by rendering it, computing statistics, exporting it, or in any other way that requires cell values to be evaluated – the entire processing chain will be invoked to evaluate raster cell data. This is performed in real-time, on-demand, as data is only ever requested over a particular spatial range and at a particular spatial resolution. A spatially restricted data request such as this can usually be responded to promptly.

Scene Assembly MVR

When you download a scene from a provider, you will be supplied with a collection of rasters – one for each spectral band (possibly at multiple spatial resolutions), as well as other rasters for QA data, ancillary data like sun angle, and pre-rendered imagery.

After importing that scene into a scene database, you will have one additional raster – this is the scene assembly MVR. From then on, ProRaster will interact with the scene rasters through this virtual raster. It provides a host of services including –

Directing the raster engine to load the source rasters through a specific driver, if needed.
Supplying the raster engine with driver preferences for each source raster.
Directing the raster engine to build overview level pyramid caches, if needed.
Gathers the spectral bands into multi-banded fields.
Implements transformations to convert DN to TOA radiance and reflectance.
Implements transformations to convert thermal DN to TOA temperature.
Implements transformations to convert DN to BOA reflectance and temperature.
Gathers the ancillary raster data into multi-banded fields.
Picks out the individual QA values for each pixel and exposes them in multi-banded fields.
Builds a mask field that uses the QA data to identify pixels of different classification and quality.

The scene assembly MVR will correct historical and current processing mistakes. For example –

Landsat data used to ship without a defined null value, injecting zero values into the raster.
Landsat COG files were shipped with the zero values incorrectly included in the overview levels, making that data technically incorrect.
Sentinel 2, which uses JPEG2000 as a raster format, currently delivers overview levels for QA data that are technically incorrect. The overviews blend data values that cannot be combined. ProRaster computes a new overview level cache for the QA data in Sentinel 2 scenes.

Importing Scenes

ProRaster Scientific provides automatic import for scenes from directly supported platforms. This makes importing easy and the assembly MVR is automatically generated for the scene. For unsupported platforms, there is a manual import process. You will be required to supply some key information about the platform and the scene, as well as identify all the source rasters and link them to spectral channels. Most multispectral platforms can be supported via the manual import process.

TOA Corrections

Landsat Level 1 scenes will be corrected to TOA in the scene collection MVR. This is performed for spectral bands and thermal bands. The Landsat online documentation details how these corrections should be made for each of the Landsat birds. Each correction is based on constants that are supplied as metadata for each scene.

To compute reflectance involves a correction for the sun angle. Historically, this was supplied as a single value at the centre of the scene and was applied to all pixels in the scene. Collection 2 data includes a raster for the sun angle across the scene. When supplied, this is used by ProRaster to compute a per-pixel sun angle correction.