ProRaster Help – Data Transform Editor
On this page, you will find general help for the ProRaster product family including links to documentation, instructional videos, and training videos.
Previous Topic: The Color Table Editor
Next Topic: The Data Conditioning Editor
Back to: ProRaster Help
Back to: ProRaster
Go to: ProRaster Essential help page, ProRaster Premium help page, ProRaster Scientific help page.
The ProRaster User Guide is available for download as a PDF.
Using the Data Transform Editor dialog
A data transform is used to convert from raster data to a value between zero and one, which is then scaled to an integer index used to acquire a color from a look-up table.
The data transform is a mathematical function. It may be expressed mathematically, like Linear, Logarithmic, or Sigmoid, or it may be defined by a table of coordinates (raster data, scaled index 0 – 1) which the software can use to estimate the scaled index for any input data value. Equal Area is an example of the latter, and the editor provides you with a set of tools to create and edit your own function tables.
The component property pages provide a rich set of data transforms that are easy to define and likely to be sufficient for the needs of most users. The Data Transform Editor dialog provides a more advanced capability with additional flexibility. In particular:
- For the standard transforms, there are more flexible methods of defining the bandpass clipping extents.
- The bandpass clipping extents can be decoupled from the transform.
- For logarithmic transforms you can edit both the log base and linear transition value.
- Some transforms can be reversed.
- You can use the full color table for range transforms.
- A rich set of table-based transforms are provided.
In addition, the editor dialog provides a preview of the transform on a raster of your choice. For this raster you can see the summary statistics presented in a property page as well as the distribution histogram presented either as a spreadsheet or graphical view.
The data transform editor dialog presents you with a list of known transforms. First in the list, above the dotted line, will be any transforms that have been loaded from the current MRD algorithm. Below the dotted line will be any transforms that you have defined and saved via the editor dialog. Select a transform to edit it.
Buttons are provided to add a new transform, delete the current transform, or clone the current transform. You would use the clone operation to save a copy of a transform that you have loaded from an MRD file. You can also delete all transforms. If you do this accidentally, make sure you hit Cancel to exit the editor dialog. No changes are saved until you exit the dialog by hitting the OK button.
When you add a new transform, you must immediately give it a unique name, because transforms are referred to by name in the algorithm XML. As you type the name, the dialog will indicate whether it is unique and valid or non-unique.
You can then set the data transform type, by selecting from the drop-list provided. Once you select a transform type a property sheet will be displayed allowing you to edit the properties of the transform.
Pass Scaled: Your data values are assumed to lie between 0 and 1. The data transform will simply convert this value to a color index.
Pass Index: Your data values are assumed to be an unsigned zero-based integer matching the number of colors in the color table. The data transform uses the data value to look up the color.
Pass Value: Your data value is a color. No modification to the data is made.
Rotate Index: Supply the number of colors through which to rotate. Your data values are consecutive integers. The data transform is the modulus of the data value and the number of colors.
Color bits linear: Supply the number of bits (N) of color, creating a linear transform from 0 to 2^N-1. For example, imagery is typically in 8-bit color.
Linear: A linear transform between two values. These values are determined by the “Bandpass” settings, which can be “None”, “Data Values”, “Percentage”, or “Percentile”. If you select “None” then the bandpass values are acquired from statistics (minimum and maximum cell values). Select “Data Values” to enter the minimum and maximum bandpass values manually. Otherwise, you can express the values as a percentage (0 – 1) or percentile (0 – 1). Note that you can reverse the transform by reversing the minimum and maximum bandpass values.
If you choose None, Percentage, or Percentile, then you can also specify the number of histogram bins to clip at the bottom and top of the range. This can be a useful way to clip out the outliers at the edges of the dataset without having to work out what those outliers are. Just clip off one or two bins at the bottom and top and improve the data – color transform.
You can reverse a linear transform easily enough by swapping the minimum and maximum bandpass values. For this transform and some others, you can also toggle this with the “Reverse transform” button. In many cases it is more convenient to reverse the color table than reverse the transform.
Logarithmic: A logarithmic transform between two values, see Linear. Here, you can specify the base of the logarithm (the default is base 10). The logarithmic transform is undefined about zero, so you can define a range about zero where the transform is linear. Enter this as the “Minimum”.
Sigmoid: A sigmoid transform between two values, see Linear. Specify the sigmoid K factor between -1 and +1.
Equal Area: An equal area (or histogram equalised) transform between two values, see Linear. You can scale the severity of the transform up and down – from linear to equal area – by adjusting the Percentage Scale between 0 and 1.
Stable Equal Area: An equal area (or histogram equalised) transform between two values, see Linear. You can scale the severity of the transform up and down – from linear to equal area – by adjusting the Percentage Scale between 0 and 1.
Equal Area uses an “Equal Width” histogram which results in faster rendering. Stable Equal Area uses a “Equal Frequency” histogram which can produce higher quality transforms, particularly in the presence of outliers or highly skewed distributions.
Ranges: Spread about the mean: Enter the number of ranges (N). The transform will generate (N/2) ranges spread evenly from the mean to the maximum bandpass value, and (N/2) ranges spread evenly from the minimum bandpass value to the mean.
If you specify N ranges, then you are usually indicating that you want to display N colors, and you will probably supply a color table with N colors in it. You can supply a color table with more than N colors. In that scenario, you can check the button to use the “Full color table” and the transform will interpolate between your ranges to use all the colors.
Ranges: Spread about the median: As previously, spread about the median.
Ranges: Spread about the mode: As previously, spread about the mode.
Ranges: Deviations about the mean: Enter the number of ranges (N). The transform will generate (N/2) ranges above and below the mean. The width of each range is equal to the one standard deviation.
Ranges: Jenks Natural Breaks: Enter the number of ranges. The Jenks natural breaks algorithm is used to compute the best position of the breaks and the width of each range.
Table: The table options create a table of coordinates on the data transform where the x-coordinate is the data value, and it maps to the y-coordinate which is a scaled index between 0 – 1. However, ProRaster provides you with multiple ways to define the data values, see below.
Table: Index over range: In this table the data values are not entered manually. Instead, you define the minimum and maximum value by using the bandpass parameter tools (see Linear, above), and the data values are assigned linearly across this range. All you edit is the scaled index for each entry in the table.
When you choose one of these table options, you will see a “Table” property page appear in the property sheet. On this page is a spreadsheet which you can edit. At the base of the sheet are controls to set the number of rows and to reset the table to default values for those rows. An undo mechanism is provided so you can undo changes.
You can add and delete rows in the spreadsheet by editing. Add a row by moving to the rightmost column and hitting the return key. The row will be duplicated below the selected row. Delete a row by moving to the leftmost column and hitting the delete key. This will delete the row above the selected row.
Note that the scaled index values do not have to be monotonically increasing.
Table: Index for value: In this table you edit both the data value and the scaled index, so you have complete manual control over the table.
The data values need to be monotonically increasing, but the scaled index values do not.
Table: Index for percentage: In this table a percentage, between 0 and 1, is used to map to data values. This percentage is stretched between the bandpass limits.
The percentage values need to be monotonically increasing, but the scaled index values do not.
Table: Index for percentile: : In this table a percentile, between 0 and 1, is used to map to data values. This percentile is stretched between the bandpass limits. Recall that a percentile is a percentage of the number of samples, equivalent to a statistical quantile for a data distribution.
The percentile values need to be monotonically increasing, but the scaled index values do not.
Table: Match value: This table, and all the following tables, are referred to as legend transforms. In these tables the scaled index does not need to be declared and is not editable. You will almost certainly use these tables with a color table that has a matching number of colors (the same number as the rows in the table) and it follows that the color for each row in the table will be the same as the zero-based row index. In fact, you may find it more convenient to create a legend color table map in the Color Table Editor dialog which creates a transform in exactly the same way, but also allows you to specify the color for each row.
This table declares data values which map to individual colors. Data values that do not match the values listed in the table will not be assigned a color appropriately. You ought to use a table like this for integer data, like classification codes for example, rather than continuously variable data like terrain.
The data values need to be monotonically increasing.
Table: Match value range: In this table you declare a set of data ranges, from minimum to maximum for each range. Data values that fall into a range are assigned the color for that range.
Within a range, the minimum value ought to be <= the maximum. However, ranges do not have to be monotonically increasing and they can overlap. Where a data value can be assigned to multiple ranges, it will be assigned to the first in the table.
Table: Match continuous value range: In this table we make sure that the ranges are monotonically increasing and do not overlap. A data value must be >= the minimum value and < the maximum value t be assigned to a range.
Table: Match value string: In this table we supply text strings as the data values. You would use these to transform against a string field in a raster. This might occur in a classified raster, for example.
You can apply your transform to a raster in real-time as you design it. The right-hand side of the dialog is a preview map window in which you can pan and zoom in the usual way. A button is provided to open and close the preview window (it cannot be undocked). Select the raster to view in the preview by opening the “Raster” property page and selecting a raster source. You can select the field, band, and time range using the “Preview Settings” menu button.
The preview raster will be displayed using a LUT Color layer algorithm. The current data transform is applied to convert the raster data to scaled value, and then a standard “pseudocolor” color table is used to provide color. If you open the data transform editor when you are working on an algorithm, the raster you were using in the algorithm will be selected as the default.
You can access statistics for the raster source you have selected for the preview. Select the “Statistics” property page to see summary statistics of the raster. This includes the following information:
Sample Count: Total number of samples, Number of valid samples, Number of invalid samples, Number of invalid samples that were Not A Number, Number of invalid samples that are empty, Number of invalid samples that are null.
Summary: Minimum value, Maximum value, Mean, Variance, Standard deviation, and Signal to Noise ratio.
Distribution: Lower quartile (25%), Upper quartile (75%), Interquartile Range, Median, and Mode.
Spatial: Minimum, Maximum, Mean, Variance and Standard deviation. These are not standard statistical measures – they measure the difference between a cell value and the adjacent cell value.
Select the “Histogram” property page to see distribution statistics. These can be presented as a spreadsheet or as a graph. You can apply a logarithmic stretch to the vertical axis of the graph.
The spreadsheet contains the distribution histogram data. Each row of the sheet corresponds to one histogram bin. In ProRaster, histograms are either “Equal width” where all the bins have the same width and there are a large number of bins, or “Equal frequency” where bins have a variable width, tend to have the same number of samples in each bin, and there are fewer bins.
Each bin covers a defined data range. This is reflected in the Bottom and Top columns in the sheet. Even so, the data samples within that range may only cover a subset of the range. This is reflected in the Minimum and Maximum columns in the sheet. The number of samples recorded in each bin is shown in the Count column. The Cumulative Count column records the percentage of samples up to and including that bin (from 0 – 1).
Once you have defined your data transform, you will be able to select it by name in the component “Color” property page under the “Stored Transforms” category.