Large Raster Rendering Performance in MapInfo Pro
Author: Sam Roberts
Published: November 27th, 2020
Updated: October 14th, 2021
MapInfo Pro can render huge rasters and promises to do so in constant time at any scale. But, if you try it, you might run into problems. In this article I want to share some tips for achieving high quality and high-performance rendering of huge rasters.
[Update: This article was originally written for MapInfo Pro 2019.3 and has now been updated after testing with a late beta copy of MapInfo Pro 2021 (Release Build 170). The critical difference, for the purposes of this article, is that Pro 2021 no longer forces a map using a Bing Maps layer to be in the Popular Visualization CRS coordinate system and it will reproject the Bing Maps layer into the coordinate system of the map.]
MapInfo Pro has always had the ability to render grids and rasters via handler DLL’s. This is an ‘arm’s length’ arrangement where MapInfo asks the handler to load and render the raster and to send back a bitmap, which it renders into the map. Users can exercise some degree of control over this by determining which handlers are invoked and the order in which they are invoked. Different handlers might load a raster differently or render it differently.
When MapInfo Pro transitioned to 64-bit, a new raster rendering, processing, and analysis plug-in was developed. This continues to be sold as a plug-in addition to MapInfo Pro called “MapInfo Pro Advanced”. Advanced added a new raster handler that supports a wide variety of both grid and raster formats and brought a new sophistication to raster rendering. It also brought support for a new raster format, called MRR, and provides support for rendering huge rasters at any scale. The MRR format provides an ability to effectively store and create huge rasters.
[Update: MapInfo Pro 2021 will be sold using a subscription model and will include the Advanced raster plug-in, so you will not have to purchase it separately.]
It is important to note that, when you install MapInfo Pro, you get the new Advanced raster handler and all its rendering capabilities, regardless of whether you have a license for Advanced. In fact, you don’t even need a license for MapInfo – even the MapInfo Pro Free Viewer will display huge rasters!
One of the key design goals of the Advanced raster engine was to provide the highest quality rendering of all rasters, regardless of the format, in constant time, and at any scale. In other words, as you zoom in and out on a huge raster, the raster engine must provide the highest quality rendering in real time and in constant time regardless of the level of zoom.
For a raster in MRR format, this is achieved by pre-generating overview resolution levels and storing them in the MRR file for retrieval when zooming out, and generating and caching underview resolution levels (from the base resolution level) on the fly when zooming in. Base resolution level statistics are stored in the MRR file so that high quality rendering can be achieved. For rasters in other formats, the performance and quality goals are achieved by pre-generating cache files for data (PERC or PPRC files) and for statistics (GHX files).
I have listed below my definition of a small, large, very large, and huge raster. This simple definition is based on the number of cells and assumes the raster is single banded, with a single field and single event. For a discussion of raster size, see the article “Raster Size Concepts Explained – How big is my raster?”.
- Small < 10 Megacells
- Large < 10 Gigacells
- Very Large < 1 Terracells
- Huge > 1 Terracells
From now on I am going to use the MERIT DEM as my example raster. You can download this raster yourself – see my article “MERIT DEM now available for download in MRR format”. I am also using the latest version of MapInfo Pro which, at the time of writing, is 2019.3 Build 43.
If you open the MERIT raster in MapInfo Pro the Raster Info dialog will report that the raster has a dimension of 433152 x 173056 cells – or almost 75 gigacells. The raster Statistics dialog provides more detailed information – in fact, the raster only contains 27 gigacells of which only 22 gigacells contain valid data. Regardless of how you count them, this is a “very large” raster by my definition. I am going to start with just the MRR raster file. If you open the MRR in MapInfo it will create a TAB file and you should see the raster displayed in a map, much like the image below. Immediately, you can see there is a quality problem. On the top and right edge, there are dark lines that are not a part of the raster. You might see a similar pattern, depending on how you adjust the map zoom.
To resolve this issue, MapInfo have provided advice to enable a transparent color for the raster and set the color to white. This can be achieved by adding the following lines to the TAB file.
RasterStyle 4 1
RasterStyle 7 16777215
RasterStyle 9 1
The key information is the “RasterStyle 7” declaration, followed by the color which in this case is 0x00FFFFFF, equivalent to transparent white. However, this does not completely solve the problem and I still see these lines around the edge of a raster from time to time. MapInfo need to address this.
One of the first things you might do when you display MERIT DEM in a map is add a Bing maps layer. When you do, you might immediately get an error which indicates the raster handler has tried to allocate too much memory.
To resolve this problem, you need to do several things.
- Enable block access.
Go to the Options page in the backstage area and open the “MapInfo Pro Raster” preferences dialog. In the “Display” dialog, turn on the “Enable Block Access” option. In a clean installation it ought to be enabled by default.
- Display Resampling Method Default
On the same dialog, you need to set an appropriate value for the “Display Resampling Method”, and “None” is not appropriate! You need to choose one of the other options – Nearest Neighbor, Bilinear, Cubic Spline (Local) or Cubic Spline (Global). When you are viewing a raster and zoom in, at some point the engine will have to start generating “underview” resolution tiles from the base resolution level. It does this by interpolation, and this is the default method used. Note that “Nearest Neighbor” does not do any interpolation, but this is not the same as “None”!
- Display Resampling Method Override
You can override the default for any raster by changing the “Raster Quality” setting on the Raster ribbon whilst the target raster is selected in the Explorer window. (Note that this control is only visible when you have enabled block access). This setting gets written into the GHX file for the raster and you need to make sure this is not set to “None” as well. You may need to check this for rasters you have opened in MapInfo in the past.
Whilst we are in the raster preferences dialog, you should also set “Generate Raster Overview Cache” to “Always” and on the “Memory and Performance” page, set “Memory Cache Size” to “Maximum”. The next thing you need to check is the reprojection setting in the map. Open the “Map Options” dialog and hit the “Image Processing” button. Make sure that reprojection is set to “Optimized”.
This will ensure that rasters are reprojected on the fly when rendered into a map that has a different coordinate system to the raster. It also prevents resampling when the coordinate systems of the map and the raster are equivalent. You can make this setting the default for all new maps from the Options page in the backstage area via the “Image Processing” dialog.
You have now done everything you can to ensure MapInfo Pro renders huge rasters quickly, efficiently and at the highest quality. Great – let’s try it!
Test 1
Display MERIT DEM and begin zooming in by using the mouse wheel. Each time you zoom in it seems to take a little bit longer to render the map. After about 10 zoom operation it starts to take a noticeably long time and it gets exponentially slower from then on.
[Update: Testing on MapInfo pro 2021 shows exactly the same performance degradation as you zoom in.]
Test 2
Display MERIT DEM and begin zooming in using the marquee zoom tool (i.e. drag out a zoom rectangle). Start zooming in on Scotland. After a few zoom operations you will start to notice the same performance degradation. Now use the scroll wheel to zoom back out again. You will find that the raster is now vertically stretched! Compare this to how it looks if you “View Entire Layer”.
Test 3
Display MERIT DEM and now add Bing imagery to the map. You might want to set some transparency on the raster layer so you can see one through the other. Firstly, you will notice that the map coordinate system changes – it is now fixed on Popular Visualization CRS. Now start zooming in again using the scroll wheel. You will see the same performance degradation, except that now it is substantially worse. This map is basically unusable after only 5 zoom operations!
[Update: When you perform the same operation in MapInfo 2021, the map coordinate system is not changed to Popular Visualization CRS. It remains in the coordinate system of the raster and the Bing imagery is reprojected on the fly to match. The performance is much the same as in Test 1 – you can zoom in 10 times before performance starts to exponentially degrade. If you change the map coordinate system to Popular Visualization CRS, you still see the same performance. This is a considerable improvement over MapInfo Pro 2019.3. The only comment I would make against this is that the reprojection of the Bing Maps imagery is not of the highest quality and it does result in text distortion which you might find unsightly.]
So, we have crippling and unacceptable performance problems and some odd stretching to boot. Let’s look at this stretching issue first as the explanation also impacts on the performance issue.
In MapInfo Pro, if the map has a longitude/latitude coordinate system then it is actually displayed with Equidistant Cylindrical Projection with a standard parallel set at the center of the map. MERIT DEM is Geodetic using the WGS84 spheroid. When you display this raster in a map, MapInfo secretly uses Equidistant Cylindrical Projection. This does two things. Firstly, it triggers on-the-fly reprojection of the raster. It now has a different projection to the map, so MapInfo reprojects the raster. Secondly, the map projection changes each time you pan or zoom – but inconsistently! When you use the scroll wheel to zoom the map projection does not get changed. So, when you first display MERIT the standard parallel is likely near the equator. After you have zoomed in on Scotland the standard parallel is closer to 60 degrees North. When you zoom back out again this projection stretches the image vertically.
One way to solve this is to go back to the map reprojection settings and change it to “None”. MapInfo now sets the standard parallel at zero (the equator) and keeps it there. But it is very dangerous to use this setting in a GIS system and I cannot recommend it. We need a better solution.
Let’s now consider the rendering performance problem, which gets worse as we zoom in on a large raster. This is also caused by MapInfo reprojecting the raster and doing a bad job of it. We need to prevent MapInfo from invoking its reprojection engine on the raster, but to do so would be unacceptable in a GIS system. What is to be done?
One solution is to use MapInfo Pro Advanced to reproject the raster into the map coordinate system. Come back tomorrow after you have finished running a reprojection operation on MERIT DEM! For large rasters, executing a reprojection processing operation just so we can display the raster efficiently is a bogus solution.
The solution we need is to tell MapInfo that the raster is in the same coordinate system as the map and to perform the reprojection outside of MapInfo and to supply the bitmaps back to MapInfo (via the raster handler) in the map projection and ready to be used without any further modification. To do this, we need to invoke the Raster Algorithm Rendering Engine (RARE).
RARE is a new raster rendering engine that is inside the raster engine itself. In other words, it is not implemented in the raster handler. Whereas the raster handler renders a single raster according to rendering settings in the GHX file, RARE renders an algorithm which is substantially more flexible and powerful and can refer to one or to many rasters. We can invoke RARE via a backdoor into the rendering engine called an MRD. An MRD is a small text file containing a rendering algorithm written in XML format.
When you load an MRD in MapInfo Pro, the raster engine invokes a raster driver for it that populates the raster on the fly by calling upon RARE. In other words, the rendering is performed in the background and output as an Imagery raster. This raster comes through the raster handler unscathed and provides a mechanism to use RARE instead of the raster handler rendering engine. There are many good reasons to do this, and one is that RARE can perform reprojection on-the-fly and so it gives us a mechanism to prevent MapInfo reprojecting the raster.
Let’s revisit displaying MERIT DEM in a map with Bing imagery. We now know we have to display the raster via an MRD and we need to use Popular Visualisation CRS as the coordinate system for the MRD. The MRD file (MERIT_DEM_PopCRS.mrd) contains only a few lines of XML which name the source raster and the required coordinate system. There is no actual algorithm defined, and so the system creates a default one for you. You will need to edit the MRD file (in any text editor) to make sure that the raster filename is correct. If you place the MRD file in the same directory as your raster you will not need to specify a path. You can download a ZIP file containing all the example MRD files here.
Open this MRD file in MapInfo via Open Table – the same way that you would any other raster, like an MRR. MapInfo will create a TAB file for it and a GHX file. Display it in a map and then add the Bing imagery and repeat the zooming exercise. You should find that performance is vastly improved.
Now consider the other case where the map is silently using Equidistant Cylindrical Projection but the standard parallel gets changed every time you pan or zoom. The best thing to do now is to set the MRD coordinate system to Equidistant Cylindrical Projection with a standard parallel at the equator. Try this yourself using MERIT_DEM_EqdCy.mrd.
If the default algorithm that RARE creates for your raster is not to your liking, there are two options. The first is to use MapInfo Pro to render the raster directly and to set the required rendering parameters (color table, color stretch, etc.) in the “Advanced Color” dialog. These rendering parameters get saved into the GHX file for the raster. You can then direct RARE to use these rendering parameters as the basis for the rendering algorithm. Use the RasterGHX tag to identify the raster and, in this case, you need to specify the full path. See the example MERIT_DEM_PopCRS_GHX.mrd.
The second option is to manually define the algorithm XML in the MRD file. Doing this is beyond the scope of this article, but you can find assistance in the Precisely MapInfo Pro Community forum. Search for articles by me (Sam Roberts).
[Update: A third option is to create a MapInfo Virtual Raster (MVR file) which performs an on-the-fly reprojection. Doing this is beyond the scope of this article.]
I hope this article helps you to achieve your goals and work effectively with large rasters in MapInfo Pro. You can download a copy of this article as a PDF here.
0 Comments