Jump to content
Odainsaker

DEM Comparisons: SRTM, ASTER, ALOS, Copernicus DEM Map Renderings

3 posts in this topic Last Reply

Highlighted Posts

Posted:
Last Online:  
 

Excellent and insightful post. Thanks! *:thumb:

  • Yes 1

Chance favors the prepared mind. ― Louis Pasteur  
Remember, a few hours of trial and error can save you several minutes of looking at the README. -- I Am Devloper (on Twitter)

Clickable ---> The Best of Cori's Posts  (scroll down a wee bit there)    Something fun: MySimtropolis - Invitation to become a SimCity 4 MySim

Are you new here? Check out the Introduction and Guide to Simtropolis.

Share this post


Link to post
Share on other sites
  • Original Poster
  • Posted:
    Last Online:  
     

    In the initial post I mentioned settling for the recent Copernicus DEM GLO-30 dataset as the best generally available set of global DEMs, but we also saw that it still showed clutter from ground structures, even if to a degree less than some of the other datasets.  Because in SimCity 4 we will build or plop our own buildings and trees, we want our DEM maps to be less of a digital surface model (DSM), which includes all surfaces such as buildings and tree canopies, and more of a digital terrain model, which should just be terrain elevations.  If your map is small, you can easily just edit the map by hand with GIMP tools and an eye on Google Maps as a guide, but this may not be practical for large maps or large numbers of maps.  I'm next going to examine two potential solutions to bring the Copernicus DEM GLO-30 dataset closer to being a accurate digital terrain model by filtering out the buildings and trees.

     

    GIMP SELECTIVE GAUSSIAN BLUR FILTER

     

    The first potential solution is to filter the set ourselves.  Short of elaborate GIS tools and cross-reference databases, our best available tool is GIMP, which has a variety of filters of which perhaps the most promising is the Selective Gaussian Blur filter.  Selective Gaussian Blur will blur a specified radius of pixels if the maximum difference in the values between those pixels is within a specified threshold.  This allows us some control as to where and to what degree blurring and smoothing occurs, and if we carefully select our parameters we can generally blur undesirable DEM spikes from buildings while preserving desirable DEM changes from terrain.  Trial and error is the most straightforward means to find he parameters for the Selective Gaussian Blur function, recognizing that it is still a grossly imprecise method as it cannot directly identify buildings or trees, but instead only general differences in pixel values across and area.

     

    FOREST AND BUILDINGS REMOVED COPERNICUS DEM

     

    The second potential solution I stumbled upon while doing trial and error testing of the first.  Researchers as the University of Bristol have actually applied GIS tools, machine learning, and and a variety of databases to the Copernicus DEM to more narrowly target urban structures and forest canopies to be removed in a way that brings the new DTM to more closely match known ground and lidar-measured elevations.  The result of their efforts is the Forest And Buildings removed Copernicus DEM (FABDEM), made available in December 2021.  Having much more sophisticated tools and data to work with than any SimCity fan is likely to possess, they have already done much of the hard work for us.  Details can be read in the paper "A 30 m global map of elevation with forests and buildings removed" on IOP.

    FABDEM is downloadable from the University of Bristol FABDEM Dataset page.  Registration is not required.  The data are downloaded as 10° x 10° mosaic groups of 1° x 1° map tiles at 1 arc second grid spacing, so the download size will be large.  The auxiliary mask files that originally come bundled with Copernicus DEM GLO-30 tiles are not included.

     

     

    I'm going to do another SimCity rendered map comparison again, this time of the original Copernicus DEM, a version of the Copernicus DEM edited with GIMP's Selective Gaussian Blur filter, and FABDEM.  The same Tokyo region map and SimCity terrain mods were used as in the initial post.  After trial and error, parameters for the Selective Gaussian Blur were narrowed down to Radius: 3 and Max Delta: 0.002 as having the best balance of effectiveness at removing undesirable buildings while not excessively removing desirable terrain.

    Again, I will show the complete regions and then various focal areas.  I have set SimCity 4 to display at 1920 x 1080 resolution and taken screenshots at that resolution.  Images were converted to the lossy JPG format for upload to Imgur at a reasonable size.  Region images were reduced 25% in size for display, as free Imgur will automatically resize large images down to a maximum of 5,000 pixels width.  The Simtropolis Forums will resize embedded large images to maintain forum layout, so you may have to expand the images to see some details.

     

    FULL REGION VIEW

    COP-DEM GLO-30

    fME6BTi.jpg

    SELECTIVE GAUSSIAN BLUR

    5wPLmRi.jpg

    FABDEM

    4TUG2iV.jpg

    Note how abstracted and almost cartoonish the low-lying areas have become.  It should be remembered that these are SC4 Mapper colors of elevation, while actually rendered SimCity maps should not have that effect.  Perhaps it does help emphasize that these are largely sedimentary plains and river drainage, and this DEM was developed specifically with an eye towards improved modeling of drainage and flooding.

     

     

    MOUNT FUJI

    COP-DEM GLO-30

    GLtds9C.jpg

    Buildings and forests should not exist at the summit of Mount Fuji, so we should expect minimal to zero effects from any correction at the summit.  Mount Fuji does have forests at its base, so we definitely want to compare and see what changes have occurred there.

    SELECTIVE GAUSSIAN BLUR

    Xgt51Qo.jpg

    Note that shallower, fainter crevices at the base and periphery of the mountain begin to disappear, while the straight line of a power transmission line cutting through the forests at the lower left are largely unaffected.

    FABDEM

    xovElY8.jpg

    The straight path of the power transmission lines has largely disappeared as the high canopy of the forest they cut through has been reduced.

     

     

    MOUNT FUJI SLOPE AND SUMMIT

    COP-DEM GLO-30

    3irI9Pl.jpg

    SELECTIVE GAUSSIAN BLUR

    eZYa5Ua.jpg

    FABDEM

    Swhc6bX.jpg

     

     

    BASE OF MOUNT FUJI

    COP-DEM GLO-30

    s1ufSFo.jpg

    SELECTIVE GAUSSIAN BLUR

    GpI7kRo.jpg

    FABDEM

    UvgKzKO.jpg

    The paper supporting this this DEM indicate they have further removed artifact pits still found in the Copernicus DEM, many of which no doubt are artifacts of forests and buildings.

     

     

    CENTRAL TOKYO

    COP-DEM GLO-30

    nOQHGml.jpg

    Note that lumpiness clusters are especially visible around the Imperial Palace, Shinjuku, and Shibuya.  The lumps are the results of buildings being detected by satellite sensors and remaining as artifacts in the DEM.

    SELECTIVE GAUSSIAN BLUR

    0nNG2u5.jpg

    FABDEM

    1RE3IIq.jpg

    Note that the process to remove the effects of building elevations in the DEM has resulted in terrain now showing as below sea level.  This actually is correct for this area of Tokyo, which can be at -1.0 m at low tide.  This does create a problem for SimCity 4, which does not model sea level in a way that would not submerge such low-lying elevation, and it will be necessary to further edit this area of the map to be above sea leave to be playable.

     

     

    IMPERIAL PALACE

    COP-DEM GLO-30

    J5WMClI.jpg

    Note the lumpiness from artifacts of buildings in the original Copernicus DEM.  The starkest lump seems to be the Tokyo Midtown Hibiya tower and there is broad, pitted clustering around Tokyo Station.  Toranomon Hills and the Ark Hills Sengokuyama Mori Tower would both be in the area covered by this map, but must not have been built when the data for this DEM was collected or were otherwise not as exaggerated as the Hibiya tower.  Mount Atago is a hill at the middle of the southern edge of the map and which historically was one of the scenic high viewing points for panoramas of old Edo and old Tokyo.

    SELECTIVE GAUSSIAN BLUR

    ikvr51i.jpg

    Note how the Imperial Palace is losing its northern moat and definition, while the Tokyo Midtown Hibiya tower starkly remains.   This suggests that further smoothing using this method risks erasing the moats altogether while still not fully erasing the tower.

    FABDEM

    uKivrMf.jpg

    Note that the Imperial Palace moats are still recognizable, particularly along the northern section of the palace, while the Tokyo Midtown Hibiya tower as well all the towers around Tokyo Station have been significantly reduced in prominence.  Unfortunately, the historic Mount Atago, today somewhat hemmed in by taller buildings, may have been erased into faint obscurity.

     

     

    TOKYO HARBOR

    COP-DEM GLO-30

    DFNVcIX.jpg

    Note that the Minato Ward area also includes numerous skyscrapers and Tokyo Tower, though Tokyo Tower is not significantly picked out in the DEM.

    SELECTIVE GAUSSIAN BLUR

    4BRNThX.jpg

    Note that while a water selection mask was used, land areas on the periphery of that mask were affected by the smoothing such that they still inadvertently became submerged.

    FABDEM

    D9CfUff.jpg

    Note that while the researchers behind FABDEM also explicity barred areas approaching Copernicus DEM water masks from being affected by their elevation correcting process in order to preserve water body edges, that safeguard also doesn't seem to have fully held, as this map has lost some water edges, numerous islands and islets, and breakwaters.  In this map, the historical naval defense battery island Dai-Roku Daiba has completely vanished.  On the other hand, the large skyscrapers of Minato Ward have been effectively smoothed away.

     

     

    SHIMIZU

    COP-DEM GLO-30

    MXS2dLG.jpg

    Note that a section of the elevated Tomei Expressway is clearly traceable before it enters what would be the Okitsu Tunnel through the mountains at one end and the Satta Tunnel at the other end nearer the coast.  Further to the right in the image a smaller section of the Tokaido Shinkansen can be seen before it too enters a tunnel.  Also recall that the Copernicus DEM was already missing the breakwater leading into Shimizu Port.

    SELECTIVE GAUSSIAN BLUR

    NW2mWyZ.jpg

    FABDEM

    bR53xFU.jpg

     

     

    TONE RIVER

    COP-DEM GLO-30

    IJdB0Ss.jpg

    SELECTIVE GAUSSIAN BLUR

    12gbb9t.jpg

    FABDEM

    XUgvptv.jpg

    Note that again large sections of the lowlands around the Tone River have become submerged.  This stretch of the Tone River does have high levees to prevent flooding of the surrounding farmlands, much of which are rice paddies.  Much of the submerged area was historically reclaimed land as major efforts began in the Edo period to control the course of the Tone River.

     

     

    RICE PADDIES

    COP-DEM GLO-30

    7VZiT9m.jpg

    SELECTIVE GAUSSIAN BLUR

    Wg4P27V.jpg

    FABDEM

    8ofDNsH.jpg

     

     

    CONCLUSIONS

    GIMP's Selective Gaussian Blur offers us a method to input our own thresholds, but, because it is untargeted, the results are average and it is at the extremes that it fails.  It cannot fully erase large buildings without also erasing terrain features we might want to keep.  The erasure of the Imperial Palace or the shallower crevices of Mount Fuji are not acceptable to me, and while such features can be masked so that a stronger smoothing blur can be applied to the urban areas, there comes point where the effort at careful masking and filtering begs the question why not go with a dataset where that masking and filtering has already been done at a much more sophisticated level.

    FABDEM does effectively remove buildings and forest canopies and creates map readily playable for SimCity without the need to use God Mod terrain editing tools to further level areas, but it introduces numerous below-sea-level issues for lowland areas for SimCity.  Thankfully, Copernicus DEM GLO-30 does come with auxiliary water masks and even transparency, so it is possible to carefully use these masks along with layers to restore coastal features or infill low-lying areas.  Broadly, I worry that FABDEM might have overzealously smoothed some areas, particularly urban areas where small water channels have been erased as the protection of the water mask was imperfect.  Fortunately, these are not irrepairable.

    FABDEM, is...ahem...a fabulous recent upgrade of Copernicus DEM making it much more ready for SimCity.  It will take some additional work in addressing water body and lowland issues, but, especially in heavily built-up urban areas, it has already solved for SimCity mapmakers numerous issues with strategic DEM smoothing.

     

    • Like 4

    Share this post


    Link to post
    Share on other sites

    Sign In or register to comment...

    To comment in reply, you must be a community member

    Sign In  

    Already have an account? Sign in here.

    Sign In Now

    Create an Account  

    Sign up to join our friendly community. It's easy!  

    Register a New Account


    ×

    Thank You for the Continued Support!

    Simtropolis depends on donations to fund site maintenance costs.
    Without your support, we just would not be in our 24th year online!  You really help make this a great community. *:thumb:

    But we still need your support to stay online. If you're able to, please consider a donation to help us stay up and running. This helps sustain a platform where we can share our community creations for years to come.

    Make a Donation, Get a Gift!

    Expand your city with the best from the Simtropolis Exchange.
    Make a Donation and get one or all three discs today!

    STEX Collections

    By way of a "Thank You" gift, we'd like to send you our STEX Collector's DVD. It's some of the best buildings, lots, maps and mods collected for you over the years. Check out the STEX Collections for more info.

    Each donation helps keep Simtropolis online, open and free!

    Thank you for reading and enjoy the site!

    More About STEX Collections