Grayscale Imaging.

[Verran 0]

Good morning, ladies and gentlemen.

After a few days of searching through Google, and a search through the forums here, I can't find an answer to my inquiry. Some have gotten close, none have hit the nail on the head.

At present, I'm having difficulty importing a rather large region into Sim City. The grayscale height map is in order, and I get "correct" topography when it comes in, yet, not quite. Once everything is said and done, I'm finding that what is white on the map is not creating true maximum elevation, nor is black creating the true bottom of the ocean. For example: Are you familiar that most developers say sea level is about, oh, 84? One-hundred six to 107 is just above sea level for me. To further illustrate, after the map imports with my "mountains," I can proceed to Sim City's internal terraformer, and raise a range much, much taller than what was just imported--but the map was pure white.

Would love some help here, I'm working on a big assed project. One-hundred twenty-four kilometers by sixty-eight kilometers, to be exact. What's so damned big...? Vana'diel.

[A Nonny Moose 8,261]

It sounds to me as though you have a height mod somewhere in your plugins that is affecting the sea-level. If so, you must remove it temporarily while rendering the map.

[cogeo 960]

With the default settings, a grayscale value of 83 is "water", while 84 is land. On import, each pixel's grayscale value is multiplied by 3 (default value of the ImageImportScaleFactor property in the Terrain Properties exemplar). As the sea level is set to 250m, while 83x3=249 and 84x3=252, you can easily figure out why.

So, to check what's really going on, I would recommend that you download and install the Extra Cheats DLL. Then press Ctrl+X, right-click on the edit box and select "Terrain Properties"; the mouse cursor will be showing terrain information (the elevation is the "Y" value). Then make a small region grayscale image and config (to be imported faster), and set some quite large areas to certain values, eg 83 (should be just underwater), 84 (just above water), or 100 (should become 100x3=300m). If this doesn't work as expected, check the following:

- First of all check if you have some "Height Mod" installed in your Plugins folder. These are special mods, affecting how grayscale images are imported. They basically modify the ImageImportScaleFactor, usually to overcome the height limitations (255x3=765m), or to achieve a better accuracy in importing "lowlands" maps; they may also modify the Sea Level. If you have such a mod installed, uninstall it or modify it to your liking.

- Another reason I would think is use of paletted (indexed) grayscale image formats. JPEGs (the format usually used by most uploaders) isn't paletted, however its compression is "lossy" (causes "artifacts") and this results in inacurracies in map importing. A lossless format (PNG, GIF, RLE-compressed BMP) is preferable imo, but this may involve another possible cause of troubles, which is the palette. A palette is a mapping of a palette index value (0..255) to the actual image colour to be displayed (ie an RGB value). Many image processing programs may "optimize" the palette, which means eliminating the palette entries not used in the image (only to save a few bytes). So the index value of 84 may not correspond to the RGB value of (84,84,84). Check the palette in your image processing program, and see if palette indices correspond to the actual colours. Keep in mind that the value SC4 reads is the palette index, not the colour value! Which means that you may have set a certain area to the (84,84,84) colour, but your image processing program may have assigned this colour to another palette entry eg 25, and that's the value SC4 reads. So my suggestion is save the image as non-paletted, not full RGB though, as this requires three bytes per pixel and results in a huge image file. The preferred format would be non-paletted "single-plane" grayscale (8 bits per pixel). Or make sure that your file uses a full-blown (non-optimized) grayscale palette.

Please also note that SC4 performs certain modifications to the data being imported, among others "erosion", so as to achieve a more "realistic" look. This can result in much of the "lowlands" be "lost" to the sea, therefore I would suggest turning erosion off. You can find such mods, as well as custom "Height Mods" in any of my maps I have uploaded to the STEX. I would recommend the West Macedonia Lakes maps (includes several "No-Erosion" mods that work in conjunction with certain "Terrain Mods" as well as SC4 default terrain settings).

Hope this helps.

[Verran 0]

I appreciate the detail you've added to your reply, Cogeo, so please forgive the brevity of my response:

The only plugin I'm currently running is the NAM; does that include a height mod?

[aysk8er 17]

What programs are best for editing grayscale images?? I would like to avoid buying Adobe Photoshop as it is super pricey.

I'm not sure what COGEO meant with the "Erosion Feature" I have noticed some of my imported maps from SC4Mapper have really low valleys usually sand color in the game... How can I turn this feature off as mentioned above??

Also I was able to find regional grayscale images usually large files (some over 100mb) on the USGS site... goto www.nationalmap.gov and use the "Earth Explorer" to find any location, set selection to "use map" and then choose DEM ASTER files and click results tab. This process has given me the best results for specific grayscale files. They are also very large which I believe is what COGEO was referring to as being lossless format.

[jason leo 485]

I appreciate the detail you've added to your reply, Cogeo, so please forgive the brevity of my response:

The only plugin I'm currently running is the NAM; does that include a height mod?

NAM is a transportation system related mod, not a height mod.

[Verran 0]

Yes, I thought as much, however stranger things have been true, no?

Cogeo, I've uninstalled the only mod I was running (NAM,) out of an abundance of caution.

I've also ensured that photoshop is saving bitmaps with indexed colours.

No improvement.

[cogeo 960]

What programs are best for editing grayscale images?? I would like to avoid buying Adobe Photoshop as it is super pricey.

All image processing programs can edit grayscale images. The most known is of course Photoshop, which as you said is expensive. It's the industry standard though, a must for professional imaging and graphics. Not that you can't do your job with another one, but it's the standard exchange format, and a graphics professional is in most cases required to know Photoshop (to a large extend it's driven by the forces of "inertia"). There are some few interesting alternatives though. Check the following:

- The GIMP. It's free software, intended as a Photoshop replacement. You may find its user-interface weird. I would suggest installing the single-window version.

- PaintShop Pro. A very affordable alternative, supporting more than 90% of Photoshop's features. Very popular, not industry standard however, although many professional graphics designers do use it!

I'm not sure what COGEO meant with the "Erosion Feature" I have noticed some of my imported maps from SC4Mapper have really low valleys usually sand color in the game... How can I turn this feature off as mentioned above??

To turn off erosion install one of my "No-Erosion" mods. This will affect only maps rendered in SC4 (pressing <Alt><Ctrl><Shift>R) though, not with SC4Mapper. SC4Mapper creates the city files from the data stored in the .SC4M file, while this erosion is an intrinsic SC4 feature. The "sand color" you mention is normal, it's how SC4 paints these very-low-elevation areas (if you lower the terrain just a little, it will turn to water). For a different look install a "Terrain Mod", or a "Beach Mod". Also the "Terrain Properties" exemplar contains several properties that control the appearance of beaches, eg how far from water you get the "beach" textures.

Also I was able to find regional grayscale images usually large files (some over 100mb) on the USGS site... goto www.nationalmap.gov and use the "Earth Explorer" to find any location, set selection to "use map" and then choose DEM ASTER files and click results tab. This process has given me the best results for specific grayscale files. They are also very large which I believe is what COGEO was referring to as being lossless format.

DEM files are so large not because of lossless compression, but because they are usually... uncompressed. Plus they are not 8-bit grayscales either, they are 16-bit .TIFFs; they contain elevation data as two-byte integers (-32768..32767) representing the elevation in whole meters. 16-bit images are not recognisable by many programs though, neither they can be directly imported by SC4 (you have to convert them to 8-bit grayscale or .SC4M).

The only plugin I'm currently running is the NAM; does that include a height mod?

As jason leo said, NAM does not contain any Height Mod.

I've also ensured that photoshop is saving bitmaps with indexed colours.

No improvement.

I didn't say make an indexed image! The opposite actually! I read my post again just to make sure that I didn't misguide you, but I didn't find anything wrong. My suggestion was use a non-indexed (or else "non-paletted") format, or alternatively make sure that the image uses a full-sized (non-optimised) palette, and indices correspond to the same RGB value, eg palette entry no 84 is colour (84,84,84). The grayscale images in my maps are RLE-compressed, paletted .BMPs, but this is neither necessary, nor "the best". Try out grayscale .PNGs: no palette (check the savefile options), lossless encoding, good compression. Otherwise send me the file (or a part of it) to take a look.

[Verran 0]

Here's an example of the image I'm using. This one is a grayscale bitmap, RLE compressed. I've also attempted importing both as a Pong, JPG, and a GIF. I must retrace my previous statement; after further examination the elevation data was correct when I imported from an indexed image. However, if I convert this bitmap, for example, to an indexed image, the colors get darker, requiring me to create an image from scratch in that mode (which means no smoothing.) This has me thinking there is some misalignment with the palette, but I don't know how to fix that.

I really thought this was as simple as going 8-bit grayscale mode and painting away.

Untitled-1.bmp

Edit: Max elevation was 750ish and minimum was 0. Something edited sea level in my game, and I have no mods running. Now this is getting interesting.

Wait, I lied again. 84/84/84 gives me a 192m elevation, below sea level. 0/0/0 gives me 0m, and 255/255/255 gives me 750ish.

  Edited January 13, 2013 by Verran  

[cogeo 960]

Here's an example of the image I'm using. This one is a grayscale bitmap, RLE compressed. I've also attempted importing both as a Pong, JPG, and a GIF. I must retrace my previous statement; after further examination the elevation data was correct when I imported from an indexed image. However, if I convert this bitmap, for example, to an indexed image, the colors get darker, requiring me to create an image from scratch in that mode (which means no smoothing.) This has me thinking there is some misalignment with the palette, but I don't know how to fix that.

Again, there is no reason to necessarily use an indexed/plaetted image. SC4 can import almost any type of image file, and it doesn't need to be indexed or not, compressed or not, or of any specific format. It only needs to be 8-bit (it can't import 16-bit images, so you have to convert them yourself). The format of the image you have attached is very much OK: it has no palette, it is 8-bit, it's lossless, and features some compression too. I have also saved it as a non-paletted, grayscale .PNG, which has a much better compression. Both images contain the same data and work the same, and it would even work the same if you had saved it as an uncompressed .BMP (only the filesize would have been much bigger).

I really thought this was as simple as going 8-bit grayscale mode and painting away.

Max elevation was 750ish and minimum was 0. Something edited sea level in my game, and I have no mods running. Now this is getting interesting.

Wait, I lied again. 84/84/84 gives me a 192m elevation, below sea level. 0/0/0 gives me 0m, and 255/255/255 gives me 750ish.

I checked your file (it was OK), and tried rendering it, after adding two special rectangular areas, painted with #84 and #83 gray shades. These correspond to the elevations just above and below the sea level respectively (252m and 249m) in an installation without a height mod installed. This is what I got (I have superimposed the terrain query dialogs):

It's just fine, the first area is "land" (you can build/zone/etc on it), the second is "water" (you can't build there), while the gray area around the island gave 192m ingame (64x3=192), and white gave 765m. So this is absolutely acceptable. You can find the modified file in .BMP and .PNG format here.

[Verran 0]

So here is what I found. I opened the bitmap you gave me, and it was saved as an indexed image. I opened the pong you gave me, and it was saved as a grayscale. Both are the same image, but the pong looks lighter in photoshop than does the bitmap. For giggles I saved a pure #84 bitmap in photoshop, and opened it in MS Paint. The color was saved as #63. I've got something deep inside photoshop screwing this all up, don't I?

Started screwing with the color profile inside photoshop earlier. Got close. #82 is giving me below sea level, #83 is going above.

  Edited January 17, 2013 by Verran  

[cogeo 960]

This is really strange! Both files are grayscale, not indexed, ie neither the .BMP nor the .PNG has a palette. When I open any of the files in PaintShop Pro and select Colors->Edit Palette, I get a warning that says "Changing the palette of a greyscale image will force the image to be converted to 8-bit indexed color". This means that the file is grayscale, and it either contains no palette (the grayscale values directly identify the intensity, a number from 0 to 255) or maybe the file contains the "default" grayscale palette, ie (0,0,0), (1,1,1)...(255,255,255), ie including all 256 intensity levels in increasing order (with each palette entry having all three colour components equal to the palette index). And both files look identical and behave the same (the .PNG is smaller because the PNG compression is better than RLE - both compression types are lossless though). Actually untitled-1.bmp (your file) is of exactly the same format as untitled-2.bmp, so these files should behave and look the same in Photoshop (except of course for the two rectangular areas I painted over). Not to mention that both the .BMP and .PNG look identical in the Windows Picture Viewer as well as MS-Paint.

So there must be some (implicit) conversion taking place in Photoshop. Can you check the palette? Particularly if palette entry #83 is RGB(83,83,83) and #84 RGB(84,84,84)? Or maybe some option like "Automatic Gamma Correction", or "Automatic Intensity Adjustment" or "Automatic Contrast Adjustment" or so? Or "Palette Optimization"? Don't know. But still, why does it behave differently for the .PNG and .BMP? And finally, if it works for the .PNG and not the .BMP, why not save your work on a .PNG? PNGs are smaller, after all. They are lossless too, and the format is simple grayscale. And use of an indexed (paletted) format isn't a requirement for SC4 either. Finding "what's going on with .BMPs" may be interesting for your Photoshop installation, but not a requirement for importing in SC4.

Still, you can post such a "mangled" file, to take a look.

[ex_p4t 0]

I was having this problem aswell with Photoshop - above sea level seemed to be precisely at gray value 107. The problem lies in the numerous color management options available in Photoshop, all of which are basically useless and confusing for purely computer-side image manipulation. Exporting a grayscale Photoshop file as bmp will darken the image with the default color management options.

 

How to solve the problem in Photoshop CS6, Windows version: Go to Edit -> Color Settings. In the Settings box, load Monitor Color. Ignore any warnings. This will disable any color profiles for future images you create in PS. If you have a previously created image you would like to use in the future, open the image, go to Edit -> Assign profile, and select "Don't Color Manage This Document". Hope this solves your problem.

[omnimutant 2]

I was having this problem aswell with Photoshop - above sea level seemed to be precisely at gray value 107. The problem lies in the numerous color management options available in Photoshop, all of which are basically useless and confusing for purely computer-side image manipulation. Exporting a grayscale Photoshop file as bmp will darken the image with the default color management options.

 

How to solve the problem in Photoshop CS6, Windows version: Go to Edit -> Color Settings. In the Settings box, load Monitor Color. Ignore any warnings. This will disable any color profiles for future images you create in PS. If you have a previously created image you would like to use in the future, open the image, go to Edit -> Assign profile, and select "Don't Color Manage This Document". Hope this solves your problem.

^ This is a huge life saver. I too was having the exact same problems as the op. I figured it was something quarky with PS because it kept changing the values on me. Now it's smooth sailing. I will mention that PNG's and Jpegs don't open for me in game though, but BMP's work perfectly fine as illustrated below: