FiSHMan: An FSH image management tool

[DarkMatter 21]

Builderman, since I posted the source, I don't really care if people reverse engineer it. (And, btw, have you TRIED to reverse-engineer MSIL? Friggin pain in the ass...)

Ralphy, thanks for posting it, but I hope you don't mind keeping after it too. I'll be making regular updates to my site, as I add new features.

7th, I didn't think I even HAD an add new bitmap option in the menus. Currently, all FiSHMan does is open FSH files, save them out using 32bit pixel format, and allow you to export the bitmaps inside an FSH to one of several image formats.

FiSHMan is still VERY VERY VERY beta, and its missing most of the features I've talked about. It does not yet allow you to create FSH files from scratch. It currently doesn't support the pixel format SC4 uses, which is the two DXT formats 1 and 3. It also doesn't have full support for palettes either. There is still a LOT of work to be done on the FiSHMan program itself. Most of the work I have completed is on the FSHLib.dll file thats included. That file is the main library for manipulating FSH files, and is where most of the dirty work with them is done.

I'm working on the BAT, FiSHMan, and a .dat editor all at the same time, as they are all kind of interrelated, so there will be lots of updates fairly frequently.

[the7trumpets 10]

thanks DM - I didn't mean to be pushy, I understand it's early beta.

 

I was just being hopeful   I haven't been able to get FSH Tool to compress the textures I have made so far, so I'll keep a close eye on the development.  Thanks alot again,  the program looks great!

[Buggi 685]

----------------

On 7/21/2003 5:25:57 PM DarkMatter wrote:

FiSHMan is still VERY VERY VERY beta, and its missing most of the features I've talked about. It does not yet allow you to create FSH files from scratch. It currently doesn't support the pixel format

SC4

uses, which is the two DXT formats 1 and 3. It also doesn't have full support for palettes either.

----------------

So... what DOES it do?

J/K!!!

[Norwegian Neko 12]

All hail DM! He is the Mod God! err, Squad!

[DarkMatter 21]

*SMACKS BUGGI*

When I get home today, I'll try to start working on the FSH creation tools. I hadn't intended to release anything yet, but everyone had gotten to the point of begging me for something...so....

It'll be pretty simple. Each folder in the tree...bitmaps, palettes, etc. will have a context menu. You right-click the appropriate node and select new from the menu to add an item. There will be a palette editor built in, and you will be able to save palettes to a custom format. Until I crack the DXT compression code, you'll only be able to save in either 32bit or 24bit color (32bit includes the alpha channel). I'm working on reduction code so you can save to 16bit formats (with and without alpha). FSH files support what is called a 4x4 format, where there are 4 bits for each color component and 4 bits for alpha, for a total of 16bits. I found some FSH files on the net that used this format, so I'm going to try and include support for this one too.

The most important formats, though, the 8bit indexed and DXT formats, are also the most difficult to implement. For 8bit indexed images, I need to learn how to do dithering, which takes the full 24bit color for each pixel and creates a table using the most common colors, and reducing similar colors to use one of the colors in the palette. Its a fairly daunting and complex process to produce a quality, dithered image that uses only 256 colors when it originally used 16.8 million. The DXT compression code is alomst done, I'm just having a few problems converting it to .NET compatible code.

Keep tabs on my site over the next few days for updates to the binary. I'm going to try to get these base features implemented quickly so that FiSHMan can be useful. I won't update the source code until I have all of these features implemented. Once thats done, I'll start working on the more advanced features like viewing animation files (FSH, MCO, and AVPFs), the slicing tool to load up a full, rendered image of a building and slice it down to 256x256 pixel tiles, etc.

[DarkMatter 21]

Ok. Updated the binary on my site (same links). Its not much, but it now saves out FSH files that have DXT compressed images. Right now, the only real use for this is converting a compressed FSH to an uncompressed FSH, as I havn't added any options to the save feature. I'll try to add a filter dialog to the save process so you can configure how the FSH saves (weather its compressed or not, editable header options, etc.)

 

The DXT compression was the main reason I hadn't added any FSH creation features. Now that's out of the way, I can add the tools to create FSH images from scratch. Thats going to be my big project tomorrow, as I finally have a day off of work and more than a couple hours to spare on this project.

[Equinox 1,218]

Can custom graphics be stored in a .SC4LOT file?  I.E. can you download a custom lot from the LEX that has custom graphics and have them show up in your game?

[DarkMatter 21]

Hmm. Interesting question. My guess is that you would need the .dat file with the custom textures in it placed in your plugins directory. We might want to consider making a single .dat file where people can put their additional textures and overlays. We could have a single person maintain this .dat file, like 7th, and just have people submit their textures to him. That way, there would be a single download for all the additional textures we create, making it easier to manage them.

[the7trumpets 10]

----------------

On 7/24/2003 3:48:06 PM DarkMatter wrote:

Hmm. Interesting question. My guess is that you would need the .dat file with the custom textures in it placed in your plugins directory. We might want to consider making a single .dat file where people can put their additional textures and overlays. We could have a single person maintain this .dat file, like 7th, and just have people submit their textures to him. That way, there would be a single download for all the additional textures we create, making it easier to manage them.

----------------

wheee!!!!    

I'd love to be in charge of organizing such a file.  It would help alot if it were centrally maintained file, to ensure that no two IDs are used, and to ensure that it is properly indexed.  Consider this my volunteer slip

 

However, I have messed around a little using FSHTool to try to recompress some FSH files and insert them into the dat file, and they have not shown up as available textures in the lot editor.  So, either I'm not using FSHTool correctly to compress them (they are showing up with a "N" no flag in the reader under the compressed column), or there is some sort of indexing file that stores locations of all the textures that needs to be edited in order to insert new textures.  I'll keep pecking at this, but if anyone has any ideas, let me know.  I'll keep you guys posted

[ilive 36]

There are also two programs made which handle fsh format : it's FshEd and EAGE

 

you can find them at

http://dynamic2.gamespy.com/~nbalive/phpBB2/viewforum.php?f=20

 

perhaps one of them could resolve your problem , 7th ...

[GouRou 2]

you can put custom graphics in a lot file, they are simply .dat files I think, but with the different extension, more people are likely to be able to differentiate between them and the normal plugins. also good for places like the exchange so that it can load just the lots.

[GouRou 2]

I just checked a hunch, the .sc4 map files are readable by the reader, meaning they are .dat too, you could package your custom lots in with your city if it doesnt already

[GouRou 2]

I opened a city that used mods, the mods are in the city file, simcity apparently packages it all together, good to know that when we exchange cities, you wont have to snag the lots and buildings and mods other people used. might be a good idea from now on to package the props in with the lots, then someone wont need to have the building prop plugin file to use the lots.

[DarkMatter 21]

Ok. I added some basic FSH creation to FiSHMan. You can now create a new .FSH file, add bitmaps, and save it to disk. Currently, supported image formats are DXT1, DXT3, 32bit color and 24bit color. I havn't yet worked out the 16bit formats, and the 8bit format will probably elude me for a while (it works, but you have to manually create or load palettes, so automatically reducing a 32bit image to an 8bit image and generating the palette in the process isn't supported). I'll probably have the 16bit formats ready later today.

 

Alpha maps are fully supported in the insert bitmap dialog. If you don't choose an alpha map, a full white map will be generated for you. The program doesn't yet let you configure the order of the bitmaps after they have been added to the FSH. Thats a feature I hope to include eventually, but its not particularly critical. I'm working on a tool to edit palettes and a tool to replace a selected color in a bitmap with an alternate color, which I think WILL be useful.

 

Keep in mind, SC4 makes use of the DXT compressed image format. The DXTC format, developed by Microsoft for DirectX, is fully hardware accelerated. The default format when adding an image is DXT1, but you can choose DXT3 if you need alpha. So you should use the DXT formats for any textures you create.

[the7trumpets 10]

I couldn't get the insert -->bitmap to do anything.

 

I did notice that I have .NET framework 1.1 instead of 1.0.  Is that a problem?

[DarkMatter 21]

Sorry, I forgot to mention that I havn't uploaded it yet. I'm still working out some bugs. I'll upload it shortly and let you all know when I do.

 

About the .NET version, there isn't any particular reason 1.1 won't work, but it has changed a bit from 1.0, so I can't guarantee everything. I myself don't have 1.1, so I can't test with it. I guess you'll be my official .NET 1.1 beta tester.

[builderman 3]

I think the .NET assemblies take care of that automatically, to reduce "DLL Hell."

[DarkMatter 21]

Well, when it comes to the runtime, not even Microsoft will guarantee that .NET 1.0 programs will run perfectly in 1.1. There are certain core changes that sometimes require you to make a small adjustment. The DLL hell thing lets you run multiple versions of a .dll side-by-side, so you could have .NET 1.0 AND 1.1 installed at teh same time, and you could use programs that use both, at the same time.

 

Since I havn't tested FiSHMan with the 1.1 runtime, I don't know if certain changes in it will cause unexpected behavior in my program.

 

I've uploaded the latest version. Its on my site for those who want it. This version adds the insert bitmap dialog, and the ability to save FSH files using the Save As... file menu option. I havn't yet added code for the regular Save option.

[DarkMatter 21]

Ralphael, do you think you could upload the latest version to SimTropolis? Since my site isn't guaranteed to work, it would be nice to have it in both places. Thanks.

[Karybdis 34]

Previous response to GouRou

You are correct sir. We have been working on decoding Sc4 savegame DAT files for about 2 months now and have been able to read them for about 2 and a half. They're quite complex though. I hope to have a map of the savegame dat ready within a month or so.

[DarkMatter 21]

I just thought I'd let everyone know, I'm having some trouble with the 16bit formats. I don't have any reference FSH files that have 16bit images in them, so I have no way of testing either my load or save code. So, I'm going to officially mark the 16bit formats as unsupported now, until I manage to find some 16bit FHS files. I don't think this will be much of a concern, though, since we need to use DXT formats for SC4 FSH files.

[RalphaelNinja 244]

The latest FiSHMan tool is now (finally) in the Mods Section....

 

Ralphael

[DarkMatter 21]

Yeah, dude! Thanks for the update! Heahaha!

[DarkMatter 21]

Ralphy, the version number is v0.1.2. I used 0_1_2 in the file name so I didn't have to use multiple periods. If you could change the name in the mods section to 0.1.2, I'd really appreciate it. Thanks.

[RalphaelNinja 244]

Dirk's mod section doesn't like multiple periods either....

 

Edit: okay... the header I seem to edit back to 0.1.2....

[DarkMatter 21]

Ok all FiSHMan fans. I've made another update, up to version 0.1.3. This update adds a batch image processor. This was a request from GouRou, who is planning on making a LOT of new textures and overlays. This batch processor will allow you to convert a nearly unlimited number of 128x128 bitmaps (and their 128x128 alpha maps if they exist) to 5 FSH files and 5 related .TGI files. You can then import the fsh files using the reader, and create a new .dat file with your new base and overlay textures.

 

The batch processor is still a work in process, but it currently works. There are some rules you will need to follow to allow the batch processor to work. First, your input files will all need to have the same base name, with a 4 digit number attached. For example, if your images are named texture, each will need a number, starting at 0000, and increasing by 1 from there. The alpha maps for each image should have the same name, but instead of just having a 4 digit number, they should have the following: -a000. The -a before the number indicates the file is an alpha map. Assuming you have 5 textures to convert, the names would be:

 

texture0000.bmp

texture-a0000.bmp

texture0001.bmp

texture-a0001.bmp

texture0002.bmp

texture-a0002.bmp

texture0003.bmp

texture-a0003.bmp

texture0004.bmp

texture-a0004.bmp

 

When you pick a file to do a batch convert, you will need to remove the trailing numbers, as the program will automatically append numbers for you. When you process the batch, the program will take your base 128x128 pixel texture, and generate 5 .fsh files from it. The files will be several sizes, and will have specific names. Using the textures from above, the output filenames would be (assuming you typed texture.fsh as your output fsh file):

 

texture0000_s0.fsh texture0000_s1.fsh texture0000_s2.fsh texture0000_s3.fsh texture0000_s4.fsh

texture0000_s0.fsh.TGI texture0000_s1.fsh.TGI texture0000_s2.fsh.TGI texture0000_s3.fsh.TGI texture0000_s4.fsh.TGI

texture0001_s0.fsh texture0001_s1.fsh texture0001_s2.fsh texture0001_s3.fsh texture0001_s4.fsh

texture0001_s0.fsh.TGI texture0001_s1.fsh.TGI texture0001_s2.fsh.TGI texture0001_s3.fsh.TGI texture0001_s4.fsh.TGI

texture0002_s0.fsh texture0002_s1.fsh texture0002_s2.fsh texture0002_s3.fsh texture0002_s4.fsh

texture0002_s0.fsh.TGI texture0002_s1.fsh.TGI texture0002_s2.fsh.TGI texture0002_s3.fsh.TGI texture0002_s4.fsh.TGI

texture0003_s0.fsh texture0003_s1.fsh texture0003_s2.fsh texture0003_s3.fsh texture0003_s4.fsh

texture0003_s0.fsh.TGI texture0003_s1.fsh.TGI texture0003_s2.fsh.TGI texture0003_s3.fsh.TGI texture0003_s4.fsh.TGI

texture0004_s0.fsh texture0004_s1.fsh texture0004_s2.fsh texture0004_s3.fsh texture0004_s4.fsh

texture0004_s0.fsh.TGI texture0004_s1.fsh.TGI texture0004_s2.fsh.TGI texture0004_s3.fsh.TGI texture0004_s4.fsh.TGI

 

The last part of each file, the _sN, represents the sequence number. All base and overlay textures are a sequence of 5 files, 0 through 4. The image shown at the farthest zoom is 0, the closest zoom 4. These numbers will help you import the images into a .dat. The batch processor will also create .TGI files based on a starting instance number and increment.

 

The latest version with the first incarnation of batch processing is on my site and also posted here on simtropolis in the mods pages.

[the7trumpets 10]

you ROCK darkmatter!!!       

 

 

One question...any idea how to easily create a batch import file to use the "batch import" feature in the reader, assigning instance IDs automatically?

[DarkMatter 21]

Yeah, I can do that. Its almost done, actually.

[DarkMatter 21]

Ok, another update. FiSHMan 0.1.4 is beeing uploaded now. It will be available on my site and here on simtropolis. This one is a bit more streamlined than 0.1.3 in the way batch processing works. Here are the changes:

 

1) You no longer have to abide to scrict naming rules. If you only wish to use 1 digit in a filename, then the batch processor will read files 0 through 9. You can use any form of numbering system, like 00, 01, 02, or 0001, 0002, 0003. You no longer have to start at zero, either. You can pick a number in the middle of a sequence if you wish.

 

2) The option to generate a sequence of .fsh files has been added. If you just wish to convert single images to single .fsh files, you can uncheck the Generate Sequences checkbox in the Batch File Add dialog. On the other hand, if you DO wish to generate sequences, keep it checked, and make sure your input files are 128x128 pixels. The batch processor will automatically scale that down to 64x64, 32x32, 16x16, and 8x8 for you.

 

3) Clicking the Batch Add button in the Batch File Add dialog no longer processes. Instead, you can batch add several sequences of files to the list, then click the process button above the list in the list window to process the whole thing.

 

4) You can how save the list out to one of two batch formats. A comma-dilimeted format proprietary to FiSHMan, or to Ilives batch.txt format. Using ilives batch format, you can convert a batch of images, then import the whole batch in a few steps. This should make the batch processor much more useful.

 

5) You can also add individual files to the batch process list. You can perform batch add and single adds numerous times without having to close the Batch File Add dialog or create a new list.

 

I think thats about it for now. This should hold everyone for a while. I'll work on streamlining some of my code, and make another source code update. I'll try to add some basic editing tools to FiSHMan in the next release, provided that I don't have to make any bug fix releases in the mean time.

[ilive 36]

Is there a way to open a FSH in the FishMan by just double clicking on this file ?

It opens Fishman but no file is opened (the goal is to open fsh files from the Reader)