Possible "OpenSC4" ?

[wheeeze 90]

On 5/25/2022 at 12:08 AM, nos.17 said:

Looks promising! I'll keep tabs on the project, and if you need any advice on the structure of DBPF files, I'll be happy to offer up what I know.

Would you happen to know where I can find the documentation on the savegame's terrain subfile (0xa9dd6ff4 0xe98f9525 0x00000001) format? Wiki says it's been fully decoded, but no link there

[rsc204 21,806]

13 hours ago, Pakistani890 said:

hmm ok that explanation was life changing and what about unreal and also the C:S dev didnt even think of optimisng the game what about that and one more thing i am very supportive of Open sc4 i am just giving suggestions that might help or ease the process

To understand why using an off-the-shelf engine, not designed to create the type of game you are making is always going to lead to compromises, you need to have a grounding in software development/coding. Even if I wrote an essay here explaining it in depth, which I'm not going to do, it's not likely you will fully understand it anyhow. Incidentally my previous response applies to the Unreal engine as much as it does Unity, not to mention any engines not intended for simulation games. Can you use one to make a functioning simulation, clearly you can, it's just less than ideal and comes with drawbacks.

I'll note too I didn't say the CS developers didn't think of optimising the game, simply that it isn't well optimised. Part of this may come from actual limitations, things that can not be altered if you've chosen to use such a game Engine, but it is also likely that part of this relates to cost too. If you licence a 3rd party engine to save spending say 200k making your own (good coders get paid very well), you aren't likely to want to spend a lot of money 'remaking' it. So what you tend to end up with is a product that is flawed and inefficient, which would IMO be a fair description of CS in this context. To put that 200k in context some large developers spend way more creating just the game engine, but often that will be used by more than one game so the investment eventually pays off. Literally that's why you can licence Unreal, because Epic made it for their games but saw a way to make more profit by licensing the engine than selling games. 

I get that for a small company like Colossal Order, making their own engine was probably never an option, simply because it would have been too expensive. Licensing one intended for Simulators was probably next to impossible, I don't know of one personally, so they were simply doing what they had too.

[wheeeze 90]

By the way, allow me to shill for the small Discord server I've made to discuss my OpenSC4 project: https://discord.gg/heDXuJvNf7!

[TheMurderousCricket 5,045]

5 hours ago, rsc204 said:

Can you use one to make a functioning simulation, clearly you can, it's just less than ideal and comes with drawbacks.

Like @Pakistani890 I am neither a videogame engine expert but I can see one devastating drawback of changing the game engine for SC4...

I believe that this would completely derail any custom textures and prop sets that we take for granted now...  

After all, it's all about rendering the game's objects. Different method and engine architecture would definitely throw all this down the PEG's garbage chute. I'm not sure if or how conversion would be possible...

[wheeeze 90]

1 minute ago, TheMurderousCricket said:

Like @Pakistani890 I am neither a videogame engine expert but I can see one devastating drawback of changing the game engine for SC4...

I believe that this would completely derail any custom textures and prop sets that we take for granted now...  

After all, it's all about rendering the game's objects. Different method and engine architecture would definitely throw all this down the PEG's garbage chute. I'm not sure if or how conversion would be possible...

There is no reason for that to happen. Nothing has to change at all.

[Pakistani890 122]

7 hours ago, rsc204 said:

20 hours ago, Pakistani890 said:

hmm ok that explanation was life changing and what about unreal and also the C:S dev didnt even think of optimisng the game what about that and one more thing i am very supportive of Open sc4 i am just giving suggestions that might help or ease the process

To understand why using an off-the-shelf engine, not designed to create the type of game you are making is always going to lead to compromises, you need to have a grounding in software development/coding. Even if I wrote an essay here explaining it in depth, which I'm not going to do, it's not likely you will fully understand it anyhow. Incidentally my previous response applies to the Unreal engine as much as it does Unity, not to mention any engines not intended for simulation games. Can you use one to make a functioning simulation, clearly you can, it's just less than ideal and comes with drawbacks.

I'll note too I didn't say the CS developers didn't think of optimising the game, simply that it isn't well optimised. Part of this may come from actual limitations, things that can not be altered if you've chosen to use such a game Engine, but it is also likely that part of this relates to cost too. If you licence a 3rd party engine to save spending say 200k making your own (good coders get paid very well), you aren't likely to want to spend a lot of money 'remaking' it. So what you tend to end up with is a product that is flawed and inefficient, which would IMO be a fair description of CS in this context. To put that 200k in context some large developers spend way more creating just the game engine, but often that will be used by more than one game so the investment eventually pays off. Literally that's why you can licence Unreal, because Epic made it for their games but saw a way to make more profit by licensing the engine than selling games. 

I get that for a small company like Colossal Order, making their own engine was probably never an option, simply because it would have been too expensive. Licensing one intended for Simulators was probably next to impossible, I don't know of one personally, so they were simply doing what they had too.

Oh ok even though i am just a beginner in coding and game dev. I now clearly understand your point, a inhouse game engine performs well than a 3rd party one. Ok no problem but then what about GODOT game engine. Stll thx for explaining me and also will the textures and ambiance of SC4 still be there

[Ulisse Wolf 4,399]

2 hours ago, TheMurderousCricket said:

Like @Pakistani890 I am neither a videogame engine expert but I can see one devastating drawback of changing the game engine for SC4...

I believe that this would completely derail any custom textures and prop sets that we take for granted now...  

After all, it's all about rendering the game's objects. Different method and engine architecture would definitely throw all this down the PEG's garbage chute. I'm not sure if or how conversion would be possible...

You have to wait and have a working copy of this project to understand if the current mods are compatible with this open source version. There are advantages and disadvantages but surely having SC4 that supports modern video cards, managing more cores and using more RAM is very useful. Of course I recommend you to also publish on SC4Devotion as there is a lot of technical information about SC4 modding there. I know there is a game engine reimplementation project (Consider that they are two projects as SimCity 4 and The Sims 2 use the same game engine)

[nos.17 5,112]

16 hours ago, wheeeze said:

Would you happen to know where I can find the documentation on the savegame's terrain subfile (0xa9dd6ff4 0xe98f9525 0x00000001) format? Wiki says it's been fully decoded, but no link there

Check out this for parsing savegame files: https://github.com/Killeroo/SC4Parser

[Leo -- 186]

Since we are talking about game engines and how mainstream engines have a underwhelming ability to handle simulations that are needed for a SimCity, I was wondering how does UE 5 hold up on a theoretical standpoint? Since it's written C++, a mid to low language it offers a significant speed advantage over something like JS or C#, however UE 4's codebase is more geared to FPSs (not sure how it is in UE 5 but there could be a performance hit there).

UE 5 also has Mass AI and Mass Entity which appears to be similar to the agent visualization and simulation in GlassBox. And by my understanding there are some "sub features" if you will that could also useful for city simulations like ZoneGraphs which looks that they could be used a cheaper version of navmeshes for networks.

Also the World Partitioning System looks like it could be used to allow larger cities by dividing them up and only rendering what's needed with less overhead, and nanite could be used to help with LODs.

But considering that UE 5 in a prerelease state there is a high likelihood that there will be many performance issues which might also negate some of these possible simulation tools. Also UE 5 is not open source only source available so it wouldn't be an OpenSC4. But if I understand UE 5's feature set correctly it looks like building an in-depth city builder with both a good deep, complex, extensible, scalable simulation, next-gen 3D graphics, all with acceptable performance, is practical.

But I don't have any experience with game engines or game development so I might be misunderstanding these features and what they are for, nor is this a pitch for a SimCity remake with UE 5. Just a question on how well it could hold up with city simulation game.

[wheeeze 90]

7 hours ago, nos.17 said:

Check out this for parsing savegame files: https://github.com/Killeroo/SC4Parser

Thank you! Looks like the terrain map was really simple after all

[Tarkus 6,932]

Since there's discussion of engines, I thought I'd add that @simmaster07 is working on reimplementing Maxis' original engine through his SCION project.

-Tarkus

[wheeeze 90]

3 hours ago, Tarkus said:

Since there's discussion of engines, I thought I'd add that @simmaster07 is working on reimplementing Maxis' original engine through his SCION project.

-Tarkus

Oh crap, that means I'm working on a redundant project

[Ulisse Wolf 4,399]

2 hours ago, wheeeze said:

Oh crap, that means I'm working on a redundant project

All is not lost. You can contact @simmaster07 and join efforts to create openSC4

[TheMurderousCricket 5,045]

7 hours ago, wheeeze said:

Oh crap, that means I'm working on a redundant project

I wouldn't say so @wheeeze. This Community has proven, time and again, that no effort is lost in exploring the game's possibilities and inner workings.

Even if a solution is not universally appreciated, I guarantee that a handful of people will show up who will definitely find a use for and appreciate your work.

I'll keep my fingers crossed for both of you.  It's always good to have two options rather than one!

[Cyclone Boom 16,852]

@wheeeze

I echo the above two replies.

We encourage you to keep striving forwards with your project. Each of us have different skill sets ranging from technical to artistic, and see unique ways to solve any given puzzle. It is healthy for the community how there are people like yourself still exploring these sorts of projects. Then even as ulisse says, perhaps at some stage it could be worth collaborating with simmaster and seeing what could be developed in some form of joint effort. There's no pressure on that though.

In the meantime do keep at it, because it's only a positive for all such projects when they do come to fruition.

[cydonianmystery 96]

I'm not an expert on game engines, but something like UE5 would be massive overkill for a game with SC4's scope. The engine alone would probably be bigger than all of SC4's game files, which comes just over a gig.

Gonna agree with @rsc204 about Unity though, as someone who's sunk over 1,200 hours into Cities Skylines. With all major expansions CSL needs at least 16gb of RAM, and that's with no custom assets or mods. Unity is probably the last engine you want to use for a simulator game, but was probably the best Collossal Order could license at the time. If the rumors of a CSL2 are true (they've been ongoing for years, so make of that what you will), I'd expect CO to ditch Unity.

EDIT: Oh I seem to have missed a whole page of additional replies. Ignore me.

  Edited June 3, 2022 by cydonianmystery  

[Leo -- 186]

20 hours ago, cydonianmystery said:

I'm not an expert on game engines, but something like UE5 would be massive overkill for a game with SC4's scope.

Yeah I was just asking how would it work for a game like CS

[wheeeze 90]

A bit of an update since this thread hasn't been posted in for over a week: the project is not dead, I just recently found out UIs were defined in the game data, so I'm trying to read them and avoid having to recreate all interfaces by hand. However there are dozens of different UI definitions in the game files, and knowing which is which is far from easy.

[CorinaMarie 29,574]

12 minutes ago, wheeeze said:

However there are dozens of different UI definitions in the game files, and knowing which is which is far from easy.

Does this help?

UIs Split Out - For wheeeze.zip

[wheeeze 90]

3 hours ago, CorinaMarie said:

Does this help?

UIs Split Out - For wheeeze.zip

If I they contain the proper TGIs, yes definitely! Thank you

[CorinaMarie 29,574]

44 minutes ago, wheeeze said:

If I they contain the proper TGIs, yes definitely! Thank you

They are simply parsed out of SimCity_1.dat with no changes and put in categories which made sense to me. There's at least one that prolly needs merged into another, but RL has become erratic for me and the project we were developing is on hold so that's my first sorting draft version.

[wheeeze 90]

Small update: while I've been on vacation for two weeks in the first half of august, work hasn't stopped! Special thanks to @wibbert for implementing most of the base terrain rendering in the city view, plus some other fixes on my code, here's what it looks like :

On my end, I've done quite the progress on UI rendering and handling, here's what the region UI looks like. Some buttons are functional and some aren't, the positioning isn't perfect, and there are, for some reason I ignore, extra images, but the hard work on the UI rendering is behind us :

 

[Naomi57 2,985]

12 hours ago, wheeeze said:

Small update: while I've been on vacation for two weeks in the first half of august, work hasn't stopped! Special thanks to @wibbert for implementing most of the base terrain rendering in the city view, plus some other fixes on my code, here's what it looks like :

Great job with rendering, @wheeeze.

I have a software development background, with a specialisation in data architecture.  It's quite awesome to see concrete progress on OpenSC4.

While I know your work is coming from a different (and exciting) angle, I see some functional overlap between your work thus far and some map rendering mods, which might be a useful reference point for identifying what SC4 map data is already known:

• Region View mods:
    https://community.simtropolis.com/files/file/11580-the-melbourne-transport-map-mod/ 
    https://community.simtropolis.com/files/file/14219-the-metropolitan-transport-mod-v1/ 
    https://community.simtropolis.com/files/file/17605-region-transport-legend/ 
    https://community.simtropolis.com/files/file/11402-edmonton-transportation-map-mod/ 
• SC4 Terraformer (wouanagaine)
    https://sc4devotion.com/csxlex/lex_filedesc.php?lotGET=731
• SC4 Cartographer (wouanagaine)
    https://sc4devotion.com/csxlex/lex_filedesc.php?lotGET=4019
    https://community.simtropolis.com/files/file/33956-sc4cartographer/

In particular, wouanagaine's work is in Python, so his source code would likely be invaluable, if you haven't seen it yet.  There's some other C# and Python tools that dig into the SC4 data, both in published tools and experimental projects.

Quote

https://community.simtropolis.com/forums/topic/59708-show-us-your-city-transit-maps/?page=43&tab=comments#comment-1740684
Edvarz - Posted November 23, 2020

Trying out the new SC4 Cartographer for the region I've been working on for I don't know how long: ...

 

[Turnstyled 485]

I'm so excited about OpenSC4! I really hope your minimum viable product is ready by the game's 20th anniversary like you hope! Sometimes, I feel like I'm a small fry here with people who have such vast knowledge of the inner workings of SimCity 4 and who make such amazing creations when I just play for fun. It's all good, though. I can't wait until something is playable!

[wheeeze 90]

14 minutes ago, Turnstyled said:

I'm so excited about OpenSC4! I really hope your minimum viable product is ready by the game's 20th anniversary like you hope! Sometimes, I feel like I'm a small fry here with people who have such vast knowledge of the inner workings of SimCity 4 and who make such amazing creations when I just play for fun. It's all good, though. I can't wait until something is playable!

If it makes you feel any better, I had zero ideas of the inner workings of SC4 too, until I started working on the project

[TheMurderousCricket 5,045]

As a sailor I have to say that the waves are moving at near gale and should probably be slowed down and sized down a bit. 

Otherwise than that the terrain looks nice. It's not an easy task given how blocky the transitions tend to be in the original game.

 

[Naomi57 2,985]

3 hours ago, TheMurderousCricket said:

As a sailor I have to say that the waves are moving at near gale and should probably be slowed down and sized down a bit. 

Otherwise than that the terrain looks nice. It's not an easy task given how blocky the transitions tend to be in the original game.

I rather enjoyed the "gale" look of the water.  It provides some interesting possibilities for weather effects!

[TheMurderousCricket 5,045]

Just now, Naomi57 said:

I rather enjoyed the "gale" look of the water.  It provides some interesting possibilities for weather effects!

Well, it would certainly be a blast to have a controllable weather in SC4O!

[Pakistani890 122]

On 8/21/2022 at 2:54 PM, wheeeze said:

Small update: while I've been on vacation for two weeks in the first half of august, work hasn't stopped! Special thanks to @wibbert for implementing most of the base terrain rendering in the city view, plus some other fixes on my code, here's what it looks like :

On my end, I've done quite the progress on UI rendering and handling, here's what the region UI looks like. Some buttons are functional and some aren't, the positioning isn't perfect, and there are, for some reason I ignore, extra images, but the hard work on the UI rendering is behind us :

 

OMG! That's amazing but will it have water physics

[Turnstyled 485]

When I said that I "feel like a small fry," what I truly meant is that I'm humbled by the work of everyone in this community. "Imposter syndrome" isn't my style.

Because I don't like to deviate from a thread's purpose, I'll say that I've been checking both the Discord server and the GitHub repo. Rotating in city view has been added!