Scripting Reader to place sidewalk textures on overpassed networks

[delta9 24]

Hello Simtrop! Boy this place has changed. Recently getting back into the game after many years, and a lot more on the modding side this time around too, thanks to all the wonderful things GoFSH can do to customize your game (can't thank rivit and MGB/rsc and everyone else enough for that). It's been crazy trying to catch up on how far things have come, much less trying to remember where I left off.

So, much to my joy I learned that RHW can go over NWM now, and much to my dismay I ran into my old nemesis, the sad, bare network textures under overpasses. I searched high and low for a mod or script or anything that could do this for me. The closest I got was MGB's underbridge and the JPN Facelift, which while brilliant, were narrow misses. I already adventured through the Reader figuring out how to change base textures on lots (note to self: make alphas to GoFSH the grass on Clayhurst lots) so I figured there has to be a way to do this.

After much digging (and lots of glass shattering sounds because of Simtrop's Xmas CSS...) I found this SC4D thread by ebina from almost 10 years ago telling how to add base textures to a puzzle piece. Followed the steps, and with a little trial and error due to missing imageshack pictures and never messing with S3D files before, I got it to work! First on the Road, then the TLA-5:

I'm using Gobias's Brooklyn Brick (HD) and this directly references the mid-wealth sidewalk texture Instance. Of course, now I have to figure out why the Road one is scaled down to SD and properly shaded while the TLA-5 one is at the proper resolution but needs color correction. My first mistaken attempt at the Road had the sidewalk layered on top, but it was HD with the puzzle piece brightness, so I'm not sure what happened.

So, to the main point, that's fine and dandy, but I don't want to do this by hand. I want to script it, but try as I might I can't figure out how to muck with anims/meshes/frames. The code for textures makes sense to me and I can get it to work, but not frames and meshes. Part of the confusion for me is not knowing exactly what a "block" refers to on the anims tab. "s3d_Count(s3d, 5)" returns that there's only 1 anim element. Aren't the meshes the anim elements, given that textures are mats elements? And are frames elements of meshes as the script code and Reader UI would indicate?

The plan is to export relevant S3Ds to separate dat(s) and process those. The exact steps that need to be performed (aka automated) on each S3D file is as follows:

 - Mats > Add Texture

 - Set Texture >> 0x08200004

 - Set Flags: depth test, texturing, backface culling; Alpha Func: greater than; Depth Func: less than or equal; Src Blend: one; Dest Blend: zero; Alpha Threshold: 0; Wrap Mode U: clamp V: clamp, magni filter: bilinear; mini filter: linear mipmap linear (trilinear)

 - Anims > [last mesh = 4th mesh = L0 network i.e. Road] >> first/only frame >> change/increment Mat Block from 3 to 4

 - Add Group >> first/only frame >> set vertex block = 3, index block = 0, prim block = 0, mat block = 3 (same as L0 network's initial settings)

 - Save S3D

Seems simple enough but I just can't crack it, and scripting Reader doesn't seem to have ever been a popular topic, at least on anything Google has managed to crawl. Thanks for any help you can offer.

Sidebar funny/tragic story: through some mix of confusion and swapping addons back and forth and figuring out how to GoFSH everything, I nailed one of my other ancient enemies: SAM-7 textures for connecting to NWM. I spent several hours learning GoFSH and making a full texture replacement for all SAM-7 x Ortho&Diag NWM only to learn Tarkus had already done a better job with his Default Asphalt Streets... which I was already aware of and had used at that point! I think trying to use TGN and TRK-NAM at the same time and/or in wrong loading order, and later removing TRK-NAM had convinced me they still didn't exist. oof

[Tarkus 6,932]

Good to see you around, @delta9!  As far as the Reader's scripting capability in the Version 1.x releases, I messed around with it for awhile when it first came out, but ran into similar sorts of issues.  And having as many tricks for expediting the manual process as I do, I kind of gave up on it, as it wasn't proving to be a time-saver for me.  I don't know of too many who have really used it successfully--the NAMites who were big into scripting approaches (e.g. jondor and memo) instead went for the jugular and used Java libraries.

cogeo's SC4 Model Tweaker tool is a big time-saver, as it can add another S3D component to groups of S3Ds in one fell swoop.  It's usually my go-to for this sort of thing.  The Reader's "clone" feature, and in particular, its ability to automatically increment the IID, is also a tool I regularly use in this sort of process.  The fact that the surface network crossings in the RHW IID scheme occupy consecutive blocks of Piece IDs (10-13 for Maxis networks, 27-35 for NWM networks) also helps immensely.

One thing you might also need to watch out for is the prospect of z-fighting, if you're using a full tile-size plane to add the sidewalks.  The tendency for it to show up depends on how one's graphics card handles those situations.  We found that out the hard way when we tried to wealthify the old NWM Transition Puzzle Pieces in 2010. 

If I were to make a mod like this (or add it to the RHW from the get-go--which is something I'd consider), I'd probably try to manipulate the S3Ds for the roadway and the sidewalk such that they didn't overlap--which is much more labor-intensive (and specific to each undercrossing network), but it would be the best way to ensure the z-fighting didn't crop up.

Hope that helps!

-Tarkus

[delta9 24]

Hello @Tarkus! Definitely someone I was eager to find to a reply from.

Yeah, I came across other reports of broken functions in that same thread once I stopped tunnel visioning the posts with code. I took a peek at the ReaderLua.cpp but my C++ skills are too rudimentary and rusty to really know how to troubleshoot the relevant functions right now. Then again, it's reassuring that I'm not as bad at coding as the experience led me to think. After figuring out the structure of the mat and anim blocks by looking at the hex in Reader I'm actually thinking about directly processing the S3Ds, but my hope was to make a flexible and relatively user-friendly method of doing this to distribute, like GoFSHing your own SWN and TGN, but hex editing isn't even user friendly for me 

Model Tweaker is actually one tool I neglected to check out. I did try SC4Reader, but it crashes when it tries to display an S3D, and I haven't seen any solution floating around. Taking a quick glance right now, it looks perfect! Seems I'd just have to make a separate sidewalk S3D and merge to all, then run a texture script (which seems to be the only type that works) in reader to increment the L0 texture's mat block. EDIT: Actually, I'm not sure that makes sense. I'll have to see how the processed S3D turns out. The only thing that is important is that the Sidewalk layer is below the L0 Network layer in the Anim tab in Reader, and switching the mat blocks up was the cleanest way of doing it.

On z-fighting, I was reminded of the potential for it in my digging but had neglected to test for it. Just tested on some hilly terrain and the Road and TLA-5 I have now Works On My Computer(tm) as far as the z-axis goes, and I'm just using a laptop with Intel HD 620 (hey, it's enough for vanilla WoW). For other users who may have z-fighting problems, is there a known ideal distance to shift the L0 network layer up to minimize it without causing visual misalignment? If that's not feasible, I can just do it quick and dirty for immediate personal use, and work on assembling some alpha textures for the sidewalk layer. In fact, I know just the place to look...  Actually, you know what, if I can appropriate @rsc204's alphas for the new sidewalk layer, it could both solve the z-fighting and also allow one to choose whether they wanted low or high density sidewalk alphas for those pieces where it makes a difference, chiefly diagonals. Ideally I want to make a method flexible enough for people to pick and choose which crossings or categories of crossings they want Sidewalk'd and in which density, since it's all baked in. Forgive me for the stream of thought there.

Your post has been a great help! Really got the noggin joggin. I do still have a question, though: Why is the sidewalk on my road piece scaled down to SD and properly colored, while the TLA one is HD and has puzzle piece brightening? Is it because NWM is model based? This isn't a problem for my own quick-and-dirty personal use, but I would like to maintain visual consistency as much as possible for any sort of end user. Losing HD on the Maxis networks is fine if it can't feasibly be avoided. If all NWM underpasses will have this pp brightening, then I guess the idea is to package the mod with a default base texture that the user mods to match their sidewalks.

Edit: I also just had the nightmarish idea of making these cosmetic PPs. Not sure if I need more eggnog or less.

[CT14 2,486]

Great work so far!

I've successfully scripted Reader by using AutoIT to record a process, then editing the recording to optimize/reduce delays. Only took as long as doing 10-20 of the processes by hand... and once you have a working script it's a huge timesaver. Caveat: the script will depend on Reader's window being in the same size/place on screen.

[rsc204 21,836]

Looking at the problem, the simplest possible solution is just to bake sidewalks into the network textures, which is exactly how I approached the problem with respect of my SWN mod. (repository for doing this attached. Untested, since right now I'm on a bare-bones system, but hopefully I've correctly edited the script for use stand-alone.

On 26/12/2018 at 8:26 AM, delta9 said:

Of course, now I have to figure out why the Road one is scaled down to SD and properly shaded while the TLA-5 one is at the proper resolution but needs color correction.

If you are using SWN (i.e. my automated texture mod), no doubt it's why the road already works, since this texture is covered by that. However, the currently released scripts for this feature in particular are very rudimentary, I've since re-worked it from my side.

Bear in mind, when linking to HD textures on S3D pieces, they are automatically re-scaled to fit a single tile. But when running the SWN scripts, if you used a HD texture as part of the process, the scaling might not be correct. You should ideally run those scripts with standard 128px textures for best results, otherwise the process might not give you the intended result.

It might be helpful to know, as a general rule, a Puzzle-Piece texture in the NAM, uses the same ID as the regular network texture, only ending in E instead of 4. I.e., the road straight texture is 0x00004B04, whereas the Puzzle variant uses the ID 0x00004B0E. Settings of the S3D piece that displays the texture can enable or disable the Alpha Channel, enabling transparency in-game. But, by including actual sidewalks as part of the texture, we can force a sidewalk to be displayed instead. Some special cases may require editing the Mats settings of the S3D models, but not in the case of the NWM ones we're talking about.

When you edited your NWM textures, it links to the default sidewalks, the only way to colour correct those is with an additional "diffuse" layer, such as I've used on my UnderBridge models. You need to take care to split that, because if it covers the network texture (a 16x16m plane), you'll mess up the road texture whilst correcting the sidewalks. I.e. everything will end up darkened 

9 hours ago, delta9 said:

For other users who may have z-fighting problems, is there a known ideal distance to shift the L0 network layer up to minimize it without causing visual misalignment?

.2m is generally the minimum to prevent problems. If you look at the Underbridge S3Ds, instead I separated the S3D planes to avoid such issues. This post (where Tarkus alluded to the issue), explains a bit about how I achieved that:

So whilst you could use this method of altering S3D models, you'll need a way to calculate the Vertex settings including U/V mapping. I can pass that on if you want to go this route, but it's not so complex if you understand how that all works. You're free to re-use those models if that helps you out, they would be pretty ideal templates.

But if you wanted to script this process, it would take an enormous amount of work, I can't really see the value in it frankly. Sure it would be nice if all NAM legacy pieces could show adaptive sidewalks, but each piece has unique properties, that scripting to allow for becomes extremely complex. Like Tarkus, I can work much quicker by simply editing those things I need manually. I often use Excel formulas and other tricks to speed things up though, no one likes repetition. IMHO, the scripting options of GoFSH make the custom texture solution the logical choice, it's just vastly less work. The only downside, if you change sidewalks, you need to re-run the texture scripts.

Of course, don't let me put you off if you really want to go the extra mile. Quite the opposite, feel free to ask if there's anything else I can help you with.

SWN-E Textures Repo.7z