(to Devs) Unique folder for mapmodels textures

[Cleaner]

While busy converting the mapmodels to md3 format I noticed that quite a few mapmodels are using the same skin image file so I though it could be a good idea to reorganize this and put all the image files in the same folder (like a folder called "skins")and then tell the model where to pick it's skin from....

It could give way more space for new mapmodels in the official package, just as an example I've done a little screenie, for instance in here I made for the fun a crushed barrel, some torned and fallen streetlamps and europalette upright (made earlier for a request).

Note that for comparison I have taken figures from the actual package vs the new package in md3.

Actual packge

4 streetlamp tris.md2 38.04kb
4 streetlamp skin.jpg 207.6kb
2 barrel tris.md2 24.1kb
2 barrel skin.jpg 128.9kb
2 eurpalette tris.md2 26kb
2 europalette skin.jpg 68.2kb

total: 493.74kb

Revised package

4 streetlamp tris.md3 17.92kb
1 streetlamp.jpg 51.9kb
2 barrel tris.md3 10.18kb
1 barrel.jpg 64.45kb
2 eurpalette tris.md3 19.96kb
1 europalette.jpg 34.1kb

total: 164.41kb

gain: 329.33kb

[tempest]

Yes, definitely! This saves not just disk space, but also video memory.
However, I would suggest to leave the skins at least somewhere in the models directory.

[ExodusS]

And what happen to all maps who have a custom .cfg file?

[Dementium4ever]

The probably best would be, to get all textures together in 2 or 3 2048x2048 files. The alpha ones would get a extra file.

This makes everything easier for the graphics card.

[tempest]

And what happen to all maps who have a custom .cfg file?

Nothing, why? This stuff happens in the model config.

The probably best would be, to get all textures together in 2 or 3 2048x2048 files. The alpha ones would get a extra file.

This makes everything easier for the graphics card.

Well, I guess some people still use graphics hardware with hwtexsize 1024 :P
That left aside, the benefit of such measures is questionable. How expensive texture switches are depends entirely on the graphics hardware. I'd guess graphics cards are designed with those things in mind, so they should be optimized for it. Batched rendering does make sense, because it doesn't really cost anything and also saves shader switches, VBO switches etc. On the other hand, using such large texture "sprite sheets" may actually have nasty side effects, e.g. killing cache performance on the GPU. In case the card runs out of video memory, it may even end up swapping those huge textures in form the main memory for every frame - ultimate performance killer guaranteed. I'm by no means experienced with computer graphics, let alone with such questions that can only be reliably answered by the hardware vendor, but I guess there's a reason why such techniques aren't widely used - not that I'd heard of it, anyway.

And something else I forgot: setting this up would be quite a pain in the ass, because it means we'd either have to calculate the actual UV coordinates while loading the textures (nontrivial job especially with non-square textures), or change them on each model by hand.

[Dementium4ever]

Well, ok, thanks for that :D
Yeah, I'm not so into hardware. But quite a few games I know are doing that...

[Cleaner]

Yes, definitely! This saves not just disk space, but also video memory.
However, I would suggest to leave the skins at least somewhere in the models directory.

It is what actually I have done packages/models/mapmodels/skins.

In fact I have nearly finished with the whole repackaging where all models are now in md3 and all skin.jpg in one folder, there is still some models that I need to rename for it to work.

The good part of this is also that devs don't need to bother about coding nor scripting :p so I guess that including a few tests to make sure everything is fine that should be ready by end of the week.

@.ExodusS* indeed exactly what tempest said, it all happen in the model cfg.
The thing is that with this system we can now make plenty versions of existing mapmodels (as simplistically shown on the screenie) for cheap since we do not longer need the heavy image file.

Well, now I hope devs will see the bright side of it and give the green light to this, if so there will be also some modifications to do to the docs.

[ExodusS]

And what happen to all maps who have a custom .cfg file?

Nothing, why? This stuff happens in the model config.

[SELECT ALL] Code:

mapmodel 1 4 0 0 "toca/guardrail2/gr2pole"    // 106    Use guardrail2 with this (Model 68)

Because all files ways are writed in.

[Joe Smith]

nice work

[Billybob]

And what happen to all maps who have a custom .cfg file?

Nothing, why? This stuff happens in the model config.

[SELECT ALL] Code:

mapmodel 1 4 0 0 "toca/guardrail2/gr2pole"    // 106    Use guardrail2 with this (Model 68)

Because all files ways are writed in.

If i understand this correctly "toca/guardrail2/gr2pole" only points to the folder that the model is stored in, it doesn't matter if that model is in md2 or md3 format. If the model is in md3 format it will read the tris.md3 and apply the md3.cfg to it (if its in the same folder as the model). The md3.cfg will state where the skin image is loaded from. So any custom cfg will not be affected.

EDIT:

Reading more into i think the md3.cfg file is read first which then reads the tris.md3 and the skin.

[SELECT ALL] Code:

md3.cfg
md3load tris.md3
md3skin torre skin2.jpg

[FANTOMAS.UA]

So any custom cfg will not be affected.

I think custom cfg (md3.cfg) will owerride default one (default_map_settings.cfg)

[tempest]

In fact I have nearly finished with the whole repackaging where all models are now in md3 and all skin.jpg in one folder

Aargh! Please, please don't do that! It might work now, but once auto-downloadable packages are added and see some use, this will become a nightmare. Just remove the duplicates except from one folder, leave the skin in there, and load it from the other ones. Having all skins in one folder will confuse everyone, and it won't work well with zipped model packages.

I think custom cfg (md3.cfg) will owerride default one (default_map_settings.cfg)

All those two have to do with each other is that default_map_settings.cfg (or the map config) declares which md3.cfg's and md2.cfg's should be loaded, and what their index is.

[Cleaner]

Aargh! Please, please don't do that! It might work now, but once auto-downloadable packages are added and see some use, this will become a nightmare. Just remove the duplicates except from one folder, leave the skin in there, and load it from the other ones. Having all skins in one folder will confuse everyone, and it won't work well with zipped model packages.

I'm looking the other way round mate and this is why I am hush hush to fix that before feeding Luc@s host.

Confuse? confuse whom? those involved in mapping,modelling and so on? thb they surely will have read this and expect some minor changes (put your skin in the skin folder).

About the zip part, I don't get it either, if packaged properly where is the trouble?

[tempest]

Confuse? confuse whom? those involved in mapping,modelling and so on? thb they surely will have read this and expect some minor changes (put your skin in the skin folder).

Well, there used to be a directory for each model which contained all the stuff - model, config, skin(s). Now you take all the skins and put them into one separate directory. Idk, but I'd say some people will be confused.

About the zip part, I don't get it either, if packaged properly where is the trouble?

The way this is done right now in the code, is that you just take that model directory and zip it up, and that can then be downloaded and unpacked properly. With your setup, we'd need something more advanced, like starting the whole thing at the packages/ level.

[Mr.Floppy]

First off, sorry Cleaner for not being around last weekend as I planned and told you. Friday night was just way too long and it took the whole weekend to become a living beeing again.

The main issue I came across here is the use of a relative path ("../folder"). How will this work with *YourFiles* directories? Tempest, I guess you can answer this for us. The problem could be, that the linked skin file is located at the default *Programs* directory and the custom model, requiring a default skin, at the *YourFiles* directory. This shouldn't work with relative paths. Does it?

The second thing is pretty much like tempest said already. One seperated skin-folder could lead to several inconvinient arrangements, just like the packaging for the auto-download feature. Also the folder structure could become a pain in the neck. Like, will we mirror the structure from the model's folder or just use one folder and naming the skin files like the folders where the models are located? Either way isn't going to be a much better solution than just keeping one skin file at the origin model's folder, in my opinion.

[Cleaner]

@ Mr.Floppy & tempest, I guess you will need to see the revised package to fully grab the simplicity of the whole process and the benefits.

We now have a 'skins' folder in the mapmodels folder, images are no longer named as 'skin.jpg' but by the model's name like for example 'bench.jpg'...
All there is to do now is adding in the model's cfg "md3skin bench ../skins/bench".
Now just imagine I want to make a broken bench model I can use that same skin just by adding the command "md3skin broken_bench ../skins/bench" in it's cfg, simple as that.
See also the particularity of this system, one could even create mapmodels using textures from several image files in the 'skins' folder, or even the gastank's base using bridge.jpg, in brief the possibilities are vast.

About packaging zipped files will only have an extra 'skins' folder so I don't think that will create such a mess and in the case of a re-used skin the zip file will be even lighter.

I suppose that explaining this in the docs won't be to complicated.

___________________________________

Now regarding the auto download feature you guys certainly have pointed out a serious question but I don't know how it actually works so right now I cannot come up with a solution for this but I will get in touch with Luc@s to work on this (it works with other games so...).

All I know is that so far only the devs and I are allowed to commit material for auto download, I have asked Luc@s to put it on hold for a bit just in case some of the packaging propositions I've offered are agreed (or not).
If agreed and if the auto download issue is resolved I don't mind taking care of double checking all material to be committed for optimisation.

I agree that it is quite a bit of a shake up but I seriously think that if it can be implemented it will definitely be a nice improvement and maps will have a lot more details and less repetitions in mapmodels.

If issue cannot be resolved well, at least I did have tried.

[tempest]

[...] This shouldn't work with relative paths. Does it?

I'm not sure if it does, but it should, and otherwise it should be fixed. For everything the engine loads, it checks in the home directory first, then in any additional package dirs, and only then in the installation itself.

@ Mr.Floppy & tempest, I guess you will need to see the revised package to fully grab the simplicity of the whole process and the benefits.
[...]
See also the particularity of this system, one could even create mapmodels using textures from several image files in the 'skins' folder, or even the gastank's base using bridge.jpg, in brief the possibilities are vast.

Well, I don't know, but IMO it just makes more sense to keep things together that belong together, e.g. model and skin.
An idea would be to use a more specific structure for models which share a skin (and therefore are probably similar in other respects as well), if they already exist. Like this:

[SELECT ALL] Code:

mapmodels
>| benches
  >| bench
    |  md3.cfg
    |  bench.md3
  >| broken_bench
    |  md3.cfg
    |  broken_bench.md3
  |  skin.jpg

And you'd use it as md3skin "../skin.png". Otherwise, if you want to add such an additonal model later, you'd just put it into a separate broken_bench directory and use md3skin ../bench/skin.png And if you want to use the bridge texture on the gas tank, that would be possible as well.

About packaging zipped files will only have an extra 'skins' folder so I don't think that will create such a mess and in the case of a re-used skin the zip file will be even lighter.
[...]
Now regarding the auto download feature you guys certainly have pointed out a serious question but I don't know how it actually works so right now I cannot come up with a solution for this but I will get in touch with Luc@s to work on this (it works with other games so...).

The problem with this is how it currently works: it makes a first run to determine which packages (maps, sounds, textures, models) are missing. Then, it attempts to download them all. Once that has finished, it attempts to load all the stuff again, including what it just downloaded. With your setup, it would notice only now that the model textures are missing, so we'd need to add another break to download them.

[Cleaner]

Well, I don't know, but IMO it just makes more sense to keep things together that belong together, e.g. model and skin.
An idea would be to use a more specific structure for models which share a skin (and therefore are probably similar in other respects as well), if they already exist. Like this:

[SELECT ALL] Code:

mapmodels
>| benches
  >| bench
    |  md3.cfg
    |  bench.md3
  >| broken_bench
    |  md3.cfg
    |  broken_bench.md3
  |  skin.jpg

And you'd use it as md3skin "../skin.png". Otherwise, if you want to add such an additonal model later, you'd just put it into a separate broken_bench directory and use md3skin ../bench/skin.png And if you want to use the bridge texture on the gas tank, that would be possible as well.

tbh it won't really make much a difference since the whole process would still be the same, on the other hand (now it's the modeller talking) having all skins in a single folder will be easier to find a texture.

The problem with this is how it currently works

As I said I had no idea on how it works, now you've explained it a bit but I need more infos, I'll try to get hold of you folks on irc.

[Felix-The-Ghost]

I approve of all of this, though I'm oblivious to "auto-downloads" as to what they are, etc.

[Cleaner]

>>> Auto-download <<<

[Felix-The-Ghost]

Wow, that's pretty awesome. At least in theory. I assume everything is being done to optimize it in practice?

[FANTOMAS.UA]

I have another idea. It does not work now, I checked. Because it probably requires a change in code. So:

mapmodels/
  | bench/
    | md3.cfg
    | bench.md3
    | skin.jpg

  | broken_bench/
    | md3.cfg
    | broken_bench.md3
    | skin.lnk // link to bench skin.jpg

Idea is to put a link (windows/mac shortcut) to skin file instead of original texture. Link is smaller than jpg/png, isn't?

[Cleaner]

Nice of you to try to fix this issue but then again as I mentioned for tempest's code it won't make no difference at all, process still the same.

As he said "With your setup ... we'd need to add another break to download them." whatever that means it doesn't sound that hard to put in place.

btw, I've found something else... "minelift" mapmodel atm use a skin of 1024x512 of 169kb, one part of this model is the classic "hook" model so I broke it in two groups and...

[SELECT ALL] Code:

md3load tris.md3
md3skin hook ../skins/hook.jpg
md3skin minelift ../skins/minelift.jpg

There is no savings too little XD


Expect the whole package for either today or tomorrow for tests/controls/revisions...


EDIT: What is actually telling the auto download what content is missing? does it reads the map's cfg? Enlighten me on this please.

[Cleaner]

Ok, it took more time than I though because all models had to be renamed within the actual 3D program for all this to work but it's finally done.

I have taken the opportunity to fix some bits here and there like welding vertex on some mapmodels, in my 'Cleaner' folder I also have slightly modify the minecart to reduce polys.

So... Again I understand that it's a totally new approach in files storage but look at the benefits, the official mapmodels package is 23Mb whilst the revised one is at a mere 8,9Mb.

As I explained previously, since the skins are already there, this leave plenty room for extra modified official mapmodels like the ones I have shown, mapmodels weight hardly nothing (between 2Kb and 20Kb for the heaviest).

I leave it now to you to check it out, if you noticed something I missed or some changes need to be made let me know.

>>> Download Revised Mapmodels Package <<<

EDIT: Download link updated (few error fixed)

[D1551D3N7]

Thanks for this Cleaner! So I replace my mapmodels folder with this?

[Cleaner]

Thanks for this Cleaner! So I replace my mapmodels folder with this?

Indeed, it should work as fine as the original mapmodels folder but it hasn't been double checked for bugs (or mistakes) so don't replace it just yet, make a backup of the original folder first.

[D1551D3N7]

I have 3 installs and they umm act as backup for each other. Im using it and works fine :D

[Cleaner]

Fine, just go through as many official maps as you can and if you notice anything odd let me know ;)

[D1551D3N7]

Will do Cleaner! Ive played all deserts and douze so far.
Edit: Toxicity knocked over barrel no texture.

Iceroad knocked-over barrel light posts along with lots of error messages

EDIT: Yes go ahead make fun of my graphics card. Look closely for a project hint :P

[Cleaner]

OK, I was expecting some minor mishaps like these, as asked to D1551D3N7 please give me that in text, screenies are less useful unless it's about a axis problem.

So...
barrel2 cfg (fixed)
md3.cfg missing on elektro
streetlamp still in md2

be sorted for tomorrow, thanks for helping ;)