[Tool] Map .cfg Cleaner

[Bukz]

I've created a very simple site that allows you to copy/paste map .cfgs and submit them to be automatically "cleaned":

http://bukz.xrd.me/mapcfg/

The PHP script will do the following to your input:

• Removes all comments.
  
• Removes all leading and trailing whitespace.
  
• Removes all blank lines.
  
• Removes all lines containing "junk" non-ascii characters.

Not only does this process usually greatly reduce the file sizes of map .cfgs, but also can sometimes "fix" older, buggy configs because of the fact that it removes "junk" lines:

Example Input (~31.3 KB)
Example Output (~15.1 KB)

Take note of the "junk" data at the end of the input .cfg

You may submit map .cfgs up to 128 kilobytes in size. This should be more than enough for any AssaultCube map config. If you run into a scenario where it isn't, I want to hear about it. In a futile attempt to prevent people/bots from spamming the site, you must submit at least 4 kilobytes worth of data for the script to do anything.

Questions/comments/feedback are all very much welcome. I hope someone out there will find this as useful as I do. Cheers :)

P.S. I may be willing to share the PHP source code with you if you are interested, just send me a PM asking about it. Also, please report any bugs or weird errors you may notice in the output, thanks!

[Cleaner]

Nice one Bukzy!

Watch out for copyrights (cfgCleaner) :p

[RandumKiwi]

Nice, I've been doing this for years with a simple Linux script. I like how you copyrighted the idea and didn't credit me for it :P

[R4zor]

I love a good, juicy, inter-dev skirmish

[Bukz]

Ye, linux (maybe even mac?) users have the luxury of being able to do this natively from the terminal. I made this to help non-linux users. My copyright isn't on the concept at all (as anyone can make a PHP script that does similar actions to a text file/string), its on the content of the website and the PHP behind it.

If I credited everyone involved in making this possible, the page would be unbearably long. (the creators of the personal computer/PHP/HTML/CSS deserve it the most I guess..)

[Andrez]

Nice one, but I don't see that usefulness. It can save an unimportant amount of space, okay, but blank spaces/lines are usually used to keep it ordered and more readable!

[Nightmare]

Spoiler

I love a good, juicy, inter-dev skirmish

Yay. RR's pycube has a cfg cleaner that tracks down what you're using and what you aren't, and kills the latter.(at least I think it does)

Nice way for a quick clean-up though. :D Doing this by hand sucks and takes a long time for complicated maps. gj team.

[Bukz]

@Andrez map .cfgs are not really meant to be easily readable by humans. /config/default_map_settings.cfg has all the formatting/comments one would need to understand whats going on in them. As unimportant as the space savings sounds it actually makes quite a big difference when we're talking sendmap/getmap's, especially from 56k internet connections. In theory the cleaned example output .cfg would transfer twice as fast as the input .cfg over the network. Not to mention the less data the .cfg reader has to go through the better.

The best part about it IMO is that it removes the junk lines that tend to accumulate at the end of map .cfgs over time. These lines can cause lots of issues in a map .cfg especially if you're trying to modify it. (adding/removing mapmodels/textures/etc.) I found this out the hard way when modifying an old copy of ac_alcove a week or so ago. Random edits to the file that should have had no effect on the map would screw up a few textures on the map. After acquiring a clean copy of the .cfg all problems went away; modifying the .cfg worked fine and didn't negatively effect the map at random.

Don't gotta use it guys, its just something that I find quite useful so I thought I would share with everyone in hopes that someone else will too.

[Ronald_Reagan]

Yay. RR's pycube has a cfg cleaner that tracks down what you're using and what you aren't, and kills the latter.(at least I think it does)

https://github.com/RonaldReagan/Pyssault...fgclean.py

[DogDancing]

useful, good work!

[Bukz]

Thanks DogDancing :)

I did a little update to the site and you can now upload .cfg's rather than paste them. Please report any issues you might find with it.

[Lord Vader]

You should cull your code to only remove the 'garbage' that spawns from nowhere on the end of the cfg files. Removing the licence and authors name and possible instructions seems like an oversight in your ability to code.

[Cemer]

You should cull your code to only remove the 'garbage' that spawns from nowhere on the end of the cfg files. Removing the licence and authors name and possible instructions seems like an oversight in your ability to code.

lol hows your high horse doing these days

[Bukz]

Its not really an oversight at all. The script removes all comments from the .cfg, period. It does what it says.

The user obviously has a copy of the .cfg with all comments in-tact, when they submit it to the site in the first place. If they want to add their name/licensy stuff after cleaning the .cfg it is their decision.

Sure I could make it only delete comments after a command has been detected, so that comments at the top of .cfgs would remain in-tact, maybe I'll implement that someday. Until then I just don't see it a huge deal. Thanks for questioning my ability to code though. <3

[RandumKiwi]

Just cause I feel like being one-step ahead of Bukz, I decided to be two steps ahead of Bukz...

The Linux script now does the following:
1) Convert newlines to Linux format.
2) Remove comments.
3) Remove leading/trailing spaces/tabs.
4) Removes invalid commands (i.e. shows ONLY lines starting with: loadnotexture loadsky mapmodelreset mapmodel texturereset texture fog fogcolour mapsoundreset mapsound watercolour shadowyaw)
5) Remove blank lines.
6) OPTIONALLY: Preserves any comments made before the mapmodelreset command.

Code available here.

(Lines 53 to 88)

[ExodusS]

[SELECT ALL] Code:

! ! %$!&%#$"&&!#%'),1
?
l[IkZJjZJgWHfVGhWHlZLhUGaVDeVF}iZeN@iXAbQ=}lX|kYxdI~jOkS9oU<kQ:sZClW@wbKxeRhUxdGm[K{f\kSBiOHaKAt`Rw^QfH;pO@…aTnVD^M9jVBpQAtWB…hS|^MlO<jO9_F1rXHcH6pSFSC3lTD`O>uZGr^Gr[HfTF`L=zdVv]NuYMw[MjN@w\KjO>fK9{`LkP<v\Fv[FkP<qVCcUAmP€




€ûûûûûûûûûûûû€þþþþþ

Those lines are added at the end of almost all .cfgs when we send the map in-game, why?

BTW: Is it possible to delete all unused files' lines (model/sounds) in relation with a .cgz? (i.e. "citynoisebirds.ogg" is not in the .cgz, si it can be deleted from the .cfg)

[Bukz]

.ExodusS* that is the million dollar question, and one of the reasons tools like these are necessary in the first place. I'm sure somebody can track the cause of it. I'm clueless though.

As far as removing unused content goes, it sounds like Ronald_Reagan has already created a python script for doing that:

http://forum.cubers.net/thread-6492.html#post_122690

[Ronald_Reagan]

Yes, my script does exactly that. It removes every unused item from the cfg and modifies the cgz to reflect the changes.

[ExodusS]

and modifies the cgz to reflect the changes.

dayyyum I didn't know :P

[Luc@s]

and modifies the cgz to reflect the changes.

dayyyum I didn't know :P

Yet, the the link was already given in this thread