I have A LOT going on with working on trying to efficiently gather the log data and put it in a more usable form. Right now I have a php script opening a pipe and starting the server itself so its essentially running the server inside itself and it logs every line the server spits out directly to mysql. I plan to expand that to actually parse those lines and spit it into different database tables for chat and other stats. This seems to be working ok even with a full server osok 18 players max.
I have been reading and pondering what to say in this thread and I have a few suggestions. XML while being a very good format for readability and has a ton of tool sets to parse it is a very innificient storage format and has a TON of overhead and bloat. Sure it can be minimized by keeping the tags simple but its still very bloaty. A more efficient format might be json and json currently has a lot of tool sets too in many languages. That would be my first choice. A second choice would be llsd which is one of the more efficient xml format might be an ok choice too. Again xml has bloat in pretty much any format.
Outside of creating a new log format or an additional output the demos have been a discussion in irc since this thread was started and I think maybe working on a way to parse out the demos might be a good solution also. I think the only reason it isn't done yet is the lack of documentation or examples on how that could be done and the fact that its in binary format.
What we have to keep in mind is that any "formatted" storage is going to be more bloaty then a plain output of the logs. So that should be a given, we just have to figure out how much bloat and to what gain...
I also wanted to comment on the trashy projects thing. Sometimes making things simple can fill an area with a lot of trash but you also create a competition to be better among the more capable ones. The trash will be weeded out and things like ladders will get more efficient and ran better. The current choice of ladders is less like a choice and more like all you have and could be improved on. When theres alternatives attitudes change and the push to be the best grows. its just like when a gas station opens across the street from the over priced one that used to be by itself. Prices go down and service gets better. So I think making it easier and more efficient to do the things the community wants to do will only be benificial even if it allows for more trash.
I have been reading and pondering what to say in this thread and I have a few suggestions. XML while being a very good format for readability and has a ton of tool sets to parse it is a very innificient storage format and has a TON of overhead and bloat. Sure it can be minimized by keeping the tags simple but its still very bloaty. A more efficient format might be json and json currently has a lot of tool sets too in many languages. That would be my first choice. A second choice would be llsd which is one of the more efficient xml format might be an ok choice too. Again xml has bloat in pretty much any format.
Outside of creating a new log format or an additional output the demos have been a discussion in irc since this thread was started and I think maybe working on a way to parse out the demos might be a good solution also. I think the only reason it isn't done yet is the lack of documentation or examples on how that could be done and the fact that its in binary format.
What we have to keep in mind is that any "formatted" storage is going to be more bloaty then a plain output of the logs. So that should be a given, we just have to figure out how much bloat and to what gain...
I also wanted to comment on the trashy projects thing. Sometimes making things simple can fill an area with a lot of trash but you also create a competition to be better among the more capable ones. The trash will be weeded out and things like ladders will get more efficient and ran better. The current choice of ladders is less like a choice and more like all you have and could be improved on. When theres alternatives attitudes change and the push to be the best grows. its just like when a gas station opens across the street from the over priced one that used to be by itself. Prices go down and service gets better. So I think making it easier and more efficient to do the things the community wants to do will only be benificial even if it allows for more trash.