Assaultcube Server Stats (aka sekret MOOM)

[Ronald_Reagan]

In my workings with python, I came across an idea to collect data on a massive scale, and thus further my knowledge of the language.

My knowledge has grown, and now I have a link to give you.

But before I give this link, let me talk more about this data collection.

I aimed this project to take the form of a server stat collector. It would pull data off of servers using extern info and perform calculations on it. It would also query the master-server to retrieve the serverlist, and then use that list to total the amount of servers and players.

And thus it was made.

I then decided I needed some easy way to display and manipulate the data so I could share the love. And increase my knowledge of python.

And thus it was made. in a very hackish style, was tired.

MOOM

And on the third day I slept (and went to work, I need money :P).

Now, I am no mathematician, so my calculations are pretty rudimentary and hopefully correct (I checked with Wikipedia, I think they are).
So, if anyone armed with a knowledge of python (or basic coding knowledge) with a sidearm of statistics wants to check and further my work, I will let them in on my soon to be revised source.

This data is one whole day (and a bit more, my math as wrong). Have at it: http://pbclan.tk/serverstats/

Some interesting servers to look at:
http://pbclan.tk/serverstats/servers/Your_mom's_house_[[24=7_T=OSOK]]_http:==ymhclan.com=:28763.html
http://pbclan.tk/serverstats/servers/[AC_Ladder]_%7BTyD%7DPennywise:28765.html

(V-Edit: FML, BBCode despises those URLs. Good luck.)

http://pbclan.tk/serverstats/servers/=My...28763.html
http://pbclan.tk/serverstats/servers/189...11111.html

http://pbclan.tk/serverstats/servers/=My...20070.html
(at least the most common mode is COOP, but ac_desert?)

http://pbclan.tk/serverstats/servers/82....11111.html
(GOoooooo AC!)

Interesting purely because they have some data to show.

Thanks goes out to B}Verse for hosting the server for data collection (and some math). HyPE|GDM and HyPE|Joesmith for help with the website end of things (CSS). The rest of the B} Clan, and MM10 for feedback. Thanks to Fiz for a PHP script that helped me form a basis to work off of.

Feel free to give as much helpful feedback as you want. Really that is what will make this better in the end.

[V-Man]

Aha! Great job!
If there's any trouble with math, I know Kirin is always up for helping with that.
And so am I!
Is Yarukinasu still around?

[Orynge]

(at least the most common mode is COOP, but ac_desert?)

The only times it was played was in TOSOK and DM. :P

[tempest]

1. Good job!
2. You really need to fix the HTML output. Not only do quotes mess up the links, but XSS attacks are also possible even though you escape <. For example, I can still name my server

[SELECT ALL] Code:

#" onclick="alert('oops')" rel="

and it will even be valid HTML :P (not sure about that, but doesn't matter)
3. Your use of floating-point values in some places made me chuckle. Like "Most players: 173.0" :D
4. The readability of dark grey on dark red is horrible. Either make the text or the background lighter.
But those are peanuts.

~ Always yours when it comes to nagging.

[Nightmare]

MOOM!
Good job, RR. :D

[Joe Smith]

This project of RR's is sexy, and has room to grow into something greater ( not that it isnt great now ) than it is right now. I see this project being useful to alot of people, and ac as a whole. Gl RR

[Verse]

I do love data. DATA ARE WONDERFUL.
I think that over time this could be a wonderful window into the health of the AC community.

[flowtron]

during a few IRC-talks that went slowly I did some python prototyping myself, a while back. you might remember me quoting some percentages on map/mode-combos and the used-servers to registered-servers ratio ;-)
Well, the code was hardly thought out, just on-the-fly .. the polling bit was of more interest to me anyway - I even included some geoIP stuff. Retrieving and displaying nicely (monospace FTW!) the maprot and stuff like that; wanted to even go as far as querying the whole range of EXTping (score-by-player and stuff)...but never made time for it.
The stats turned out to get more skewed as the errors I was doing on-the-fly accumulated; so I broke off that experiment.

My guess now would be to try to simply collect stats on an individual server - so you'll need some way to uniquely identify them.
You can always provide a "last accumulated version" of the stats for each server, if you combine all of those to one chunk you can then run (weighted!(*)) analysis on the "entire network" with an acceptable range of deviation.
_____
*: what you weigh on is the tough choice, but you can't simply add any specific values coming from a server online 24/7 during your last 3 weeks of polling data from the network with the pendant from one that just popped up yesterday and has been online 7.25 hours out of those 45.5 since you first saw it.

[Ronald_Reagan]

Only replying to tempest for now. Sorry! (Will get to the rest of the text later)

As I said, this is a project to learn things, so I didn't put tons of time into that. Towards the end, I was just wanting to finish it. My code to actually compile all the stats into HTML is really gross and disgusting. I'm not releasing that because I would be too embarrassed.

I can get you a list of HTML escaped characters, but really I didn't put as much effort as I could've found (a std module to use) an easier way. I did the basics still. No one knew enough about it to be able to safely do that. During data collection only myself Verse and jamz had a notion about what was going on. None of them knew exactly what was happening in the end. Nor did I a little. Security through obscurity is how this one worked out.

Valid HTML, psh. Overrated.
I will try to see if I can get it somewhere close though.

I could've sworn I cut out all of those floating point numbers. The AVG and STDDEV is also numbers to laugh at. If you can name some instances, that would help me. I did try to remove them (this may be an old revision of the HTML possibly?).

I was advised to use that color schema rather than the one I cooked up by myself. The one I cooked up myself was easy to read, but apparently the read burned holes in peoples eyes :>
I like red...

Biggest problem I see as well is the HTML output as well. I will double check sometime and make sure I have the hurtful characters escaped. IIRC I do. At least the stuff in common server names (and some lesser common characters).

When I'm more away and intelligent I'll respond more in depth about the statistics and the stuff that requires more brain power later :\

[tempest]

Security through obscurity is how this one worked out.

Yeh, I just wanted to have it pointed out (I think you're hosting forums on the same domain?...)

Valid HTML, psh. Overrated.
I will try to see if I can get it somewhere close though.

No, I meant that my injected stuff resulted in valid HTML :P

I could've sworn I cut out all of those floating point numbers.

I was primarily referring to this page, but now I noticed that it's out of date anyway, so probably an old version.

On a sidenote though, I noticed that you don't count "no players" as 0? See http://pbclan.tk/serverstats/servers/|FO...28763.html, for example.

I like red...

AFAIK red is not a good color for website backgrounds. IANAWebDesigner though.
But if you just change the font color to something like #EEE, the text becomes a lot more readable.

[Ronald_Reagan]

Ah yes. I missed that page. I focused mainly on the server logs.

The reason for this is early on in the process of making this script, I didn't realize python made a difference between int and float. So I had to do some un-thought-out changes to the script.

One thing to realize is that these stats are not live. These are all one day. I plan on getting some regular scheduling for updates and such, but first I have to make many improvements.

Here is the escape table:

[SELECT ALL] Code:

html_escape_table = {
    "&": "&",
    '"': """,
    "'": "'",
    ">": ">",
    "<": "&lt;",
    }

Meaning this:
#" onclick="alert('oops')" rel="
Would become this:
#&quot; onclick=&quot;alert(&apos;oops&apos;)&quot; rel=&apos;&apos;

again, I'll be making improvements on this front.


Another note is that all of the data compilation (HTML making) happens off of the server on my home computer. This is because I would make the site, then revise it and rerun the script.

I'll play around with the CSS some more with my web-guru's. I dont have a terribly great eye for color :>

---

@Flow
I recall your stats that you pulled out of your hat, and I wondered at the time how you masterfully produced those.
I have a feeling my classes (I made them so they dont have to be specific to this script) are about in the same stage as yours were, you were probably farther (just re-read your post). I plan on extending and getting more out of the info when I can figure more out about it.

On the topic of extern info, it might be nice if some documentation was written up. I know I do not possess enough knowledge to write up a sufficient piece of documentation. I just never learned some of the stuff I use. (Now I am reminded that I need to thank Fiz up in the OP).

Ahha, I plan on using geoIP on the next version. That sounds quite fun.
I also plan on showing the list of servers in some kind of order. Most likely the highest ratio of time up/time played. I would have to think about temp servers only put up to play 4 games or so.

I am quite sorry that I may not of adressed as much as you did.

---

To make my intentions about the project clear:
I plan on making this script more secure and reliable (meaning preventing injection and such)
I plan on figuring out some server to collect data once a week.
I plan on making everything fully automated.
I do not plan on making this a constant data collection process like a ladder.

However, if I do get a server resource that will allow me to constantly collect data, I may rethink the last one. It would require something very different though.