Map popularity experiment 2 results

[SKB]

Hello fellow cubers,

I came up with this new way to see which maps are more popular by counting people's leaves and joins. What you in the end get, is NET value of people the map/mode combo brings to the server. Pretty much all the data comes from TyD ladder and spans the 2011-2012 period (out of 3.3GB, 3.1GB is just TyD ladder logs, so take all of this as trends on TyD ladder. All the combos, which had less than 10 loads have been removed from the data set.

Highlights:

• Douze brings all the boys to the yard;
  
• PF / SURV / TOSOK should usually empty the server;
  
• People dislike aqueous (sorry, rk);
  
• In general, custom maps suffer from not being able to attract new players to the server;
  
• I can't explain DM iceroad being #4;
  
• Nor OSOK shine (#5).
  
• Due to a certain way server handles first player joins, maps which get loaded on player join rank low.
  

Data can be found here: WEB table ~ ODS

A slightly improved tracking will be present in the next version of ACstats. In addition to this, map votes and their outcomes are going to be tracked, too.

[ExodusS]

Server owners are influencing this experiment, and especially from {TyD}...

[Nightmare]

Cool, thanks skb. :3

Good Guy Dev : Hello, thanks for doing this. We will remove all the unpopular modes and maps, and add the popular ones. And hey, we might actually look at the ideas thread and add some of those too. We love our players. <3

Reality Dev : Too bad, smd. /htf ac_rattrap umad umad trololtrolo. ctf douze/gema/twin towers are super popular and 95% players love them? Let's get rid of douze's flags, and anti-bug map restrictions to kill all of that. Now people will be forced to play pr0 1337 maps that the devs themselves made!!!111!! There's an ideas thread?

inb4 banned 4 life.

[Cho]

Devs want us to hate them :/

[Luc@s]

Server owners and influencing this experiment, and especially from {TyD}...

No...

[InitialSAW]

Good Guy Dev : Hello, thanks for doing this. We will remove all the unpopular modes and maps, and add the popular ones. And hey, we might actually look at the ideas thread and add some of those too. We love our players. <3

Reality Dev : Too bad, smd. /htf ac_rattrap umad umad trololtrolo. ctf douze/gema/twin towers are super popular and 95% players love them? Let's get rid of douze's flags, and anti-bug map restrictions to kill all of that. Now people will be forced to play pr0 1337 maps that the devs themselves made!!!111!! There's an ideas thread?

inb4 banned 4 life.

Let's be honest here, you wouldn't want to take suggestions from people either if all they did was try to reverse every decision you've made and complain about how crappy of a job you're doing at developing. Not trying to single you out here or "suck up" to anyone, but come on, have some sympathy. You said yourself there needed to be more respect both ways, yet here you are making of mocking of the devs' choices.

[Oracle]

^Erm, he was joking (though I'll admit, he sounded just a wee bit silly).

[Nightmare]

Just saying. :)
SKB takes the time to do this. We get to see what maps are loved and hated. But nothing will come of it to improve the game...unless every server runs rattrap 24/7, then it'll all of the sudden become a major problem and must be destroyed.

and the poor players who pour hours of gametime each day into ctf douze. Now they'll have to play ctf mini douze instead. :(

[SKB]

SKB takes the time to do this. We get to see what maps are loved and hated. But nothing will come of it to improve the game...

Are you sure? In a small scale, this can help server owners better understand what attracts players to a server. In turn, this can allow for better crafted maprot, which makes people (esp newbs, whenever they decide to try something else than bloody douze) stick around, or so I believe.


There's obviously many variables to UX when trying to make people stick around. What these popularity results tell me, however, is that someone needs to pick up Lucas' work on game/map media downloader. 'MISSING CONTENT' everywhere is a huge no no if we want more map variety on the MS. This, however, should not be feature creep, but rather a new feature in next (point?) release.

and the poor players who pour hours of gametime each day into ctf douze.

They only have themselves to blame.

---

Server owners and influencing this experiment, and especially from {TyD}...

I'm sorry, I don't quite understand what you are trying to say. Influencing how?

[Nightmare]

dm iceroad gogo server owners.

meant from the dev side of things of realizing rattrap is evil and doing something about it instead of trolling whatever the most popular map/mode is currently. somewhat off topic though so nvm, sorry. <3

[Fate]

IMHO, (Load + Join) - Leave = "Popularity score" [the greater the score the better] is a better way of seeing how popular a map and mode are.
Surely popular modes and maps such as CTF @ ac_shine is vastly more popular than DM @ ac_iceroad.

DM @ ac_iceroad = 33 "popularity score"
CTF @ ac_shine = 5982 "popularity score"

[SKB]

IMHO, (Load + Join) - Leave = "Popularity score" [the greater the score the better] is a better way of seeing how popular a map and mode are.
Surely popular modes and maps such as CTF @ ac_shine is vastly more popular than DM @ ac_iceroad.

DM @ ac_iceroad = 33 "popularity score"
CTF @ ac_shine = 5982 "popularity score"

It can be written as Score = Loads + Joins - Leaves, which is plain silly. It has huge bias towards how many times a map got loaded. E.g. DM scaffold with 9934 map loads of which 8525 are from playground alone. Most of those map loads are loads when a player joins the server. Sure, due to some issues with how server handles first joiners the map has NET loss of people when using my "formula". It surely fails to attract, as nobody likes to wait for other players to come, so they just leave. But going with your formula, because server loaded the map thousands of times it makes it popular, which it's not.

[ExodusS]

Server owners and influencing this experiment, and especially from {TyD}...

No...

I don't get it, many players are staying on {TyD} or {BoB} servers for the ladder, is the map supposed to be attractive? It's kinda obvious that mods like PF is not attractive at all, but the map is not the reason why players stay or not. Unfair teams, no way to shuffle, no way to kick, teamkillers, I go to eat, the server admin ban me be cause I 360° HS him ... and much more reasons.

[Ronald_Reagan]

I don't get it, many players are staying on {TyD} or {BoB} servers for the ladder, is the map supposed to be attractive? It's kinda obvious that mods like PF is not attractive at all, but the map is not the reason why players stay or not. Unfair teams, no way to shuffle, no way to kick, teamkillers, I go to eat, the server admin ban me be cause I 360° HS him ... and much more reasons.

Yes, we are assuming that the things that cause a player to leave early that doesn't include "I don't like the map" happen equally across all maps. The fact that it is a ladder doesn't change throughout the games, it is constant.

The main issue I see with this study is the variable sample size. When some maps get loaded ~10 times and other maps 9000 times, you can't expect to have accurate results. What about maps that get loaded by vote? And the maps that get un-loaded within the first minute in favor of douze?

Not saying SKB hasn't thought about these things, I just don't see a mention about them.

[ExodusS]

When some maps get loaded ~10 times and other maps 9000 times, you can't expect to have accurate results.

You can take exemple with apollo's maps that are loaded over 9000 times a day on {TyD} ladder. (jk but kinda true)

IMO: The best value is from the map vote, even if it's also biased cause of the lack of knowledge from newcomers and a very high number of maps on AC.

[Stranz]

i don't know if you could implement this but if somehow consider a set amount of people on the server so it ignores the amount of times a map is played or loaded when there's just 1-2 people on the server. the average should be around 8 or higher so the data is not misleading, if not nevermind lol

[SKB]

What about maps that get loaded by vote? And the maps that get un-loaded within the first minute in favor of douze?

Not saying SKB hasn't thought about these things, I just don't see a mention about them.

If a map got loaded - it got loaded. "Unloading" makes no sense. Also, if a person got kicked or banned their leave would not be counted.

When some maps get loaded ~10 times and other maps 9000 times, you can't expect to have accurate results.

Technically yes, the bigger the sample the better. It doesn't really matter, though, as all NET values are "normalised" by map loads. Originally, the limit was at least 5 map loads, but seeing DM urban having NET value of 4.8 made me reconsider the sample size.

You can take exemple with apollo's maps that are loaded over 9000 times a day on {TyD} ladder. (jk but kinda true)

If you cared to look at the results, you'd see that apollo maps have negative NET values. It doesn't even have that many map loads as you seem to suggest. (besides abbey)

[RealViking]

and the poor players who pour hours of gametime each day into ctf douze.

They only have themselves to blame.

the ctf douzers, mega/twintowers fans, tosoking brahs and gemaers contributes the most to this game by accutally playing it. you and your little experiments are nothing compared to us

[Lantry]

I'm not sure I understand what you are tracking here, please correct me if I'm wrong: 'loads' counts how many times that mode+map are loaded in a server, 'joins' counts how many players join while that mode+map is running, and 'leaves' counts how many players leave while that map+mode is running.

I think that the net people per load isn't the best way to rank the map+modes.
Net players (joins - leaves) provides a more meaningful ranking because it shows how many people joined (or left) the server because of the map+mode.
Sorting by number of loads is interesting also, especially for high ranked map+modes with negative net values.

It will be interesting to see the results from tracking map votes.

[GDM]

When some maps get loaded ~10 times and other maps 9000 times, you can't expect to have accurate results.

Technically yes, the bigger the sample the better. It doesn't really matter, though, as all NET values are "normalised" by map loads. Originally, the limit was at least 5 map loads, but seeing DM urban having NET value of 4.8 made me reconsider the sample size.

Even with normalization by #map loads, you can't reasonably compare 1078 loads for the 3rd place with 13 loads for the 4th place. You have made no determination of the # of map loads needed for a representative sample. I don't remember much stats, but iirc you could probably calculate a #of samples needed to achieve some confidence in the average?

Also, I haven't thought about this too much, but normalizing for time-of-day might also clarify the results. My vague idea is that different maps may not be evenly distributed across time => different times w/ different #s of players and different timezones of players online may have different leaving/joining behaviors.

*Edit: I like the current results for mode. #map load normalization doesn't matter so much there, so those are interesting right now.

[SKB]

I think that the net people per load isn't the best way to rank the map+modes.
Net players (joins - leaves) provides a more meaningful ranking because it shows how many people joined (or left) the server because of the map+mode.

I can't disagree more. Non normalised NET varies in sample/frame/whatever it is size, therefore the values are not correct and should not be compared. Now, when you normalise by respective number of map loads, you get a constant 'NET per map load' size, which can be compared easily.

Even with normalization by #map loads, you can't reasonably compare 1078 loads for the 3rd place with 13 loads for the 4th place.

Some of the samples are way too small (10 vs 1000), yes, and I whole heartedly agree, but if I went with something like geometric mean (~86 loads) about half the mode/map cobos would not be part of the data set. The data set would definitely have a lot more "confidence" associated to it, though.

You have made no determination of the # of map loads needed for a representative sample. I don't remember much stats, but iirc you could probably calculate a #of samples needed to achieve some confidence in the average?

If only I knew how :| I am in no way a statistician, so that's why I've released a spreadsheet so people experienced in the field could mess around with the data.

[1Cap]

"Statistics is the art of squeezing the numbers until the numbers confess."
The formula to measure the popularity of the map determines the outcome .. obvious.

Some element "variable" are very important:

Beginning players leave the map when there are good players in it. (sometimes)
Players with clantag like to play with other good players, even on public servers.

Players who are friends enjoy playing together and generally leave the map when your friend comes out too.

Few servers run the maps in modes like TDM and HTF.

Not all players know how to load a map or make a server.

In general new players seeking servers with other players clantag. This is a way to learn and search how join a clan in AC.

There are many other possibilities ...

The map more popular is Depot! (yes-no?)
The most popular map is defined by maprotation, priority.

If there were no restrictions for maprotation, we would be surprised!

[#M|A#Wolf]

The map more popular is Depot! (yes-no?)
The most popular map is defined by maprotation, priority.

The most popular map is douze, you mad.

[Lantry]

I can't disagree more. Non normalised NET varies in sample/frame/whatever it is size, therefore the values are not correct and should not be compared. Now, when you normalise by respective number of map loads, you get a constant 'NET per map load' size, which can be compared easily.

There is clearly something wrong with using net people per load to rank by popularity. OSOK+ac_shine is the fifth most popular according to this ranking, despite it only having 10 loads and 31 joins.
Non normalized net ranks them based on sheer popularity. It is saying, "Over the course of the study, 3323 players joined the server to play ctf on ac_shine". Load count is an entirely separate metric.

I tried combining the results by giving each map+mode a score for it's ranking by load count and by net players (142 for first, 141 for second, down to 1 for last) then summing the two scores. I am pretty satisfied with the results, I think that this more accurately reflects map popularity.