Statistics[of TWHL] Created 8 years ago2016-01-20 01:24:46 UTC by Striker

This was a shower idea.
I am now pissed off. I wrote an entire forum post, it's 03:00 AM and accidentally pressed the back button on my browser. ARGH, you'd think these days the browser would remember what you wrote in a form!

I've been recently playing with nodejs and I slapped together some code for a web scraper.
I targeted the "Members" part of TWHL, and since we have over 6500 members registered, I thought I could infer some data from that. Considering the fact that I'm making consecutive requests at 150ms delay, this should have taken about 100 minutes, yet idk what magical thing happens in my code because it took only about 10 minutes. I can't go under 150ms because after 15-20 records the TWHL.info server gives me a 503 service unavailable.

I've discovered that we have some non-existent members: 1922 2611 5921 5968 5969 5974 5994 5996 6002 6003 6005 6006 6006 6007 6008 6009 6366 6366 6367 6369 6402 6431 (these are IDs). The ban hammer was strong with these ones :)). We have about 1000 banned members.

The statistics are a bit unusual and not what you'd expect because the way I took the data( I used jQuery to build a CSV file), and because I only took it from the "Members" area(for now).

TOPs
(EDIT) This is not really a top. For sure the last result has the maximum metric, but you can guess it's just the code logic that gave this output.

I'm so lazy right now that I'm just going to paste the output of the code I used. Figure for yourself what each top means:

New champ! Andy has 1 maps uploaded!
New champ! SlayerA has 8 maps uploaded!
New champ! Seventh-Monkey has 18 maps uploaded!
New champ! Soup Miner has 35 maps uploaded!
New champ! Angry Carrot has 40 maps uploaded!
New champ! zeeba-G has 50 maps uploaded!
New champ! Rimrook has 57 maps uploaded!

atom has 545 logins uploaded!
Andy has 1235 logins uploaded!
SlayerA has 3294 logins uploaded!
Seventh-Monkey has 12321 logins uploaded!
ZombieLoffe has 13524 logins uploaded!
Mariowned has 33858 logins uploaded!

atom has 101 forum posts!
Andy has 1132 forum posts!
Seventh-Monkey has 16814 forum posts!

atom has 1 journals!
Andy has 8 journals!
Tlax has 9 journals!
Seventh-Monkey has 152 journals!
AJ has 175 journals!
Habboi has 188 journals!
Madcow has 205 journals!

Andy has 81 map vault comments!
Seventh-Monkey has 266 map vault comments!
Captain P has 330 map vault comments!
Soup Miner has 341 map vault comments!
Angry Carrot has 429 map vault comments!
Habboi has 872 map vault comments!
rowleybob has 1051 map vault comments!

Charts

I used plot.ly for this. Keep in mind the plots represent some summed metrics from members that subscribed in the same year.

Member Registrations by year

Finished Maps contributions by people who signed up the same year

Forum posts contributions by members who signed up the same year

Journal entries by members who signed up the same year

Map Vault comments by members who signed up the same year

Total logins from people who signed up in a given year

Seems like 2004 was a prolific year for TWHL, and its freshly registered members gave the roots for this community.

I want to take more information though. For one thing, I want to make some map rankings and comparisons based on different metrics. I'd also like to explore more what can be done with this data to present it in an interesting fashion. If that's ok with Penguinboy. I don't know if making multiple requests does any damage to the server, it doesn't seem to as it has some rules to protect itself.

Interesting idea! But... well... you could have just asked me to run some SQL queries on the database

I'll think about including some leaderboard stats in TWHL4.

Interestingly those graphs seem to approximate a P-distribution, as does the daily view count graph on the YouTube video for Gangnam Style: