Sign in to follow this  
Followers 0
DeinFreund

How to get started with the meth (API, Datasets?)

20 posts in this topic

So this is the forum to go when you're tired of WG forums? I'll keep this short in case I'm posting in the wrong place.

Maybe I haven't done my research properly, but I somehow failed to find out how to access the battle API of WOT. I've seen there are a lot of stat based metrics for WOT and am disappointed at how much winning games is being neglected. You have probably tried Elo rating before and I wouldn't be astonished if the large teams made it a pain to work with. As I have just implemented Whole History Rating (publication) for my favourite open source game, I thought I might as well see how it does on WOT. It generally converges much faster than Elo and might thus have a chance of coping with WOT's big teams, although I can't make any guarantees.

WHR is a bayesian rating system like Elo. It is time dependent, meaning your rating function is optimized for each point in time, not just your most recent or average game. It also adds an uncertainty to each point in time, similarly to Glicko or Trueskill (they only store one data point). The only thing that improves your rating is if you win games, so it's much harder to exploit than ratings like WNR and winrate. I will explain this in more detail later. You can also read this thread if you're interested. If you're wondering who else is using this system, take a look at Go ratings.

How does one access the WG API? Are there any battle data sets(containing a list of battles with usernames of each battle and who won) available that I could test this on? Let me know if you need more information.

Share this post


Link to post
Share on other sites
40 minutes ago, DeinFreund said:

The only thing that improves your rating is if you win games, so it's much harder to exploit than ratings like WNR and winrate.

So, you would have rating system that only takes into account how much you win, and against what players, if I understood it correctly? While that sounds interesting, I don't see how it would differentiate from the normal WR rating in the long run, unless it can take into account platoons, what tiers you play and what tanks you play. Win rate itself is probably the easiest stat to manipulate, and in 15 v 15 what skill level players you play against becomes irrelevant in the long run, because average would be same for all after enough games (no skill-based mm). More over, current statistics systems don't take WR into account because they are meant to compliment that, not provide stand-alone metric, exception being WG PR. In my opinion, after many different rating systems that we have seen, WOT is kind of a game where you can't have absolutely accurate rating, there are too many variables in the game and too many ways to play the game. And on the other hand, basically every rating we have gives you enough accuracy in most cases, or should I say as much accuracy as you can expect with all the variables we have. 

Anyway, will be interesting to read what you have in mind :) I did check the links but too lazy to really study them.

Share this post


Link to post
Share on other sites

It is only based on winning, just like win rate. But winrate is pretty much the most jittery rating you can get, needing thousands of games to average out its errors. It's also susceptible to platooning or WG manipulating the Matchmaker to not be random.

Elo is the basic step-up. Instead of only considering whether you won or not, it'll also check your enemies and allies. If you play with a strong team (possibly caused by platooning), you'll only earn little reward for winning. If you play against a strong team, you'll only lose little on a defeat and gain a lot on a victory. Everybody starts at the same rating, and over time it'll get a good approximate of each player's influence on winning games. TrueSkill, Glicko and WHR are all improvements based on the same idea. I wouldn't worry too much about bottom/center/top tier, I've seen similar things have little impact in other games. Especially with the new MM it shouldn't be much of an issue.

Not incorporating any in-game stats is part of the foundation of all these ratings. Once you start rewarding other things than winning, there are always loopholes and uncovered cases. So in short, these systems would be a possible replacement for winrate that reduce impact of platooning and bad MM. It could be applied per-tank, per-tier or per-player. I am doubtful whether WG actually provides the API required for such ratings though. The rating could still be used for sites where you upload your battle history.

I think I've seen some similar topics before, so I expected this to be a familiar sight. Because I already have the code to test it, I thought I'd just give it a go if you have some data sets available.

Share this post


Link to post
Share on other sites

Not going into technical difficulties as I don't really know what WG API includes, it would definitely need to take into account not only who you play with and against (again, excluding platoons this will average out in the long run, so only use would be for short term session statistics), but also what tiers and tanks you play. Even per tier would not tell much as there are so big differences in what kind of impact different tanks have in the games. But, if your rating would just take into account platoons it would be a big improvement over current overall WR. Then, you could have expected WR for each tank and use each player adjusted per tank WRs to determine "real" WR that takes into account how much you have platooned and what tanks you have won with...that would be pretty good rating I think. And for per tank WR it might be even meaningful to use skill levels of your and enemy teams as overall game numbers are usually not that high, although probably not worth to do it.

Share this post


Link to post
Share on other sites

There are definitely a lot of drawbacks caused by how different tanks and tank compositions affect the outcome. I can't really say anything about whether and for what use case this would work if at all. WNR is probably much more precise, but also much more exploitable. However I might as well try it.

Share this post


Link to post
Share on other sites

Why?

I'm not trying to be a dick, but hasn't this poor old horse been beaten badly enough?  This is not chess, even though superficially it might feel analogous - its not.  In chess you don't let some addled cretin off of the street play your rook for lulz.  Trying to create yet another rating metric based on the most easily skewed variable in the game is just silly.

Do something productive like write an app that lets me read all of those click-bait ranker lists in a single page without ads.  Now that would raise your ELO brother, that would make you purple at life.

WoT does not need another rating scale, we can already pretty much discern who is good at the game as it is. 

Share this post


Link to post
Share on other sites

Obviously no one needs another rating, I understood that OP is doing it just out of interest, rather than to answer some need. And if you are interested in that kind of stuff, why not? Every rating from EFF has been good enough.

Share this post


Link to post
Share on other sites

@Haswell  Thanks, it seems individual battles aren't available in the API so WHR wouldn't work. If a moderator could move this into the right section that'd be nice.

Share this post


Link to post
Share on other sites

Would you be able to use the dossier? 

Share this post


Link to post
Share on other sites

It seems even the dossier doesn't contain any information about other players you played with. In order to establish such a rating I'd need to know about individual battles, who participated in them and who won. Apparently this is only available in replays. WOT would need some kind of battle database that I could use.

The rating would be very suited for esports, for both teams and individual players. So maybe I could use it with the esport section from wotreplays. Not sure how much the site owners like scrapers though. Also WOT seems to be rather new to esports, I'm not sure whether there are enough tournaments available for a rating to be sensible.

Share this post


Link to post
Share on other sites
On 07/08/2017 at 8:02 PM, DeinFreund said:

Also WOT seems to be rather new to esports

ehh. No.

 

Its more like WG don't really give a shit about esports, regardless of what they've said publicly. Just look at their balancing decisions. 

Share this post


Link to post
Share on other sites

Uhm, sorry if I miss something but:

If you have a sufficient number of battles then it is VERY likely that your average team mates and the average opponent winrate is THE SAME for everyone.

So how is using that is making a good rating even if you could access the info?

Share this post


Link to post
Share on other sites
On 8/8/2017 at 3:25 PM, MagicalFlyingFox said:

ehh. No.

Apparently I got that wrong. Just found https://worldoftanks.eu/en/tournaments. Is there an API for these or would I have to write a scraper? I'm not sure if it's really worth doing though, is there an interest in tournament ratings? 

 

On 8/8/2017 at 4:10 PM, Jaegaer said:

Uhm, sorry if I miss something but:

If you have a sufficient number of battles then it is VERY likely that your average team mates and the average opponent winrate is THE SAME for everyone.

So how is using that is making a good rating even if you could access the info?

I'm not quite sure what you're trying to say, but the idea of a good rating system is to require small amounts of battles to get good results. So you can keep track of a player's skill development and estimate his current strength. The currently used ratings mostly seem to concentrate on getting a good estimate of a player's average skill. For a four year old players this average represents how he played two years ago instead of how he plays now.. You can't just use recent winrate because it is too random, this is where rating systems come in.

 

@Private_Miros tanks

Share this post


Link to post
Share on other sites
18 hours ago, DeinFreund said:

the idea of a good rating system is to require small amounts of battles to get good results

All rating systems that work that way adjust the MM according to the results of previous matches.

In other words: ELO works because the MM uses the ELO-rating to create matches.

This is not true for WoT. Furthermore you can not poll the WoT API for matches, only for overall and per tanks. So you can't even construct a recent rating from the API alone (you would need to poll tank for tank, save the results and then poll tank for tank again and measure the differences - quite a lot of effort).

So it all boils down to the fact that the effort to construct a database for a "good" rating is way too much effort for the improvement of quality over WN8.

Share this post


Link to post
Share on other sites
45 minutes ago, Jaegaer said:

All rating systems that work that way adjust the MM according to the results of previous matches.In other words: ELO works because the MM uses the ELO-rating to create matches.

No, Bayesian rating systems are completely independent of the matchmaking. They are designed to account for any kind of matches. For example with Elo: If you are put in matches that you will lose nine out of ten times and do lose nine out of ten times, your rating will stay constant. Chess tournaments aren't made to put people of similar Elo against each other. That may be the case in the first round for some tournaments, but after that it's completely open. I've implemented WHR for zero-k without even needing a matchmaker.

Of course most systems will work better if you have good matchmaking. It's easier to learn a player's strength when he is put against equal opponents. It's impossible to determine somebody's strength if he wins/loses every match. Random MM as we have it in WOT is already more than adequate for this. 

Quote

 

This is not true for WoT. Furthermore you can not poll the WoT API for matches, only for overall and per tanks. So you can't even construct a recent rating from the API alone (you would need to poll tank for tank, save the results and then poll tank for tank again and measure the differences - quite a lot of effort).

So it all boils down to the fact that the effort to construct a database for a "good" rating is way too much effort for the improvement of quality over WN8.

 

I've noticed this. There would be no point trying to make this a rating for the general WOT player. This is why I suggested using it for competitive/esports only. But I doubt there actually is any interest in having a rating for WOT tournaments. It seems this would be of much more use for games like csgo or dota.

Share this post


Link to post
Share on other sites
1 hour ago, DeinFreund said:

But I doubt there actually is any interest in having a rating for WOT tournaments. It seems this would be of much more use for games like csgo or dota.

Yes, indeed. WoT is mainly paid by the older generation (like me) who do not follow an esports scene too keenly. On top of that WoT also doesn't lend itself for competitive esports because of the randomness involved as well as the general format.

Share this post


Link to post
Share on other sites

It would be a most excellent system. Especially if you disregarded tank selection, in effect making tank selection a part of the metric.

Some internalized adjustment would need to be made for tier played I think.

Unfortunately, WG doesn't give you the data to make such a thing happen. :(

Praetor, Richard Nixon (same person perhaps ??? :) ) and I discussed this but it's not possible with the data provided.

 

IMHO, WG gives a flawed API in order to prevent an accurate rating system from being developed. 

There is a reason Poker is more popular than Chess. Providing a game where most everyone can delude themselves into thinking they are worthy is very important

to the success of a game on a monetary level. My .02

Share this post


Link to post
Share on other sites

@Buckyball Thanks for your positivity, I hope you enjoyed reading the paper!

I've already tried different weighting systems for Teams in Zero-K, even though there are no tiers(More weight to newbies and the like). It should be easy to use this for tiers. From my experience it'd probably be best though to completely ignore tiers and tanks in the rating. This might sound weird, but in many games those things only really make a minor impact. The enemy might have 50% more HP, but skill and teamwork have a far bigger impact. Too sad WOT is a closed-source for-profit game. :(

There's also an easy and effective way to compare different iterations of or completely different rating systems. Take the probability p with which the system predicted the actual outcome, then score the system by `1 + log2(p)`. This gives the system a score of 0 for 0.5(50:50) probability, 1 for perfect prediction and negative infinity if it was completely wrong. Average this for all games and you get a fairly good score by which you can compare the different systems with smallish sample sets (much better than taking correct percentage).

If you think you could use this rating somewhere just let me know. Of course you're also free to copy my Java and C# implementations.

Share this post


Link to post
Share on other sites
Sign in to follow this  
Followers 0

  • Recently Browsing   0 members

    No registered users viewing this page.