Kalith

New effort to create tank specific values for wn8 rating

102 posts in this topic

I'm Kalith from wottactic.com, and I'm posting to let people know I'm starting an effort to create a new tank specific data set for the wn8 rating. I'll base myself on the practices explained here: http://wiki.wnefficiency.net/pages/WN8#Data_Sourcing. I'm still looking into it, but when I'm done I'll go into detail on the exact process. I'll also release all code.

I am doing this, because I disagree with both the v29 and v30/v31 average values. v29 because of the ridiculous expected values manually assigned to certain tanks. I'm not opposed to a little tweaking, but it has to be well founded. v30/v31 abandons tank specific ratings altogether. This Brings us back to days before wn8, where people where boosting heir stats by playing the most op tank they could find.

Yes wg balancing tanks and shifting their tiers makes the rating less reliable. Yes the values for reward tanks and certain rare tanks are skewed and need to be tweaked manually, which is statistically iffy. Still wn8 is still the best tool we have and it's fundamentals are sound, but it needs up to date expected values, so that's what I'm offering.

If you want to help or offer any advice, feel free to contact me.   

Share this post


Link to post
Share on other sites

Generating WN8 values from data is an interesting exercise as you can tell from a few of the threads I've started here. My script (in R) is posted in one of the older ones. If you are successful in generating the values using the right method - and I assume you would be - the problem is what happens when, some months from now, something changes (new tanks, tier changes, buffs/nerfs, etc)  - and you are afk. We'd be here all over again. The reason we went to the averages was sustainability - no-one wants to do all these endless updates. Reinforcing that decision is that the change to most player's WN8 by moving to averages was less than 100; it's a 'so what' change. Finally, if it makes players happy to pad, have at it. Life is too short to play bad tanks, right?

The one thing I'd ask is that if you do go down this road, follow the method published here and start with the code we used. If you use different filters, methods, or - worst of all - don't use linear models to align back per-tank performance to account level performance then you'll have a baseless WN8 spinoff like KTTC. 

You might want to consider getting WN9 going instead. If you could get THAT up and running (RichardNixon the creator is long gone) then you'd be a rock star.

EDIT: just heard that apparently our friends at xvm are ignoring our v30/31 values and creating their own. Hey, wouldn't it be great if every site created their own values? o.O 

Share this post


Link to post
Share on other sites

I'd be happy to use your R script if it still works, that would be a great help. Can you give me a link ? And how does it like it's data ?

I have quite a bit of experience scraping the statistics data from WG, for wottactic stats I basically make a list off all active accounts and re-fetch the data for all accounts that have played battles that week. ~200K players on average. I can't use that data set of course because it's created from all players that have ever been looked up on wottactic or are in a clan that has been looked up. 

About wn9, I did a correlation test between wn8/winrate and wn9/winrate on my data set and i got:

corr wn9-wr: 0.7867520119707153
corr wn8-wr: 0.8046118979857809

This could be because of outdated data, but wn8 did seem the better rating.

As for maintaining it, we'll see when we get there. I have to admit I don't have a lot of free time, I'll have to see to what extend I can streamline the process. 

Anyway I'm focusing my efforts on the eu server for now. To create an unbiased list of players with more than 1k battles, I'm just asking for the battle counts of all players between 500000000 and 540000000 (100 at a time), I've never seen an eu wg id much over 530000000, so that should catch them all. Anyway this alone will take approx 16h as I can't send >10 request a second without them getting dropped. But at least then I'll know how many players we're dealing with and if I can just fetch all their stats or if I need to work with a random sample.

Anyway I don't necessary want to do this alone, and I would be happy with any advice or help I can get. I do have solid base of statistics and a more solid base on programming.

As a side note, I think it would be better to start with the "random" stats instead of the "all" stats. Random stats seem to be basically "all" - tank company stats. I know wn9 uses it and it just makes more sense, wouldn't it be better to move wn8 to the random stats ?

 

Share this post


Link to post
Share on other sites
13 minutes ago, Kalith said:

About wn9, I did a correlation test between wn8/winrate and wn9/winrate on my data set and i got:

corr wn9-wr: 0.7867520119707153
corr wn8-wr: 0.8046118979857809

This could be because of outdated data, but wn8 did seem the better rating.

Didn't WN9 include a variable to factor out the effects of platooning on WR?  If so, that might account for some of the variance, though probably not that much.

You do realize that the skewed values in V29.0 was a deliberate prank, right?

Kudos to your attempt, but it just seems like yet another attempt to reinvent the wheel with an incomplete toolbox.

Share this post


Link to post
Share on other sites

Sure I realize it's a prank. But it's been taken over by most rating websites and used over the last 4 months or so. I think it has run it's course.

Anyway I'm really not trying to re-invent the wheel here. I would be fine with with somewhat accurate and updated expected values for wn8. I'm not creating a new rating, I am just trying to update the expected values for wn8.

Share this post


Link to post
Share on other sites

Lipstick on a pig bro. WG wants their data dirty and invalidated by history, so why try to futilely clean their sty? I know it's flippant, but I cannot think of a reason unless you're trying to practice munging and modeling for fun. In terms of outcomes...there's nothing more to be squeezed from that dataset.

Share this post


Link to post
Share on other sites

Wn8 is dead, this game is dead.  There is no such thing as end game play, clan wars is gone, strongholds are a joke, the only end game now is League which is highly constricted to a select few elite.

GIVE UP.  IN 2 YEARS OR LESS THIS GAME WILL BE GONE

Share this post


Link to post
Share on other sites
5 hours ago, Kalith said:

IYes wg balancing tanks and shifting their tiers makes the rating less reliable.

Less reliable? Try practically unreliable. Any metric that relies on history and perusal of the overall record to generate expected values is by the very nature of the data set fundamentally flawed. As we say: GIGO (garbage in, garbage out)

WN8 has run its course and deserves to be given a dignified retirement. As Elsa would say...

 

 

Share this post


Link to post
Share on other sites

For posterity and historical interest, a recent version of the expected values update R script is below.

You need a dataset to read in at start.

You also need to manually add to the output file estimated values for new tanks.

FYI, 'compDescr' is the unique tank ID, also known as IDNum on some sites.

#WN8 Expected Values Updater by Gryphon

#load data from csv file on HDD
dataMaster <- read.csv("~/R/WN8/dataMaster_2017-04-25.csv") #this is the datafile of user accounts, one row per user/tank
any(is.na(dataMaster))
head(dataMaster)
nrow(dataMaster)

#apply filters as needed
userTankStats <- dataMaster[dataMaster$battles > 50,]
userTankStats$damage_dealt <- as.double(userTankStats$damage_dealt)
userTankStats <- userTankStats[,c("userid", "compDescr","title", 
                                  "type", "tier", "countryid", "battles",
                                  "victories","damage_dealt","frags",
                                  "spotted","defence_points")]
userTankStats$userid <- as.factor(userTankStats$userid)
any(is.na(userTankStats))

# number of battles in dataset
sum(userTankStats$battles)

#calc actuals
userTankStats$aFRAG <- userTankStats$frags/userTankStats$battles
userTankStats$aDAMAGE <- userTankStats$damage_dealt/userTankStats$battles
userTankStats$aSPOT <- userTankStats$spotted/userTankStats$battles
userTankStats$aDEF <- userTankStats$defence_points/userTankStats$battles
userTankStats$aWIN <- 100*userTankStats$victories/userTankStats$battles
any(is.na(userTankStats))


#load current expected values from wnefficiency.net - currently version 30
wnefficiencyURL <- "http://www.wnefficiency.net/exp/expected_tank_values_30.csv"
expectedValues <- read.csv(wnefficiencyURL)
names(expectedValues) <- c("compDescr", "eFRAG", "eDAMAGE","eSPOT", "eDEF", "eWIN")

head(expectedValues)
any(is.na(expectedValues))

# add the expected values data to the user tanks data
require(dplyr)
userTankStats <- inner_join(x=userTankStats, y=expectedValues, by = c("compDescr") )

# fix chars that upset file naming
userTankStats$title <- chartr("*/", "_-", userTankStats$title)
any(is.na(userTankStats))

# calculate the user rSTATS
userTankStats$rFRAG <- userTankStats$aFRAG/userTankStats$eFRAG
userTankStats$rDAMAGE <- userTankStats$aDAMAGE/userTankStats$eDAMAGE
userTankStats$rSPOT <- userTankStats$aSPOT/userTankStats$eSPOT
userTankStats$rDEF <- userTankStats$aDEF/userTankStats$eDEF
userTankStats$rWIN <- userTankStats$aWIN/userTankStats$eWIN
userTankStats$rFRAGproduct <- userTankStats$rFRAG * userTankStats$battles
userTankStats$rDAMAGEproduct <- userTankStats$rDAMAGE * userTankStats$battles
userTankStats$rSPOTproduct <- userTankStats$rSPOT * userTankStats$battles
userTankStats$rDEFproduct <- userTankStats$rDEF * userTankStats$battles
userTankStats$rWINproduct <- userTankStats$rWIN * userTankStats$battles
any(is.na(userTankStats))

# calculate the user rSTATc's
userTankStats$rWINc <- pmax(0,(userTankStats$rWIN - 0.71)/(1 - 0.71))
userTankStats$rDAMAGEc <- pmax(0,(userTankStats$rDAMAGE - 0.22)/(1 - 0.22))
userTankStats$rFRAGc <- pmax(0,pmin(userTankStats$rDAMAGEc + 0.2,((userTankStats$rFRAG - 0.12)/(1 - 0.12))))
userTankStats$rSPOTc <- pmax(0,pmin(userTankStats$rDAMAGEc + 0.1,((userTankStats$rSPOT - 0.38)/(1 - 0.38))))
userTankStats$rDEFc <- pmax(0,pmin(userTankStats$rDAMAGEc + 0.1,((userTankStats$rDEF - 0.10)/(1 - 0.10))))
userTankStats$rWINcproduct <- userTankStats$rWINc * userTankStats$battles
userTankStats$rDAMAGEcproduct <- userTankStats$rDAMAGEc * userTankStats$battles
userTankStats$rFRAGcproduct <- userTankStats$rFRAGc * userTankStats$battles
userTankStats$rSPOTcproduct <- userTankStats$rSPOTc * userTankStats$battles
userTankStats$rDEFcproduct <- userTankStats$rDEFc * userTankStats$battles
any(is.na(userTankStats))

# calculate the user WN8 per tank 
userTankStats$WN8 <- with(userTankStats, 980*rDAMAGEc + 210*rDAMAGEc*rFRAGc + 155*rFRAGc*rSPOTc + 75*rDEFc*rFRAGc + 145*pmin(1.8,rWINc))
userTankStats$WN8product <- userTankStats$battles * userTankStats$WN8
any(is.na(userTankStats))

# filter out all tanks where WN8 is below median WN8 for every users' tanks
require(dplyr)
median.userTankStatsWN8 <- summarize(group_by(userTankStats,userid), median_WN8 = median(WN8, na.rm=TRUE))
userTankStatsFiltered <- inner_join(x=userTankStats, y=median.userTankStatsWN8, by = "userid")
userTankStatsFiltered <- userTankStatsFiltered[userTankStatsFiltered$WN8 >= userTankStatsFiltered$median_WN8,]
nrow(userTankStatsFiltered)
any(is.na(userTankStatsFiltered))
rm(median.userTankStatsWN8)

#calculate the user account WN8, rSTATs, and rSTATSc
require(dplyr)
userAccountStats <- summarize(group_by(userTankStatsFiltered, userid), 
                              WN8product = sum(WN8product),
                              rWINproduct = sum(rWINproduct), 
                              rDAMAGEproduct = sum(rDAMAGEproduct),
                              rFRAGproduct = sum(rFRAGproduct), 
                              rSPOTproduct = sum(rSPOTproduct), 
                              rDEFproduct = sum(rDEFproduct),
                              rWINcproduct = sum(rWINcproduct), 
                              rDAMAGEcproduct = sum(rDAMAGEcproduct),
                              rFRAGcproduct = sum(rFRAGcproduct), 
                              rSPOTcproduct = sum(rSPOTcproduct), 
                              rDEFcproduct = sum(rDEFcproduct),
                              battles = sum(battles))

userAccountStats$user_WN8 <- userAccountStats$WN8product / userAccountStats$battles
userAccountStats$user_rWIN <- userAccountStats$rWINproduct / userAccountStats$battles
userAccountStats$user_rDAMAGE <- userAccountStats$rDAMAGEproduct / userAccountStats$battles
userAccountStats$user_rFRAG <- userAccountStats$rFRAGproduct / userAccountStats$battles
userAccountStats$user_rSPOT <- userAccountStats$rSPOTproduct / userAccountStats$battles
userAccountStats$user_rDEF <- userAccountStats$rDEFproduct / userAccountStats$battles
userAccountStats$user_rWINc <- userAccountStats$rWINcproduct / userAccountStats$battles
userAccountStats$user_rDAMAGEc <- userAccountStats$rDAMAGEcproduct / userAccountStats$battles
userAccountStats$user_rFRAGc <- userAccountStats$rFRAGcproduct / userAccountStats$battles
userAccountStats$user_rSPOTc <- userAccountStats$rSPOTcproduct / userAccountStats$battles
userAccountStats$user_rDEFc <- userAccountStats$rDEFcproduct / userAccountStats$battles

userAccountStats <- userAccountStats[,c("userid",  "user_WN8", "user_rWIN", "user_rDAMAGE", "user_rFRAG", "user_rSPOT", "user_rDEF", "user_rWINc", "user_rDAMAGEc", "user_rFRAGc", "user_rSPOTc", "user_rDEFc")]
any(is.na(userAccountStats))

#merge back
require(dplyr)
userTankStatsFiltered <- inner_join(x=userTankStatsFiltered, y=userAccountStats, by = c("userid"))
any(is.na(userTankStatsFiltered))

# create table of compDescr and title as index for the loop
require(dplyr)
listOfTanks <- summarize(group_by(userTankStatsFiltered, compDescr, title ), users = n() )
any(is.na(listOfTanks))

# loop to do linear regression for each rSTAT vs user account rSTAT, derive corrected expected values
newExpectedValues <- expectedValues
for (i in listOfTanks$compDescr){
    sample <- userTankStatsFiltered[userTankStatsFiltered$compDescr == i,]
    rDAMAGEmodel <- lm(rDAMAGE ~ user_rDAMAGE, data=sample)
    rDAMAGEcorrection <- rDAMAGEmodel$coef[[1]] + rDAMAGEmodel$coef[[2]]
    eDAMAGE_new <- round(rDAMAGEcorrection * expectedValues$eDAMAGE[expectedValues$compDescr == i], 2)
    newExpectedValues$eDAMAGE[newExpectedValues$compDescr == i] <- eDAMAGE_new
    rFRAGmodel <- lm(rFRAG ~ user_rFRAG, data=sample)
    rFRAGcorrection <- rFRAGmodel$coef[[1]] + rFRAGmodel$coef[[2]]
    eFRAG_new <- round(rFRAGcorrection * expectedValues$eFRAG[expectedValues$compDescr == i], 2)
    newExpectedValues$eFRAG[newExpectedValues$compDescr == i] <- eFRAG_new
    rSPOTmodel <- lm(rSPOT ~ user_rSPOT, data=sample)
    rSPOTcorrection <- rSPOTmodel$coef[[1]] + rSPOTmodel$coef[[2]]
    eSPOT_new <- round(rSPOTcorrection * expectedValues$eSPOT[expectedValues$compDescr == i], 2)
    newExpectedValues$eSPOT[newExpectedValues$compDescr == i] <- eSPOT_new
    rDEFmodel <- lm(rDEF ~ user_rDEF, data=sample)
    rDEFcorrection <- rDEFmodel$coef[[1]] + rDEFmodel$coef[[2]]
    eDEF_new <- round(rDEFcorrection * expectedValues$eDEF[expectedValues$compDescr == i], 2)
    newExpectedValues$eDEF[newExpectedValues$compDescr == i] <- eDEF_new
    rWINmodel <- lm(rWIN ~ user_rWIN, data=sample)
    rWINcorrection <- rWINmodel$coef[[1]] + rWINmodel$coef[[2]]
    eWIN_new <- round(rWINcorrection * expectedValues$eWIN[expectedValues$compDescr == i], 2)
    newExpectedValues$eWIN[newExpectedValues$compDescr == i] <- eWIN_new
}
any(is.na(newExpectedValues))
newExpectedValues <- newExpectedValues[,c("compDescr","eFRAG", "eDAMAGE", "eSPOT", "eDEF",  "eWIN")]

#export new values
date <- as.Date(Sys.Date(), "%m/%d/%Y" )
expected_value_filename <- paste("~/R/WN8/expected_values_",date,".csv")
write.csv(x=newExpectedValues,file=expected_value_filename ,row.names = FALSE)

Its best to check the results graphically by plotting the linear models and inspecting them for damage at least (500+ graphs). Only for those with a strong constitution...

Share this post


Link to post
Share on other sites

Ok the script that is making the account list is over half way done. It found 1.7mil accounts with >1000 battles and 1.3 mil accounts with over 2500 battles, so I'm going to assume around 3mil eu accounts when it's all done. Tbh that's not so bad, according to my calculations I should be able to fetch those in +- 5 days without getting banned from the wg api :).

Anyway I'll make sure to do a random shuffle on the wg ids before I start downloading the stats, so anything it does download can be used as a sample.

I got your script to work fine. on a very small sample of stats I downloaded. It does spit out some expected values that make sense at first sight. A few NA's in there but I presume that my small sample of 5 accounts doesn't cover every tank.

Anyway here's sample csv file I generated, your script seems to accept it, but just to make sure, can you have a look to see if I haven't made any mistakes. For example I don't know exactly what title is supposed to be. Here is the link: https://karellodewijk.github.io/other/input.csv

As for inspecting the output, can you post the code to create the scatter plot with the linear regression line. I can probably figure it out, but it's been a while since I used R.

But tbh manually eliminating outliers as you seem to be doing is time consuming and even a little iffy. I'd rather give it more data so the outliers matter less or at least automate the process using some fixed rules.  I also think these outliers are a result of accounts with a low number of battles (>50) on a certain tank. Wouldn't it help a lot already if you weighted your linear regression again with the number of battles, or are you already doing this ?

Anyway Thanks for the help Gryphon, I appreciate it.

Share this post


Link to post
Share on other sites

Here is the plotter function, with a few calls to run important views. Note: the first two calls crank out over 500 tanks every time you run it.

The purpose of me posting this is to educate you and others on how WN8 worked so that we can figure out what changes we need to get WN9 going. I have no interest in resurrecting the WN8 update process unless a site (maybe wottactic, maybe xvm) figures out a way to do it automatically AND PUBLISHES THE RESULTS FOR EVERY OTHER SITE TO USE. Please lets not all have our own sets of values...!

However, when you consider that WN8 now has averaged expected values, and that WN9 starts with averaged expected values and then applies per tank scaling factors, it makes sense to leave WN8 as is, and get WN9 going for those who want the extra detail. 

Key to implementing WN9 is understanding the scaling factors RN developed. Might be able to pull those straight out of the linear model that R generates. WN9 also had different filters on the dataset, different formula, and I think it dropped the change from rSTAT to rSTATc.

#WN8 rSTAT Plotter by Gryphon

#plot function
require(ggplot2)
ggplotRegression <- function (fit, title) {
    ggplot(fit$model, aes_string(x = names(fit$model)[2], y = names(fit$model)[1])) + 
        geom_point() +
        stat_smooth(method = "lm", col = "red") +
        coord_cartesian(ylim = c(0,2), xlim = c(0,2)) +
        labs(title = paste(title,":", names(fit$model)[1], "=", round(fit$coef[[1]],2), "+", + round(fit$coef[[2]],2), "*", names(fit$model)[2],
                           "; Adj R2=", round(summary(fit)$adj.r.squared, 2)))
}

# run rSTATc vs user_rSTATc plots for all tanks
for (i in listOfTanks$compDescr){
    sample <- userTankStatsFiltered[userTankStatsFiltered$compDescr == i,]
######set model parameters###########
    model <- lm(rDAMAGEc ~ user_rDAMAGEc, data=sample)
#####################################
    jpegString <- paste("~/R/WN8/plot/", names(model$model)[1],"_vs_",names(model$model)[2], "_v30_%s.jpg")
    ggplotRegression(model, listOfTanks$title[listOfTanks$compDescr == i] )
    savedPlot <- sprintf(jpegString, listOfTanks$title[listOfTanks$compDescr == i])
    ggsave(file=savedPlot, width=7, height=7)
}


# run WN8 vs user WN8 plots for all tanks
for (i in listOfTanks$compDescr){
    sample <- userTankStatsFiltered[userTankStatsFiltered$compDescr == i,]
    ######set model parameters###########
    model <- lm(WN8 ~ user_WN8, data=sample)
    #####################################
    jpegString <- paste("~/R/WN8/plot/", names(model$model)[1],"_vs_",names(model$model)[2], "_v30_%s.jpg")
    ggplotRegression(model, listOfTanks$title[listOfTanks$compDescr == i] )
    savedPlot <- sprintf(jpegString, listOfTanks$title[listOfTanks$compDescr == i])
    ggsave(file=savedPlot, width=7, height=7)
}


# user_WN8 ~ user_rWINc for all accounts
usermodel <- lm(user_WN8 ~ user_rWINc, data=userAccountStats)
ggplot(usermodel$model, aes_string(x = names(usermodel$model)[2], y = names(usermodel$model)[1])) + 
    geom_point() +
    stat_smooth(method = "lm", col = "red", size = 1 ) +
    coord_cartesian(ylim = c(0, 3000), xlim = c(0,2)) + 
    geom_abline(intercept = 0, slope = 1565, col = "blue", size = 1) +
    labs(title = paste(names(usermodel$model)[1], "vs",names(usermodel$model)[2],"; Adj R2=", round(summary(usermodel)$adj.r.squared, 2)))
savedPlot <- paste("~/R/WN8/plot/",names(usermodel$model)[1],"vs", names(usermodel$model)[2],"_v30.jpg")
ggsave(file=savedPlot, width=7, height=7)

 

Share this post


Link to post
Share on other sites

Well I'm a realist, wn8 is used today and very widespread. I also think it's fundamentally sound, with per tank expected values. And from my limited testing it holds up well even against wn9. I do feel it should use "random" stats published by WG and not the "all" stats as a base. The "all" stats by wg are random stats + tank companies + clan wars before a certain date. wn9 does use the "random" stats. But I'm not going to implement that myself without some kind of consensus.

Anyway I fully intend to release my expected values with all code and the process I've followed to get there. 

As for wn9, I'll release my code to scrape stat data and if you want I can create a raw dataset in any format with any filters you want. When browsing around I see you use a bash script with wget. That works obviously but by slightly modifying the code I've already written to keep wottatic stats up to date, I  can do it an order of magnitude faster. If you or someone else with experience with wn9 could then do the post processing, that would be great.

Share this post


Link to post
Share on other sites

maybe we should just make an entirely new metric based on dmg, kills, assist dmg, stun dmg, and dmg blocked.... just an idea

Share this post


Link to post
Share on other sites
19 minutes ago, saru_richard said:

maybe we should just make an entirely new metric based on dmg, kills, assist dmg, stun dmg, and dmg blocked.... just an idea

Please do that then.

Share this post


Link to post
Share on other sites
Just now, Stige said:

Please do that then.

it was just an idea....

Share this post


Link to post
Share on other sites
1 hour ago, Kalith said:

 I also think it's fundamentally sound, with per tank expected values.

It isn't. You may think that, but it isn't.

Share this post


Link to post
Share on other sites

Tank based values in theory would work, until its leaked how to manipulate them to make you 'pad' your scores. WN9 was the best attempt at this, given how your WN9 score scales with more damage, but even its flawed and wont be accepted

WN* is dead. Long live WN*

Share this post


Link to post
Share on other sites

Two thing

here is a list of all eu players with >1000 battles and their battle count: https://karellodewijk.github.io/other/players_eu.zip (2464513 in total), should be useful when calculating any rating, so you know which accounts exist.

I've also uploaded my code and explanation so far to github: https://github.com/karellodewijk/wn8_expected.

I've shuffled the players and started downloading their stats. It will take the better part of a week to download them all. But I don't think I need all 2.5 milion. A random sample of a few 100k accounts should be plenty. 

Share this post


Link to post
Share on other sites
18 hours ago, Shifty_101st said:

snip

fuck right off with your game is dead bullshit. tired of hearing this pathetic excuse for trying to make something better. if everyone who says "lawl gaem ded" was right it would have failed in 2013

Share this post


Link to post
Share on other sites

@Gryphon_, mostly

I'm not finished yet, but I tried running the data I had gathered so far and see what I got.

Basically I generated a data set:that your R script seems to be ok with: http://forum.wottactic.com/other/input.zip

That will give me a set of expected values, I added the tank name to it to make it easier to talk about: http://forum.wottactic.com/other/expected 2017-05-14 .csv

And I've also used your plot script to generate your scatter plots: http://forum.wottactic.com/other/plots.zip

 

Am I doing ok so far ?

 

Now what would be the next step. By investigating the scatter plots I can see some tanks are quite rare and people who played 50 games in them are even rarer. It will get a little bit better when the stats of more people are gathered, but there probably will be tanks where data is so sparse the linear regression is very unreliable. 

So I guess that for those rare tanks it's best to fix their expected values to the most similar tank I can find, right ?

Then there is the issue of tanks that were part of some kind of skill based reward. I'm thinking cw campaign rewards tanks, maybe the t-22, others ? They are not necessarily very rare but their raw calculated stats are super biased because only somewhat skilled people own them.

I guess for those I do the same and fix their stats to a very similar tank ?

Then there are tanks that have received significant buffs/nerfs/tier changes. I'm somewhat inclined to ignore this, because it's very subjective.

Am I on the right track or is there something else I need to do ? 

Share this post


Link to post
Share on other sites

The rare tanks are going to be less of a problem if you are pulling loads of data. We used data on 20k accounts from vbaddict for all the runs we did ,so if you have way more than that it should fill the gaps. Plots with less than 100 dots will be error prone. You might want to modify the script to 'pass' on calculating value corrections if there are less than 100 datapoints for the tank in the 'userTankStatsFiltered' dataframe.

For future reference regarding WN9, note that each graph, in the title gives the intercept and slope of the line of best fit. So if you wanted to generate the 'scaling' values for WN9, you can see where the data comes from (but you have to run the plots AFTER applying the new expected values first).

We used to assign values to some reward tanks and rare tanks using a pluglist where the community estimate an 'equivalent' tank for each , ie M60 = M48, and the script then read across those values from a pluglist before writing the results to a expected values file. Note: upkeep of the pluglist is a major PITA and that's one reason  we went to averaged values. The list I posted would need a lot of work to figure out which tanks could be removed from it (enough data exists) and which need to be added.

The code snippet to run a pluglist is here, and the last pluglist we used is attached.

any(is.na(newExpectedValues))
newExpectedValues <- newExpectedValues[,c("compDescr","eFRAG", "eDAMAGE", "eSPOT", "eDEF",  "eWIN")]

#make substitutions for rare tanks
expectedValuesAdjusted <- expectedValues # newExpectedValues
pluglist <- read.csv("~/R/WN8/pluglist.csv")
for (t in pluglist$tank_id)
{
    tank <- pluglist$tank_id[pluglist$tank_id == t]
    substitute <- pluglist$substitute_tankid[pluglist$tank_id == t]
    expectedValuesAdjusted[expectedValuesAdjusted$compDescr 
                           == tank,names(expectedValuesAdjusted) != "compDescr"] <- 
        expectedValuesAdjusted[expectedValuesAdjusted$compDescr 
                               == substitute,names(expectedValuesAdjusted) != "compDescr"]
}
any(is.na(expectedValuesAdjusted))
newExpectedValues <- expectedValuesAdjusted

 

pluglist30.csv

Share this post


Link to post
Share on other sites

Ok that makes sense, I'm just going to download all the accounts, it will take a few days but I can live with that, i hope WG is too :). And it will make my life easier in the long run.

I can't seem to download your attachment though, "The page you are trying to access is not available for your account.". Maybe just mail it to me [email protected]

I did just have to restart, I noticed I was missing some pretty popular tanks, looks like it was skipping over tanks it couldn't find name/tier/etc from. That together with "/wot/encyclopedia/vehicles" being very incomplete, made sure I missed a bunch of tanks. I switched to getting the tank info from "/wot/encyclopedia/tanks/", which is deprecated but at least more complete, and just to be sure if it encounters a tank that is missing it will just fill in some dummy values, it the id that counts anyway, such fun.

Anyway Plots with less than 100, got it, use community created pluglist file.

Thanks again

Share this post


Link to post
Share on other sites

I'm probably missing something here.  My question Kalith is how your stats gathering code accounts for recent battles vs. overall account totals?  Were the WN8 expected values always based on the account overall stats or was it somehow based on recent stats only?  My understanding is vbaddict's database only keeps the last 30-60 days of data or so, thus it's all recent and that was the primary data source used until v30.

Share this post


Link to post
Share on other sites

The WG api has no concept of recent battles. It has only totals.

The way stat websites calculate recent performance is, they take a snapshot of your tank stats, then a little later they take another snapshot and they calculate the difference.

Share this post


Link to post
Share on other sites

Well I finished gathering the data and creating new expected value files.

First of all here is a useful database for anyone debarking on something similar: http://forum.wottactic.com/other/input.7z. It contains a large csv file with lines such as:

"userid","compDescr","title","type","tier","countryid","battles","victories","damage_dealt","frags","spotted","defence_points"

for every tank of every player with > 1000 battles on the eu server.

 

As for processing the data: I've taken two approached. One identical to Gryphon's method.

For CW reward tanks, I've used the closest related tank, when in doubt I picked one that's probably a little bit better.

15617,Object 907,16897,Object 140
15905,M60,14113,M48A1 Patton
55841,T95E6,14113,M48A1 Patton
58641,VK7201,9489,E 100
63537,121B,4145,121
58881,IS-5,5377,IS-3
11809,T23E3,1569,T20
 

Then there were some exceedingly rare or new tanks. partly based on the community pluglist, partly just common sense:

54273,SU-76I,2369,FCM 36 Pak 40
59425,T34 B,2849,T34
12577,M4 Improved,52257,M4A2E4 Sherman
13905,FV4005 Stage II,9297,FV215b (183)
53793,T95E2,5921,M26 Pershing
58369,Object 260 mod. 1945,7169,IS-7
59665,Grosstraktor - Krupp,2385,Vickers Medium Mk. III
61457,Pz.Kpfw. III Ausf. K,6417,Pz.Kpfw. III/IV
54033,Pz.Kpfw. V\/IV Alpha,51473,Pz.Kpfw. V\/IV
49409,IS-6 B,9217,IS-6
64273,Panzer 58 Mutz,49937,Schwarzpanzer 58
59681,M4A3E8 Thunderbolt VII,1313,M4A3E8

Then there are a few very new or very rare tanks that I didn't have enough data, I've gone with the average values from v30

5681,0.68,1100.00,2.15,0.63,51.27    
4737,1.12,2153.40,0.71,0.64,49.22
13905,1.12,2153.40,0.71,0.64,49.22
15441,0.88,1162.12,1.31,0.91,51.62
18705,0.90,1579.22,0.98,0.68,51.03    
19473,0.95,1918.96,1.08,0.64,49.34
49921,1.00,1106.68,1.02,0.92,54.23
52513,0.92,1283.61,1.07,0.81,52.12    
52225,1.21,290.59,1.71,1.32,57.68    
19201,0.68,1400,2.15,0.63,51.27
 

except for the pz2j because there is nothing average about this tank. I just went on the limited data i have: 

  
51729,1.82,353.81,1.56,1.78,65.19

 

Then I've tried an alternative approach that removes the 50 minimum battle requirements, but when doing the linear regression weights the entries with the amount of battles. For popular tanks it yields very similar results, but for rarer tanks, there is a lot more data to work with.

I have again replaced the CW reward tanks with their closest counterparts

15617,Object 907,16897,Object 140
15905,M60,14113,M48A1 Patton
55841,T95E6,14113,M48A1 Patton
58641,VK7201,9489,E 100
63537,121B,4145,121
58881,IS-5,5377,IS-3
11809,T23E3,1569,T20

There were no relevant entries in the pluglist. And only these 9 tanks had insufficient.

15441, 17217, 18705, 19457, 51473, 52993, 19201, 62785, 63233

JITT, Chieftain\/T95, KV-4 Kreslavskiy, Mäuschen, AMX M4 mle. 49, Pz.Kpfw. V\/IV, A-32, T-100 LT

 

Resulting expected values: 

 

regular: http://forum.wottactic.com/other/expected_v32.csvhttp://forum.wottactic.com/other/expected_v32.json

weighted: http://forum.wottactic.com/other/expected_v31w.csvhttp://forum.wottactic.com/other/expected_v31w.json

 

Code and detailed procedures at: https://github.com/karellodewijk/wn8_expected

 

So what do you think ?

 

Share this post


Link to post
Share on other sites

  • Recently Browsing   0 members

    No registered users viewing this page.