Kalith

New effort to create tank specific values for wn8 rating

102 posts in this topic

Version v31 is already taken, that's the consolidated table by type and tier, which wotlabs will adopt in next 24 hours. So please repost as v32 to avoid confusion. 

My comments:

a) You need to add the missing tanks - especially AMX M4 mle. 49,  A-32, and T-100 LT - to the pluglist, so that the values when published are complete. Then you have to be prepared to do this again every time they issue a new tank or change something.

b) I dont support dropping the 50 battle filter and applying weights else the guy who played 18000 battles in his Maus distorts the regression towards his results. All that matters is how each player does on each tank compared to how each player did on all tanks (post filter)

c) The changes that we are seeing in player WN8 from v29 (normal) to v30 (averaged) is minimal. For example, mine changed from 1630 to 1635. So it looks like the averages work well enough and require no further work. How would you justify the work involved (you and the other websites) in going back to the normal method?

& Floor is open for community to comment on the values you published..

 

Share this post


Link to post
Share on other sites

I'll add the rating to wottactic as wn8 experimental or something.

I had another look at the tank list and looked them up. swarzpanzer 58 is the mutz black edition and the is-6B is the is-6 black edition, so those are obvious.

Pz.Kpfw. V/IV is apparantly the Pz.Kpfw. V/IV alpha, which is identical to the regular Pz.Kpfw. V/IV

The chieftain T95 will be a cw reward tank.

The BT-SV and pz2J are just very rare premium tanks.

The M6A2E1 was apparently available as a pre-order package for wot a long time ago.

The M4A3E8 Thunderbolt VII, Strv 103B, mausschen, T-100 LT and wz-132A, FV4005 Stage II have been released but are just new.

Not sure how the T26E5 made it onto this list, that's the patriot, I thought it was pretty popular but I only have 14 people on record playing more than 50 battles in it am I missing something ?

Anyway can use the easy 8 value for the thunderbolt those are pretty similar. But I don't know how comfortable I am guessing values for the Strv 103B, mausschen, T-100 LT and wz-132A, FV4005, they are not sufficiently similar to other tanks imo, maybe I should use the values from the weighted version.

 

 

 

Share this post


Link to post
Share on other sites

Ok I'll use that for the remaining tanks. For the tanks in the 'new' pluglist already, it's definitely a better approximation, also still can't open attachments on this forum :)

Share this post


Link to post
Share on other sites
14 hours ago, Kalith said:

I've now plugged in the values for the remaining tanks, I've updated the post accordingly. You can check out the stats of a player with the new rating here: http://wottactic.com/wn8_standalone.html

It's a little basic, but it gives you some idea.

 

 

Cant get that portal to return any results

Share this post


Link to post
Share on other sites

works as advertised for me

Share this post


Link to post
Share on other sites

You forgot the ISU-130.

Share this post


Link to post
Share on other sites

Not sure if something's wrong with the way I'm reading the CSV but not all T8 LTs are listed.  LTs are listed at their old tiers and no T9-10 LTs are listed.  The Lorraine 40t is listed as T9 still even though it's T8 now.

The portal works and shows these vehicles, however.  It does not list their tier so I can't confirm that aspect but I assume it's correct.  My overall WN8 is currently 2017 on wotlabs and 2064 on your portal (the forum account is my SEA account, I also play on NA), a difference of 47.

Thanks for putting in the effort!  Thoughts:

  • Did you say this is complete data for all players on all clusters (NA, EU, RU, SEA) or a sample/subset?  What would be most interesting to me would be comprehensive data across all clusters to use for data mining (eg: global WR for tank X, top WR tank at tier Y) vs. always relying on subsets of data like vbaddict, noobmeter, wot-news, etc.
  • Some of the biggest deltas are the SPGs mostly going up.  Seems like it doesn't make sense given the recent class-wide change to arty which should result in lower DPG across the board.  But then again it will take time for this change to be reflected in the overall account-wide data.
  • Along these lines some system that incorporates recent values would seem to provide the most accurate and relevant data about the current abilities of all players.

Share this post


Link to post
Share on other sites
4 hours ago, bounceplink said:

Not sure if something's wrong with the way I'm reading the CSV but not all T8 LTs are listed.  LTs are listed at their old tiers and no T9-10 LTs are listed.  The Lorraine 40t is listed as T9 still even though it's T8 now.

The portal works and shows these vehicles, however.  It does not list their tier so I can't confirm that aspect but I assume it's correct.  My overall WN8 is currently 2017 on wotlabs and 2064 on your portal (the forum account is my SEA account, I also play on NA), a difference of 47.

Thanks for putting in the effort!  Thoughts:

  • Did you say this is complete data for all players on all clusters (NA, EU, RU, SEA) or a sample/subset?  What would be most interesting to me would be comprehensive data across all clusters to use for data mining (eg: global WR for tank X, top WR tank at tier Y) vs. always relying on subsets of data like vbaddict, noobmeter, wot-news, etc.
  • Some of the biggest deltas are the SPGs mostly going up.  Seems like it doesn't make sense given the recent class-wide change to arty which should result in lower DPG across the board.  But then again it will take time for this change to be reflected in the overall account-wide data.
  • Along these lines some system that incorporates recent values would seem to provide the most accurate and relevant data about the current abilities of all players.

What exactly do you consider "comprehensive data across all clusters"? Because for WNR I'm pulling all tank data from all servers of all players who have played in the last 60 days. That's a little over 8.7 million players, 6.2 million of which are coming from the RU server. The mining has been pulling (optimally) 12.5 accounts per second for the past 7 days (with only a few spots of downtime to fix some program errors) and I've got about 2.7 million accounts left to pull. The expected optimum time to pull all the data was about 8 days, but the RU server is under very heavy load around 4am eastern and the collection average goes down to almost half of optimal during that time due to the number of retries for 504 errors.

If I actually tried to pull account data from every player who has account data, that would be 46.2 million accounts and my optimal expected mining time would be about 43 days, which isn't practical.

Share this post


Link to post
Share on other sites

@User

I checked for the ISU-130 and it is listed: 58625,1.01,1369.06,0.69,0.98,51.12

On what account did it mess up the stats ?

@bounceplink

I make no assumptions based on tier. The expected values for a tank are based on the performance of players on that tank compared to their performance on other tanks. This is plain wn8, I didn't change anything. It will take quite a long time for the expected values to reflect recent changes. I've not modified the way the expected values are calculated except for the tanks explicitly mentioned, I've even used the same code. I've simply applied it to a larger dataset. Also try to compare against v29 as it's the last version that had per tank values.

It is possible a tank is not picked up if it wasn't mentioned in v30 and nobody has played > 50 battles on it.

Fluctuations are normal. There isn't even a guarantee that it remains the same on average, the entire scale could have shifted up a bit. I sort of expected as much as previous expected values were calculated from a player list taken from a rating website which is slightly biased towards better players.

As I mentioned the expected values are calculated on all EU accounts with > 1000 random battles (random columns on API) taken about a week ago now.  2464513 players in total. I purposely didn't rely on rating sites because they are always biased towards people who look up their stats. I built the player list by just asking for the random battle count of all accounts ids 500000000-550000000. It's really not so bad as you can query a 100 accounts in a single request. Of course many accounts come back empty because they don't exist.

@Android25

It took me about 1 day to create the list of players and 5 days to download the data. I basically sent a request every 150ms or 7 a second to stay below the WG recommended maximum of 10 requests a seconds. Requests that bounced for whatever reason are automatically retried.

Yes it takes some time to gather the data if WG is reading this, if you release a database dump every few months or so, that would be very handy and we wouldn't have to harass your api servers quite as much :).

Share this post


Link to post
Share on other sites
On 5/23/2017 at 2:37 AM, Gryphon_ said:

 

c) The changes that we are seeing in player WN8 from v29 (normal) to v30 (averaged) is minimal. For example, mine changed from 1630 to 1635. So it looks like the averages work well enough and require no further work. How would you justify the work involved (you and the other websites) in going back to the normal method?

The difference between versions now is less important than the difference once people start selecting for the metric.  With previous versions light tanks were a padding choice, with tier averages the padding tanks are the ones with the biggest delta over average.

 

its an open question whether the work is worth it, but you can't simply say "it looks fine now" as though they are going to remain "close enough" in perpetuity.

Share this post


Link to post
Share on other sites
On 5/25/2017 at 10:49 AM, Kalith said:

@User

I checked for the ISU-130 and it is listed: 58625,1.01,1369.06,0.69,0.98,51.12

On what account did it mess up the stats ?

I entered my accountID on your website and searched for "130" and the ISU-130 wasn't listed by name.

Share this post


Link to post
Share on other sites

@User

Can you five me your wg id, so I can take a look ?

I've moved the stats for wottactic.com to these new expected values. I don't want to fragment wn8 unnecessarily. But  for reason mentioned earlier I also won't implement the v30 expected values.  

Everyone is free to adopt/use/(re)publish/improve or modify the tank specific wn8 expected values I've published, and I'll update them occasionally for the foreseeable future.  I think tank specific values are important for a more accurate and less padable rating and I hope people will adopt them or come up with another set of tank specific expected values to use. 

Share this post


Link to post
Share on other sites

ITT: Stats are dead, game is dead, why even try for anything, effort is pointless.

Pretty positive community you got here my dudes.

Share this post


Link to post
Share on other sites

Just wanted to thank the OP for his efforts to keep the metric going. A question - how hard would it be to automate the whole process of updating WN8 values? It would have to automatically figure out when a new tank is added (I assume if tanks are moved between tiers the moved tank gets a new key value in WG system, no?), then collect data on it, then calculate the expected values. For example, a tier 8 AMX 13 90 it would show up as AMX 13 90 v1 in expected values table, the current tier 9 one as AMX 13 90 v2. Both would have separate expected values.

To be honest I think with the dawn of the official WG mod portal the developers have better chances than ever to make WG help you automate this process by structuring their data better and automatically notify 3rd parties of new tanks via API.

 

PS - haven't seen the game as healthy for years as it is at the moment, at least on EU server.

Share this post


Link to post
Share on other sites
Quote

How hard would it be to automate the whole process of updating WN8 values? It would have to automatically figure out when a new tank is added.

Basically the script will pick any tank that a player has > 50 battles on. The problem is that it takes some time for enough people to reach 50 battles on a tank to be statistically useful.

Quote

I assume if tanks are moved between tiers the moved tank gets a new key value in WG system, no

Mostly no, there is no real way to tell them apart, except maybe taking a snapshot now and using the difference in stats rather than absolute stats. It's not ideal, I'll admit. I believe Android25 is collecting a stats dump of all players atm. If we have that we can use that as base, we can use that to calculate expected values based on difference between that and some point in the future. Hell we could include spotting damage/tracking damage/armor use in the rating as well then if wg does not change how they are calculated in between.

 

 

Share this post


Link to post
Share on other sites

If i am reading this right, your expected values are based off of ONLY the EU server, So My 1420ish wn8 of v29 is showing at 1482 with your data, If i am to believe this, then the claims of NA being a poor server pool of talent has just been proven incorrect. IMO the expected values should have been pulled from all available servers and done that way, otherwise the data set is just too small to offer a realistic view of what is actually average. Have you checked your data and compared it to other versions of wn8 expected data and checked to see what differences there are?

Share this post


Link to post
Share on other sites

The expected values have actually always been based on only the EU server, I've just used a much larger sample of players.

I've noticed on average wn8 with my expected values seems a bit higher overall. I can't really say for certain why as I don't know how/which data was gathered to calculate the expected values before.  

Anyway I'll be starting an update probably next weekend. I do plan this time to gather data across servers instead. 

Share this post


Link to post
Share on other sites
On 28/06/2017 at 7:32 PM, Kalith said:

The expected values have actually always been based on only the EU server, I've just used a much larger sample of players.

I've noticed on average wn8 with my expected values seems a bit higher overall. I can't really say for certain why as I don't know how/which data was gathered to calculate the expected values before.  

Anyway I'll be starting an update probably next weekend. I do plan this time to gather data across servers instead. 

What makes you say they were only based on EU in the past?  To my knowledge they were basing the data largely on vbaddict which has users from all regions.

Share this post


Link to post
Share on other sites

@bounceplink

I remember reading it somewhere, anyway Gryphon_ will know.

Anyway I've started on an update. Like before I'm starting by gathering a player list first but this time I'm building one with players across all servers. It might take some time to gather but I'll use that player list again in the future and publish it for other people to use.

 

Share this post


Link to post
Share on other sites

I gathered the player list. This list should contain all players across all servers who played 1 or more RANDOM battles in world of tanks. with the amount of random battles played. (~47.32 mil players). 

http://forum.wottactic.com/other/players_jul_12_2017.7z

I shuffled that list and am now downloading the stats for all players with > 1000 battles to generate the wn8 values. I probably will not end up downloading them all but a sufficiently large sample.

Share this post


Link to post
Share on other sites

Hi, I'm sirmax from XVM team.

I made automatic calculation of expected values for WN8 and I need your help to confirm that I'm doing its right.

I rewrote Gryphon's scripts from R to SQL and used our players database as data source.

Here you can see results and plots: https://yadi.sk/d/V0HRYfLs3NJDj9

The data recalculating every day using previous expected values as basis. For calculation I take all players who was active last 3 month (who played at least 1 game during 90 days), filtered by 1000+ battles on account. Tanks are filtered by 50+ battles.

New tanks are skipped from calculation. When the number of players of the tank will be more then 10000, it will be added to calculation automatically. For the starting expected values I made this script:

 
CREATE OR REPLACE FUNCTION wn8.fill_missed_expected_values()
RETURNS VOID
LANGUAGE plpgsql
AS $$
DECLARE
    i int := 0;
    l_total int;
    l_tank_ids int[];
    l_tank_id int;
    l_cnt int;
    l_ofs_lo int;
    l_ofs_hi int;
    l_battles bigint;
    l_tag text;
    e_dmg float8;
    e_frg float8;
    e_spo float8;
    e_def float8;
    e_win float8;
BEGIN
    RAISE LOG '[WN8] === calculating initial wn8.exp for new tanks ===';
    l_tank_ids := ARRAY(SELECT tank_id FROM xvm.vehicles_list ORDER BY tank_id);
    l_total := array_length(l_tank_ids, 1);
    RAISE LOG '[WN8] % tanks in the vehicles list', l_total;
    i := 0;
    FOREACH l_tank_id IN ARRAY l_tank_ids LOOP
        i := i + 1;
        IF EXISTS(SELECT 1 FROM wn8.exp WHERE tank_id = l_tank_id LIMIT 1) THEN
            CONTINUE;
        END IF;
        -- new tank
        SELECT
            COUNT(tank_id),
            SUM(battles)
        FROM wn8.data
        WHERE tank_id = l_tank_id
        INTO l_cnt, l_battles;
        l_ofs_lo := l_cnt / 100; -- 1%
        l_ofs_hi := l_cnt / 10000; -- 0.01%
        l_tag := tag FROM xvm.vehicles_list WHERE tank_id = l_tank_id;
        RAISE LOG '[WN8] tank_id=% cnt=% offset=%..% battles=% %', l_tank_id, l_cnt, l_ofs_lo, l_ofs_hi, l_battles, l_tag;
        IF l_cnt < 10000 THEN
            RAISE LOG '        skip: low players count';
            CONTINUE;
        END IF;
        WITH
            xx AS (
                SELECT
                    (dmg::float8 / battles) AS a_dmg,
                    (frg::float8 / battles) AS a_frg,
                    (spo::float8 / battles) AS a_spo,
                    (def::float8 / battles) AS a_def,
                    (100::float8 * wins / battles) AS a_win
                FROM wn8.data WHERE tank_id=l_tank_id),
            ld AS (SELECT a_dmg as v FROM xx ORDER BY a_dmg DESC LIMIT 1 OFFSET l_ofs_lo),
            lf AS (SELECT a_frg as v FROM xx ORDER BY a_frg DESC LIMIT 1 OFFSET l_ofs_lo),
            ls AS (SELECT a_spo as v FROM xx ORDER BY a_spo DESC LIMIT 1 OFFSET l_ofs_lo),
            le AS (SELECT a_def as v FROM xx ORDER BY a_def DESC LIMIT 1 OFFSET l_ofs_lo),
            lw AS (SELECT a_win as v FROM xx ORDER BY a_win DESC LIMIT 1 OFFSET l_ofs_lo),
            hd AS (SELECT a_dmg as v FROM xx ORDER BY a_dmg DESC LIMIT 1 OFFSET l_ofs_hi),
            hf AS (SELECT a_frg as v FROM xx ORDER BY a_frg DESC LIMIT 1 OFFSET l_ofs_hi),
            hs AS (SELECT a_spo as v FROM xx ORDER BY a_spo DESC LIMIT 1 OFFSET l_ofs_hi),
            he AS (SELECT a_def as v FROM xx ORDER BY a_def DESC LIMIT 1 OFFSET l_ofs_hi),
            hw AS (SELECT a_win as v FROM xx ORDER BY a_win DESC LIMIT 1 OFFSET l_ofs_hi),
            mv AS (
                SELECT
                    median(a_dmg) as m_dmg,
                    median(a_frg) as m_frg,
                    median(a_spo) as m_spo,
                    median(a_def) as m_def,
                    median(a_win) as m_win
                FROM xx, ld, lf, ls, le, lw, hd, hf, hs, he, hw
                WHERE xx.a_dmg >= ld.v AND xx.a_dmg <= hd.v
                   OR xx.a_frg >= lf.v AND xx.a_frg <= hf.v
                   OR xx.a_spo >= ls.v AND xx.a_spo <= hs.v
                   OR xx.a_def >= le.v AND xx.a_def <= he.v
                   OR xx.a_win >= lw.v AND xx.a_win <= hw.v)
        SELECT m_dmg, m_frg, m_spo, m_def, m_win FROM mv
        INTO e_dmg, e_frg, e_spo, e_def, e_win;
        RAISE LOG '[WN8] %/% e_dmg=% e_frg=% e_spo=% e_def=% e_win=% tank_id=% %',
            i, l_total, e_dmg::numeric(9,2), e_frg::numeric(9,2), e_spo::numeric(9,2), e_def::numeric(9,2), e_win::numeric(9,2), l_tank_id, l_tag;
        INSERT INTO wn8.exp(partkey, tank_id, battles, dmg, frg, spo, def, win)
        VALUES ('epoch'::date, l_tank_id, l_battles, e_dmg, e_frg, e_spo, e_def, e_win);
    END LOOP;
END;
$$;

 

It gives rough result which then refined during calculation.

 

Share this post


Link to post
Share on other sites

@sirmax: welcome to wotlabs

I'm not able to read SQL so wanted to check a few things that are key to the expected values calculation. First you apply the 1000 battles per account filter, then the 50 battles per tank filter. Then using the last set of expected values, you calculate the rSTATs and then the tank WN8 for every account's tanks, then the weighted average rSTATs and WN8 for each account, based on the tanks they have left in the dataset.

Then - most important - you find the median tank by WN8 for every account, and filter out all the tanks in every account below their median. THIS CUTS THE DATASET IN HALF but leaves you with data on tanks that each account has played well. The 'worst half' of every accounts' tanks are GONE and not used. If you dont do this step - its not WN8. Looking at your script, I don't know if you are doing this or not.

The new expected values are then calculated by plotting, for every tank, each rSTAT vs the same rSTAT at the account level. Every dot on the graph of rDAMAGE vs user_rDAMAGE represents how one user scored in that tank vs how that user scored at the account level. The line of best fit should go through (1,1) if the expected values are correct. The script needs to recalculate the expected value so that the line of best fit WOULD go through (1,1) - and that is the new expected value for that rSTAT. Repeat for all rSTATS and all tanks. Note: it looks from your plots like you are doing this with rSTATc values not rSTAT - don't use the rSTATc for this, as the rSTATc values can be altered by other rSTATS (see the wiki) - keep it clean.

The problems you will find that missing tanks in the expected values have to be estimated before you can start. I suggest you use the v31 (averaged values) table as they will never have any missing tanks. Each time you run your script, you will be calculating per tank values using the v31 averages as a baseline. That should work well.

Finally, using just the data left after the filters described above, plot WN8 vs rWINc for all accounts. If you have done it right, you should have a plot where the line of best fit (red) goes through (1565,1.0) and the Adjusted R squared is about 0.88 (which is what it is with the averaged expected values graph below). A lower R Squared means the method isn't good.

With all our attempts to improve WN8, we couldn't find one that gave an Adjusted R squared of more than 0.89, which is why we went to averaged expected values: the huge amount of extra work required isn't worth it. Good luck.

 

 

user_WN8 vs user_rWINc _v30.jpg

Share this post


Link to post
Share on other sites

  • Recently Browsing   0 members

    No registered users viewing this page.