Android25

Setting PR to a scale

106 posts in this topic

PR could be padded hard in Team battles a few patches ago. Not sure if it still does but I know 

1. A shitter account i played got boosted 100 PR in one day

2. Mpro

Share this post


Link to post
Share on other sites

Mind explaining why PR unicum @ NA isn't at ~9600?

The chart was initially scaled to match the curve of the population percentile, then I adjusted the position of the curve (not the shape of the curve) to match the current percentage (ie. number, not percentile) of unicums on the NA server. The intent was to keep the same number of unicums per rating on the NA server. The definition of a unicum is supposed to be the top 0.01%, however, it's not as easy as that, because each server scales differently. In theory, there should be the same percentage of players in every bracket across every server. However, in practice, there is noticeably a higher saturation of unicums on the NA server than there are on the RU server (and the only reason that the RU server is as high as it is is because we cut a ton of low WN accounts out of the equation). I personally haven't gotten to play with the EU data, so I'm not sure how it compares.

 

However, if you ask why Unicum PR isn't at 9600, you may as well ask the other question presented by the data, 'why don't we raise the Unicum threshold on the NA server to 2850'? Which is why Nixon doesn't like percentile data, and I can agree with him for the most part on that point, because regardless of what the data tells you, percentile data is unable to account for skill in almost any way. Ok, we have a higher percentage of unicums on the NA server, does that mean we should raise the bar? No, because 2450 is a generally accepted point where people can clearly see that you play at a level that is associated with "Unicum". And no percentile data can argue with that association.

Share this post


Link to post
Share on other sites

It's placed to match the WN8 unicum mark by population percentage. The current WN8 unicum mark is nowhere near 99.9% for NA players with 1k+ battles: The previous mark was derived from something like RU players with 2k+ battles and an arbitrary filter at 146 WN8.

This adjustment hasn't been applied to the other boundaries, so the chart is internally inconsistent. Hence the blunicum range is only 500 points.

I'm having trouble understanding why someone would get server-wide stats and still use outdated boundaries.

Share this post


Link to post
Share on other sites

I'm having trouble understanding why someone would get server-wide stats and still use outdated boundaries.

The NA server isn't generally accepted as the server to update boundaries with, and nobody currently seems interested in updating the boundaries. In fact, I'm not even sure how the current color scale (the one of the front page) came about... I just know it was mostly based off the 26 pages of collaboration using data from the RU server. But that's a whole different story about trying not to hurt bad players feelings.

Anyway, the point of the project wasn't to update the WN8 bounds. I've already spent my time at the center of that can of worms, and I don't plan to go there again with WN8.

Share this post


Link to post
Share on other sites

The chart was initially scaled to match the curve of the population percentile, then I adjusted the position of the curve (not the shape of the curve) to match the current percentage (ie. number, not percentile) of unicums on the NA server. The intent was to keep the same number of unicums per rating on the NA server.

>> Ok. The data in the tables you posted in this and other thread is what confused me.

The definition of a unicum is supposed to be the top 0.01%, however, it's not as easy as that, because each server scales differently. In theory, there should be the same percentage of players in every bracket across every server.

>> That seems to be the case.

However, in practice, there is noticeably a higher saturation of unicums on the NA server than there are on the RU server (and the only reason that the RU server is as high as it is is because we cut a ton of low WN accounts out of the equation). I personally haven't gotten to play with the EU data, so I'm not sure how it compares.

>> Who is we and when? As far as i recall seriych does his xvm calculations on the entire RU with >1k battles (link).

However, if you ask why Unicum PR isn't at 9600, you may as well ask the other question presented by the data, 'why don't we raise the Unicum threshold on the NA server to 2850'? Which is why Nixon doesn't like percentile data, and I can agree with him for the most part on that point, because regardless of what the data tells you, percentile data is unable to account for skill in almost any way. Ok, we have a higher percentage of unicums on the NA server, does that mean we should raise the bar? No, because 2450 is a generally accepted point where people can clearly see that you play at a level that is associated with "Unicum". And no percentile data can argue with that association.

>>> Wouldn't 2.8k WN8 bring it over 10k PR? Not raise but put it where it belongs, but that's another story, I guess, considering with the multitude of opinions in the last thread on colour scale.

I've already spent my time at the center of that can of worms, and I don't plan to go there again with WN8.

I remember. It was quite a nightmare.

Share this post


Link to post
Share on other sites

This thread is pure gold. Especially Kuro going apeshit ...:happy:  What his arguments all boil down to: "PR sucks because it doesn't reward my sealclubbing with unicum stats ...":QQ:

 

And those are exactly the reasons why I like PR:

  • It shits on pro-sealclubbers.
  • It doesn't reward rerolls.

Seems like an honest rating to me. :P

 

P.s.:

Because the damage components are tiny at tier 2 regardless of your performance, competent sealclubbing is under-rewarded compared to incompetent sealclubbing. This is probably intentional.

"Competent sealclubbing" is an oxymoron. There is no true competence required if you use your pimped out tanks with multi-skill crews to beat the shit out of newbies lacking even the most basic knowledge of the game.

 

P.p.s.: Just check Kuroialty's performance in higher tier tanks and you will immediatley see that PR rates him in exactly the way he deserves to be rated:

  • T30: 1815 DPG, 55.38% WR
  • T28 Proto: 1767 DPG, 51.97% WR
  • FCM: 1614 DPG, 58.34% WR
  • T25/5: 1061 DPG, 55.38% WR

Do those stats say unicum to you?

Share this post


Link to post
Share on other sites

PR could be padded hard in Team battles a few patches ago. Not sure if it still does but I know 

1. A shitter account i played got boosted 100 PR in one day

2. Mpro

Yeah, definitely something odd with Mpro. On random + CC I get an absolute maximum calculated PR (= no premium at all) of ~11400, and the team battles shouldn't help because they have much lower average damage. It's possible that it's a dumb bug, like adding up the team damage but not including the team battles in the divisor.

Of course, because it's WG, there's no way of telling them about dumb bugs.

Share this post


Link to post
Share on other sites

1. That seems to be the case.

 

2. Who is we and when? As far as i recall seriych does his xvm calculations on the entire RU with >1k battles (link).

 

3. Wouldn't 2.8k WN8 bring it over 10k PR? Not raise but put it where it belongs, but that's another story, I guess, considering with the multitude of opinions in the last thread on colour scale.

 

1. It's not the case. When I ran RU account data based on xvm provided Json files for all players with 1000+ battles, the 99.9% point came out around 2650, whereas they come out to 2867 on NA. Its not a huge percentage difference, but because all unicum numbers fall within the top 0.1%, it ends up being a rather large difference.

2. "we" refers mostly to myself, as I was doing most of the math for updating the charts, but because I was trying to keep multiple people happy with the output, and taking a lot of constructive feedback, it was a collective effort between myself, Crab, bjshnog, gryphon... hell, even pauli contributed a lot to the numbers part of the discussion. "we" adjusted the percentiles a lot like the way expected tank stats are adjusted in an attempt to take out the noise from both the top and bottom end (you want to remove reroll accounts just as much as you want to remove bots because WN8 is an overall statistic). So "I" cut off the top end (top 2%) and the bottom end (bottom 50%) and then redrew them based on a fit to the data that was left. The zero percentile intersected WN8 at 169, and was assumed to be the point mathematically where you aren't contributing at all. It was defined as the "bot baseline" something crab was intent in utilizing, and so all accounts with a WN8 at or lower than this were cut. A new 50% to 98% data group was taken based on the percentiles without any accounts with a WN8 lower than 170. (it wasn't really much different than the group with it, it just pushed all the numbers up by less than 50 points).

The post that put the cutoff at 2450 is 3 from the bottom http://forum.wotlabs.net/index.php?/topic/10758-official-wn8-colour-categories-update/&page=16

It likely won't be changed for 2 reasons. 1. nobody wants to see it changed. 2. it uses the 50% to 98% group (arguably the most stable group on the server) to fit the top 2% and bottom 50% (arguably the groups most prone to changes due to noise).

3. 2.8k WN8 is the 99.9% point on the NA server (unicum). The 99.9% point on PR is 9865. So no, 3450ish brings PR over 10k. Even on the adjusted data fit line I posted in the OP, PR doesn't hit 10K until 3052 WN8.

Share this post


Link to post
Share on other sites

 

3. 2.8k WN8 is the 99.9% point on the NA server (unicum). The 99.9% point on PR is 9865. So no, 3450ish brings PR over 10k. Even on the adjusted data fit line I posted in the OP, PR doesn't hit 10K until 3052 WN8.

Now you've got me curious about the EU points. :P

Share this post


Link to post
Share on other sites

Now you've got me curious about the EU points. :P

I should have the full EU database by tomorrow.

A fun thing to do would be to graph the distributions against each other with some basic scaling. I suspect you'll see the effects of RU having a far higher bot count.

 

Share this post


Link to post
Share on other sites

why don't we raise the Unicum threshold on the NA server to 2850'?

Stop raising minimum unicum WN8...

Share this post


Link to post
Share on other sites

Stop raising minimum unicum WN8...

If more and more ppl reach a certain threshold its simply needed to keep it at a 1% ratio. Else we all gonna be unicums soon, which doesn't make sense in the original way of being an unicum :gaben: 

Share this post


Link to post
Share on other sites

 

Stop raising minimum unicum WN8...

I was using that as an example against raising unicum expected PR. Like I said in my last post in this thread, it's very unlikely that the WN8 unicum bound will ever be changed again. We'll just move to a new statistic.

Share this post


Link to post
Share on other sites

Here's the WN8 to PR chart based on average PR per WN8 point. The data is very stable up until around 1800, where it becomes incredibly noisy. There are a lot of points in the High WN8/Low PR range where their WN8 unicum status is questionable, like low battle count (under 5000) and very low tier (seal clubbing). However, some accounts seem to be genuinely unicums but just have a lower than normal PR.

8nDpAlW.png

I might run the numbers again capped at 5k battles instead of 1k, but it takes several hours to process the data and I enjoy using my computer for other things as well.

Share this post


Link to post
Share on other sites
Quoting myself from a topic in Tech Support because nobody who was looking for anything remotely similar would ever find it there.

It's actually a little more complicated than that. WG PR gives you something of a base score if you will.

The base score is made up of Win Rate, Survival Rate, Damage, XP per battle without prem since 8.8 patch, Average spotting since 8.8 patch, Average track damage since 8.8 patch.

Once you have the base score, the formula looks like this, where x is total number of battles

(540*x^0.37) * tanh(0.00163*x^-0.37(Base Score))

The above formula can be graphed in google, provided you have a base score. The base score will always be higher than your actual PR as the formula asymptotes at 88.02% of the base score.

If you play around with the graph and the base score a little, you can see that the higher the base, the more battles it takes to reach X% of the top end.

So, the lower your score is, the fewer battles you have to play to hit 90% of your top end.

At the lowest Base score on the EU server (111), it takes less than 1 game for your PR to hit 90% of the top end, and only a little more than 1 game to post 99%. The highest number would be reached on the 10th game.

At a low end unicum player level, 9100 PR after 10000 games (11671 Base). Without improving skill wise, the player would move up 100 points in about 1500 games. However, going up 500 points would take 13,150 games.

The chances that you don't improve while playing at that level in 13k games, is incredibly slim. However, as the players base goes up, so do the number of games to raise the PR number x percentage of the base, and the more games you play, the less effective playing a game is on your PR.

Overall, PR, even while using battles, is still a great 'overall account' metric (mostly because WG uses numbers we simply don't have access to), which is what you need in XVM since there is no recent.

I wouldn't say PR takes number of battles "very heavily" into account. Except at the clearly padding levels of play, number of battles makes up no more than 10% of your number as long as you have over about 8k games. Which is the percentage WR is supposed to account for in the WN metric.

 

Share this post


Link to post
Share on other sites

If you play around with the graph and the base score a little, you can see that the higher the base, the more battles it takes to reach X% of the top end.

In case people don't want to make their own graphs, here's one I made (much) earlier:

AbfcrGD.png

I'd probably have chopped that at 12k if I'd checked where the unicum point was.

Share this post


Link to post
Share on other sites

Here's the same graph but with a few more points. The max Base PR on the EU server is in the 16k range while the max PR is just below 12k

The unicum range based on population is at a Final PR of around 9100

wyftzw0uc2.png

The lines, from top to bottom are # of battles:

100,000 (Purple)

80,000 (Green)

40,000 (Blue)

20,000 (Red)

10,000 (Orange)

5,000 (Blue)

2,000 (Black)

1,000 (Green)

500 (Purple)

200 (Orange)

100 (Red)

The equation pretty much goes 'If you're bad, it doesn't matter how many games you've played, you're bad. If you're good, you have to be good for a lot of battles to prove you're good, but once you've played a lot of battles, it matters less and less'

If a very good player played on a bad account with less than about 5k battles, he could raise the PR dramatically because the limit would be on skill not battles, but if a very good player played on a very good account with less than 5k battles, it would move more so because of battles than because of improvement of skill, but would still go up a lot slower than in the first scenario.

 

Share this post


Link to post
Share on other sites

EU Server [All players with Battles  1000]

WN8 against base PR. Each point is representative of the average Base PR for players with WN8 of that point ±25

jpbq8sepyw.png

This would put the Unicum PR intercept at 9000 for 20,000 battles.

EU Server [All players with Battles 5000]

WN8 against PR. Each point is representative of the average PR for players with WN8 of that point ±25

c0e2gooqli.png

 

Both graphs show 9000 as a reasonable point for Unicum PR

Share this post


Link to post
Share on other sites

What about graphing the maximum difference between PR and WN8, possibly with a 5k or 10k battle cutoff to exclude the rerolls.  Are there any high WN8 or high PR players that are rated noticeably different by the other metric (or bad players rated differently)?

Share this post


Link to post
Share on other sites

I might run the numbers again capped at 5k battles instead of 1k, but it takes several hours to process the data and I enjoy using my computer for other things as well.

Let it run at night (or during the day if you sleep then) when you are sleeping?

Share this post


Link to post
Share on other sites

Let it run at night (or during the day if you sleep then) when you are sleeping?

It's a lot faster when I run it like I did above. It only takes 120 samples, rather than 3000, but gives pretty much the exact same data. There isn't really any reason to graph average anything per point, a 50 point spread with 2 overlaps per point is fine.

It grabs average PR for WN8 of 0-49, 25-74, 50-99, etc. And only takes about 10 min.

 

What about graphing the maximum difference between PR and WN8, possibly with a 5k or 10k battle cutoff to exclude the rerolls.  Are there any high WN8 or high PR players that are rated noticeably different by the other metric (or bad players rated differently)?

I could check when I get up tonight. That would be a little more difficult to write a macro to determine but I can already think of a few ways to do it. Thank god for While and For statements.

There are obviously several descrepencies; the last per point wn8 to PR scatter plot I posted shows that, but most of the biggest offenders come from low battle counts and seal clubbers (who can post superuni WN8 but fail to hit 8k PR) and I'm fine with weeding both out.

Share this post


Link to post
Share on other sites

EU Server [All players with Battles  5000]

CDCZwRx.png

Data Including account ID's of highest/lowest differences:

https://docs.google.com/spreadsheets/d/1GmQZjrAS9MRKq2vVnlZUCUYwZZ50Ia07uy90a2s8lHk/edit?usp=sharing

 

The highest deviation, that point on the 1200 line is Hami... who plays 75% of his battles in tank companies...

The other large deviation, on the 2800 line, is Tomylw, but I'm not entirely sure why he's such an outlier.

The biggest deviation, on the 4200 line, is Mustiala, I'm not sure why the account is rated so low.

 

Edit:

Same data averaged over each 50 points of data to reduce noise. "Average" is the average across each set of 50 points but only the highest and lowest number in each 50 point set were used.

dz6i6it.png

 

Edit 2:

Here's the top graph but with my OP proposed lines. It looks like Unicum and Super Unicum fall nicely around the average line, but the cutoffs below those could be changed around a bit.

c1hmPi4.png

 

Share this post


Link to post
Share on other sites

Thanks Android.

 

I don't know how to represent the data, but it would be good to know how the distribution is for a given point from lowest to highest difference.  Maybe the standard deviation?

 

Is it a really sharp peak near the center (ie mostly accurate with some outliers) or is it a flat bell curve (ie a wide spread) for a given WN8?  Does that distribution change at higher or lower ends of the scale?

 

I'm not sure if any of this is meaningful either, just trying to think of ways to assess where either WN8 or PR breaks down in the face of more extreme gameplay.

Share this post


Link to post
Share on other sites

The other large deviation, on the 2800 line, is Tomylw, but I'm not entirely sure why he's such an outlier.

The biggest deviation, on the 4200 line, is Mustiala, I'm not sure why the account is rated so low.

Mustiala is a dormant account since 8.8 and so has zero assisted damage. You need to filter by last battle played for WG-PR. For reference, he does demonstrate that the 8.10 WG-PR formula is exactly correct, at least up to the post-8.8 section.

Tomylw appears to be glitched on WG-PR (should be in the 10-11k range). Maybe something to do with the high clan battles count.

Share this post


Link to post
Share on other sites

Ahh, much cleaner.

With this chart I removed all accounts who had not played a battle since before Dec. 25, 2013. Which was during the 8.9 patch, so everybody has at least a few points in the 8.8 part of the equation. I also removed all points on the graph where there were only between 1 and 10 (inclusive) accounts counted, to remove noise due to lack of data.

TRZ3hPT.png

Here's the same chart but with the last battle req set to the first day of this year (2015) instead.

r3AoHZt.png

Share this post


Link to post
Share on other sites

  • Recently Browsing   0 members

    No registered users viewing this page.