Jump to content
Sign in to follow this  
erickHV

The future of rating systems after the Great Rebalance

Recommended Posts

Hi,

I just wanted to see what brighter minds than mine make of the future of any current or planned ratings system when the Great Rebalance hits. Recently there was the infamous Sandbox Server leak:

https://thearmoredpatrol.com/2016/06/07/some-stats-of-rebalanced-tanks-on-sandbox/
https://thearmoredpatrol.com/2016/06/08/sandbox-leak/

Many individual tanks and whole classes are rebalanced in a way that makes the infamous arty nerf of 8.6 that screwed up arty WN8 seem like a child's play. It is all naturally subject to change, etc., but nevertheless we can be fairly certain that the changes will be extensive and involve all vehicle classes.

1. Is it even possible to use any sort of overall/historical rating after this rebalance?
2. Do we need to "restart" rating systems such as WN9 to only count results starting from from the rebalance patch?
3. Is it possible or practical to devise a rating system that assigns values on a per patch basis, "locking in" results for each patch separately and comparing those to the average values of the playerbase in a string of per-patch WNx values?
 

 

Share this post


Link to post
Share on other sites

I didn't even follow the specifics, however I thought WG was pretty damn clear on the matter. So I'll try and rephrase it:

Stop worrying about the shit they do on the Sandbox server. Stop worrying about leaked info. Stop worrying oh my God!!!

This is not a supertest server. The "changes" you see there are not even close to "not final", they're pure fantasy and are meant to test specific things out.

This seems like an actually very good approach from them and if they do the work properly, it might be the tool that allows them to really understand why some mechanics / maps are flawed (and hopefully fix them).

 

edit: After re-reading, I think I may have misinterpreted what you meant. AFAIK, both WN8 and WN9 will adapt to mostly any type of balance change because all expected values are updated and since changes are the same for all players, a "global increase" or "global decrease" in WNx would be of a similar level proportionally for everybody. The weight of some specific factors may have to be adjusted though, for example to account for changing balance between LTs and other vehicles or SPGs and other vehicles.

Edited by NThirtyTwo

Share this post


Link to post
Share on other sites

I didn't read this as OMGWHATSHALLWEDO?-thread.

It's legitimate to wonder in the light of a rebalance that will be there (but probably very different from the sandbox), will make it difficult for ratings to correctly cover both current as well as past play.

His three questions are basically very valid outside of the sandbox (leaks).

Share this post


Link to post
Share on other sites
3 minutes ago, Private_Miros said:

I didn't read this as OMGWHATSHALLWEDO?-thread.

It's legitimate to wonder in the light of a rebalance that will be there (but probably very different from the sandbox), will make it difficult for ratings to correctly cover both current as well as past play.

His three questions are basically very valid outside of the sandbox (leaks).

Yes, I agree and edited my post immediately after re-reading it. I don't think this will require specially important changes in the rating, however I am far from the most qualified person to give accurate, specific answers.

Share this post


Link to post
Share on other sites
3 minutes ago, Private_Miros said:

I didn't read this as OMGWHATSHALLWEDO?-thread.

It's legitimate to wonder in the light of a rebalance that will be there (but probably very different from the sandbox), will make it difficult for ratings to correctly cover both current as well as past play.

His three questions are basically very valid outside of the sandbox (leaks).

Doesn't WN9 take up WN8 problem with recent/overall by cutting off stats to certain point? Is it possible that after a number of games played you'll enter a zone where your stats pre-rebalance won't be as significant?

Share this post


Link to post
Share on other sites
1 minute ago, no_name_cro said:

Doesn't WN9 take up WN8 problem with recent/overall by cutting off stats to certain point? Is it possible that after a number of games played you'll enter a zone where your stats pre-rebalance won't be as significant?

AFAIK, only the "worst" is going to be cut off. Therefore, if the re-balance makes WN9 go up for some players, then I suppose technically, yes, at some point, their older "lower" stats will be cut off, and account WN9 will mostly / entirely reflect post-re-balance reality.

However, if a different player finds himself with lower stats than before following the re-balance, then it is going to be the newer stats that are going to be cut off, and his account WN9 will mostly show how he was prior to the re-balance.

Share this post


Link to post
Share on other sites

It all depends on how WG will handle their book keeping when the rebalance hits e.g mix old and new stats, start from scratch. This will dictate how stat tracking sites and the guys who create(d) ratings can respond. At this point there is nothing else to say.
 

And knowing WGs history with book keeping, I 've a bad feeling about this. Same goes for this dice rolling contest they call sandbox.

Share this post


Link to post
Share on other sites
2 minutes ago, Folterknecht said:

It all depends on how WG will handle their book keeping when the rebalance hits e.g mix old and new stats, start from scratch. This will dictate how stat tracking sites and the guys who create(d) ratings can respond. At this point there is nothing else to say.
 

Padding stats in tank companies never forget. :D

Share this post


Link to post
Share on other sites
1 hour ago, erickHV said:

Many individual tanks and whole classes are rebalanced in a way that makes the infamous arty nerf of 8.6 that screwed up arty WN8 seem like a child's play. It is all naturally subject to change, etc., but nevertheless we can be fairly certain that the changes will be extensive and involve all vehicle classes.

I'd be surprised. The 8.6 arty nerf was statistically spectacular, with some SPGs having their capability cut by more than 50%. The only way they could approach that for other classes without blatantly wrecking the game would be to change all the HP values.

Recent WN9 is based on recent data, so it'll work fine after 10 weeks unless WG actually break the API. Account WN9 assumes that tanks were played at their strongest historical state, so it would still work to an extent, although there would be an argument for cutting the battle percentage used, and eventually there may be little point in keeping it if enough tanks are shuffled around.

Share this post


Link to post
Share on other sites
2 minutes ago, SoliDeoGloria said:

Look at those turret traverse nerfs. It'll be like every tank is a hellcat, and has to spin its hull to point its gun anywhere.

That was the part that jumped out at me the most. Snapshots are going to take a serious hit if turret traverse drops by that much (as is circling/brawling).

Share this post


Link to post
Share on other sites
36 minutes ago, SoliDeoGloria said:

Look at those turret traverse nerfs. It'll be like every tank is a hellcat, and has to spin its hull to point its gun anywhere.

I get that it is the sand box, but what is the point of nerfing turret traverse so hard?

Share this post


Link to post
Share on other sites

Interesting to look at these changes.  While they're certainly not what we'll see in the game one day, they do represent some of WG's lines of thinking.  Interesting to see them playing with an idea that had been bounced around for years:  Lower damage to offset the higher pen on gold ammo.  Logical conclusion would be the elimination of gold rounds, allowing both rounds to have the same cost, and rewarding proper use.  Also interesting to see what must be severely inflated values for Russian DPM and German accuracy.  I wonder if we might see a shift away from corridor meta, if they combine these changes with the proposed vision rebalancing?  Might be a new golden age for LTs.  

But who knows what these changes will look like, if they ever even reach the live servers...

 

 

As far as the OP's question goes... I'm curious as well.  There have been some jokes around here about my overall going purple because I quit, and let the ratings catch up with me.  I wonder if more of the same could happen.

Share this post


Link to post
Share on other sites
6 hours ago, Korb3n_Dallas said:

I get that it is the sand box, but what is the point of nerfing turret traverse so hard?

My guess is to make hull traverse more meaningful, and also introduces the idea of combining hull+turret traverse when you absolutely need to get gun on target in a timely fashion, even at the expense of screwing up your armor angling.

The nerf to turret traverse will also reward players who actively seek to flank and situational awareness becomes absolutely valuable since you have to pre-empt your gun facing.

Share this post


Link to post
Share on other sites
19 minutes ago, neokai said:

My guess is to make hull traverse more meaningful, and also introduces the idea of combining hull+turret traverse when you absolutely need to get gun on target in a timely fashion, even at the expense of screwing up your armor angling.

The nerf to turret traverse will also reward players who actively seek to flank and situational awareness becomes absolutely valuable since you have to pre-empt your gun facing.

Eh, just seems like it would be painful and not fun to play like that. I mean, I know they aren't going to settle on those numbers and they are just playing around (at least I hope so).

Share this post


Link to post
Share on other sites

But is it possible to create a rating system that is completely impervious to various rebalances? Meaning it would assign players within a given percentile or rank or rating value based on their performance compared to their peers in a given patch, and then carry it forward "as-is"?


EXAMPLE: A hypothetical player playing Leopard 1 (I'm not so good at math so I leave out weighing by battles played, for the sake of simplicity)

Current system:

9.14: WNx 3000 (player picks up Leopard 1 for 100 games and no longer plays it)
overall: 3000
9.15: WNx 3100 (Leo 1 gets slightly nerfed so player gets a +100 boost to WNx compared to the suckers playing it post-nerf)
overall: 3100
9.16: WNx 3250 (Leo 1 gets nerfed again, this time a bit harder, so another +150 boost to WNx compared to the suckers who play it in nerfed state)
overall: 3250

...
10.0: WNx 2250 (Leo 1 gets buffed so hard it can now kick Chuck Norris's ass, but the player cannot play the game for some months due to personal matters. All suckers who played it before this epic buff get their WNx score nerfed to hell, -1000 WNx in this case)
overall: 2250

A few months forward, nobody believes that he once played like a superunicum in this tank as his WNx rating no longer reflects that.

 

Proposed system:

 

9.14: WNx 3000 (player picks up Leopard 1, and plays during each following patch)
overall: 3000
9.15: WNx 3250 (Leo 1 receives slight nerf which affects average player's results more than his due to his unique playstyle, making him perform better)
overall: (3000+3250)/2 = 3125
9.16: WNx 2900 (Leo 1 receives another nerf, which this time makes is more difficult to play for this player than others)
overall: (3125+2900)/2= 3012.5

Player drops Leopard 1. The 10.0 patch sees it hugely buffed the playerbase can now greatly inflate the expected results. For some reason, the player is unable or doesn't want to play the tank.

10.0: WNx 0
overall: (3012.5 + NA)=3012.5

Player's overall rating results still show that he has historically played Leo 1 at a very high level, and he retains his superunicum ranking.

Share this post


Link to post
Share on other sites

Not really, because WG API doesn't tell you when the battles were played. So there is no way to tell whether someone's T-10 stats are from the old IS-4 at tier 9, the tier 9 IS-8 or the current buffed T-10. Same for M4 in its earliest HEspam incarnation, the HEATspam meta or the current nerfed setting. Same for arty.

Now you could conceivably make such a system going forward but you'd have to discard old stats, and keep snapshots of everyone's stats at every patch. That's a lot of space and processing power...

tl;dr WG's bookkeeping sucks

Share this post


Link to post
Share on other sites

Technically, how hard would it be to take a snapshot of all tracked player's stats at the end of the patch, analyze them and only store the final values of per tank WNx for that patch?

Let's say roughly 500 tanks, and a single WNx value per each, stored in plaintext. I tried it out and it should take no more than 8KB per player, making the total file something like only 9GB for all 11M+ tracked players on the EU server. Since only updating the file is required at the end of each patch, it should be quite doable. For all players in the world, assuming there's about 70M active accounts maybe, it would be a <65GB file. This is rather small by today's standards. For updating, it would help if the server had modern, fast flash storage with nice IO. Intel 750 series NVMe SSDs can do 500MB/s+ speeds when randomly writing 4K files, in theory, if not bottlenecked by anything else, you could update that whole 65GB file in 2 minutes and 10 seconds.

 

EDIT: this is not directly relevant, but just to give you a perspective - here are the logging stats for the industry-leading software company I work for:

 

  • In the average busy site about a 100 million games are played per day.
  • Every game generates around 250 lines of log on average because of details needed to capture.
  • This means more than a million lines of log per second.
  • This translates to about 7TB of plain text log per day.
  • The daily log is zipped to 700GB for permanent storage.
  • About 20% of application server CPU is consumed by logging.
  • About 50% of the storage cost is related to logging, which is about 10% of the whole site’s cost in the current solution.

Share this post


Link to post
Share on other sites

There are methods to deal with this without keeping extensive logs and saving huge amounts of computing power; similar to what WG does to calculate marks of excellence.

Basically, instead of calculating the values for the entire account every single time, you log a baseline and calculate on the fly

Taking your Leo example:

- The player has 3000 WN8 in his Leo 1. It is buffed / nerfed whatever.

- This is stored in the log: "Player X", "Leo 1 ID", "3000 WN8", "2800 DPG", "1.5 KPG", ETC., "850 games"

- Buffs happen, nerfs happen, shit happens, but the player does NOT play the Leo 1 in that time frame.

- When a user stats refresh is requested, instead of calculating everything, the system looks at each tank to see if new games were played.

- It sees the user has played 850 games in the Leo 1. It looks in the log entry and sees the previous entry was also 850 games. The WN8 is not updated.

 

- Now suppose the player now takes his Leo 1 for a spin. The tank has been nerfed into the ground, so it's terrible. He plays 50 games with it and gets 2000 WN8 in that session.

- If he updates his stats, the system looks again: Current games played: 900, logged entry: 850.

- It will calculate the WN8 of the new 50 games played based on the difference with what was in the logged entry. It will find out that the WN8 for those 50 games is 2000.

- It will simply calculate (3000*850 + 2000*50) / 900 = 2944

- It will update the log with the new values: "PLayer X", "Leo 1 ID", "2944 WN8", "xyz DPG", "xyz KPG", ETC. "900 games"

And so on.

 

In other words, keep a baseline log of the stats that matter and calculate the current stats based on the differences between current vs. log and proportionally update the log as required.

It's possible this requires even less computing power than what is needed right now, and also, would only require tiny log space per user, since you don't log the entire history, but only a baseline.

 

Thoughts? From the stats folks?

Share this post


Link to post
Share on other sites

Keeping a pre-patch baseline is very doable, and if the nerds are substantial it would be a nice reference point.

 

Also not too difficult to have per-patch deltas if someone wanted to put the time in.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...