Jump to content
Sexy_Dave

What is a proper sample size for stats?

Recommended Posts

Just out of curiosity , another poster asked on another thread, and it has made me think about sample sizes again.

I am convinced (however stupid it may be) that some accounts are simply "luckier" than others, and I have one of the unlucky ones. (Because like anyone posting about this would say they were lucky...)

AS an example, while playing the T-54 across several accounts, I noticed I rarely hit for 300 damage, and (it seemed) barely ever for the 320 the gun is supposed to "average", while I constantly had shots in the 270's.

So I kept tracked for all my shots for just over 100 games iirc, until I had 2000 shots, discounted all fires/ammo racks/kill shots, and discovered I was in fact averaging 297 damage per shot.

Posted on the forums and was told that this was a statistically explainable, as 2K shots was far to small a sample size to use,

Resigned myself to the obvious fact that the game has decided to flag me for punishment, and thought no more of it , until today, when I read the other guys question and thought yeah, what do you math guys consider a significant sample size? I mean 100 games seems to be a popular number for things, but that's still hugely open to chance

Share this post


Link to post
Share on other sites

What was your methodology for pulling data on these 2000 shots?

Did you count ammo type (HE, obviously)?

Did you count kill shots (the damage output will be lower)?

If my understanding of RNG is correct, then over 2000 shots, your average should pretty much always be in the 318-322 range (calculated with a SD of 20, for simplicity).

For this reason, I am inclined to believe there was a mistake in the way you did it. If you want proper measurement, you need to only select shots done with AP / APCR / HEAT and do not count kill shots.

 

Share this post


Link to post
Share on other sites

Without seeing your data set its difficult to tell, but 2K would seem to be enough based on the fact that sigma for damage is capped and you can't have outrageous flyers.  The lower your SD, the smaller sample size you can make statistically valid.  If your data is in excel, I can run a 95% UCL this week some time and get you the results.

HTF did you do it, hand pulling from replays???  You're a fucking masochist man, the game is not punishing you - its obviously rewarding you.

 

Share this post


Link to post
Share on other sites
56 minutes ago, Archaic_One said:

Without seeing your data set its difficult to tell, but 2K would seem to be enough based on the fact that sigma for damage is capped and you can't have outrageous flyers.  The lower your SD, the smaller sample size you can make statistically valid.  If your data is in excel, I can run a 95% UCL this week some time and get you the results.

HTF did you do it, hand pulling from replays???  You're a fucking masochist man, the game is not punishing you - its obviously rewarding you.

 

It's cool- it was a while ago and I don't have the documents anymore.

All I did was screenshot the damage log at teh end of the game and discount any shot that was marked as fire damage/ammo rack/ or that killed a target. Then just dropped all the numbers into excel and ran it. (To be honest the GF actually did it, I'm useless with computers) I asked if anyone had a a mod that could export the data, but apparently it doesn't exist. (Which seems odd to me, but again->suck at computers)

31 minutes ago, engineered said:

@NThirtyTwo, he says "discounted all fires/ammo racks/kill shots"

@OP , if you think the account is being punished with bad RNG, why not let a Unicum play the account and see if the theory holds up?

LOL because I know it's a ridiculous assumption- it just seems that way.

~ Although to be fair, in my Ferdinand (newest tank) I have yet to break 550 damage in a shot, yet have had multiple 370ish hits. So maybe...

Thing is I played a game called Asheron's Call ages ago when they had something called the Wi Flag, something that we used to complain about and were always told by numbers guys that "that's not the way computers worked". Apparently it assigned a number to your character when you created that character, and this resulted in mobs directing their attacks at you, rather than randomly like everyone assumed they were.

Totally different systems of course, and I've never heard of another game doing it. I'd  be staggered if it happened here, just one of those things-

I've *never* had a game where any tank rolls higher than average damage for the entire game (any game in which I shoot more than a couple shots anyway) whereas I've had dozens where the shots all do below average damage.

 

Still- back on topic- Thanks! I just wasn't sure if 2K was enough, I know that the more the better (obviously) not really important, just if I ever decided to go crazy and do something like this again, wanted to know.

Share this post


Link to post
Share on other sites
38 minutes ago, engineered said:

@NThirtyTwo, he says "discounted all fires/ammo racks/kill shots"

@OP , if you think the account is being punished with bad RNG, why not let a Unicum play the account and see if the theory holds up?

My mistake, I had overlooked that part in his original post.

My point still stands though, I am quite sure that statistically speaking, 2000 shots is a sufficiently high number that his result (297 damage per shot) is best explained by an error in the data gathering methodology.

8 minutes ago, Sexy_Dave said:

It's cool- it was a while ago and I don't have the documents anymore.

All I did was screenshot the damage log at teh end of the game and discount any shot that was marked as fire damage/ammo rack/ or that killed a target. Then just dropped all the numbers into excel and ran it. (To be honest the GF actually did it, I'm useless with computers) I asked if anyone had a a mod that could export the data, but apparently it doesn't exist. (Which seems odd to me, but again->suck at computers)

When you say damage logs, do you mean logs produced by an in-game mod (xvm or non-xvm type)? Or did you get it from a replay analysis type of software? In any case, it's possible you are missing some things like damage done to invisible targets, or done by ramming, or other things I'm not thinking of right now.

Share this post


Link to post
Share on other sites

The damage log I was using simply out down each hit I did and whether that hit resulted in a fire, ammo rack, or kill shot. All three types I discounted.

I just took screenshots of the damage log at the end of the fight (or replay if I missed it).  

I'm ashamed to admit I never used HE because ..well I was stupid "That E75 has 2 HP's left, why can't I pen it, Why do my cap reset shots always bounce??Stupid RNG" 

So the issues (that I see) with my methodology are that it tracked damage done per tank, so if I happened to hit a tank for 350,350,350 and then killed it- those shots were all not counted. 

My idea (again sucks at math and computers) was that even discounting damage done to tanks I ended up destroying, the result should still be fairly accurate. 

 

Share this post


Link to post
Share on other sites
1 hour ago, Sexy_Dave said:

The damage log I was using simply out down each hit I did and whether that hit resulted in a fire, ammo rack, or kill shot. All three types I discounted.

I just took screenshots of the damage log at the end of the fight (or replay if I missed it).  

I'm ashamed to admit I never used HE because ..well I was stupid "That E75 has 2 HP's left, why can't I pen it, Why do my cap reset shots always bounce??Stupid RNG" 

So the issues (that I see) with my methodology are that it tracked damage done per tank, so if I happened to hit a tank for 350,350,350 and then killed it- those shots were all not counted. 

My idea (again sucks at math and computers) was that even discounting damage done to tanks I ended up destroying, the result should still be fairly accurate. 

 

Sounds like your slave (gf) could have took also wrong numbers, fire ticks, kills etc? Or she wants you stop playing because game is rigged...

Share this post


Link to post
Share on other sites
5 hours ago, Sexy_Dave said:

The damage log I was using simply out down each hit I did and whether that hit resulted in a fire, ammo rack, or kill shot. All three types I discounted.

The usual problem is that you also need to discount potential kill shots. Consider what happens if a target has 320hp remaining and you hit him with a 320 alpha shell. On a high roll, you're discarding the result, but on a low roll you're counting it towards the average.

 

6 hours ago, Sexy_Dave said:

So the issues (that I see) with my methodology are that it tracked damage done per tank, so if I happened to hit a tank for 350,350,350 and then killed it- those shots were all not counted.

Similar problem. If you low-roll a tank, then you're less likely to get the kill.

There's no worthwhile way to use battle results for this test. Replay-staring works if you discount potential kill shots, but it obviously takes a while.

Share this post


Link to post
Share on other sites
22 hours ago, RichardNixon said:

The usual problem is that you also need to discount potential kill shots. Consider what happens if a target has 320hp remaining and you hit him with a 320 alpha shell. On a high roll, you're discarding the result, but on a low roll you're counting it towards the average.

 

Similar problem. If you low-roll a tank, then you're less likely to get the kill.

There's no worthwhile way to use battle results for this test. Replay-staring works if you discount potential kill shots, but it obviously takes a while.

Yes, but for tracking average damage per shot, I'm guessing the number of tanks I killed with rols from 320-400 were minimal. I doubt they were enough to bring the average damage up to 320.

 

I'm thinking of trying the tracking again now with the T54E1, It's been a few games since I've had more than 400 damage per shot, and just managed  a 10 penetration 3571 damage game with no kills.

Maybe just T54's have it out for me...

Share this post


Link to post
Share on other sites
32 minutes ago, Sexy_Dave said:

Yes, but for tracking average damage per shot, I'm guessing the number of tanks I killed with rols from 320-400 were minimal. I doubt they were enough to bring the average damage up to 320.

Not discounting potential kill shots is a bias of around 3 damage points for a T-54. The other effect (where chains of low rolls are likely to lead to someone else killing the target) is difficult to model and may be larger, although I would be surprised if the total was anywhere near 20 points.

If you have a deficit anywhere near 20 points then you don't need many battles to statistically confirm it. I'll do the hard work. All you need to do is announce immediately before you play the first game (this step is important for verification), and then collect the next ~20 replays in that tank. Batch them up and I'll check them shot-by-shot.

Share this post


Link to post
Share on other sites

Cool, will do. (Although I don't expect you to do anything, just forward through the boring play parts- and no judging the stupidity,...)

Share this post


Link to post
Share on other sites

Mathematically the lowest number you can draw a statistical result from is 23-27 or so (e.g. T value). 

As always,  the larger the sample size,  the less likely variation will lean any which way. 

 

 

Share this post


Link to post
Share on other sites

Maybe it´s just me, and please do take this with a grain of salt: imo 320 alpha guns have the most retarded rolls, they always seem to roll below average.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...