“Power” to detect statistically significant effects based on sample size and magnitudes of effects
I was going through magnitude-based inferences materials by Will Hopkins and I am playing with R simulations. I wanted to see how many times I am able to detect statisticaly significant effects (p<0.05) depending on magnitude of effects (expressed as Cohen’s D, and using Will Hopkins levels) and sample sizes.
What I did is created a baseline group (mean = 100, SD = 10), and 5 more groups based on magnitude of difference (Trivial, Small, Medium, Large, Very Large) and repeated this for different number of subjects. Then I calculated p values using t test between baseline group and 5 other groups for each number of subjects. Then I repeated this process 1000 times and counder significant effects (p<0.05).
The result is the table showing how many times (percentage) in those 1000 resampling I was able to detect statisticly significant effect depending on the number of subjest of magnitude of change (from baseline group).
Here is the code and the resulting table:
Unlock a world of knowledge with our exclusive membership! With Standard Plan, you’ll instantly get a free access to Google Sheets for Sports Science and Excel Tricks for S&C Coaches courses, But that’s not all! The first 100 members who join us will receive an extra special bonus – Strength Card Builder, absolutely FREE! Unbeatable value at just $35/month for Standard membership, offering so much more in return. Don’t miss out on this incredible opportunity! And Claim The First Month Bonuses Only spots left!Limited offer!
With Complementary Training You Always Get More!
Lean Annual Planner, and the highly sought-after AthleteSR app!Start Your Memberhip Today
The fastest 100 members who join us get a 1-month membership, ALL the above bonuses + Strength Card Builder
Hurry Up
