The use of a statistical test, such as the chi-squared test, to determine if selection has occurred within a sport has been used frequently in research. The assumed distribution of a sample could influence the occurrence of significant outcomes. The occurrence of significance is generally interpreted as RAE and explained as a result of selection within the sport. Most studies in this field have been done using a cross-sectional design. Therefore, the purpose of this study was to examine the influence of using different types of distribution when testing for significance, in swimming as an example, over a nine-year period of six cohorts in age by gender groups. 1/10 Results show that using either an assumed uniformed distribution or a proportional distribution of the national population distribution will lead to an increased number of significant results, in comparison to using either a distribution of the actual sample of the specific age by gender group or the distribution of the previous year within the age by gender group. In addition, when using a longitudinal design over a nine-year period, the occurrence of significance decreased over time. In order to interpret significant results as a consequence of selection within a sport the use of a sport specific and age by gender distribution and a longitudinal design is proposed.