More annotations for ANOVA results

My ANOVAs usually involve many means. When an ANOVA tells me that the means are significantly different, I can take a look at the post hoc tests. Of course, with lots of means I also have lots of comparisons each with its own "Prob." value. In order to be able to easily spot the significantly different means, I add an extra column after the "Prob." value column. This column contains a conditional expression which assigns each "Prob." value

1) a word ("yes" or "no" for a significant difference or no significant difference based on a certain "Prob." value threshold)
or
2) a symbol (*, ** or *** for significant differences, very significant differences or highly significant differences based on certain ranges for the "Prob." value).

Just for ease's sake it would be good to have an option for producing this extra column with qualitative or semi-quantitative information using statistiXL.

Comments

  • This seems like a good suggestion. I am a bit cautious about highlighting different 'degrees' of significance though. What would people suggest as levels for significant, very significant and highly significant differences? Presumably significant would be <0.05. Would very significant be <0.001 and highly <0.001? Anyone have other recommendations?
  • Hello Alan,

    Your intuitive suggestions are pretty much what I know as a sort of consensus regarding this issue. I am a biochemist and in my field scientists generally use three degrees of significance, which are described in the excerpt below. The following excerpt is a quotation from a freely downloadable electronic PDF document called "Analyzing Data with [...]" by Harvey Motulsky:

    "Once you have set a threshold P value for statistical significance, every result is either statistically significant or is not statistically significant. Degrees of statistical significance are not distinguished. Some statisticians feel very strongly about this. Many scientists are not so rigid, and refer to results as being "[] significant", "very significant" or "extremely significant". [...] summarizes the P value using the words in the middle column of this table. Many scientists label graphs with the symbols of the third column.
    These definitions are not entirely standard. If you report the results in this way, you should define the symbols in your figure legend.

    Statistical significance in science

    P value Wording Summary
    >0.05 Not significant ns
    0.01 to 0.05 Significant *
    0.001 to 0.01 Very significant **
    < 0.001 Extremely significant *** "

    Although these degrees of significance are not an established standard (as it is stated above) and it is finally up to the investigator himself what he considers to be significant, very significant and highly/extremely significant, these symbols with their according P value ranges are very common in the life sciences and can be seen in many publications.

    Excelian
  • An alternative to using degrees of significance would be to let the user specify a P threshold value and then assign only "yes" or "no" to each comparison (according to this criterion).
Sign In or Register to comment.