#Nejm format full
#Nejm format series

Why would we worry about validity of multiplicity adjustments here and not in overall study analyses? Where did this notion come from? The interaction test is the only thing that is safe in a subgroup assessment-the only thing that exposes the difficulty of the task. Therefore, in most cases, no P values for interaction should be provided in the forest plots. The guidelines go so far as to say 'A list of P values for treatment by subgroup interactions is subject to the problems of multiplicity and has limited value for inference. The guidelines imply that forest plots should be based on actual subgroups and not derived from a unified statistical model. Such non-covariate-adjusted analysis that invites the use of improper subgroups (e.g., вЂњolder patientsвЂќ) are highly problematic and statistically non-competitive.

The guidelines condones the use of subgroup analysis.

Researchers should not be penalized for trying to answer more questions, as long as the priority ordering and context is rigidly maintained.

Reporting of P-values вЂњonly until the last comparison for which the P value was statistically significantвЂќ is silly.

Silly straw-man large non-inferiority margins are appearing in the journal routinely.

The guidelines fail to state that non-inferiority margins must be approved by patients.

The precision of estimating A should not always depend on whether B was estimated.

The guideline states that вЂњconfidence intervals should be adjusted to match any adjustment made to significance levelsвЂќ.

#Nejm format full

The excellent advice on handling missing data did not include the more unified full Bayesian modeling approach.The P value is a probability about data if the treatment is ineffective, not a probability about the true treatment effect.Дelving deeper into the revised guidelines, I have a few other concerns: The statement that вЂњClinicians and regulatory agencies must make decisions about which treatment to use or to allow to be marketed, and P values interpreted by reliably calculated thresholds subjected to appropriate adjustments have a role in those decisionsвЂќ is not backed up by theory or evidence. It is merely the chance of interpreting a finding as вЂњpositiveвЂќ if indeed the treatment has exactly zero effect and cannot harm patients. The guidelines do not recognize that type I error is far from the chance that a positive interpretation of a study is wrong. I would have liked to have seen NEJM recognize the validity of the Cook and Farewell approach. It should be fine to report the 11th endpoint in a study in a paper in which the results of the 10 previous endpoints are also summarized. As eloquently described by Cook and Farewell a very reasonable approach is to have a strong pre-specified priority ordering for hypotheses, which also specifies a reporting order and leads to context preservation.

#Nejm format series

More importantly, multiplicity adjustments do not recognize that many studies attempt to provide marginal information, i.e., to answer a series of separate questions that may be interpreted separately. Multiplicity adjustments are notoriously ad hoc, because recipes for these adjustments do not follow from statistical theory and the adjustments are arbitrary. The guidelines, as in the past, place much emphasis on multiplicity adjustments. The increased emphasis on confidence intervals will hopefully help, even if the revered statistical advisors are overruled by medical writers. NEJM has routinely made the absence of evidence is not evidence of absence error using wording such as вЂњtreatment X did not benefit patients with вЂ¦вЂќ in these situations, as if it is a shame to admit that we still donвЂ™t know if the treatment is beneficial. So is the continued emphasis on pre-specification.ДЄ particular applaudable statement is вЂњthe notion that a treatment is effective for a particular outcome if P0.05. The NEJMвЂ™s new emphasis on confidence intervals and openness to Bayesian methods is also welcomed. A major emphasis of the revision is moving away from bright-line criteria for deeming a treatment to be effective or ineffective, consistent with the American Statistical AssociationвЂ™s initiative. The highly revered statistical consultants to the New England Journal of Medicine in conjunction with senior medical leaders have revised guidelines for statistical reporting in the journal.