The ecotoxicity of binary chemical mixtures with a common mode of action is often predicted with the concentration addition model. The assumption of concentration addition is commonly tested statistically based on results of toxicity experiments with the two single chemicals and their binary mixture. The present simulation study shows that if not all these experiments are performed simultaneously, one has a 20–80% chance of concluding synergism or antagonism while the mixture is actually additive (false positive rate). Truly synergistic or antagonistic mixtures have a 10–50% chance of being falsely categorized as additive (false negative rate). Additionally, false positive rates decrease with increasing experimental error, while false negative rates increase with increasing experimental error. Based on these results, we put forward a number of recommendations for future mixture ecotoxicity evaluation.