Configurable MDE and confidence for A/B tests
under review
Robert O'Rourke
There's a default Minimum Detectable Effect of 5% for A/B tests, and a required 99% confidence threshold for detecting a winner. It would be great if these were configurable as the amount of traffic can impact the values you may want here. Even better would be a feature that can suggest optimal test configuration & duration based on existing site traffic patterns.
Noel Tock
under review
Noel Tock
This is a good shout for large sites wanting better granularity for tests. In the meantime, I suggest we wait for this demand to come in, as well as dig into this question when speaking to users.