Thread:Brandon Rhea/@comment-26923519-20170430184849/@comment-20644-20170430191423

Because it's an A/B test.

When we start the test, 50% of visitors will see the current large version of the module. That's what's known as the control group. It's the group where their interaction with the site doesn't change because they're getting the same experience they had been getting. The other 50% of visitors will see the smaller version of the module. This is the test group.

Once we get enough data, we'll be able to compare the two groups. From there we can see if visitor behavior changed at all in the test group. Do they click on articles more than their control group counterparts? Do they click less? Does it not change anything at all?

If they clicked on it more or it didn't change traffic at all, then we would consider that a successful test and release the change to all visitors. If they clicked on articles less, we'd have to revaluate the design.

We did this same thing with the new headers. 50% of logged out users saw the new headers, and 50% saw the old ones. That way we could compare data.