Board Thread:New Features/@comment-20644-20170406222059/@comment-20644-20170421001554

Reguyla wrote: In most instances that's true. But if people are not using the site because they don't like how things display, then it will take time for them to come around again and learn it's changed. They sure aren't reading these forums and learning about the update. But you could be right. There's really no way to know until you all flick the switch. I still think not doing the most important part of it kinda makes the test pointless though. What do you mean by "not doing the most important part of it"? Do you mean shrinking the size of the module? We're doing that in the test.

When we start the test, 50% of visitors will see the current large version of the module. That's what's known as the control group. It's the group where their interaction with the site doesn't change because they're getting the same experience they had been getting. The other 50% of visitors will see the smaller version of the module. This is the test group.

Once we get enough data, we'll be able to compare the two groups. From there we can see if visitor behavior changed at all in the test group. Do they click on articles more than their control group counterparts? Do they click less? Does it not change anything at all?

If they clicked on it more or it didn't change traffic at all, then we would consider that a successful test and release the change to all visitors. If they clicked on articles less, we'd have to revaluate the design.

We'll know more in a couple of weeks!