User blog comment:Rupert Giles/Global Navigation Update/@comment-5275700-20141204194147/@comment-452-20141205144929


 * What kind of results do you expect to get out of this "testing phase"

This is a variant of an A/B test, so the results they expect will be in line with the normal results of an A/B test.

As it says in the blog post:
 * " At a high level, we are tracking total searches and total page views per user. But we’re specifically looking at how each of these metrics move within user segments."

The results they expect to get out of the test are to see if the behaviour of those 3 groups changes, specifically:
 * Whether they search more.
 * Whether they view more pages.

We will likely never know exactly how they boil down the results, whether their report generator just spits out
 * "More people searched, success!"

or whether they get a more detailed:
 * "Out of the 10000 most active users, 5001 searched more, 3000 searched less, 1000 stopped searching and 999 stopped editing altogether".

It's also unclear whether they would care if the 999 of the top 10000 users stopped editing altogether, so long as 5001 of them were searching more.

As usual, they do not appear to be measuring whether or not the changes result in more edits, and a member of Staff will likely reply to any such inquiry to copy/paste the standard response of: "If you have more views, then more people might edit". (While true, they have the power to measure it, and apparently don't.)