User blog comment:Jenburton/Coming Soon: Message Wall in Labs/@comment-655-20111025210449

There's a thing or two to say about the metrics you suggest:


 * Are communities that are using Message Wall talking more?
 * I think simply "talking more" is not necessarily a good thing, unless the Wiki in question is a purely social one. An increase in number of "user talk edits" can mean that people socialize more, but it could also mean that "non-community users" spam more - or that it takes more edits than before to get a message across. To be fair, if your initial worries about talk pages were correct, and many of those edits are just error corrections, we should actually see a decrease in edits. All in all, it's not very clear if a change (in whatever direction) is good or bad.
 * Are those communities seeing an increase in edits?
 * "More edits" can be a bad thing, too - if half of those additional edits are spam invited by an easier access to messaging functions, and the other half is reverting those edits, nothing is really won.
 * Are communities seeing a growth in new visitors and editors?
 * Similarly again, while getting new editors is normally a good thing, new people that just edit in form of message wall entries are not. They "cost" in form of administrative overhead without giving back anything.

To end on a somewhat constructive note, I'd like to suggest some other metrics - which will be more complicated to compute, but probably a better indicator of MW functionality, too:


 * Increase in User: namespace edits after MW has been activated. (Are users moving their talk page to a non-talk namespace?)
 * Relative increase in edits to other Talk: namespaces. (Are users using the remaining "old-style" talk pages more?)
 * Increase in "user talk" edits, but grouped by user groups: anonymous, new, autoconfirmed, admin. (Is the new feature accepted by "old" users, or just used by new ones?)
 * Increase in content edits by new users. (Does having a different conversation experience help visitors and new users become editors?)