User blog:Daniel Baran/Search Developments: Big Picture

Hi everyone,

My name is Dan Baran, and I’m the Director of what we call the “Platform Team” here at Wikia. We focus on systems that support multiple front end features -- one of which is search. The recent improvements we’ve made represent many months of hard work and careful decision making. As with many foundational systems, it was necessary to weigh the options of integrating an external search service, versus developing our own (based on a Solr/Lucene core, for you tech types). It was not an easy decision, but in the end, we decided that having fine-grained control over our search tool outweighed all other factors. Wikis, as documents, do things a little differently than other content systems, and the only way we’re going to be able to respond to your requests is to be the ones turning the wrenches.

So that brings us to today. The recently released changes represent what we consider to be the first step in an ongoing commitment to evolving our search into a great experience. Behind the scenes we’ve done things like:


 * Relevance tuning. We’ve worked to ensure your search yields the best and most pertinent content that exists.
 * Reduced the time it takes us to refresh the index that powers search results. This is not something you’re likely to notice, but it allows us to develop features, and fix problems much more rapidly.
 * Lots of hardware upgrades and re-configurations.

This is just the beginning of an ongoing dialogue we’re looking forward to having with you. One of the biggest challenges we’re working on is creating an experience that works for both advanced, highly skilled founders and new users seeing Wikia (or even wikis) for the first time. Everyone has something to contribute and search is a fundamental entry point into the wiki experience. Getting the discovery experience right, while maintaining mission control for advanced editors is what we’re committed to achieving.

As far as our next steps, we know we have lots of work to do to ensure optimal performance, integrate with other content types, and continue tuning relevance (it’s a journey, not a destination). Some specific things we’ve heard about from you and which we’ll be attacking immediately are:


 * Proper handling of redirects. We’re aware that redirects have been used as a powerful tool to guide people to the right places in your wikis. We’re working to integrate the thought they represent into our search.
 * Pulling all other content types into the fold, most significantly videos and photos.
 * Faceted search. This is the ability to refine your search results based on the information contained within the resulting pages.

So that’s an overview of where we are with search. We’re excited about what we’ll be able to do from now on and we’re definitely looking forward to working with all of you to figure it out!