User blog:Daniel Baran/Search Developments: Big Picture

Hi everyone,

My name is Dan Baran, and I’m the Director of what we call the “Platform Team” here at Wikia. We focus on systems that support multiple front end features -- one of which is search. The recent improvements we’ve made represent many months of hard work and careful decision making. As with many foundational systems, it was necessary to weigh the options of integrating an external search service, versus developing our own (based on a Solr/Lucene core, for you tech types). It was not an easy decision, but in the end, we decided that having fine-grained control over our search tool outweighed all other factors. Wikis, as documents, do things a little differently than other content systems, and the only way we’re going to be able to respond to your requests is to be the ones turning the wrenches.

So that brings us to today. The recently released changes represent what we consider to be the first step in an ongoing commitment to evolving our search into a great experience. Behind the scenes we’ve done things like:


 * Relevance tuning. We’ve worked to ensure your search yields the best and most pertinent content that exists.
 * Reduced the time it takes us to refresh the index that powers search results. This is not something you’re likely to notice, but it allows us to develop features, and fix problems much more rapidly.
 * Lots of hardware upgrades and re-configurations.

This is just the beginning of an ongoing dialogue we’re looking forward to having with you. One of the biggest challenges we’re working on is creating an experience that works for both advanced, highly skilled founders and new users seeing Wikia (or even wikis) for the first time. Everyone has something to contribute and search is a fundamental entry point into the wiki experience. Getting the discovery experience right, while maintaining mission control for advanced editors is what we’re committed to achieving.

As far as our next steps, we know we have lots of work to do to ensure optimal performance, integrate with other content types, and continue tuning relevance (it’s a journey, not a destination). Some specific things we’ve heard about from you and which we’ll be attacking immediately are:


 * Proper handling of redirects. We’re aware that redirects have been used as a powerful tool to guide people to the right places in your wikis. We’re working to integrate the thought they represent into our search.
 * Pulling all other content types into the fold, most significantly videos and photos.
 * Faceted search. This is the ability to refine your search results based on the information contained within the resulting pages.

So that’s an overview of where we are with search. We’re excited about what we’ll be able to do from now on and we’re definitely looking forward to working with all of you to figure it out!

UPDATE 5/11/12: Adding a Preference, Improving Search Suggestions

Thanks for all the great feedback you've given us about the changes we're making with Search. We've discussed many of your comments internally, and considered them along with our other requirements for this project. As mentioned in the post above, we're committed to getting the discovery experience of search right, while maintaining mission control for advanced editors. With those goals in mind, we want to update you on the following areas:


 * 1) We are adding a personal preference that enables "Go" search for individuals. If you prefer to go directly to pages when you hit "search" (or hit “enter” after entering a search term), you'll be able to activate that feature for yourself through a checkbox in Special:Preferences. We'll release this change as soon as it's ready.


 * 2) We're working on ways to add support for redirects in Search Suggestions, so your curated content recommendations will be front and center while visitors enter their search terms.


 * 3) We're also reducing the delay on Search Suggestions so they're quicker and more responsive as you type.

We believe these changes move us toward our common goals of creating a functional and effective search tool for everyone, and we look forward to continuing to discuss the progress of this work with you. Policy note: We're aware that some wikis have used Javascript to change their search field so that it includes two buttons and/or goes directly to articles (basically, replacing “Go”). Since search is a critical part of the Wikia interface, it's a violation of our Terms of Use to make changes to it that affect other users. We need to ask these wikis to remove those changes. People who use this code only for their own personal experience, however, can leave it as is -- that's the beauty of personal customization at Wikia.