User blog:Brandon Rhea/Technical Updates: August 25, 2023

Hey everyone! Check out the latest Technical Updates since August 11th. We’ll continue to update the editor community on bug fixes and product development updates on a bi-weekly basis!

In this update, we go into a couple of deep dives – the first one is about an upcoming change to the global navigation colors, and the second is about a feature we’ve been working on called Quick Answers. Be sure to check them out!

Notable Changes

 * In the edit window, when a wiki editor tried to add or remove text at the cursor position, text that was added or removed shifted at least one character to the left or right. This issue has been resolved by our User-Generated Content (UGC) team.
 * The Video team fixed a bug where, when video content was completed and if the mini/sticky-player is closed, the viewport would snap the user back up to the top of the page.

Known Issues

 * Admins have reported issues with deleting replies in Discussions, where removing large numbers of replies leads to an error message saying they are unable to delete more.

User-Generated Content (UGC) team

 * Thanks extension. The Thanks extension continues to be in testing on a subset of communities, as we work to make it available as an opt-in feature. There’s still some data we need to analyze about the extension’s behavior against platform engagement metrics, and we’ll continue to update the community as we make progress on that.
 * Reported Post Notifications. Notifications for reported posts are still in testing on 10% of our top 6,000 wikis. The next step is to ramp this up to even more test communities. Through this ramp up, we’ve discovered a bug that we’ll need to resolve before we roll this out further to make sure the experience is as seamless as possible. More info to come!

User Experience (UX) team
Global nav color change for logged out users. We recently ran an experiment to see what would happen if we changed the logged out global nav color from yellow to the logged in colors (dark purple for dark mode, white for light mode). Our Product Design and User Experience Research teams have been giving a lot of thought to the global nav lately and how to improve it, and this seemed like an easy area to experiment on because a) we know the yellow color is something the community never particularly liked and b) our original goal behind the color (drawing more attention/use to the global nav) wasn’t successful. So it was an easy choice to experiment with making it consistent with the logged out experience.

Our goal with the experiment was to simply “do not harm” so we could make sure changing the nav wouldn’t negatively impact our platform engagement goals. Not only did we do no harm, but the experiment actually yielded positive results with people in the test group reading even more pages during their sessions. So, we’re going to officially roll that change out to most logged out users (we’re holding back on a small subset in order to continue monitoring engagement for a period of time).

The team is expected to start work on implementing this soon, and we’ll let the community know about this with even more detail as we get closer to the release. We’re also going to be looking at some additional global nav improvements as well. More info as we have it!

Traffic team
Quick Answers rollout. Last week, you may have noticed that we began a ramp-up of a previously experimental feature called Quick Answers, which we removed over the weekend after a number of reports from the community about the accuracy of the content. I wanted to take this opportunity to share more information about the feature and what’s coming up next after we’ve spent time reviewing the previous rollout.

Quick Answers is a feature we first teased at Community Connect, back in April, as part of our “Canonical Pages: The Ultimate Portal to the World of Fandom” session. The gist was simple: we wanted to format information from wiki pages into a Q&A (questions & answers) format that would appear as a module on the wiki page it's based on. The goal is to increase SEO and, therefore making it easier for readers to discover your pages on Google. To achieve the scale needed to make this have an impact across the site, we said we would leverage GenAI in order to create the initial content.

We began experimenting with that throughout the spring and summer months, using a GenAI partner to leverage information from Fandom to create an initial set of answers to questions garnered from related Google searches. The questions and answers that were live during the experiment were reviewed by Fandom staff for accuracy, and then posted once they were reviewed.

We saw promising results from this experiment, including a bump in traffic to the site. You can even see in the image on the right how that was starting to get picked up by Google Search Engine Results Pages (SERPs). Once we saw those results, we decided to prioritize starting to scale this feature out even further, with a set of questions and answers generated for 6,500 character pages across the platform. The goals of this next rollout included testing for accuracy in the model that was being used, as well as seeing how the rollout impacted traffic numbers on an even bigger scale. That was released to those pages last Thursday, and we started seeing that traffic bump we were looking for!

Then, the weekend happened.

On Saturday, we received a number of reports from the community about the accuracy of the content. I was online at the time and saw some of those issues coming in, particularly on Discord. At first, not seeing the full impact of the issues yet, I asked that any inaccuracies be reported either to the wiki teams or to our Customer Support team. But throughout the day, it became apparent that this was a growing issue, so I got in touch with engineering and product leaders at Fandom, and we decided we would turn the feature off site-wide until we could assess the issue and figure out the next steps.

As we reviewed the issue, we saw that while many of the questions and answers were accurate, the inaccuracies that were seen were glaring enough that it was clear that the initial method of generating the questions and answers was not the right approach. When you’re testing new features out, that’s not the worst thing in the world. We learned from it (both what went right with traffic and wrong with accuracy), we adapted, and we’ll try something else. Fortunately, we had already been working on a different model (let’s call this “method B”) where both questions and answers were generated directly from the wiki pages themselves. Compared to the previous approach (“method A”), method B’s approach improved the accuracy of the questions and answers.

So, here’s what’s coming next. Next week, we’re planning to re-release the initial experimental and staff-reviewed questions and answers from method A as well as an initial set of questions and answers from method B. This initial set of method B content has also been reviewed by Fandom staff, and we are confident that they are as accurate as possible. We’ll be testing to see what the SEO impact is for method B questions vs our previous attempts at method A, to see if this is the right path forward to achieve scale and accuracy all at the same time. The total number of pages these will all appear on next week is less than 300.

This feature will go through changes over time, which will add some level of quality assurance capabilities from the community. That was always going to be something we looked at, but given the issues that cropped up last weekend, we’ve spent the last week talking about ideas for how we can make that happen. More info to come there.

For now, if you spot any accuracy issues with the questions and answers, please let us know by contacting a staff member or emailing our Customer Support team. If you look at the questions and feel they’re a bit bland, that’s okay, and we know. Since it’s based on existing content, it may seem somewhat repetitive, but we want to test if formatting that content in this way helps SEO and whether it brings value to readers. The questions themselves will certainly be improved over time, particularly once there’s a UGC component to the feature.

We’ll continue to share information about how this next phase of the rollout goes, what we’ve learned, where we’ll go next, and the UGC decision that we make. So stay tuned for that!

Performance Task Force
What’s this? A Performance Task Force, you say? Recently, we brought together a team of Fandom staff from all of our development teams to work on a single focus: improving performance, like page speed, across the wiki platform. I’ve mentioned platform engagement metrics a few times now, and one of the most impactful things we can do to make the platform even better for people to spend time on is improving things like page load times on your wikis.

We’ll periodically update you on performance improvements that we’re making, and we’ll have a deeper dive into the work of the Performance Task Force later on. Here are some of the improvements that have recently been made:


 * Experimentation tools. We experiment a lot on the site, with all development teams focused on testing out different ideas and features on the site to see what works and what doesn’t. So adding experimentation tools naturally leads to more things being loaded on wiki pages. To help mitigate that, we’ve recently released some changes focused on improving the performance of these tools.
 * Lazy loading pictures. We’ve added lazy loading to most pictures across the site, which will improve page load times.
 * Video performance. We’re working on reducing the size of video components across the platform, which should improve the overall page performance when videos are present. We’ll be doing this by reducing the amount of code a feature like this requires on a page, in order to make the pages lighter and reduce JS and CSS asset sizes – all without changing any functionality. This is our approach to performance improvements in general right now.

If there’s anything you still think needs to be fixed that hasn’t yet, please be sure to either contact your Wiki Representative or, if your wiki doesn’t have one, send us a bug report!