Forum:Server work today

A general update about the server work today for those not on the mailing list. I've copied the main parts of John's messages to the list and IRC here, so this is just me parroting him:

Last night we switched over to San Jose taking 100% of page requests. Things are looking good but there is a built-in delay which will effect the site a bit today. In order to conduct this test safely, the traffic still goes to Tampa and then gets routed to San Jose for fulfillment. Because of the way DNS works (how you find our servers on the internet), we wanted to have San Jose run overnight via Tampa... so page requests will take roughly twice as long since the data is going back and forth. This morning we are working on having your requests go straight to the new servers in San Jose which will bring the page response back to normal.

That being said, we are seeing one server in San Jose that is temporarily getting too much traffic sent to it, which makes it respond slower than the others... so we're working on that now.

[Later:]

One of the new servers was getting overloaded, which is why it slowed down for a little bit... we've brought more apaches on-line and things appear to be back to normal. Again, we're still double-routing our traffic from San Jose via Tampa, so pages will serve a bit slower than usual until we are done getting everything going directly from San Jose. We're working on that now.

[Later:]

For the next few hours pages should only be loading slightly more slowly than usual due to the double-routing I mentioned earlier. We're currently starting to point traffic directly to our new colo, so each to which that happens will return to normal speed.

sannse (talk) 19:45, 17 October 2006 (UTC) (and John Q)

[Later:]

The DB in San Jose is just responding a bit slowly to requests, so we're focusing entirely on bring it back down to normal. John Q. Zuirdj 20:53, 17 October 2006 (UTC)

[Later:]

We found the issue and adjusted it and everything seems to be running quite nicely now. Sorry for the inconvience! John Q. Zuirdj 22:11, 17 October 2006 (UTC)