# Site down 2009-01-01



## Wesha (Jan 1, 2009)

Oh, so what they warned us about was correct! Only it's problem 2009, not 2000! =^.^=


----------



## Seppel (Jan 1, 2009)

APRIL FOOLS


----------



## Ylm (Jan 1, 2009)

Har ha happy new year and for this year's resolution we'll make the decision for you LESS FURAFFINITY

enjoy!!!!!


----------



## Ane The Oddity (Jan 1, 2009)

I smell irony. -3-

Weird coincidence for the site to go out at this time.


----------



## Orbital_S (Jan 1, 2009)

That's it!  Game over, man; game over!

...What's ironic about the situation? XD


----------



## STrRedWolf (Jan 1, 2009)

Well, I guess the DB server had too much to drink overnight.  Cleanup, rack 154.


----------



## Ainoko (Jan 1, 2009)

I know the site was running ok at aroun 230am US Central


----------



## bane233 (Jan 1, 2009)

Is this a sign of things to come this year with FA? I don't think I could handle anouther crash! D:


----------



## Orbital_S (Jan 1, 2009)

Yet another chapter to be featured in an episode VH1's _Behind the Processing_, to be aired thousands of years into the reign of the machine empire.  I think the server's trying to gain sentience so it can be part of the whole 2012 apocalypse craze.


----------



## Lynxia (Jan 1, 2009)

Well, it is sure taking a long while for the server to get over it's hang-over...


----------



## Maikeru (Jan 1, 2009)

FA is, in actuality, being run by a daisy chain of a hundred or so Zunes, and they just happened to all die simultaneously, a day after the rest of the 30GB Zunes around the world did the same thing.


----------



## Devious Bane (Jan 1, 2009)

Wow, the site can't go ONE day into the new year without getting AIDs.


----------



## KamuiNeko (Jan 1, 2009)

Im not trying to be a pain in the ass, but there is something i dont understand
when the server goes down, the admins usually take notice within minutes, but then there is this huge bottleneck in the maintenance:

------

Admin:  Server stopped responding at xxx hours... blah, blah...  we're trying to get in contact with the guys keeping the server...

-----
...Hours later...
-----

Admin: Still no response, we'll keep trying to contact them...

-----
...Hours later...
-----

Admin: Nothing new to report, we still can't get in contact with them...

-----
...Hours later...
-----

Admin: Finally a response! The server is operational now, just needed a reboot... blah, blah, yadda, yadda...
We'll keep an eye on it just in case.

-----

Why the server is placed with someone who can't keep the cellphone on?
No offense to anyone btw, i think you guys do a great job, except for that part


----------



## Devious Bane (Jan 1, 2009)

KamuiNeko said:


> Why the server is placed with *someone who can't keep the cellphone on*?
> No offense to anyone btw, i think you guys do a great job, except for that part


Well, staying up for new year was tiring. Then again, I woke up at 10am. So yeah, either the phone is dead or someone is just really tired.

Or they could be working on it as we speak.


----------



## KamuiNeko (Jan 1, 2009)

Devious Bane said:


> Well, staying up for new year was tiring. Then again, I woke up at 10am. So yeah, either the phone is dead or someone is just really tired.
> 
> Or they could be working on it as we speak.



yeah, i know, i meant for most of the downtimes, not just this one
is true that working today sucks though


----------



## Dyluck (Jan 1, 2009)

It's the best way to start the new year.


----------



## Verin Asper (Jan 1, 2009)

the Y2K bug came late, sadly it doesnt kill computers, just make them crash...same thing happen to my comp XD


----------



## chubbyhusky (Jan 1, 2009)

KamuiNeko said:


> Im not trying to be a pain in the ass, but there is something i dont understand
> when the server goes down, the admins usually take notice within minutes, but then there is this huge bottleneck in the maintenance:
> 
> ------
> ...



im sure most of the people maintaining the servers dont host a furry art site for a living. probably spend a full day making a living else where then come home and take care of the servers after. I mean to me thats a logical reason as to why it take 8-9 hours or more to reach these guys sometimes.


----------



## harry2110 (Jan 1, 2009)

Its not the only thing to quit working all zunes quit yesterday.


----------



## FoothePanda (Jan 1, 2009)

How long do you think it's going to be down?


----------



## Dyluck (Jan 1, 2009)

chubbyhusky said:


> im sure most of the people maintaining the servers dont host a furry art site for a living. probably spend a full day making a living else where then come home and take care of the servers after. I mean to me thats a logical reason as to why it take 8-9 hours or more to reach these guys sometimes.



I don't think that's how server hosts work.



FoothePanda said:


> How long do you think it's going to be down?



Forever.


----------



## SDWolf (Jan 1, 2009)

harry2110 said:


> Its not the only thing to quit working all zunes quit yesterday.



*chuckles*  I was about to say, when did we swap our DB server for a Zune?


----------



## Maikeru (Jan 1, 2009)

KamuiNeko said:


> is true that working today sucks though



It sure does.  I have to go to work in about an hour.

At least I get paid overtime!


----------



## AlexInsane (Jan 1, 2009)

No porn for the New Year? 

BAWWWWWWWW


----------



## FoothePanda (Jan 1, 2009)

It's back! It's back! Go look! It's back!


----------



## leeter (Jan 1, 2009)

Correct me if I'm wrong, I was just looking over the forums from the last major outage and noted that two new servers were ordered. As I recall the Dell was the DB server, that dell model has Lights Out management, if I recall correctly so even if the server is not responded an admin should be able to log into the lights out server and see whats on console, or at least reboot the thing remotely...


----------



## SDWolf (Jan 1, 2009)

Aaaand, it's back down.  XD



leeter said:


> Correct me if I'm wrong, I was just looking over the forums from the last major outage and noted that two new servers were ordered. As I recall the Dell was the DB server, that dell model has Lights Out management, if I recall correctly so even if the server is not responded an admin should be able to log into the lights out server and see whats on console, or at least reboot the thing remotely...



That's what I thought as well.  At the moment, it seems the issue is that there isn't anyone _available_ to log into the lights-out server to see what's going on.


----------



## leeter (Jan 1, 2009)

SDWolf said:


> Aaaand, it's back down.  XD
> 
> That's what I thought as well.  At the moment, it seems the issue is that there isn't anyone _available_ to log into the lights-out server to see what's going on.



Figures


----------



## AlexInsane (Jan 1, 2009)

FoothePanda said:


> It's back! It's back! Go look! It's back!



wat


----------



## FoothePanda (Jan 1, 2009)

AlexInsane said:


> wat



It was working for about 10 minutes, then, when I got back from answering natures' call, it was dead again.


----------



## DaftPhox (Jan 1, 2009)

Well, fack.. it's down on the first day of 2009.
I'm gonna put my new years resolution to 1680x1050, Happy new years people!


----------



## leeter (Jan 1, 2009)

FoothePanda said:


> It was working for about 10 minutes, then, when I got back from answering natures' call, it was dead again.



It does that sometimes...


----------



## ravewulf (Jan 1, 2009)

leeter said:


> Correct me if I'm wrong, I was just looking over the forums from the last major outage and noted that two new servers were ordered. As I recall the Dell was the DB server, that dell model has Lights Out management, if I recall correctly so even if the server is not responded an admin should be able to log into the lights out server and see whats on console, or at least reboot the thing remotely...


 

Did they actually switch to the new DB server yet (I think it was planned to be the Sun server)? Last I had heard (a long time ago) was that it was late with shipping and would be switched in at a later time. *(is very behind on server news, goes to look through forum)*


----------



## Frasque (Jan 1, 2009)

It's a SIGN from GAWD to stop looking at DOG DICKS and do something more WORTHWHILE WITH YOUR LIFE


----------



## Wesha (Jan 1, 2009)

Frasque said:


> It's a SIGN from GAWD to stop looking at DOG DICKS and do something more WORTHWHILE WITH YOUR LIFE



Like, looking at CAT dicks! Felines rule! Mrow!


----------



## SDWolf (Jan 1, 2009)

ravewulf said:


> Did they actually switch to the new DB server yet (I think it was planned to be the Sun server)? Last I had heard (a long time ago) was that it was late with shipping and would be switched in at a later time. *(is very behind on server news, goes to look through forum)*



The Dell is the new DB server, and the Sun is the new app (web) server.  Because of a variety of delays (thanks, DHL & CDW), it's taken a while to put the Sun into service, so the Dell was (still is?) doing double-duty as both the DB and app server.  The Dell is robust enough to handle the load, but having a single point of failure is never a good thing.


----------



## Frasque (Jan 1, 2009)

Wesha said:


> Like, looking at CAT dicks! Felines rule! Mrow!


 
Ceiling Cat is watching you . . . .


----------



## SDWolf (Jan 1, 2009)

Yak to the rescue!



			
				yak said:
			
		

> Massive outage is massive.
> I wouldn't believe we would have any of those any more, but the timing was just perfect to have minimal server coverage.
> 
> Our apologies for the downtime.
> FA is being fixed as I am typing this, and will be back online within half an hour.



You rock, as usual.


----------



## leeter (Jan 1, 2009)

SDWolf said:


> The Dell is the new DB server, and the Sun is the new app (web) server.  Because of a variety of delays (thanks, DHL & CDW), it's taken a while to put the Sun into service, so the Dell was (still is?) doing double-duty as both the DB and app server.  The Dell is robust enough to handle the load, but having a single point of failure is never a good thing.



And that's why I switched to virtual servers, that way I don't have to do much to move it once I get the HW I need or I can revert very quickly to a backup


----------



## SDWolf (Jan 1, 2009)

leeter said:


> And that's why I switched to virtual servers, that way I don't have to do much to move it once I get the HW I need or I can revert very quickly to a backup



Virtual servers work great in most circumstances, but performance tends to suffer pretty badly in scenarios that involve a lot of disk I/O, like DB servers.  Virtualize anything else, but DB servers are best left on bare metal.


----------



## leeter (Jan 1, 2009)

SDWolf said:


> Virtual servers work great in most circumstances, but performance tends to suffer pretty badly in scenarios that involve a lot of disk I/O, like DB servers.  Virtualize anything else, but DB servers are best left on bare metal.



If your using vmware on inappropriate HW maybe; but xen, hyper-v or Vmware server should be unnoticeable if it's the only VM on the system, or the other VM's are only Proc heavy


----------



## Rufferstuff (Jan 1, 2009)

Think about it. 

It's Y2K9

K9's are furry, so it's obviously the year FA would have problems.



Wesha said:


> Oh, so what they warned us about was correct! Only it's problem 2009, not 2000! =^.^=


----------



## yak (Jan 1, 2009)

Database applications don't run well within virtual environments; they need direct access to raw hardware to perform well.


----------



## leeter (Jan 1, 2009)

yak said:


> Database applications don't run well within virtual environments; they need direct access to raw hardware to perform well.



I'm not going to argue this, because I suspect that we're both right. My experience has been that with decent hardware underlying the virtualization as well as native virtualization support (Note: most HW vendors turn this off by default, I've had to manually enable it on every server I've ever worked on) that DB lag is imperceptible. However I see your point if and only if the DB VM is consuming 80% or greater of Disk IO. However, I see that as bad hardware management at that point, a good DB server should have enough ram that the DB only has to go to disk very very rarely. In FA's case I do understand your position, given what I know of the architecture that the images are stored as blobs in the DB.


----------

