# Persistant site slowdowns



## Erethzium (Jan 30, 2013)

So, even after the main slowdown issue with the site was fixed, it seems that the other slowdown issue still persists.

From time to time, the site will just start loading incredibly slow. Like, taking upwards of 10 seconds to load pages. This will go on for about 30-45 seconds, before returning to normal again. And this will happen roughly once or twice an hour, just about.

And insight onto what's causing it? It's been happening for months now.


----------



## Tigercougar (Jan 30, 2013)

Sheeeit, just be glad the main site is up at all with its history.


----------



## thoron (Jan 30, 2013)

Wow, 10 whole seconds to load, that certainly a big deal. Go back to I believe 2007 when it could take 5 minutes for a page to load.


----------



## Arshes Nei (Jan 30, 2013)

Hey 10 seconds is a long time! Just play super hexagon


----------



## Erethzium (Jan 30, 2013)

thoron said:


> Wow, 10 whole seconds to load, that certainly a big deal. Go back to I believe 2007 when it could take 5 minutes for a page to load.



Just five years ago, webpages took five minutes to load? Lolno. Maybe if you were using an old DOS computer with terrible dial-up. Try 1995, Dial-up internet. Hell, even then, pages only took about 15-30 seconds total to load, depending on the content of the page.

And yes, when every other website loads in under 2 seconds, 10-15 second loading times for every single page IS a big deal. Especially when it's happening consistently.



Arshes Nei said:


> Hey 10 seconds is a long time! Just play super hexagon


Glad to see that the staff don't care about server issues that have been happening for months.


----------



## thoron (Jan 30, 2013)

Erethzium said:


> Just five years ago, webpages took five minutes to load? Lolno. Maybe if you were using an old DOS computer with terrible dial-up. Try 1995, Dial-up internet. Hell, even then, pages only took about 15-30 seconds total to load, depending on the content of the page.
> 
> And yes, when every other website loads in under 2 seconds, 10-15 second loading times for every single page IS a big deal. Especially when it's happening consistently.



It wasn't on an old DOS computer, it was FA itself. In all honesties, FA is a site of leasure so why do you care some much if it takes 10 seconds to load? I'd be more understanding if FA was stock exchange site or something were every nanosecond could make a difference.


----------



## Tigercougar (Jan 30, 2013)

thoron said:


> It wasn't on an old DOS computer, it was FA itself. In all honesties, FA is a site of leasure so why do you care some much if it takes 10 seconds to load? I'd be more understanding if FA was stock exchange site or something were every nanosecond could make a difference.


Nuh uh, man. For good or ill, FA is a primary source of activity for the fandom. It may not affect the world economy, but the performance of the site DOES affect the lives of how ever many thousands of people are regular members here. As an important site to the fandom as a whole, the administration should take the responsibility for doing everything in its power to make the site run as smoothly as possible, whether that's in a site-loading or moderation sense. We as members of the site should not be throwing our hands up in the air and allowing the staff to be complacent. We have professional artists that depend on this site to showcase what allows them to make a living, the powers that be should acknowledge and return that same professionalism.


----------



## Arshes Nei (Jan 30, 2013)

Erethzium said:


> Glad to see that the staff don't care about server issues that have been happening for months.



Glad to see assumptions, it makes things so much better. You could wait for a staff member more involved in handling that issue instead of engaging in bickering. You could continue, but at a certain point it waters down your concerns being addressed.

It is up to you.


----------



## PheagleAdler (Jan 30, 2013)

I experience this behavior as well.


----------



## thoron (Jan 30, 2013)

Tigercougar said:


> Nuh uh, man. For good or ill, FA is a primary source of activity for the fandom. It may not affect the world economy, but the performance of the site DOES affect the lives of how ever many thousands of people are regular members here. As an important site to the fandom as a whole, the administration should take the responsibility for doing everything in its power to make the site run as smoothly as possible, whether that's in a site-loading or moderation sense. We as members of the site should not be throwing our hands up in the air and allowing the staff to be complacent. We have professional artists that depend on this site to showcase what allows them to make a living, the powers that be should acknowledge and return that same professionalism.



Of course they shouldn't be allowed to be complacent, but 10 seconds? Come on. Commissions aren't reliant upon 10 seconds. If the 10 second issue was going through out the entire day instead of 45 seconds at random I'd be more understanding to concern. Those 45 seconds are probably spikes in the data requested from the site.


----------



## Erethzium (Jan 31, 2013)

thoron said:


> Of course they shouldn't be allowed to be complacent, but 10 seconds? Come on. Commissions aren't reliant upon 10 seconds. If the 10 second issue was going through out the entire day instead of 45 seconds at random I'd be more understanding to concern. Those 45 seconds are probably spikes in the data requested from the site.



If you read my post, I said UPWARDS OF 10 seconds, not "10 seconds". Pages take anywhere from 10-20 seconds to load, during these bursts of slow-ness. It's rather annoying and disrupting to have one website suddenly dragging its ass.



Arshes Nei said:


> Glad to see assumptions, it makes things so much better. You could wait for a staff member more involved in handling that issue instead of engaging in bickering. You could continue, but at a certain point it waters down your concerns being addressed.
> 
> It is up to you.



This problem has been around for months. Nothing has been done about it so far. Â¯\_(ãƒ„)_/Â¯


----------



## PheagleAdler (Jan 31, 2013)

well, i guess we could see what this maintenance does to alleviate the problem.


----------



## kayfox (Feb 1, 2013)

Still observing odd behavior:

* Occasional pauses in serving requests.
* When these pauses happen, the likelyness is nearly 100% that if there is more than 3 requests waiting, 503 errors will result.
* The 503 error rate limiting behavior is counterproductive combined with these pauses in serving requests, in that, if I were to open one submission in a new tab every 2-5 seconds, if one of these pageloads gets stuck, all of them will sit waiting then error out with a 503 error.


----------



## Armaetus (Feb 1, 2013)

thoron said:


> Wow, 10 whole seconds to load, that certainly a big deal. Go back to I believe 2007 when it could take 5 minutes for a page to load.



It is unacceptable for someone with broadband for even the base of the site to take 10 or more seconds. Even myself with 30/20Mbit FIOS was experiencing slowdowns with the site last night before maintenance.


----------



## DarrylWolf (Feb 1, 2013)

I tend to think that site slowdown and the occasional shutdown for maintenance is unpleasant and annoying but it's just the cost of business for a site like this. Don't be so hard on the site administrators whenever FA screws up- instead be thankful that the site is up the vast majority of the time. They know we're addicted and they're doing their very best to cater to our love of Furry art.


----------



## thoron (Feb 1, 2013)

Glaice said:


> It is unacceptable for someone with broadband for even the base of the site to take 10 or more seconds. Even myself with 30/20Mbit FIOS was experiencing slowdowns with the site last night before maintenance.



Perhaps I'm a bit complacent, but I also know it could be a whole hell of a lot worse. Even now after maintenance I see the slow load happening time and again, I don't think its the server itself but something making large data requests.


----------



## Williamca (Feb 2, 2013)

I notice this issue tends to be most active when removing journals. Anywhere from 5-30 seconds to remove a journal.


----------



## idejtauren (Feb 2, 2013)

I find there isn't any one specific thing that's slow, just everything unpredictably.
Like right now


----------



## Williamca (Feb 2, 2013)

idejtauren said:


> I find there isn't any one specific thing that's slow, just everything unpredictably.
> Like right now



Yeah, just now for me everything is slow. Not just one specific area like it was before.


----------



## Mewtwolover (Feb 2, 2013)

Williamca said:


> Yeah, just now for me everything is slow. Not just one specific area like it was before.


Same here.


----------



## MRGamer01 (Feb 2, 2013)

Figured the site would still be in a less than desirable condition right now.  10 seconds became anywhere between 20 or almost a minute.    I'm no programmer or anything but why is it that the staff can't hire someone to help fix the site up right?


----------



## Williamca (Feb 3, 2013)

MRGamer01 said:


> Figured the site would still be in a less than desirable condition right now.  10 seconds became anywhere between 20 or almost a minute.    I'm no programmer or anything but why is it that the staff can't hire someone to help fix the site up right?



I don't want to speak for FA on this but if I recall correctly there was something about the worry of a new coder leaking security flaws in the system. Please correct me if I am wrong...

Seems the issue worsens towards night though with more activity. The maintenance even seemed to make matters worse itself.


----------



## Cloudchaser (Feb 4, 2013)

Slow to respond for me too.  not complaining because I'm not impatient, I'm just saying.


----------



## Nicthalon (Feb 4, 2013)

Williamca said:


> Seems the issue worsens towards night though with more activity. The maintenance even seemed to make matters worse itself.



1:30pm Eastern and still taking 1-2 minutes to load submissions or reload a page after trying to clear submissions or journals.  Same for viewing the main page.  Basically, everything is taking an inordinate amount of time to load other than the forums, which I assume are hosted on another server.  If I can stream video in HD without stuttering, I should be able to load a page with static images a hell of a lot faster than I am.  This has to be an issue on the FA server side, as nothing else on the web takes this long to load.


----------



## tktktk (Feb 4, 2013)

I didn't want to make a whole new topic for this, as I am rarely on the forums but it's taking anywhere from 30 seconds to two minutes to load anything on FA, including the forums, for me. It's been like this since the update.


----------



## kayfox (Feb 4, 2013)

Williamca said:


> I don't want to speak for FA on this but if I recall correctly there was something about the worry of a new coder leaking security flaws in the system. Please correct me if I am wrong...



I kinda want to say if your code is that weak, your in trouble, but lets think about the options:

Security professional X wants to help out the site, do you:
1. "Hire" them on and have them sign an NDA; or, 
2. Shun them and not have any way to retaliate when they uncover something and spread it around.

Most people would choose #1, especially in this case since it does not cost anything.  But the people who run this site are scared of what might happen, so they chose #2 and live the life of worrying about _what might happen_ rather than accepting it might get rocky, but once its done, you don't have to worry as much.  And hey, if you have a bunch of security professionals who happen to be furries around, you might have an option other than shut down everything when something hits.

But hey, what do I know, I only work in the industry.


----------



## Erethzium (Feb 4, 2013)

tktktk said:


> I didn't want to make a whole new topic for this, as I am rarely on the forums but it's taking anywhere from 30 seconds to two minutes to load anything on FA, including the forums, for me. It's been like this since the update.



Indeed; Right now pages are taking upwards of 2 minutes to load...what's goin' on? Are we getting DDoS'd again?

Seems the server just had some sort of hiccup though, because now it appears to be back to normal...albeit with lots of "Image not Found" submissions on the front page.


----------



## TeenageAngst (Feb 4, 2013)

Stuff like this makes me wish I knew networking. My friend's an industry professional and has been poking the mainsite with a stick looking for cats in the ceiling. According to him there are, but he hasn't been able to figure out how to utilize them yet to have any real effect.

I'm not even sure that's related though. IIRC the last time FA was down for a ridiculous amount of time it was due to a hard drive failure which was preceded by significant site slowdown over several days. Well, the past few days have been *really* slow. Methinks this might be another imminent hardware crash?


----------



## Venu.Shade (Feb 4, 2013)

took 5 minutes+ for it to tell me a 502 error on my end just now.. then proceeded to load when i refreshed the page after another minute


----------



## MRGamer01 (Feb 4, 2013)

Williamca said:


> I don't want to speak for FA on this but if I recall correctly there was something about the worry of a new coder leaking security flaws in the system. Please correct me if I am wrong...
> 
> Seems the issue worsens towards night though with more activity. The maintenance even seemed to make matters worse itself.



If I recall correctly that was/is their/Yak's concern.  And the issue seems somewhat sporadic now with load times only being when accessing the site from a different page.


----------



## Krazinik (Feb 4, 2013)

I've got incredibly fast internet, and FA is taking upwards of 2-5 minutes to load a single page, and most of the time they end up ending with a "503" error about the poor servers not being able to handle all the requests :-/ This got considerably worse after the "upgrade" they did just recently. This wait time is incredibly frustrating. A simple 30-minute conversation with a customer turns into a two-hour long fiasco with slow-loading reference sheets and laggy correspondence and irritation on both ends.

Is there a reason I'm getting 503 errors while everyone else seems to be getting 502s? Or are they basically the same thing? Nevermind, I looked it up; 502 is bad gateway/proxy, 503 is service unavailable due to overload/servicing...

I pulled a new page on FA as I started typing this, got distracted by a TV commercial, went and grabbed a soda, and typed this last line up, and it's still not loaded. That was a good 10-12 minutes ago. I get that more people are online now, but jeeze...


----------



## MRGamer01 (Feb 4, 2013)

Upwards of 5 minutes or more now to load even the main page.


----------



## Xadera (Feb 4, 2013)

Yeah... I'm just trying to respond to comments on a journal and instead of being done in an hour I've been going for 3+ hours now... Each reply takes a good 2-5 minutes to wait for it to post and refresh the page.


----------



## idejtauren (Feb 4, 2013)

504 Gateway Time-out.


----------



## Teal (Feb 4, 2013)

FA is broken again.


----------



## MochaCorgi (Feb 4, 2013)

I just received a 504 error after 12 minutes of loading :I


----------



## Williamca (Feb 4, 2013)

Erethzium said:


> Indeed; Right now pages are taking upwards of 2 minutes to load...what's goin' on? Are we getting DDoS'd again?
> 
> Seems the server just had some sort of hiccup though, because now it appears to be back to normal...albeit with lots of "Image not Found" submissions on the front page.



I sincerely doubt that. It's happening around peek hours. Though from my understanding of the specs of the servers that shouldn't be happening...if it's just users and not a DDoS


----------



## Luxario (Feb 4, 2013)

Aye I keep getting 503 messages constantly... taking up to 2 or more minutes to load a page & it's been happening for anywhere from a a week to nearly a month, getting progressively worse over time. It's starting to annoy me a little & it's happening at all times of the day.


----------



## PunkTiger (Feb 4, 2013)

Well, I've been getting 504 errors for the past 10 minutes (two of them... that's how long it takes before the server gives up and throws the 504 error) and before that it's been sluggish and 502s since about 8:30 EST. :/


----------



## FoxWolfie (Feb 4, 2013)

When I get a page to load, I notice the number of users online isn't even as high as usual. It's very unlikely to be the servers. All the symptoms suggest a DDoS.


----------



## Ainoko (Feb 4, 2013)

I too am getting 502, 503, and 504 errors...

after waiting 5 minutes every time


----------



## Deltaru (Feb 4, 2013)

Williamca said:


> I sincerely doubt that. It's happening around peek hours. Though from my understanding of the specs of the servers that shouldn't be happening...if it's just users and not a DDoS



A DDoS... surely peek hours is the best time to hit a website with DDoS?

Site's not even loading here, but that could be due to my browser choice - IE.


----------



## Williamca (Feb 4, 2013)

Deltaru said:


> A DDoS... surely peek hours is the best time to hit a website with DDoS?
> 
> Site's not even loading here, but that could be due to my browser choice - IE.


It is, but let's wait til they say something...and naa not your browser choice, doing the same with chrome and firefox here.


----------



## Krazinik (Feb 4, 2013)

Deltaru said:


> A DDoS... surely peek hours is the best time to hit a website with DDoS?
> 
> Site's not even loading here, but that could be due to my browser choice - IE.



It won't even load on mine anymore. It tries sooo hard for about 10 minutes, then spits out a 504. I'm on Chrome, so it's not your browser choice :-/


----------



## Calemeyr (Feb 4, 2013)

Yep, "is it down" says FA is down. Huh...is the site being worked on or something? I mean, if it was a DDoS it would go down immediately and not be slow for days, if I'm not mistaken. I don't know, I'm not an expert with computers.

Probably is a DDoS...but still...what was causing the persistent downtimes before this DDoS?


----------



## Ainoko (Feb 4, 2013)

Deltaru said:


> A DDoS... surely peek hours is the best time to hit a website with DDoS?
> 
> Site's not even loading here, but that could be due to my browser choice - IE.



I tried accessing the site from 5 different browsers and keep getting 502, 503, and 504 errors


----------



## Krazinik (Feb 4, 2013)

http://www.isitdownrightnow.com/furaffinity.net.html

The servers are not responding. So now it's no longer a matter of "Is it just me, or is it slow?"

It's a matter of "How long is it gonna be down for, why is it down, and when will it be back?"


----------



## MochaCorgi (Feb 4, 2013)

Everyone keep refreshing! That'll fix it! /sarcasm

I'm off to do things.


----------



## Ansil (Feb 4, 2013)

F-list is crashing as well, I'm sure it's likely unrelated, but just throwing it out there as well.


----------



## Tignatious (Feb 4, 2013)

I'm not even getting an error page on my mobile phone. It just loads as a blank white page. :/


----------



## JerryFoxcoon (Feb 4, 2013)

It's been slow as heck for me as well. Pages took an average 30 seconds to load, got one 502, but at the moment it's completely stuck. As if requests are received but are never processed, then the server throws a 504 after waiting for 3 minutes.

Could a DDoS simply full the server with requests but not actually crashing it? Like, if the server can process, say 10 000 requests a second, send 10 000 requests just to keep it busy?


----------



## Ainoko (Feb 4, 2013)

JerryFoxcoon said:


> It's been slow as heck for me as well. Pages took an average 30 seconds to load, got one 502, but at the moment it's completely stuck. As if requests are received but are never processed, then the server throws a 504 after waiting for 3 minutes.
> 
> Could a DDoS simply full the server with requests but not actually crashing it? Like, if the server can process, say 10 000 requests a second, send 10 000 requests just to keep it busy?



http://www.isitdownrightnow.com/furaffinity.net.html


----------



## Jardenon (Feb 4, 2013)

Thought is was down. Seems i was right.


----------



## roflcopter (Feb 4, 2013)

JerryFoxcoon said:


> Could a DDoS simply full the server with requests but not actually crashing it? Like, if the server can process, say 10 000 requests a second, send 10 000 requests just to keep it busy?



In general, a DDoS does not "crash" a server, but uses up its resources so that nobody can connect to it. So, yes, what you are suggesting is possible.


----------



## JerryFoxcoon (Feb 4, 2013)

*EDIT*


----------



## Tignatious (Feb 4, 2013)

JerryFoxcoon said:


> *EDIT*



-golf claps-


----------



## Venu.Shade (Feb 4, 2013)

now im just getting constant 504 errors


----------



## Flippy (Feb 4, 2013)

Yep. The only thing FA related I can look at is the saved tab from when I looked at my user control panel yesterday, a white screen with:
[h=1]504 Gateway Time-out[/h] nginx/1.2.6& these forums.


----------



## Ansil (Feb 4, 2013)

It looks like someone is on it now!


----------



## Ainoko (Feb 4, 2013)

Venu.Shade said:


> now im just getting constant 504 errors



ditto


----------



## MRGamer01 (Feb 4, 2013)

Image not found everywhere.  It's a start I assume to a fix.


----------



## Ansil (Feb 4, 2013)

Back up yay!


----------



## MRGamer01 (Feb 4, 2013)

Ansil said:


> Back up yay!



Still a sort of persistent slow down.  Not getting my hopes up yet just in case something goes wrong again.

Edit: And we're back to 502 errors.  Thankfully there's at least the page saying they're doing something.


----------



## Stargazer Bleu (Feb 4, 2013)

For me it changes from FA 502 that says admins are working on it, to the 504 timeout white screen.
I as well noticed lot more slow down since the update a few days ago.


----------



## ArtemisZiebenwolf (Feb 4, 2013)

Blagh. I've noticed the slowdown after the updates, too.


----------



## maxgoof (Feb 4, 2013)

A DDoS slowdown is not always the fault of the requestor. If the server is inefficient in responding to a request, it can deny service to other requests.

Frankly, I wonder if FA might be having problems with browsers which block ads, and is trying to force them on, or something, failing, and that is what is causing the slowdown. Why else would this persist for days?


----------



## Accountability (Feb 4, 2013)

It's actually more than likely that PHP or MySQL is Failing Horribly again. Or there's some sort of hardware fault. Not everything is a DDoS, people.


----------



## kayfox (Feb 4, 2013)

Interesting...
At 3 days we can see the problem is getting worse:
http://habnab.it/fa/3d/

6 months, we don't see any trend towards more users, just business as usual:
http://habnab.it/fa/6month/

So, whatever it is, its not an issue of users on the site.

Speculation:
1. The DB, or tables in the DB have reached that magic point where something happens that didn't, like writes needing to be flushed more often, etc.
2. Something is broken, maybe that RAID rebuild?
3. The rate-limiting is killing the site, more errors, more refreshes, more page hits, but with less effectiveness.

The interesting thing is how much is not being seen on the graphs, I'm wondering what is going on there.


----------



## Accountability (Feb 4, 2013)

kayfox said:


> 6 months, we don't see any trend towards more users, just business as usual:
> http://habnab.it/fa/6month/




There's no user growth, however response time gets progressively worse.

_Thaaaaaaaaaaaaaaaats_ our FA!


----------



## kayfox (Feb 4, 2013)

Im really wondering if the timeouts are killing things, consider this:

1. User makes a request, like for http://www.furaffinity.net/msg/others/
2. Request runs for 15 seconds, say using 4 seconds of SQL time.
3. Request times out, user does not get the page.
4. User hits refresh, trying to get their message center.
5. Request runs for 15 seconds...

I don't know what the term for this is, but Ive seen it keel over things.  It might not help that the load balancer is queuing things...  The attack I usually proscribe is two-pronged, increase backend timeouts and set a connection limit, trims from both ends.  As the requests go faster, you can increase the connection limit. Ala:

Connection limit: Hard drops part of the incoming connections, trimming this load off the top.
Increasing the timeouts: Increases the number of successful requests, this means less refreshes trying to get the page.


----------



## Ainoko (Feb 4, 2013)

Maybe 'neer and the admins should look into hiring a couple of network engineers to get this site properly coded. Proper coding, means a faster, more reliable site, which in turn will draw in more users. As it is, sites like SoFurry, InkBunny, Weasyl, etc. are faster and more reliable than FA and are taking away users. 

"Neer, I urge you to seriously consider hiring a network engineer, and a couple of codding professionals to fix the site. Hell, you can even make these professionals sign Non-Disclosure Agreements. Doing this not only will save the site from fading into the mists of time as one of the worst boondoggles online, but will make FA bigger and more attractive.

Somehow, I don't think that any of the users here and on the main site will bitch if you do that, in fact it may just bring in more donations as the users know that you will be on the ball resolving issues as soon as they show up.


----------



## Erethzium (Feb 5, 2013)

Ainoko said:


> Maybe 'neer and the admins should look into hiring a couple of network engineers to get this site properly coded. Proper coding, means a faster, more reliable site, which in turn will draw in more users. As it is, sites like SoFurry, InkBunny, Weasyl, etc. are faster and more reliable than FA and are taking away users.
> 
> "Neer, I urge you to seriously consider hiring a network engineer, and a couple of codding professionals to fix the site. Hell, you can even make these professionals sign Non-Disclosure Agreements. Doing this not only will save the site from fading into the mists of time as one of the worst boondoggles online, but will make FA bigger and more attractive.



They haven't made very many major changes to the site in...what, like 6 years now? I highly doubt they will, any time soon. Hell, the "UI Overhaul" that we were promised 4 years ago still isn't here, and is apparently "on the backburner"...how is something that was supposed to be released 4 years ago "on the backburner" ??


----------



## Ainoko (Feb 5, 2013)

Erethzium said:


> They haven't made very many major changes to the site in...what, like 6 years now? I highly doubt they will, any time soon. Hell, the "UI Overhaul" that we were promised 4 years ago still isn't here, and is apparently "on the backburner"...how is something that was supposed to be released 4 years ago "on the backburner" ??



Yeah, all we have been getting is teasers, but my challenge still stands


----------



## kayfox (Feb 5, 2013)

_trying to troubleshoot some other potentials issues (hardware issues)._

Im wondering if your not hitting the far end of the bathtub curve with the DB server disks, also vibration.  Has the DB server's fans been cleaned out?  Also, you should have graphs of read, write and seek latency, has that been going up?  Consider replacing the disks one by one starting with the slowest one.


----------



## CannonFodder (Feb 5, 2013)

Ainoko said:


> Yeah, all we have been getting is teasers, but my challenge still stands


Now taking bets on which will come first fixing FA and bringing it up to modern day standards or half life 3!


Accountability said:


> There's no user growth, however response time gets progressively worse.
> 
> _Thaaaaaaaaaaaaaaaats_ our FA!


genericaudiencelaughter.sfx


Accountability said:


> It's actually more than likely that PHP or MySQL is Failing Horribly again. Or there's some sort of hardware fault. Not everything is a DDoS, people.


That's what I was thinking as well.  A DDos wouldn't explain how the site for some moments can work and then can't.


----------



## PheagleAdler (Feb 5, 2013)

Yeah, disable the notes system, because that's _totally_ going to fix the problem.


----------



## Dragoneer (Feb 5, 2013)

PheagleAdler said:


> Yeah, disable the notes system, because that's _totally_ going to fix the problem.


We're trying to pinpoint where the massive delays are originating. More precisely, we can see where notes are causing an issue, and we temporarily disabled it while we investigate how much of an impact is has on performance, or or whether it's causing the locks ups we're experiencing.


----------



## Ainoko (Feb 5, 2013)

Dragoneer said:


> We're trying to pinpoint where the massive delays are originating. More precisely, we can see where notes are causing an issue, and we temporarily disabled it while we investigate how much of an impact is has on performance, or or whether it's causing the locks ups we're experiencing.



it's good that you are troubleshooting, but I do hope that you guys take my suggestion to heart


----------



## Dragoneer (Feb 5, 2013)

Ainoko said:


> it's good that you are troubleshooting, but I do hope that you guys take my suggestion to heart


If I had any money at all that would be the first thing I did. But it's not quite realistic with out exceedingly limited budget.


----------



## Nicthalon (Feb 5, 2013)

Well, from 1:52 until about 1:57, the site was loading fast as hell for me, and it's slowed back down to a crawl for me, now.  Not sure what you did in that time frame, but that's the result from my end.


----------



## Dragoneer (Feb 5, 2013)

Accountability said:


> There's no user growth, however response time gets progressively worse.
> 
> _Thaaaaaaaaaaaaaaaats_ our FA!


We're getting 554.94 new registrations daily since Jan 1, 2013. The site is growing, and /fast/. The same time last year, those numbers were about 400 a day.

That said, I can't argue the performance is getting /really/ bad.


----------



## Ainoko (Feb 5, 2013)

Dragoneer said:


> If I had any money at all that would be the first thing I did. But it's not quite realistic with out exceedingly limited budget.



Ask the members for help then. In the past there were those who had the coding and technical knowledge that offered to help clean up the sites coding and security issues out of the kindness of their hearts. Reach out to them, they are the best bet to fix most of the issues the site is experiencing.


----------



## Dragoneer (Feb 5, 2013)

Nicthalon said:


> Well, from 1:52 until about 1:57, the site was loading fast as hell for me, and it's slowed back down to a crawl for me, now.  Not sure what you did in that time frame, but that's the result from my end.


We reset PHP, which offers a temporary reprieve, but is not a solution.


----------



## AmpDragoness (Feb 5, 2013)

Maybe some account cleanup would help? By which I mean deleting accounts that have been inactive for very long periods of time. I come across accounts all the time that have been inactive for 1-2 years or more.


----------



## Nicthalon (Feb 5, 2013)

Dragoneer said:


> We reset PHP, which offers a temporary reprieve, but is not a solution.


Sadly, temporary in this case was only about 5 minutes.   Hope you find the problem.  Best of luck!


----------



## Calemeyr (Feb 5, 2013)

AmpDragoness said:


> Maybe some account cleanup would help? By which I mean deleting accounts that have been inactive for very long periods of time. I come across accounts all the time that have been inactive for 1-2 years or more.


I'm not sure accounts can be deleted as the usernames are in the code, if I'm not mistaken.


----------



## MRGamer01 (Feb 5, 2013)

Dragoneer said:


> If I had any money at all that would be the first thing I did. But it's not quite realistic with out exceedingly limited budget.



Then talk to the people who actually want to help you and the others out with the site.  Hell, some users would probably be willing to help you out completely voluntarily.


----------



## Ainoko (Feb 5, 2013)

MRGamer01 said:


> Then talk to the people who actually want to help you and the others out with the site.  Hell, some users would probably be willing to help you out completely voluntarily.



Exactly what I said.


----------



## Dragoneer (Feb 5, 2013)

MRGamer01 said:


> Then talk to the people who actually want to help you and the others out with the site.  Hell, some users would probably be willing to help you out completely voluntarily.


That will be happening. One of the things we're trying to do right now is create documentation as to procedures and policies for coding. I really, REALLY want new coding hope as that's been a limiting factor for some time.


----------



## CannonFodder (Feb 5, 2013)

Dragoneer said:


> If I had any money at all that would be the first thing I did. But it's not quite realistic with out exceedingly limited budget.


I think we all know that fa doesn't have enough money to hire anyone. . . However why not hold a recruitement drive like you did for admins for site coders?  I'm not saying to add on like 20 new coders; I am saying is why not try and find one or two people to add to the coding staff that everyone would be reasonably sure won't do anything and everyone is reasonably certain that they actually do want to help the site.  You could also set the bar higher and thus reducing the number of applicants by making the requirements that they apply with their real name, have a background in internet programming or security or such and able to prove it.  Whereas the admin recruitement you guys did had hundreds of people, if you had those two requirements then in all probability there would only be a couple dozen to apply and thus require much less time to find someone.


----------



## deragorka (Feb 5, 2013)

Could it possibly be related to continued use of the Android apps and browser extensions that were mentioned in a previous announcement? Is there any way to block access to those programs?


----------



## kayfox (Feb 5, 2013)

Dragoneer said:


> That will be happening. One of the things we're trying to do right now is create documentation as to procedures and policies for coding. I really, REALLY want new coding hope as that's been a limiting factor for some time.



You need more than coders:

1. Network engineers.
2. Usability people.
3. Security people.

You also need more than just a few of these people because this is a volunteer sort of thing and more eyes make every problem shallow.

Edit: I can, if you want, provide you with my resume and references.  If you want me to sign some NDA, I can do that too.


----------



## idejtauren (Feb 5, 2013)

Starting to slow down again, around the same time as yesterday...


----------



## MRGamer01 (Feb 5, 2013)

Edit: 502 twice.


----------



## CoyoteoftheSands (Feb 5, 2013)

The 502 error has shown its face to me twice now...


----------

