We had a big bugfix a couple weeks ago that let people look back beyond just 1000 items when browsing pages. We upped this limit to 50k, to basically give everyone infinite lookback on all content.

However this bug fix had some unintended consequences, as computing the 50k list every 15 minutes is very computationally expensive. So if you notice the site chugging sometimes, it's because the list of post ranks and comment ranks for every page is being computed all at once and it overloads the CPUs when it happens every 15 minutes, and it takes 4 solid minutes of computation to finish. So we've been working on ways to try and make it so it doesn't overload the CPUs so the site doesn't experience slowness while this necessary function is happening.

We've had some mild success with this today, d3rr limited the process to 1 core instead of letting it have all 4, and things seem a bit better. Also the DDOS attacks we had yesterday didn't help our measurements, someone came at us with around 8,000 fake IP addresses, requesting rarely-used data (which is less likely to be cached) with the intent of overloading our systems. However we simply turned on our DDOS protection, which blocked it while keeping the site functional, and they eventually stopped.

I've also recently developed much more thorough tools to detect people who are upvoting themselves. Upvoting yourself using alternate accounts is against saidit rules, and now with our new tools we can much more easily detect self-upvoting. This will help ensure saidit's front page actually represents what people want to see.

Anyway, we're working on balancing the CPU load a bit for the more heavy processes. We're also figuring out our next upgrade path, because we are growing to a size where even once we optimize these processes, we might just still need more horsepower. Saidit is getting approximately 25,000 unique IP visits a day and each visitor views about 3.9 pages, so that's approximately 100,000 pageloads every day, or 1.1 every second. We manage to run all that traffic on 3 servers, with a total cost of just $51/mo. We are currently receiving $36/mo in donations and we also received a $100 bitcoin donation so thank you for to everyone who has donated and continues to donate! You're the heroes that make this site possible, because we do not run ads, so we have no other source to pay expenses for the site.

Whether we do upgrade our hardware soon or 6 months from now, our next step of server upgrade to get more CPUs will put our total expenses up to $101/mo. This will give us 4 servers up from our current 3, with 8 cores total on our 2 main servers, instead of the 4 cores we now have on our 1 main server. So it would basically double our CPU power. The open source code used for saidit is great, because it's natively designed to be deployed on many servers like this. So it should be a straightforward upgrade when the times comes, and d3rr is already doing some testing to ensure our upgrade goes smoothly. But hopefully we can optimize the CPU loads caused by that recent bug fix, so the hardware upgrade is not necessary for a few more months, and we can save a little money. We're really trying to make each dollar stretch as far as we can! Having a financial buffer is important to saidit's longevity, and keeping spending down therefore obviously a key priority.

Thanks to everyone who has contributed to saidit so far. I recently saw the database has something like 374,000 comments and 95,000 posts stored in it. That's a lot of thoughts and opinions! This community is growing in to something great, and I wish the best to everyone who has participated and continues to participate. We're looking forward to the future of saidit. Thank you!