Question

Can speed / scalability be increased?

Answer 1

I noticed the site is using a lot of resources and in times with many people online it gets quite unresponsive (I think that is because of server response times, not local javascript).
So I was thinking how that can be improved.FireBugs YSlow Profiler and Chromiums Audit tool both mentioned: "Combine external JavaScript", "Enable gzip compression", "Leverage browser caching", "Serve static content from a cookieless domain", "Optimize the order of styles and scripts" and "Remove unused CSS rules".
But I think those are not the major problems. The biggest problem is, that every logged in user makes all those comet-requests. I know you are using some kind of framework, but somehow some changes are needed. If the site keeps growing like before, I guess you need to upgrade your server resources anyhow. Still it could be very useful to apply some modern technology like WebSockets. Unfortunately the standard is not yet complete. But most browsers support at least hybi-draft-00, except that firefox and opera have it disabled by default. The advantage with WebSockets is, that there only has to be one open connection per user, which can handle all ajax and comet communication and so you won't have to open and close a HTTP connection every second.

Answer 2

Also there has just been a new draft version released:
http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-08
I think the WebSockets standard is close to be finished.

Answer 3

WebSockets is a dangerous protocol until proven otherwise. Opera and FF have it disabled because a gaping security hole was found. We won't be using it anytime soon.

Moreover, the connection overhead itself isn't what's slowing things down, it's actually communicating all the live changes happening in a given study group. I'm going to be giving some love to the performance of the site over the next week as I get the chance, because we've definitely been noticing that. We've already taken some action to mitigate some of the load server-side, so we'll see how things perform over the next few days based on that, too.