Fog Creek Software
Discussion Board




Why is Friendster slow? Web Performance

Social networking website friendster.com has become very popular recently. Because of its sudden success, the website runs unimaginably slow most of the time. Some basic googling revealed that friendster (supposedly) uses tomcat (open source servlet and jsp web container) and MySQL (open source database).

My question is, does anybody here have any idea as to what makes friendster perform so poorly almost all the time? What is the best way to set up a website that requires frequent database reads and writes? How have people tested against many concurrent users on their own projects?

o.s.
Tuesday, January 27, 2004

"uses tomcat (open source servlet and jsp web container) "


Tuesday, January 27, 2004

tomcat is pretty slow compared something like
resin.

Other than that being fast are large loads takes
planning. They may been very rational and
said they would worry about it when it became
a problem.

son of parnas
Tuesday, January 27, 2004

Even if Tomcat is slow (Which I do not concede lacking experience and performance metrics) there are plenty of techniques to scale such an application over a bunch of servers or on a really big server.

I think it is impossible to know from the outside but attributing it to the use of a single technology is unlikely to be right.  I could run the site plenty slow using Oracle on a one processor windows NT 4 box doncha know

name withheld out of cowardice
Tuesday, January 27, 2004

The programmers involved with the project are retards. I've met them.


Tuesday, January 27, 2004

I actually think the guy who runs it likes that its so slow.  Or at least, he doesn't view friendster users as "his customers" so he sees no need to focus on performance.  No one (as customers) is paying him (although there is lots of advertising on the site).  If I'm not mistaken he has a bunch of VC too, so you'd think the VC would be interested in making the site function at least.

The truth is, the performance of friendster is abysmal.  It's not just slow.  It's unusable.  Which could be very bad considering all the competition popping up...

I've read interviews with the creator that make him sound really mean and introverted.  In fact, I remember reading somewhere he doesn't even like to do interviews.  So much potential... seemingly being squandered.

He could squash match.com and lavalife.com and all the other dating sites combined (which in truth was his initial goal - friendster as a dating site).  It works so well because no one CONSIDERS it a dating site, so women (and men) don't feel "weird" or "goofy" about putting up their profile. "Hey!  I was just doing it cuz all my friends were... I wouldn't actually MEET a guy from the internet.  How weird!"

Michael H. Pryor
Tuesday, January 27, 2004

That's interesting Michael, I hadn't thought of it as a stealth dating site.  But I wonder, what makes a person willing to put up info online at friendster, but not yahoo dating?

Is there really that big of a gap between 'looking for friends' versus 'looking for dates'?

doesn't get social networking software
Tuesday, January 27, 2004

"Is there really that big of a gap between 'looking for friends' versus 'looking for dates'? "

Yes. At least everywhere in the USA I've been. Have you not ever gone on a "date" or hung out with your friends?


Tuesday, January 27, 2004

Friendster isn't about to put match.com and lavalife out of business. For one, those two companies are making enormous profits - I saw the SEC filings for match.com for the year 2001 - they were much smaller then, and their operating costs were $15 million and their revenue was $48 million. LavaLife is privately held, but I do know that their revenue last year was in excess of $100 million.

The stigma attached to online dating is on the way out. In certain demongraphics, it never really existed. That said, yes there is definitely room for a social network that does not suck. Incidentially Match.com is getting into the social network business.

The new Orkut network looks interesting because apparently the relationships are non-binary (that is, you can quantify how much you like someone) and also because I understand that it's possible to form communities within the social network. Tribe (another social network) does this,and it's a great idea for associating with people with common interests. Something like this would be very good for organizing parties or group activities without the formal structure that a club demands.

Back on topic, I'm in general agreement that the Friendster people are essentially throwing away their venture capital. The performance of their system is inexcusable and I've heard that a lot of people have simply given up. Their implementation is, in a word, pathetic.

And as someone mentioned ealier, Friendster's CEO has certainly not been a positive asset to the company at all. He really does come off as being a real jerk; one suspects that he founded Friendster because he needed to...

Burninator
Tuesday, January 27, 2004

Online Dating just implies a certain negative connotation to a lot of people.  Less so in the major cities, but even here in NYC a lot of people have a hangup about it.  It's becoming a lot less so I believe because you have lots of people growing up on AIM and email, so flirting via the computer and meeting people that way (think blogs ... who hasn't thought someone was cool and then went to their photo album to see if they were hot too).  But there is still a stigma and women are especially conscious of that... "What if my friends saw that I was on here?  They would think I'm so desperate!"

Friendster circumvents this whole thing and owes its ENTIRE success to the fact that people do not have the IMPRESSION that it is a dating site and therefore are more willing to put up a photo and description of themselves.  Plus there friends can write funny things about them and they can amass friends in the same way you amass HP in some D&D game.

Match.com (and the other dating sites) have now added "friends" sections in direct copy of Friendster.  Clearly they are scared.  But they missed the boat.  They can't *hide* the fact that they are a dating site, whereas Friendster hides it beautifully.  And even the people who would never respond to a pick-up email are useful as they connect you to their friends who might.

Michael H. Pryor
Tuesday, January 27, 2004

Doesn't trying to get friends online reek of desperation more than trying to get a date online?

I know a couple of attractive girls that have plenty of friends but have given up trying to find what they consider 'decent' guys the old-fashioned way. 

Of course, this is Oklahoma, maybe we're just marriage-happy down here.

doesn't get social networking software
Tuesday, January 27, 2004

Back on the topic of the original question as to why Friendster is so slow. I very, very much doubt that it has anything to do with their choice of web container (Tomcat). My understanding is that they are running at least 50 servers behind a load balancer and frankly that should be more than adequate for their purposes. I know that last year, match.com had a comparable amount of hardware (I can dig up the numbers iif anyone wants the specifics - but I can tell you right now, their main site is running IIS and MS SQL Server).

I think the reason Friendster is so slow is simply because of a combination of poor design choices and even worse implementation. From what I've seen when using Friendster, the database seems to be the bottleneck - I believe they're running MySQL, not that it's particularly relevant, because I have a feeling they just don't know how to builld a cluster that works. It's a horrible implementation, I know that much. I've even seen it throw NullPointerExceptions!

Burninator
Tuesday, January 27, 2004

They also make heavy use of Servlet Filters (seen the stack traces). I believe the spec leaves filter performance and implementation requirements rather underspecified. It leaves a lot of room for competition. I expect a commercial servlet container such as Resin would make a big difference.

Robert Sayre
Tuesday, January 27, 2004

Burninator makes a good point.

Usually when the site is slow (which is always), you can still do certain things with your own account that don't have to "check" if you are connected to someone.  But for example if you try to view someone else's profile, it has to first make sure you are connected and will sometimes fail and tell you you aren't... even though you are... after spending an inordinate amount of time doing so.

But then you can check your "messages" really fast (which doesn't test this connection and will let you read msgs from people you are no longer connected to).

Another major bottleneck is their images server.  Its just way overloaded... I'll get pages to load and then have the image server just NEVER show the images.  It might also have to check to see if you are connected to the person.

All of this is rather strange (this checking to see if you are connected) because if you ADD someone as a friend, you can't immediately see their friends (sometimes for a few days).  Which makes me think they have some background processes that go around and update links in their graph.  So if there are background processes updating links, why can't it just figure out all the people you are connected to at the same time and cache it?

Michael H. Pryor
Tuesday, January 27, 2004

They're pushing it as a dating site now:

"Friendster is an online community that connects people through networks of friends for dating or making new friends.
You can use Friendster to: 

Meet new people to date, through
  your friends and their friends
...
"

doesn't get social networking software
Tuesday, January 27, 2004

It's always been marketed as a dating site.. just no one thinks of it like that.

Michael H. Pryor
Tuesday, January 27, 2004

I have been hired a few times to optimize web backends for performance. NONE of these problems had anything to do with the tech used. It was rather that they hit the limitations of original simplistic database design. In all my cases some table partitioning and caching sufficed.

It seems that rare sites are designed to scale well from the start. Especially original ones, that often start as a low-budget proof of concept. This is, to some degree, natural.

Egor Shipovalov
Tuesday, January 27, 2004

I'm gonna guess that there servers are  poorly load-balanced.  I will occasionally get timeouts, presumably being redirected to either an overloaded server (booh, must mean round-robin load balanced) , or a down server.  This is really unacceptable.  I agree that friendster could probably make a ton of money if they became a dating site, but they really need to hire competant engineers before anyone will use it. 

vince
Tuesday, January 27, 2004

I will agree with Igor 100%.  99 times out of 100 slow performance on DBMS-driven applications is due to insufficient/incorrect/incompetent DBMS design (this includes logical/physical structure and query design).

Quite often people will put these things together in their spare time with no real expert domain knowledge and as far as they are concerned things are fine (even bubble sort is fast on small record sets or on an unloaded system!).  But when you throw even small load on it (maybe dozens of concurrent users) it quickly falls over, and then they have to hire loads of consultants (like me!) that charge exorbitant rates to basically tell them that it needs re-written.  Since the system is ‘live’ there is little desire to take it down for two months while it is redone, so they keep throwing more and more hardware at it, spending tons of money while only achieving diminishing returns.

Oh, I wonder what a little bit of book learnin’ and experience would’ve cost compared to that?

MR
Tuesday, January 27, 2004

I have been in the business of performance testing and tuning for a long time. I would like to say that it is almost impossible to say what is causing the performance issues without actually testing the environment. Quite often it is the simplest of tweaks that will dramatically improve performance. One of the most frequently overlooked issues is having sufficient bandwidth - you would be amazed how heavy modern web page can be.

I have not used the site so I am at a serious disadvantage discussing the specifics of performance of the site. While it is true that Tomcat is not as fast as commercial Servlet engines - but simply switching engines rarely brings significant benefits without actually identifying the actual bottleneck(s).

But I am usually wrong when I guess - so I apologize for my mistake in advance :)

Nurul Choudhury
Tuesday, January 27, 2004

I haven't used the site in over 6 months (I toyed with it for a week or two when my friends sent me a link) and even then it was ladened down with ads. Probably a good 60-70% of screen real estate was ads. This may have something to do with it.

www.MarkTAW.com
Tuesday, January 27, 2004

> In all my cases some table partitioning and
> caching sufficed.

> Oh, I wonder what a little bit of book learnin’
> and experience would’ve cost compared to that?

> I have been in the business of performance testing
> and tuning for a long time.

To the people who wrote these posts (or others who are knowledgeable), are there any good books/articles on developing a database-driven website with performance in mind, or is this knowledge generally gained through experience? I have to admit that my motivation for the original post was not to fix friendster's problems, but rather to avoid them.

o.s.
Tuesday, January 27, 2004

Online dating is super fun.  I think everyone should try it.  I am serious about that.  I've met some really interesting people that way.  The stigma won't be there for younger generations.  They do everything online, including getting dates.  I think finding dates online will be more common than the traditional ways.

christopher baus (www.baus.net)
Tuesday, January 27, 2004

You just say that because you want a backup wife. Since your current wife is blonde and tall, you'll want a short brunette.

www.MarkTAW.com
Wednesday, January 28, 2004

I've always been curious as to how they code connections.  when you see a user's profile, if you're not a friend of that user, it will tell you all the ways in which you're connected to them.

you > billy > steve
you > zack > steve
you > fred > john > steve
you > fred > mike > steve
you > joe > Bon Jovi > steve

there is no way it generates this at runtime by cycling through users.  what would the runtime be on this, n^n^n^n^n?  many users have hundreds of friends.

if you search for user names, you can only see them if they're in your network.  there's no way i can think of to check this on the fly.

and as another user said, often the connection links are totally lost.  at that point, you will only be able to see profiles of your friends and your messages.

so they store this data somewhere, i'm just curious as to what solution we think they implement, and what the correct solution would be.

my solution would it be:

tblConnections
(
user_id int,
connection_id int,
connection_path varchar(10)
)

with the connection_path being a string storing the actual  connection, possibly comma delimited and needing to be parsed out.

when new users or friends are made, a process has to run to revisit and update these connections throughout the tree.

ignu
Thursday, January 29, 2004

Friendster performance can be addressed quite easily. It seems to be an issue of scalability. The current hardware/software configuration cannot handle the growth in useage. Or perhaps even their datacenter...

Solutions:-
1. add more hardware. if the configuration is apache/tomcat then you can scale out in a no. of ways. add more low cost PC's and have (more) dedicated web servers and application servers.

2. this may put the bottleneck on the database. if they are using mysql then they may need to port this onto a higher spec box since i am not sure how scalable mysql will go. i think its prob reached it's upper limit

soluton would be to migrate to a more robust and scalable database. oracle on linux.

3. tuning. there is loads that can be done here to squeeze every drop of performance out of the exising configuration.

Why has nothing improved since the problems have been apparent for a while...

a) mgt. issue. perhaps the cto doesnt have the cooperation of mgt to address this issue. a super solution would be to put a high performance switch in front of the servers and do intelligent load balancing.

b) any tecnical sioluton is going to take time to implement.  it would be wise to benchmark the improved performance and be able to plan for future capacity. its also a good time ( pre IPO ) to also think about high availability resilient systems.

aw
Tuesday, February 03, 2004

hey,
i'm also a friendster user. sometimes, i mean all the time, friendter seems to be very slow.  i dont know anything about tomcat except i know it is really slow. i wanna ask, what is the email add of friendster's trouble shooting administration.. thanks.

ekeke kaekekan
Tuesday, February 03, 2004

I just signed up at friendster and less than 5 minutes later I give up. Page loads are hitting the point where I'm not sure you even get a response (I just let one sit for 2 minutes with no response). So friendster is useless.

Nathan
Thursday, February 05, 2004

oracle is not even needed in this situation.  no other social networking site sees as many users as friendster does, and so all of the scalability and tuning that is involved with making a static HTML site fast don't apply.

also, getting accurate friend network numbers and searching for data within that network are two completely different problems.  building how people are connected to other people is not the problem, managing the extras on top of that (i.e. messages, bulletin board posts, testimonials, etc.) is what is expensive. Ironically, that is what separates one social networking site from another.

Smitty
Friday, February 06, 2004

It's designed poorly and looks really lame, it's only the domain name thats good.

Here are some alternatives

http://www.friendzy.com
http://www.friendsync.com
http://www.tribe.net
http://www.tickle.com
http://www.orkut.com

and many more to follow in the next few months before they are start shutting down due to lack of funds/revenue, just like the docom boom.

Not Friendster
Tuesday, February 10, 2004

if the networking part of friendster wasn't the problem, i'd love to see you code an efficient solution.

ignu
Thursday, February 12, 2004

*  Recent Topics

*  Fog Creek Home