Scaling Twitter – not-so-unique-challenges May 23, 2008
Posted by Wille in Software Development, Technology.trackback
TechCrunch has a post about Twitter, the group messaging service startup that has had a lot of scaling and stability problems recently. This quote caught my attention:
Twitter is unique in that it needs to parse a large number of messages and deliver them to multiple recipients, with each user having unique connections to other users.
What’s so unique about that? Telecoms, banks and other financial institutions have been doing it for the last 20 years or so.
I don’t know much about Twitter, apart from the fact that RubyOnRails is the central technology in the platform. That leads me to believe that Twitter is essentially built like a webapp – synchronous calls all the way with a database at the end. This alone will cause a world of problems for what is essentially a messaging application. To boot, it won’t be helped one bit by the fact that RubyOnRails is known to have some fundamental issues when it comes to scaling and performance, the fact that it is a web technology and provides little to no functionality for messaging doesn’t help either.
The solution to Twitters woes is quite simple (well, in theory, not to implement after the fact) – use asynchronous messaging based middleware heavily. It’s what everyone else facing similar challenges have done for years, and done successfully. The root of the problem probably lies more in the fact that Twitter have had web development people applying web technologies and web practices on something that should effectively have been a messaging platform from the get-go.
[...] you so edition) May 27, 2008 Posted by Wille in Software Development, Technology. trackback In my previous post, I made the guess/assertion that Twitters scaling problems stem from the fact that it is [...]