The Huffington Post

The Huffington Post portal was launched May 9, 2005 and very soon it became extremely popular.

While one reason of the apparent success of this portal is that their contributors are popular journalists and well-known personalities from politics, business, and entertainment, the other reason is that the development team has been able to provide a technical solution that was up to the challenge.

Task

To implement a news portal capable of sustaining heavy load, including peak load, and convenient for both visitors and authors of the entries.

Challenge - making definite entries appear among the results of running search engines queries, SEO optimization; - simultaneous maintaining of a large number of users; - quick generation of pages that contain dynamic and static data.

Solution

The portal is based on the MovableType platform, which has been modified to minimize the number of database queries and to improve performance. To load a server in order to process pictures, CSS and other statistic content, Amazon S3 is used. To ensure stable and efficient work under heavy and peak loads we have used the memcache technology, which is a general-purpose distributed memory caching system that allows speeding up dynamic database-driven websites by caching data and objects in memory to reduce the number of times the database must be read. For subsystem where database queries were slow but memcache couldn't be used because it wasn't a persistent storage the team used Redis NoSQL server.

 

Also Akamai caching system was used to dramatically decrease number of incoming requests approximately in 1 000 000 times , this system allows flexibly configure each page and create different caches for it.  

 

QArea developers have implemented a rather serious new functionality for this site: basing on Google Maps API it was created our own custom map that operates a large number of images generated by our team, works with large amount of data (about a million rows) and supports a huge number of users.

 

For working with a comments system the developers used several data storages, first they used MySQL with "nested set" data model than comments system was moved to MongoDB and implemented as a web service. New solution easily works with more than 100 000 000 records database.

 

For reliable message delivery and adding latest news and comments to already loaded pages was used Websocket Fronend Solution with Erlang-based server implementation. The system was configured for sending 150 000 messages per second.

 

A powerful statistics system has been implemented in the back-end: there is a statistics of tracking of entries, statistics of visits referred from different search engines. There is also statistics of the sites referring to huffingtonpost.com and general tracking of keywords/key phrases which can be used to refer the users to the site.

Result

The site has been featured on the main page of Yahoo several times. The site lives perfectly with 200+k visitors per hour.

Platform/OS: 
OS Linux
Tools: 
Perl / PHP / Javascript / Ajax / MySQL / caching techniques
Language: 
Perl / PHP / Javascript / Ajax
Category: 
news web site & weblog service
URL: 

"QArea has played a critical role in the development of HuffingtonPost.com. They have been able to become a part of the core team very quickly and develop amazing features that perform under the highest performance and demand requirements possible. They possess the highest level of business cooperation, an outstanding sense of responsibility and delivery of quality work..."

Request a Quote

* Please fill in fields with asterisk.

Request a Call

* Fields with asterisk are mandatory for filling.