Hide Notes 

Slide 1: How to scale (with ruby on rails) George Palmer george@meecard.com 3dogsbark.com
Slide 2: Overview • • • • • • • Starting out Scaling the database Scaling the web server User clusters Caching Elastic architectures Links and Questions George Palmer 26th May 2007
Slide 3: How you start out Shared Hosting Web Server DB • • • • Shared Hosting One web server and DB on same machine Application designed for one machine Volume of traffic will depend on host George Palmer 26th May 2007
Slide 4: Two servers Web Server DB • • • • Possibly still shared hosting Web server and DB on different machine Minimal changes to code Volume of traffic will depend on whether made it to dedicated machines George Palmer 26th May 2007
Slide 5: Scaling the database (1) Slave Web Server Master DB Slave Slave • DB setup more suited to read intensive applications (MySQL replication) • Should be on dedicated hosts • Minimal changes to code George Palmer 26th May 2007
Slide 6: Scaling the database (2) MySQL Cluster Master DB Web Server Master DB • DB setup more suited to equal read/write applications (MySQL cluster) • Should be on dedicated hosts • Minimal changes to code George Palmer 26th May 2007
Slide 7: Scaling the web server Web Server Worker thread Worker thread Worker thread Worker thread DB Farm • Web Server comprises of “Worker threads” that process work as it comes in George Palmer 26th May 2007
Slide 8: Load balancing App Server Load balancer App Server DB Farm App Server • App Server depends: – Rails (Mongrel, FastCGI) – PHP – J2EE • Some changes to code will be required George Palmer 26th May 2007
Slide 9: The story so far… App Server Master DB Slave Load balancer App Server Slave App Server Slave • App servers continue to scale but the database side is somewhat limited… George Palmer 26th May 2007
Slide 10: User Clusters • For each user registered on the service add a entry to a master database detailing where their user data is stored – UserID – DB Cluster – Basic authorisation details such as username, password, any NLS settings George Palmer 26th May 2007
Slide 11: User Clusters (2) SELECT * FROM users WHERE username=‘Bob’ AND … App Server user_id=91732 db_cluster=2 Master DB User clusters are themselves one of the two database setups outlined earlier User Cluster 1 User Cluster 2 George Palmer 26th May 2007
Slide 12: User Clusters (3) • ID management becomes an issue – Best to use master DB id as user_id in user cluster or uuid’s – If let cluster allocate then make sure use offset and increment (not auto_increment) • Other DBs such as session must reference a user by id and DB cluster • Serious code changes may be required • Will want to have ability to move use users between clusters George Palmer 26th May 2007
Slide 13: Architecture so far • As number of app servers grow it’s a good idea to add a database connection manager (eg SQLRelay) • Extract out session, search, translation databases onto own machines • Add background processor for long running tasks (so don’t block app servers) • Use MySQL cluster (or equivalent) for any critical database – In replication setup can make a slave a backup master George Palmer 26th May 2007
Slide 14: Non-cached architecture BackgroundRB App Server 1 Master DB Master DB App Server 2 Load balancer Session DB DB Connection Manager Search DB NLS DB Master … App Server 50 Static Files User Cluster 2 Master User Cluster 1 Slave Slave Slave Slave Slave Slave George Palmer 26th May 2007
Slide 15: Issues • Load balancer and database connection manager are single point of failure – Easy solved • 2PC needed for some operations. For example a user wants to be removed from search database – 2PC not supported in rails • Rails doesn’t support database switching for a given model – Can do explicitly on each request but expensive due to connection establishment overhead – Can get round if using connection manager but a proper solution is required (a few gems starting to emerge on this) George Palmer 26th May 2007
Slide 16: Making the most of your assets • In a lot of web applications a huge % of the hits are read only. Hence the need for caching: – Squid • A reverse-proxy (or webserver accelerator) – Memcached • Distributed memory caching solution – Language specific caching • Eg rails fragment caching George Palmer 26th May 2007
Slide 17: Squid App Server 1 Squid … Not in cache App Server 2 In cache Storage • Lookup of pages is in memory, storing of files is on disk • Can act also act as a load balancer • Pages can be expired by sending DELETE request to proxy • Can program any load balancer to pick up pages cached by your app servers (if you know the rules under which it operates) George Palmer 26th May 2007
Slide 18: Memcached Physical Machine Physical Machine App Server Memcached App Server DB Farm Memcached • • Location of data is irrespective of physical machine A really nice simple API – SET – GET – DELETE (Not in memcached) In rails only a fews LOC will make a model cached Also useful for tracking cross machine information – eg dodge user behaviour George Palmer 26th May 2007 • •
Slide 19: Cached architecture • Introduce squid or nginx • Introduce memcached – Can go on every machine that has spare memory • Best suited to application servers which have high CPU usage but low memory requirements • Introduce language specific caching George Palmer 26th May 2007
Slide 20: Cached architecture App Server 1 M C BackgroundRB Master DB Master DB Load balancer M App Server 2 C … App Server 50 Session DB DB Connection Manager Search DB NLS DB Master M C User Cluster 1 Storage User Cluster 2 Master MC=memcached Slave Slave Slave Slave Slave Slave George Palmer 26th May 2007
Slide 21: Cached architecture • Wikipedia quote a cache hit rate of 78% for squid and 7% for memcached – So only 15% of hits actually get to the DB!! • Performance is a whole new ball game but we recently gained 15-20% by optimising our rails configuration – But don’t get carried away - at some point the time you spend exceeds the money saved • Its very easy to scale this architecture down to one machine George Palmer 26th May 2007
Slide 22: Elastic architectures • Based upon Amazon EC2 – Allow you to create server images and launch instances on demand – Very cheap as you only pay for what you use • Currently no way to mount Amazon S3 – Strictly speaking there are a few projects ongoing… • Still in Beta – We’ve had network performance issues • An American VC was quoted as saying “Are you using EC2 for scaling? If not, you better have a good reason” George Palmer 26th May 2007
Slide 23: Elastic architectures App Server 1 M C M C M C M C Monitor High load EC2 Cloud App Server Image App Server 2 Load balancer App Server 3 App Server 4 produces • WeoCeo now offer a similar service George Palmer 26th May 2007
Slide 24: How far can it go? • For a truly global application, with millions of users - In order of ease: – Have a cache on each continent – Make user clusters based on user location • Distribute the clusters physically around the world – Introduce app servers on each continent – If you must replicate your site globally then use transaction replication software, eg GoldenGate George Palmer 26th May 2007
Slide 25: Useful Links • • • • http://www.squid-cache.org/ http://nginx.net/ http://www.danga.com/memcached/ http://sqlrelay.sourceforge.net/ • http://railsexpress.de/blog/ George Palmer 26th May 2007
Slide 26: Questions? George Palmer 26th May 2007