Page 227 - Open Soource Technologies 304.indd

Open Source Technologies

Notes unlimited number of objects (2,147,483,647) and database size (524,272 TB). Again we could
see the difference. Without enabling features like compression we could achieve a bigger
number of around 70000 transactions per second.
Would it Work? Probably not Long Term
Probably enabling compression and using other features to optimize performance would have
resulted in a higher figure but the problem here was that there was a limit to this. Adding
hardware or moving to newer version of a database (or even to a different database) wouldn’t
have given us the virtually unlimited (100 times 30000 transactions per minute) capacity we
were looking at. Obviously we needed some way to deploy more than one server to split
the load and thus increase the capacity to handle very high transaction loads. In simpler
terms we needed a scale out solution instead of scale up for our database. Looking around
for out of the box solutions in the market didn’t help, simply because there aren’t many
scale-out solutions available in the market to load balance SQL server. Oracle has launched
such a solution but even that requires syncing between different servers in cluster which
takes up a lot of network bandwidth thus reducing the overall effectiveness of the solution.
Scale Out–Approach
After a lot of brainstorming sessions, it was decided that it was time to create our own scale
out solution. The idea was to create a design which could help us scale out as our user base
grew but at the same time being able to handle sudden increases in transaction volumes.
But as it happens all the time, we didn’t have too much money to be spent on this. We
looked at various existing products such as SharePoint to see how they stored their data
and came up with a very simple first draft of the scale out solution. It looked very similar
to way SharePoint does load balancing for its data stores. We had a cluster of database
servers connected to the Broker Hub. Each database was configured to handle a set of users
with specific user Ids and thus held data only for those users. Merge replication was used
to replicate data from all databases to a central database which was used for all reporting.
Here’s how it looked like:

Contd...

222 LOVELY PROFESSIONAL UNIVERSITY