Skip to main content

Highly scalable web solutions

Amazon Web Services user’s group meeting last night was slightly derailed from the presenter’s topic to the discussion about current approaches to highly scalable web architecture. Question is simple: in the new world of Web/Facebook applications when millions of users could be added virtually overnight, question of scalability should be addressed from the day one of application design.

In order to make scalable Web application the most important consideration is the foundation it is build upon – choice of the database architecture. I know have seen four radically different approaches:

  • Database clusters
  • Database replication
  • Federated data layer
  • Non-relational databases

Clusters require virtually no code change, scaled up to 32 nodes and in cases of Oracle (do not know anyone else who did it right) are prohibitively expensive for start-ups

Database replication seems so appealing at the beginning… Basically forward all data update/write requests to the master and read the data from slaves. Not much of a code change, cheap/free databases could be used (think MySQL), any slave could become the master when needed… Unfortunately like with the cluster you can go only so far with that – performance will start degrading to the level of inusability after 10-15 slave nodes.

Federated data layer is what Hi5 and many other smart companies are using today. Basic idea – split data horizontally by assigning range of item ids to separate db instances and have “smart” data layer figure out where to go for data request/updates. Major problems: hard to implement “right”, aggregated queries (like selects/count across large data sets, etc) should be executed against many data sources and then aggregated before returning data to the app. Rather challenging to implement and labor intensive any non-trivial scenarios.

Non-relational approach. Sort of new kid on the block (or may be old fellow we used to call object db?). Anyway, with introduction of SimpleDb by Amazon, people start taking it more serious. Basic promise – you trade database features for scalability. Perfect if data is used just to store and retrieve objects and their properties. But what to do about data set aggregation? Navigate through the whole set of objects? I guess, we have to wait for the answer for a while…

As usual – no silver bullets, no magic wands. Well, at least software architects have job security...


Popular posts from this blog

Posting to FaceBook feed using Graph API

Graph API was announced at F8 with a promise to dramatically simplify the FB API. I checked the read access over the new interface during the presentations and to my big surprise it worked flawlessly and from the first time. When I tried, JSON-formatted info about the FaceBook page was returned (as expected).
Then I tried OAuth 2.0 way of accessing the API to post a message to the feed. And to my even bigger surprise it worked too!
Here is what you need to do to access Graph API over OAuth: 1. Create a FB app, store app properties to a file:
$appkey='7925873fbfb5347e571744515a9d2804'; $appsecret='THE SECRET'; $canvas=''; 2. Create a page that will prompt user the access permission (I am prompting for the publish_stream and offline_access permissions at the same time)
// require'settings.php';
$url=""; $url.=…

Developing Google Glass application showing the Heart Beat Rate

Wearable Sensors + Google GlassWearable sensors  are a lot of fun! I played with them for some time by now, both professionally (at work), and for the coding pleasure. This post is about my latest weekend project - native google glasses app showing heart rate. The idea is to continuously measure hear rate using the sensor and to show pulse rate in real time, for example, while running or exercising. Honestly, I did not find running with Google Glasses to be much fun (as they tend to fell off when I am running), but to see what makes my heart beat faster is an interesting and educational experience.

Google Glass (GG) Turns out to be a nice piece of hardware that is a fun to work with: reasonably fast processor, Android 4.0.1, wide range of sensors, WiFi, and Bluetooth interface!

IngredientsGG ( glass SDK ( SDK ( Sensor (…

Respect Coin

Respect I think it's time to talk about currency. Let's create a Respect Coin.
Step 1. Install OpenZeppelin library npm install zeppelin-solidity
When it comes to coins, I like to use some functions that smart people already implemented and other smart people verified. I think that Zeppelin is a nice collection of Solidity contracts that can be trusted. Let's use the StandardToken contract and use it as a parent class for our own RespectCoin contract.
Step 2. Create RespectCoin contract and store it in "contracts/RespectCoin.sol" file pragma solidity ^0.4.4; import "../node_modules/zeppelin-solidity/contracts/token/StandardToken.sol"; /** * @title RespectCoin * @dev ERC20 Token example, where all tokens are pre-assigned to th e creator. * Note they can later distribute these tokens as they wish using `transfer` and other * `StandardToken` functions. */ contract RespectCoin is StandardToken { string public constant name = "RespectCoin&quo…