Skip to main content

Highly scalable web solutions

Amazon Web Services user’s group meeting last night was slightly derailed from the presenter’s topic to the discussion about current approaches to highly scalable web architecture. Question is simple: in the new world of Web/Facebook applications when millions of users could be added virtually overnight, question of scalability should be addressed from the day one of application design.

In order to make scalable Web application the most important consideration is the foundation it is build upon – choice of the database architecture. I know have seen four radically different approaches:

  • Database clusters
  • Database replication
  • Federated data layer
  • Non-relational databases

Clusters require virtually no code change, scaled up to 32 nodes and in cases of Oracle (do not know anyone else who did it right) are prohibitively expensive for start-ups

Database replication seems so appealing at the beginning… Basically forward all data update/write requests to the master and read the data from slaves. Not much of a code change, cheap/free databases could be used (think MySQL), any slave could become the master when needed… Unfortunately like with the cluster you can go only so far with that – performance will start degrading to the level of inusability after 10-15 slave nodes.

Federated data layer is what Hi5 and many other smart companies are using today. Basic idea – split data horizontally by assigning range of item ids to separate db instances and have “smart” data layer figure out where to go for data request/updates. Major problems: hard to implement “right”, aggregated queries (like selects/count across large data sets, etc) should be executed against many data sources and then aggregated before returning data to the app. Rather challenging to implement and labor intensive any non-trivial scenarios.

Non-relational approach. Sort of new kid on the block (or may be old fellow we used to call object db?). Anyway, with introduction of SimpleDb by Amazon, people start taking it more serious. Basic promise – you trade database features for scalability. Perfect if data is used just to store and retrieve objects and their properties. But what to do about data set aggregation? Navigate through the whole set of objects? I guess, we have to wait for the answer for a while…

As usual – no silver bullets, no magic wands. Well, at least software architects have job security...


Popular posts from this blog

Posting to FaceBook feed using Graph API

Graph API was announced at F8 with a promise to dramatically simplify the FB API. I checked the read access over the new interface during the presentations and to my big surprise it worked flawlessly and from the first time. When I tried, JSON-formatted info about the FaceBook page was returned (as expected).
Then I tried OAuth 2.0 way of accessing the API to post a message to the feed. And to my even bigger surprise it worked too!
Here is what you need to do to access Graph API over OAuth: 1. Create a FB app, store app properties to a file:
$appkey='7925873fbfb5347e571744515a9d2804'; $appsecret='THE SECRET'; $canvas=''; 2. Create a page that will prompt user the access permission (I am prompting for the publish_stream and offline_access permissions at the same time)
// require'settings.php';
$url=""; $url.=…

Respect Coin

Respect I think it's time to talk about currency. Let's create a Respect Coin.
Step 1. Install OpenZeppelin library npm install zeppelin-solidity
When it comes to coins, I like to use some functions that smart people already implemented and other smart people verified. I think that Zeppelin is a nice collection of Solidity contracts that can be trusted. Let's use the StandardToken contract and use it as a parent class for our own RespectCoin contract.
Step 2. Create RespectCoin contract and store it in "contracts/RespectCoin.sol" file pragma solidity ^0.4.4; import "../node_modules/zeppelin-solidity/contracts/token/StandardToken.sol"; /** * @title RespectCoin * @dev ERC20 Token example, where all tokens are pre-assigned to th e creator. * Note they can later distribute these tokens as they wish using `transfer` and other * `StandardToken` functions. */ contract RespectCoin is StandardToken { string public constant name = "RespectCoin&quo…

"Hello World" from Solidity

How to say "Hello World" in the era of blockchain? Reflections Once upon a time, I worked in CTO role of a great startup, and in order to find talented code developers, we ran a competition for the most creative "Hello World" application. It was a lot of fun! I was not able to participate in the competition back then, but here is my delayed entry.

Step 1. Environment Configuration# Installing Truffle framework ( $ sudo npm install -g truffle# Creating project $ mkdir solidity-experiments $ cd solidity-experiments/ $ truffle init# Installing and starting Etherium local test network $ sudo npm install -g ethereumjs-testrpc $ testrpc &
Step 2. Point truffle framework to the test network Update truffle.js file created by the init to point to your test network. File content should look like this:
module.exports = { networks: { development: { host: "localhost", port: 8545, network_id: "*" // Match any netwo…