mySQL, noSQL, and Key Value datastores
Monolithic RDBMs are losing ground to key-value data stores, particularly persistent distributed in nature. mySQL mounting problems was perhaps the key reason (pun intended) people looked elsewhere. Google's brilliant engineers realized that a key/value data model can satisfy the needs of almost every class of application that needs a datastore backend.
Key/value datastores are simple to build, easy to understand, easy to optimize, easy to scale. The, now famous, CAP theorem states that it is not practically possible to guarantee consistency, availability and partitioning resilience/tolerance all at the same time; one of those traits has to be sacrificed. Again, most applications really do not require all three to function. The CAP theorem is most likely derived from the Project Triangle mode.
Most web-based applications are built on simple data models. Most web-applications eventually suffer from service capacity and availability issues(i.e scalability woes). It is trivial to scale out(vertically) application logic processing(application servers), HTTP requests processing(web servers, load balancers).
It is not easy to scale out an RDBMS. Some expensive systems(Oracle, etc) provide ways to address those issues (e.g Oracle RAC) but its expensive to deploy them, and most of them rely on a shared everything setup which just doesn't work in the long run. (Shared nothing is really the way to go).
Google released a bunch of papers ( actually, a bazillion of papers ), many of them defining and shaping the development of future related technologies. Namely, the papers describing GFS, BigTable, MapReduce (and of course, the paper the changed everything, "The Anatomy of a Large-Scale Hypertextual Web Search Engine" ) steered everyone to the right direction.
In the datastores domain, Hadoop/HBase, Radix, Cassandra and others, based on BigTable and Amazon's Dynamo papers, all relying on the simple key/value datastore model, are gaining market share - rightly so. Coupled with Memmache and similar services(in-memory key/value stores) they are solving the problems of service capacity and availability. This is a paradigm shift. Its a downhill for heavy-footprint, complex and inflexible datastore systems. They wont go away but will not be such a valuable(pun intended) component in tomorrow's technology landscape.
We are going to gradually migrate from RDBMs - though, we are not relying that much on them nowadays - to a key/value datastore (we are currently building one, also based on BigTable and Dynamo ). If nothing else, those simple systems are both simple and beautiful (for the most part).
Saturday, 13 March 2010 8:14 pm
Ideas for iPhone applications
I thought of two ideas for iPhone applications yesterday and I thought I would share it with anyone interested in pursuing the tasks. I may do it myself if time and motivation allows it, but do feel free to try your luck with them. So here goes nothing.
Life-tracker
The main principle idea is that you could use your iphone to keep track of your life through time, specifically where you have been, what have you been doing and what your thoughts on any given date. The said application would be really simple to use. You launch it, two buttons will make it possible to record your existing location (geolocation coordinates) and/or record thoughts or, say, what it is that you are doing at the moment. You can do that as many times as you wish, whenever you wish. Sometime later, you can sync all that with a web-based service. You could access your life activities through that service ( what have been thinking a year ago this very day? where have I been last week when I was on vacation in London? you want that on a google map - there you go) and so on, so forth. A more or less trivial application to build.
Javascript driven native iPhone applications
This is a no brainer, in fact I wonder why someone ( Apple even ) hasn't thought of it yet. One can expose the iPhone functionality(framework facilities) through Javascript (Javascript objects), have a simple runtime application that 'all' it does is act as the VM/runtime for javascript code. 'Everyone' knows Javascript, everyone(?) likes Javascript, why not make it possible to build real (i.e not hosted on Safari, web-apps ), native applications using the language? A developer would still submit an iPhone application ( the Javascript VM/runtime, with the javascript files and resources in the bundle ) to Apple, Apple, nor the users, would be able to tell the difference. Hey presto, a gazillion apps flood the App store - most will be crappy ( the nature of things ) some will turn out to be gems. If I could make it possible for my brother and my fellow javascript gurus at work to build any app they want as easy as they build our web-apps, that would be kinda cool. Here is what it could look like:
var myButton = new UButton();
myButton.text = "Hello World";
myButton.addListener('click', function(event)
{
alert('Your geolocation is:"+(new ULocation()).toString();
}));
thisWindow.containerView.addView(myButton);
or something.
Tuesday, 11 August 2009 10:30 pm
Random thoughts produced on the balcony
Terry Pratchett on the right to die : Pratchett is one my favorite authors. His books ooze hilarity. He seems an all around awesome person, all things considered, too. Recently, he was diagnosed with Alzheimer's and he seems brave(?) enough to wish to die before this treacherous disease disrupts his mental abilities.
Every single time I need to use Windows, it feels wrong in a profound way. Every single time.
Paypal now available for US Xbox Live Accounts : A great idea, I hope it wont be long before this option becomes available for everyone else, too. Dealing with Microsoft points is an often laborious, not to mention futile, process. You would think MSFT would make it easy for us to give them money.
Check out the water crisis article on wikipedia and HowStuffWorks's why can't manufacture water? topic.
I got to spend some more time on the SGL revision, mentioned in previous post. The Garbage Collector (initially a mark and sweep based facility with a single heap for objects allocation, but will experiment with tri-colour marking and generation based segmentation (i.e generational GC) later on ) is almost in place, the main runtime component seems to be operating as expected(values are stored in 'registers', no stack manipulation, will use thread dispatching using GCC's goto *pointer extension ). I went through the WebKit's JavascriptCore source code for ideas. Came up with lots of notes and concepts that I want to toy with, if time and motivation permits. I am looking forward to diving into Google's V8's source code next week. I am afraid I will have to freeze the project in 2 weeks though and switch gears to work on our new file system (PFS, for lack of a better name), based on design decisions applied on Gooogle's GFS and Amazon.com's Domino.
I am somewhat let down by myself; over 5 projects are left in a semi-complete state. I need to focus on one at a time, wrap it up and move to the next one. Its just that August is mostly about trying out new stuff and research. Its the least demanding month of the year for us, most of the folks at work are away on their much deserved vacations. Little to no sound is produced ( I can't deal with sound, laughter, yelling, phone calls, you name it ) and that does wonders for my (currently degraded, due to Summer) productivity.
Saturday, 8 August 2009 9:19 pm
