Multi platform, high volume data recording
essuu at ourshack.com
Sun Nov 11 10:41:21 GMT 2007
Lyle - CosmicPerl.com wrote:
> I'd appreciate some heavy criticism (without swearing or personal
> insults) and suggestions. Although some things will not be changed, for
Flash is an interesting choice of presentation medium. I particularly
loath that I can't link directly to any of your slides. A great example
of why Flash sucks from a usability point of view.
Anyway, regarding the content I'd make four comments:
1. Multiple A records. We've been doing some testing on a setup just
like this over the last couple of weeks and so far we've found that
"automatic" is something of a misnomer. The current browser crop do NOT
appear to automatically retry the 2nd IP address if the first goes down.
Some data is always lost but when the user clicks refresh the browser
does pick up the second address. So it appears to be good for automatic
failover but it's not seamless and not error free. It's certainly better
than not having a second datacentre though.
2. Fault tolerance. By having just one logging server receiving and
processing data at one time you're introducing a single point of
failure. If it goes down all the data collected goes with it.
3. DB scalability. MySQL only supports 1 master per database and given
that this is primarily a data writing application you will need to
consider partitioning your data into multiple databases so that you can
write to multiple servers. You will also need to consider how you
migrate the write master to another server if the primary server goes
down. Load balancers are quite good for this too.
4. Commercial reality. Your diagram shows 12 servers in 2 datacentres.
That's what, maybe a £30-40k spend on hardware, more if you virtualise
with shared storage and £2k a month in rack rental and transit. Plus the
admin cost of running & supporting that which is another £6-7k a month.
I don't know what you're planning to charge for this software but if I
was on the buying side, and I have been, there's no way we would spend
that kind of money on a one-man band almost no matter how good the
software was. People spending that kind of money tend to be cautious.
Or is this your plan for the hosted solution, in which case you *will*
need to hire a sysadmin to help run that lot. You alone will not be able
to write the software, build those servers, run the system and, most
importantly, sell the solution no matter how good you are. Been there,
done that. You *will* fail if you try it.
BTW - out of curiosity, what's your USP compared to Google Analytics and
Good luck with the project, it's certainly "ambitious" as Sir Humphrey
would say :-)
More information about the london.pm