The KIND meta cloud is a relatively unconventional cloud hosting platform exclusively aimed at clients of KIND Studios and my own projects. The idea is to create an easily scalable, self sufficient, automated and portable platform for web hosting that makes use of IaaS (Infrastructure as a Service) offerings like Amazon Web Services and the Rackspace cloud but is not depending on infrastructure specific details. One other aspect of the development is to examine ways of making such a formation completely self sufficient by expanding the server instances with S.M.A.R.T. Agent functionality and giving them programmatic ways to fully configure themselves and their neighbors according to events they infer from a predefined set of configurations or even from an actively populated Reinforcement Learning ML model that would gather information from the whole deployment and interpret it.
Beyond the obvious achievability concern, given that I am a single person working on this, the main concerns while planing the service were platform robustness as well as keeping the implementation as platform agnostic as possible to ensure good expandability and avoiding provider lock in as a bonus. The resulting platform should be able to handle Infrastructure failings and theoretically you should be able to take the VM images from one IaaS provider and move them to another with minimal rewrites. The premise is that even though cloud infrastructure providers generally have very good credits for reliability and service continuity their existence is relatively young and therefore You still should be cautious about going all in on one specific offering. Also it is calming to know that your applications will have only limited downtime even in the extreme case that somebody takes Ireland (where the implementation currently resides) off line or a rogue employee wreaks havoc on the data center.
STATUS:
2014 - 05 - 08
Extensions and thoughts.
The infrastruckture has been functioning solidly for more than two years now. I am I have to offload some data from time to time bacouse the 8G Volumes I originally provisioned aren't holding up that well when s3 caching and the massive logs come into play.
One other concern is that due to arch's switch to systemd I have to rewrite all startup script in the new format if I wan't to continue to use the arch repos. Even though, compiling packages on zour own is fun too! I'm interested how long I can keep rolling my own updates before it becomes too much of a hassle....
2012 - 09 - 12
Project complete.
All KIND related projects are now hosted on a meta-cloud deployment. The ml databases are gathering information with each visit and should be minable in a couple of months. DNS is still in testing but no bugs have crept up until now, any inconveniences should be config related from now on.
2012 - 08 - 27
GDC Europe and Gamescom where great! No, really they were. But that is past now so it's back to busyness. The past week I have been fiddling around with the DB implementation and it seems that the, still alpha, mysql-proxy has in a matter of speaking come of age. The replication and proxy functionality seems to be working flawlessly today.
Some more testing is obviously mandated but it looks as if its finally time to move on in the schedule (and just about time as well).
2012 - 08 - 13
Note about Delays:
Well, I'm leaving the comfortable chair of my Development Studio and my workstations behind as the date has come for me to fly north. Obviously I am not abandoning the Project, it has become too good and too close to (relative) completion to abandon. The unmistakable elephant in the room is the fact that I took on this project at a time where it was evident that many other obligations would demand my time.
The project at this point is still a self sustaining cloud formation. Once the mysql_proxy is installed the hierarchy daemons will take control of the application layer and the load balancing layer and what you end up with will in all regards be a self scaling cloud. Hopefully European recruiters will like it as much as I do :-)
2012 - 08 - 03
Long time no see they say. June and July went past without me even noticing. Between work on some secret, projects it was mainly getting to grips with the AWS APIs and finding good API libraries to work with.
I have done a lot of work with MySQL replication and am currently continuing work on implementing myslq_proxy to create a single headed replication environment and make sure that it will actually work.
On other news, for the AWS specific parts of the meta cloud I have chosen to use the boto python library. I liked Python when working with it during the facebook hacking competition and want to dive a bit deeper into it.
2012 - 06 - 01
For the time being DNS has been pushed back in order to set up a well performing autonomous and decentralized database and virtual host management/replication system more news should come in the coming month.
2012 - 05 - 17
DNS & DNSSEC are still being worked on as well as the mail fall back system
2012 - 05 - 09
Webmail, MTAs, IMAPS and alot of other tech is online (savetu.be is now completely on the meta cloud)
2012 - 05 - 03
web server infrastructure testing is complete, some projects (including this site) already are hosted on it
