Being Away Thu, 3 Sep 2009 12:58:07 -0400

Sorry for the emptiness, I have been quite busy lately. Having relocated the family to Berea, Kentucky has been very taxing on my time and energy. I have some cool stuff I'm working on in erlang and I hopw to be able to make a post about that soon. As well as maybe a C network event driven library for TCP/UDP over IPv4 and IPv6 soon.

0 comments. More...
Migrating Servers Mon, 20 Apr 2009 14:41:45 -0400

Okay, so I'm nearing a transition in life. Not only is my first baby due 1 month from yesterday, but I also have to migrate one of the largest site's I've ever managed. I can't give any details atm for what it is, or why. But I am moving everything to a cluster of servers configured in Amazon's EC2 Cloud using a few parts of their technologies, which will hopefully expand to some others, that should end up totally rockin, and way cheaper. Currently the binary database for this system is approaching 100 gigabytes. And there are other migration issues at hand in regards to the nature of the network and the data it maintains, but sufficient to say that there is some new data being imported at least every 10 minutes or less.

Each day has shown increase in load, as well to the fact that the imported data stands around 20 million records a day currently and that now we will expand the abstraction of said data from 1 table to 250+ tables. Thank goodness with EC2 I can quickly throw up more instances to handle any load and scale horizontally.

TBH this is probably the first technology that I've been really impressed with in a long time, now the only question is if it can stand up to it's promise.

This isn't my first time messing with the concept of shared resource processing networks. When I was 16 I built a beowulf cluster using computers that were being thrown away by my high school. Mostly Pentium's and 486, but considering I didn't have anything better and the electric bill was paid by my parents at the time I enjoyed the challenge. Granted I never was able to find a use for the network. Then a couple of years ago with 3tera launching their grid I began to research and came up with a couple of theories of how they had done this. I concluded they used a set of common GNU tools, the names of which have long escaped me. C'mon it was like 3 years ago!

Anyway, it's very cool and change is a comin so you better get ready for it!

0 comments. More...
New Site Mon, 12 Jan 2009 15:45:19 -0500

Okay so I formatted the server this site runs on. And in going with my latest development I have dropped PHP (finally) and have begun migrating to the Django python web development platform. This took all of an afternoon to rewrite the blog, and probably just another day to redo the few other features that were on this site. I'm excited by the prospects of finally having a good development platform to work from for my site, with as little as I do with it anyway.

_UPDATE_

Also notice that if you normally view VIA RSS, the feed has moved to /feeds/blog/, so please update accordingly.

2 comments. More...
  • Disclaimer
  • The ideas and opinions expressed here are mine.
  • I'm a Linux and BSD user, and lean heavily toward the use of OSS vs certain other commercial solutions.

View the Ninja's profile on LinkedIn

:= RSS =: