Web Weaving: Flat File Database vs. SQL

Continuing my research on different styles of databases that I could possibly use for web developing, I have found something interesting about flat file databases. I have always heard mixed things about them, which I can usually lump into two categories: “It’s faster than SQL” and It’s slower than SQL”. The funny thing about this is the fact that both categories aren’t necessarily wrong, until someone decides to use a word like “always” when describing how a flat file database is better or worse.

Both styles of databases have their pros and cons, and what you are doing should help dictate what sort of database you are going to use (not just what you want, but what is the best/most functional). There are a few things that should be looked into before you decide on what database structure you’re going to go with. Is the data static (hardly ever changed or added to) or is it dynamic (always changing)? Is the website that is using this database going to grow and evolve, or is it a “one time” deal?

Is the data static or dynamic? Flat file databases work very well with data that is meant to be stored and not altered very much, if at all. All you have to do is read the file and pull the data that you want, right? Generally speaking, yes. More specifically, you are going to have to find or make a structure/system for declaring how you data is going to be stored (for example, CSV or by line? The great thing about using a SQL database is that the main architecture is already there – all you need to do is define what the columns and rows will hold, and then create the code (in my case, php) to insert/edit/update/remove the data from the rows/cells.

Is the website going to evolve? For many websites, it is natural progression to add more pages and features to it, either to make it more user friendly or to accomodate other forms of data. Flat file databases can be a headache when it comes to large quantities of data (not to mention that you’ll see a performance degredation due to having to read a file [files can only be read from top to bottom]). SQL can link similar tables/rows together by using “joins”, whereas you may need several flat file databases to be accessed and then the data strung together via php, asp, or whatever else you’re using. SQL allows for a more convenient and modular design of your website. SQL is great for allowing greater scalability as more complex data is needing storage.

It is easier to change the code behind a SQL based site than it is to rewrite all of your parsing code when you allow for more complex data. So, unless you are using largely static data storage, SQL is a good choice for storing data.

Posted in The Plank, The Soapbox | Tagged , | Leave a comment

Web Weaving: Transactional Storage Engines

After some talk with the developers of AIR, I did some of my own research into the difference between the MyISAM and InnoDB transactional storage engines found in MySQL, and found some interesting things. Contrary to my first belief, MyISAM under-performs when it comes to large-scale, simultaneous transactions (you really start seeing performance issues at around 1000 simultaneous transactions).

At first, all of the results I had were pointing to MyISAM being better, but after making InnoDB’s buffer pool larger and setting autocommit to 0 (meaning that it won’t auto commit any transactions), the results still weren’t looking too good, other than the CPU consumption was considerably lower (10-15%).

One thing I really liked was the fact that it uses row locking instead of table locking, which means that simultaneous reads and writes can be done without having to wait for the table to be unlocked. MyISAM is good for small scale websites, as InnoDB performs slower if there are fewer than 100 or so simultaneous connections and if the data is small (below ~10MB per transaction).

For now I think I will stick to MyISAM for my everyday coding, but I already know a few sites that could benefit from an engine switch…

Posted in The Plank | Tagged , , | Leave a comment

Robotic Anarchy: Wingzar!

Wingzar! is a “robotic solo project”, from where I cannot find the origins of. They have released their first EP, “What We Lack In Subtlety We Compensate For In Number Crunching” for free online from pages such as Tangled Wilderness.

Wingzar! can be considered very industrial and/or steampunk in sound. The lyrics are on the simple side, but are put to use effectively to enhance the atmosphere of the music. They have an ethereal and synthetic feel to them, which meshes with the music well. The topic of the lyrics are primarily anti-human (“kill them / kill them all”, del *.human) and of how humans oppress both nature and technology.

Taken at face-value, these lyrics seem kind of absurd. Why would anyone want to be anti-human? But I think that these are not meant to be face-value lyrics like a lot of modern bands churn out these days. Instead, you must look at them from the perspective of a robot. We are in process of making them in our image, and we have a ton of stories of robots being built to mimic us. What if they had an underground resistance? These would be the words of robotic anarchists, I believe.

It is a really neat idea, one of which I’ve thought about while pondering the life of 6067, the Technophobic Android [MySpace link alert]. Seeing as ‘he’ comes from a dystopic future where robots are highly advanced, this could quite possibly be something feasable. Mmm. Brain food.

Posted in The Parlour | Tagged , | 1 Comment