In August Productions

I've got some code in lp:drizzle/staging right now that's on its way (barring major catastrophes) to trunk. It's not code that does anything sexy as far as the actual running server is concerned. It's a code cleanup branch.

Anyway - short story being - everything from mysys and mystrings that is actually part of public APIs has been moved into drizzled/ proper. Everything else has been moved into drizzled/internal. None of the headers from drizzled/internal are installed... so none of the headers in drizzled/ should be using any of them. Combine this with the past week's removal of both server_includes.h and global.h, and we're getting pretty close to having fully consumable headers.

Which brings me to:

In doing this, I noticed a bunch of things that either need to be fixed, still need to be deleted, or need to be put behind a namespace so that including our headers doesn't strangely and unexpectedly b0rk someone's application. To that end, I've added a bunch of new low-hanging-fruit blueprints. As always, you can look at the full list on Launchpad but I'm including links to the new ones here.

New Drizzle Low Hanging Fruit:

http://blueprints.launchpad.net/drizzle/+spec/code-cleanup-consolidate-charset-headers

http://blueprints.launchpad.net/drizzle/+spec/code-cleanup-merge-decimal

http://blueprints.launchpad.net/drizzle/+spec/code-cleanup-merge-error

http://blueprints.launchpad.net/drizzle/+spec/code-cleanup-move-sql-alloc

http://blueprints.launchpad.net/drizzle/+spec/code-cleanup-types-into-namespace

http://blueprints.launchpad.net/drizzle/+spec/code-cleanup-mem-root-namespace

http://blueprints.launchpad.net/drizzle/+spec/code-cleanup-internal-namespace

http://blueprints.launchpad.net/drizzle/+spec/code-cleanup-remove-myisam-depends

http://blueprints.launchpad.net/drizzle/+spec/code-cleanup-remove-my-files

http://blueprints.launchpad.net/drizzle/+spec/code-cleanup-remove-sql-list

Even the most casual of acquaintances knows that I really, really hate software that doesn't understand how we do things out here in the Open Source world. We have tools, really amazing ones, called ... "Package Managers" which allow us to "Install" software and describe the "Relationships" between elements. Apparently they can't seem to figure out how to deal with software dependencies and instead they opt for the "copy into tree and fork" method:

Google is forking existing FOSS code bits for Chromium like a rabbit makes babies: frequently, and usually, without much thought. Rather than leverage the existing APIs from upstream projects like icu, libjingle, and sqlite (just to name a few), they simply fork a point in time of that code and hack their API to shreds for chromium to use. This is akin to much of the Java methodology, which I can sum up as "I'd like to use this third-party code, but my application is too special to use it as is, so I covered it with Bedazzler Jewels and Neon Underlighting, then bury my blinged out copy in my application.". A fair amount of the upstream Chromium devs seem to have Java backgrounds, which may explain this behavior, but it does not excuse it. This behavior should be a last resort, not a first instinct. What should happen is this:

[google] hey, it sure would be nice if we could use sqlite in chromium for our local db needs.

[google] hmm, the sqlite API doesn't mesh 100% with how I'd like chromium to use it

[google] hey sqlite upstream, there are a few places where we'd like to see API improvements 
            so chromium can leverage it for our use-case

[sqlite_upstream] hey google, so cool that you want to use our code.

* sqlite_upstream looks at google's proposed changes to sqlite *

[sqlite_upstream] interesting, you could try using the foo_bar function for your camel launching
needs, but the rest of the changes seem okay

* sqlite_upstream commits changes to the source tree in commit rev 12345 *

[sqlite_upstream] Our next release will support that.

[google] yay! we'll tell people to either apply our patch or use commit rev 12345 or newer. 

As much as I appreciate Google's attempt at openness documented yesterday - and it REALLY is great and many steps in the right direction, it would be great if they'd also learn some of the process we already use out here. If they think it sucks and want to improve on it, then great - but start from what's there, don't just ignore it and hope we don't notice.

Jonathan Rosenberg of Google posted a rather long but thoughtful description of what it means to be Open. Although he skips over a few things with a hand waving and a "yeah, I know I just said Open is good, that it's counter to what business types normally think, but in the long run Open is better... it just doesn't apply to these few things for vague reasons" the bulk of his post is both well thought out and correct.

The crux of the whole thing: Open wins.

I agree. We all need to continue to grapple with that and to continue to fight against the urges of "oh, but this thing is the magical exception to that rule." It's not. 

Open wins. Closed can suck it.

It's real simple. If you aren't a bank or a credit card company, and I'm not logged in to my actual private account information, stop logging me out of my session due to inactivity.

This means you, airplane reservation systems.

This means you, anonymous web shopping carts.

Guess what? Sometimes when I'm shopping online, I'm doing other things. Or I pause to check some stuff out. There is almost never a time when I'm thankful that you, Lenovo, have cleared out my choices so far for my security. I mean, really? Oh - and if I add something to my wish-list - which is the closest thing I can find to "save for later" - how about you save the CONFIGURED version of the item, not the base model which no one would want anyway.

The re-enabling of the dlopen()-based dynamic plugin loading has hit the Drizzle trunk. This means that plugins can now be built as shared modules and loaded at will on server startup. We still do not support dynamically loading a plugin in a running server via the LOAD PLUGIN command, nor will we until this can be done without re-introducing global mutexes.

Since this is a feature we turned off a while ago, but that was there before, why mention it? There have been several changes to the system that I thought would be handy to mention.

First of all, Drizzle has a crap-ton of plugins and the numbers are only growing. In the tree now, the default is to build any of them we are able to, and to build them all as shared modules, rather than statically linking them into the server.  Then, from the list of built and available plugins, there is a default list of plugins that should be loaded if no other options are given.

But say you do want to load an optional plugin at server start. Do you need to list all of the standard plugins in addition to yours? NO! That would be silly.

There are two options related to plugin loading now, --plugin-load and --plugin-add. Both take comma separated lists of plugin names. --plugin-load overrides the default list of plugins and will only load the list of plugins specified. --plugin-add loads the default list of plugins in addition to any plugins given as a parameter. Any plugin that is requested that cannot load will result in the server not starting. (And yes, it will spit out an error ot that effect)

If you wanted to load the crc32 plugin, for instance, you'd simply do:

drizzled --datadir=/my/data/dir --plugin-add=crc32

If, on the other hand, you wanted to only load a few plugins you care about, you'd do:

drizzled --datadir=/my/data/dir --plugin-load=crc32,md5,transaction_log,console,multi_thread,information_engine,info_schema,errmsg_stderr

Make sure you get a scheduler in there - they're all plugins. :)

As you can probably tell, --plugin-load is almost never what you actually want. I'm also going to add a --plugin-skip in case you want to load all of the defaults except one - but I haven't gotten to that yet.