In August Productions

Dear LazyWeb,

I'm looking for a usable C++ IDE for Linux and I'm wondering if you've seen one. Before you start giving the normal suggestions (Ecliipse, NetBeans, just-use-vi) let start off by saying that I've tried Eclipse, Netbeans and Code::Blocks and KDevelop several times, and that I normally hack in some combination of vi and emacs. (yes yes, I know I'm supposed to religiously pick one and be rude to the other... consider me a postmodern hacker)

For it to be usable by me, it must be able to:

1. Handle the fact that my build is run with autoconf/automake.
2. Properly rename a method and have that show up throughout the codebase.
3. Properly encapsulate a variable with getter/setting methods.
4. Correctly answer the question "where is this method being used"
5. Run without consuming all of my RAM and CPU resources.
6. Quickly and easily open a new project/branch (I have 93 different branches of Drizzle in my source dir right now. Going through a 10 step process to open any given branch in the IDE== FAIL)

Bonus points given for:

1. Allowing me to deal with one or more bzr branches in a sane manner
2. Supporting an option like emacs where the tab key NEVER inserts a tab character and instead ALWAYS indents the line its on.
3. Figuring out by the existence of a configure.ac file that perhaps a Makefile will appear if it runs "autoreconf -f -i; ./configure"

If you think you have the answer, try this as a test:

• bzr branch lp:drizzle
• Open the drizzle directory as a "project"
• Build
• Find some method on the Session object in drizzled/session.h.  Rename it using the IDE. Build again.
• Find the method errmsg_printf in drizzled/errmsg_print.h. Find out every place that uses it. See if that matches what grep -r '\berrmsg_printf\b' would tell you.

Anybody? If you have an IDE and it can actually deal with my daily Drizzle development, I will happily blog both that it can and how to get started.

One of the things we've been working on in Drizzle is having automated performance regressions run when we push new code. Although the fully automated system isn't quite there yet, one of the pieces is, which is a Drizzle-supporting version of sysbench.

I've pushed the code to:

 lp:~drizzle-developers/sysbench/trunk

This new version of sysbench has a libdrizzle driver which can be used to run benchmarks against Drizzle, or, since libdrizzle can also talk to MySQL, to MySQL.

Getting started playing with it is pretty easy, just make sure you've installed a recent copy of libdrizzle first.

  bzr init-repo sysbench

  cd sysbench

  bzr branch  lp:~drizzle-developers/sysbench/trunk

  cd trunk

   ./autogen.sh

  ./configure

  make

  make install

From that point, you should be able to do:

  sysbench --test=oltp help

And see all of the lovely drizzle options.

When I use it against drizzle, I do this. In one shell, I go to the drizzled dir of a recently built drizzle and issue:

  ./drizzled --datadir=.  --innodb-buffer-pool=256M --key-buffer-size=16M --scheduler=multi_thread

Then in another window, I'll do:

  ./client/drizzle 'create database test' 

  sysbench --max-time=60 --max-requests=0 --test=oltp --drizzle-db=test --drizzle-port=4427 --drizzle-host=127.0.0.1 --drizzle-user=root --db-ps-mode=disable --db-driver=drizzle --drizzle-table-engine=innodb --oltp-read-only=on --oltp-table-size=10000 prepare

  sysbench --max-time=60 --max-requests=0 --test=oltp --drizzle-db=test --drizzle-port=4427 --drizzle-host=127.0.0.1 --drizzle-user=root --db-ps-mode=disable --db-driver=drizzle --drizzle-table-engine=innodb --oltp-read-only=on --oltp-table-size=10000 --num-threads=128 run

Which will run 128 client threads against a drizzle database.

This version of sysbench is based on the pre-lua version of mainline sysbench... so I don't know how soon these changes might hit mainline, but this branch is in pretty constant use by us at least.

Ok. So I just looked at two blog posts about MySQL log rotation. While not technically incorrect, I'd like to disagree with the final answer, mainly because recreating the wheel drives me crazy.
Rather than writing a custom script to do this and putting that script in cron, why not use logrotate?
We already ship fully functional log rotate scripts in the Debian MySQL packages. Here's an example:

/var/log/mysql.log /var/log/mysql/mysql.log /var/log/mysql/mysql-slow.log {
        daily
        rotate 7
        missingok
        create 640 mysql adm
        compress
        sharedscripts
        postrotate
                test -x /usr/bin/mysqladmin || exit 0
                # If this fails, check debian.conf!
                export HOME=/etc/mysql/my.cnf
                MYADMIN="/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf"
                if [ -z "`$MYADMIN ping 2>/dev/null`" ]; then
                  # Really no mysqld or rather a missing debian-sys-maint user?
                  # If this occurs and is not a error please report a bug.
                  if ps cax | grep -q mysqld; then
                    exit 1
                  fi
                else
                  $MYADMIN flush-logs
                fi
        endscript
}

This does the re-creation, flush logs and handles failure conditions gracefully. It also compresses old files. Even better - your system already has logrotate running (unless you run a really bad distro)
Now, if you aren't on Debian, don't just copy this verbatim, as you probably don't have a debian-sys-maint user or an /etc/mysql/debian.cnf file. In Debian, we make this at package install time and refresh it on upgrades. It's a system user with an autogenerated password put in a file only readable by root, that allows system scripts like this to run unattended.
Once you read up a little on logrotate, you'll realize it's really great at just about anything you need to do in the log rotation field. Have fun!

Yesterday, I used mysqlpdump to dump 300G of data. Today, as step two of the process, I need to restore that data into the new server. Every good dump tool needs a restore tool, so I wrote mysqlprestore, which is really just a modification of mysqlpdump that will spawn off threads and run your restore in parallel, processing an output directory from mysqlpdump.
I really need to stick it in version control or something, and I'm sure there are bugs, but it's working for me so far. Maybe we can merge the two into a single great tool?

I'm working on a project at the moment where we're doing an upgrade and need to do the dump-and-restore method. It's 300G, so that's never going to be fun, but I found a tool today that helped significantly.

Multi threaded mysqldump is not an utopia any more. mysqlpdump can dump all your tables and databases in parallel so it can be much faster in systems with multiple cpu’s.

I ran mysqlpdump (with one patch I'll send in soon to put quotes around table names) today with 16 threads on a 4 core system and did all 300G in ~3.5 hours. Additionally, since it wraps mysqldump but iterates over the tables, I got a sql file for each table, which is going to make writing a script to restore a piece of cake. It understands that I wanted to do --master-data and it had an option to gzip each sql file as it went.
All in all, I'm thrilled. kudos! And thanks for the tool.
mysqlpdump