Wasn't an older version of iMovie available for OS 9? If you've got a FW camcorder, I'd try to find that version. OS 9 on that hardware would likely be waaaay faster than OS X for that sort of thing. Plus, you may have more available RAM and could easily disable VM, which would speed things up even more.
As for the OS X version, it depends on which iMac you've got. If it's a tray loader, I'd stick with 10.3. If it's a slot loader, 10.4.