The requirement for 10.4 was that the machine have either AGP graphics or FireWire, but Apple decided more people would know what FireWire was than whether they had AGP graphics.
If you take out the hard drive and use another machine to install 10.4 (any except, of course, Intel machines), it'll boot and run just fine in a 350 MHz iMac.
If you want 10.4 on a first generation iMac, you have to make sure the boot partition is within the first 8 gigs or use Xpostfacto to boot OS X from an OS 9 partition. I have a 750 gig drive connected to my first generation iMac motherboard, and it's running OS X Server 10.4.11 quite well.