I'm not too sure what the problem is, but a factor could be the single network card setup involved. Usually, Internet Connection Sharing (or Network Address Translation) best works with multiple network cards. Your iBook connects to one of your iMac's network cards (which is part of your "internal network"), then the connection goes out the network card which connects to your service provider's network.

Is performance sluggish even when the iMac isn't doing anything at all (that is, its just sitting there, not downloading, not running iTunes or whatever)?