i ran tiger on my unpartitioned 120gb HDD with the help of xpostfacto for a long time without issue. xpostfacto removes the 8gb limit if the drives larger then 8gb. if you was to try to install OS X (testing using 10.2) on the 120gb hdd without xpostfacto it will not allow you to select where to install to. with xpostfacto (version for tiger) you can install on the 120gb HDD.
like he said above the limit isnt there for Mac OS, i really think it was a software implemented attempt from Apple to persuade people from installing OS X onto a Beige (old world Mac) even tho 10.2 supported the Beige. if OS X would not install on the beige that isnt in stock config (HDD bigger then 8gb) and the person didnt know about the limit then they will either A get a new Mac or stick with the old Beige and Mac OS. remember that when OS X is running it over rides the stock rom in the Beige with rom in ram so it dont matter what rom version you have.
also like he said it has to be set to master (sometimes single master mode if its a option) i found that out the hard way with my first attempt to install 10.2 on my Beige the first time. my HDD was set to master but i kept getting cant open across the screen in the openfirmware screen when it would try to boot up. then someone helped me to diagnose the issue and asked me if the HDD was put into single master mode and that if it wasnt to try it in that mode. when i did 10.2 booted up and ran fine untill the 10.2.8 update.
but my 120gb HDD could be set to single master or master with slave present and 10.2 didn't have a issue booting.
but my Beige G3 in OS 9 had stability issues as well but in OS X it ran like a champ with no issues (sept for the monitor going to sleep and not waking up after the 10.2.8 update, which there is a work around for on Apples site) i have the AIO model, the issue with the monitor going to sleep and not wake up is also there in 10.3 and 10.4 but the work around works for those as well.