Yes, go to the "Internet" tab under the "Sharing" control panel in OS X to set it up. The G4 will effectively act as a DHCP server to which you can connect other Macs via ethernet. Depending on how you are connected to the internet with your G4 you may require a PCI card if the built-in ethernet port is already in use.

Then on the older machines, simply configure TCP/IP to "get address from DHCP server" and it should work fine. If you're using traditional Mac TCP (eg in System 6) then it will be a bit more complicated to configure.