VLC is not well optimized for AltiVec. Some people have reported decent performance with Movist or CorePlayer, but I just use QuickTime 7 and Perian playing back H.264 on my Luxo G4. It works well enough.
On paper, there is no GPU codec acceleration on Power Macs, just GPU scaling of the output. However, Apple does have some private interfaces that accelerate some codecs, H.264 believed to be one; it is probably the most accelerated of the codecs, but you will only get this going through QuickTime. 3rd party apps implementing their own decoders, including ffmpeg, do not have any way of running on the GPU in PPC OS X.
