This. The 4.5V lantern batteries basically contain 3 AA cells soldered together, put in a cardboard case and then fitted with two long solder tabs. Just clip the connector and wires off the old Rayovac battery and solder them onto the terminals of the lantern battery.

Another good alternative is getting a 3xAA plastic battery holder and doing a similar thing. The beauty of this is that it is a permenant conversion to AA PRAM batteries for your Mac, so future replacements are as cheap as chips.