I don't think that the SE has much in the way of failing caps on the logic board. I could be wrong, but I don't think that going in the 68000 Compacts until the Classic. The power supply and analog board on the other hand have been known trouble-makers since Pina published his wonderfully useful books.
If the OP has a voltmeter, I suggest that he measure the 5V and 12V pins on the logic board to analog board connector while the machine is running and see if they're within tolerances. Also try it while giving the machine something disk related to do, maybe copy a big file.
Frequently, on these old machines, the voltage reads okay in idle but drops under load.
Also, there are the old standbys. Is the 2GB drive terminated properly? If the drive was pulled from an external enclosure or was a loose drive, it's entirely possible that its termination was not set prior to installation.