Turns out surprisingly it was the hard drive. It got stuck in a seek loop or something. If I let the computer sit for about 5 min it would eventually get to the folder with question mark.

If I booted with the option key, I can boot from CD-ROM.