An FPGA could certainly work, if you know how to create hardware from programmable logic or are inclined to learn. A second CPU could also work, but is kind of a waste of resources. If you really want to roll your own video card, then I'd say you're more than likely going to have to adjust your expectations down from the SVGA world. You can get some decent oldschool video generators in parts form (the TMS9918 VDP is a fairly capable little NTSC chip, and if you get the 9938 instead you even get expanded capabilities and RGB output instead of composite.) Or you could get really hardcore and build a video generator out of logic. Analog video really isn't that complicated to understand - composite video only gets complicated once you start adding color into the mix, and RGB/VGA doesn't even have that.

In any case, though, if you're really attached to the idea of high-res and high-color, you're probably better off going for a microcontroller with built-in VGA or HDMI.