Think I found my problem, not sure if there's a solution or if I just need to live with it. Turns out it's most likely related to the pre-existing 640x480 mod drawing a little extra power. When I dial down the horizontal width just barely the jittering disappears, and when I crank it up it really shows up, to the point that it actually dances on and off the edges of the CRT. I guess the screen size has a point where it's drawing just too much juice to work properly. So I'm living with a -slightly- smaller screen area but a now rock solid display. I wonder if there's anything I can do to remedy this aside from going back to 512x384?