Alas, a responsive *nix GUI on a 68K Mac is an unlikely proposition. Mac graphics cards are designed to display QuickDraw and have drivers (in ROM or INIT) that accelerate QD. 68K Macs have used literally dozens of graphics processors, most of which are proprietary. The NeXT family have graphics interfaces and drivers that optimise Display PostScript. PowerMacs sometimes get reasonable X11 graphics, but only if they the right graphics card.
A modern X11 Server for 68K and low power PPC Macs, that uses all of the wonderful goodness of classic Mac OS, makes more sense to me.