Back on the new/old OS idea: Usual disclaimers apply, I am not a coder, pie in the sky, etc. But how about this for older 68ks?
Take MachTEN's design approach. Make the "new" OS a virtual machine: a process that seizes the CPU from the co-operatively multitasking Mac OS/System software, and runs pre-emptively and with memory protection within itself. All screen drawing and other peripheral device handling is dumped back to the OS/Quickdraw, saving having to write new drivers for everything, and allowing native video acceleration, etc. Once the peripheral task is completed, the CPU is handed back to the VM. So System 6 (for example) acts as the "kernel".
eCos would be a possible candidate, whether using the above approach or running natively. (Yes, I have brought this up elsewhere. Pardon the cross-post; I couldn't find this thread at the time). For starters, there's already a 68k port, albeit for later embedded 68ks, and it is currently developed, supported, and deployed. Given the memory requirements, perhaps (disclaimer above) it could even run on a 128k.
http://en.wikipedia.org/wiki/ECos
The only M68k port I can find is for the Coldfire. As I understand it from others' comments here, the Coldfire ISA is a subset of that for 680x0s, so my very rough guess is that a backport would be feasible. To run natively, I guess there would be a lot of custom coding involved to get peripheral device support (ethernet, serial, video, SCSI etc)
Take MachTEN's design approach. Make the "new" OS a virtual machine: a process that seizes the CPU from the co-operatively multitasking Mac OS/System software, and runs pre-emptively and with memory protection within itself. All screen drawing and other peripheral device handling is dumped back to the OS/Quickdraw, saving having to write new drivers for everything, and allowing native video acceleration, etc. Once the peripheral task is completed, the CPU is handed back to the VM. So System 6 (for example) acts as the "kernel".
eCos would be a possible candidate, whether using the above approach or running natively. (Yes, I have brought this up elsewhere. Pardon the cross-post; I couldn't find this thread at the time). For starters, there's already a 68k port, albeit for later embedded 68ks, and it is currently developed, supported, and deployed. Given the memory requirements, perhaps (disclaimer above) it could even run on a 128k.
http://en.wikipedia.org/wiki/ECos
eCos Porting Guide
The only M68k port I can find is for the Coldfire. As I understand it from others' comments here, the Coldfire ISA is a subset of that for 680x0s, so my very rough guess is that a backport would be feasible. To run natively, I guess there would be a lot of custom coding involved to get peripheral device support (ethernet, serial, video, SCSI etc)