Not ejecting sounds like a bad drive to me, possibly a bad IWM chip (it's socketed, next to the 2 ROMs).  1.44MB drives will work in a non-FDHD SE, you just won't be able to read/write 1.44MB disks in it.  I had a 1.44 out of an LC in my SE for years and years until I needed a 1.44 for another machine, which finally motivated me to clean and lubricate the original 800K drive and put it back in the SE.  While you probably don't want to dismantle all your Macs, you could try this drive in the Mac Plus or try the Plus' drive in the SE.