No, that is not correct in regards to the earliest System software. And by the way, it is not "what I did" but rather "what I do." I still have two Mac 512k's with the stock 64k ROMs and the ability to run even the earliest System and Finders. The very early System software versions do NOT allow you to even "Restart" insofar as a menu selection is concerned. Nor would you return to the flashing "?" icon if you ejected the 400k internal floppy. As Mac128 points out, you should have first ejected the floppy before flipping the power switch. But the fact is, your computer was still up and running, in the Finder (not a black screen) when you flipped off that power switch, when running very early System software.
I think at this point we need to hear from Dookie, the man who started this thread. What System software are you using, Dookie?