No, you don't have to go far. I have a Mac LC whose ADB ports are dead because I accidentally hot-plugged them.
AFAIK, these are the ports not to mess around with:
- ADB
- SCSI
- power (
)- floppy disk
Again, AFAIK, these ports can be hot-plugged:
- video
- serial
Knowledge for the above list has come from personal experience, hear-say, and reading the Apple manuals.