While it is true that you have an SE whereas I have an SE/30, the body is still the same (i.e., no vents on top like the Plus and earlier). And I too removed the internal HD in my SE/30 to decrease the heat. Even so, I still feel HOT air blowing out the back (via my internal fan). Therefore, if you have no fan and no vents on top, where does that heat go? Answer: only a small amount leaks out and the rest remains inside your Mac, gradually making it hotter and hotter until one of your components goes, taking the rest of the machine with it.

So how does one resolve this problem? Easy. Just replace the noisy fan with a quiet one. In my case it was easy. I already had a stock "boxer" type Elina fan in the SE/30 so it was easy to choose another 60mm fan from Silenx to give me about the same air flow but at 1/10th the noise levels. If you have a cylindrical fan, your mounting bracket may not accommodate a boxer fan easily so it may not be an easy fix like my Mac. But I do know that some SE's came with a boxer fan so those models are easy to swap it.

Simply put, don't remove the fan and run your Mac 24/7. Indeed, I wouldn't run it more than 3 hours straight in that condition. If it had vents on top, I would say go for it, as the heat would rise up and out those vents. But for SE's and SE/30's, that heat will not escape, I am sorry to say.

Hope this helps.