Slowly but surely is the way to use a magic eraser - a little bit of water and detergent also helps, as it is basically a light abrasive. Yes, if you use it too firmly, it will kind of make a polished/smear mark on the plastic. Toothpaste may also work well here (and gives you that minty freshness we all know and love)