Magic Erasers only cause damage if used without sufficient detergent/lubrication in my experience - just take it easy, don't push it down hard and you'll be fine.  Forgot to mention in my other post, when cleaned sit out in the sun to dry.