That would be a SCSI port. I have a Macintosh ED with similar upgrades - two Dove MacSnap add-on cards that add extra memory and a SCSI port. Yours might be from a different brand, but the idea is the same.
But that's a really nice find. Talk about classics, the 128k Mac is the mother of 'em all. And you bought it for a very good price too, I'd say this model is relatively rare, especially in good working condition, so its collector's value is pretty high. I would almost advise you to remove those upgrades and bring it back to its original state, just out of principle.