The reason they're there is because after the sprues are made in the injection moulding machine, the two mould halves come apart and the sprue then needs to be pushed out of the half it usually sticks in. For this reason, lots of holes are drilled through the mould and steel rods inserted into them, so that the machine can push on those rods to eject the sprue. Hence "ejector pin marks".
You can see a block of them at 1:40 in this video, and see them do their thing at about 3:03:
As for removing them: no need to bother unless they'll be visible or will interfere with the fit of other parts. Some of the marks are recessed, in which case you can fill them with putty and then sand them down when that's dried. Sometimes, they're raised above the surface instead, which you can file, scrape and/or sand away.
.
