Of course there is nothing wrong with turning people away from petting your animals. They are your dogs and its not your obligation to allow other people to pet them.
From the perspective of socialization it would be good for your dogs to get to experience interacting with other people especially for your rescue to overcome is reservations with strangers but you are correct that this socialization should not be forced upon him. Many children do not understand how to approach dogs properly and if you feel uncomfortable with them around your dogs its probably best from a safety perspective to politely decline their interactions. But definitely do not reject a good opportunity with polite people who are willing to listen to your conditions for the interactions. If the people are willing to lower their level and allow your dogs to come to them that is always a step forward to increasing socialization, but being smothered by strangers is definitely a step backwards.