Could be anal glands. When Phoebe starts doing this, I give her a handful of All Bran cereal with her food for a couple of days and it usually stops. If it doesn't stop, then it means her glands need to be expressed, so I take her to the vet to have it done. You could try doing it yourself, but ...eewwwww! The smell and the mess are enough for me to pay the few bucks to let someone else do it.
You probably don't want to scold him for doing it indoors. It's not a behavioral thing, it's a physical issue, like scratching an itch. You'll want to keep an eye on things whether he's doing it inside or out.