One of my cats used to do that too. I don't think I would clap my hands or startle him to get him to leave as he may dig in his claws trying to get away fast. I would let him come up on my shoulder and then sit down on a sofa or let him walk off onto a cat tree shelf.
"We humans are indeed fortunate if we happen to be chosen to be owned by a cat." -- Anonymous