My dog does that too, and it drives me nuts!! I was taking her to my brother's the other day, and took her in my backyard at least 3 times to make her go. Nada.
I get to my brother's and the first thing she did was take a humungous dump right in his front yard. I could have killed her.
If you're in the habit of taking your dog home as soon as he goes, he might very well hold it in. Also, it's much more distracting outside and like little kids, they don't want to stop long enough to go.
Usually my dog goes on cue if I take her to the same spot and tell her "go potty" because she generally realizes she isn't going anywhere until she does!