It might well be Hazel, that the soil is too rich. You could try moving them plant to a different spot with sandy soil. If you decide to try again try soaking your seed for 48 hours before starting. That gives them a better chance of germinating.
Taken from : http://www.southcoasttoday.com/apps/...14/-1/life0604
Perhaps finer than any listed here are the plants imported from England known as Russell lupines. Developed by George Russell of Yorkshire, after many years of selection and hybridizing, their flowering spikes are 2 ½ to 3 ½ feet high, the upper two-thirds of the spike a closely packed cluster of gorgeous bloom. Seeds are difficult to germinate unless nicked with a file, and if left too long in the soil without germination are apt to rot. For this reason, it is better to sprout them in pure sand or in posting mix.
Lupine like full sun to partial shade. They will grow quickly to a height of 2 to 3 feet. Lupine will do well in average soils and in cool weather. The soil should be loose and lime-free(they do not like lime). Keep the soil moist to feed their quick growth. Water them during dry periods, once or twice per week. Adding a general purpose fertilizer once a month before blooming will result in bigger plants and blooms.
Deadhead spent blooms to prolong the blooming period. Insects and disease are not a major problem. If problems occur, treat early with organic or chemical insect repellents and fungicide.
Chris - You are right, I am lucky. Where's your hubby? I'll put a bug in his ear!
And the being scared - I figure if it's meant to work out, it will. If not, try again. Doesn't mean I don't get upset. I do. But I can't change anything so I don't stress too much! My lantana will be a good size for next year too. I will have to buy more seed this winter and start more.