Well since it's next to impossible to actually get IE off a windows system without butchering other applications... I've always been stuck with it on the harddrive.
I rarely ever use it (IE) - like I said.. only when I hit a total roadblock with a non-Mozilla compatible site. I haven't run into any issues with having both installed on the system (as long as I have Firefox set as the default browser)... But running them simultaneously can get hairy - so Firefox gets shut down when I have to open IE.
(Oh - I didn't mention the other cool thing about switching to Firefox! As part of the installation you can automatically move all settings, bookmarks, etc. to Firefox from IE. (Same thing with Thunderbird... All outlook mail and addresses simply get moved over to Thunderbird) So there's no starting over.)