You can also MASSIVELY cut down on heating costs with those insulation things you put around doorways and windows - if anyone knows what I'm talking about and has a link, please share.
Space heaters can work really well - my room is very poorly insulated and gets a ton of wind, compared to the rest of the house, so if the rest of the house is at 70 degrees my bedroom is a nice chilly 56, and the space heater I run for an hour or two before bed and it's nice and toasty. No idea what running one as a main heat source would do to an electric bill, but I don't think it's too bad.