Excellent points from Herschel. Just a little something else...when you associate good things, toys, yummy treats, whatever his favorite things are with being ouside, make sure to give him treats when he is not cowering, but when he's a little bit braver. Try to reinforce the feelings of confidence that he does have, however small they may be now. And yes, try to act normal and unconcerned. Also, I don't think once or twice a week is enough exposure to new people and dogs. I think every day a dog should get out and see something new and be sure it's a pleasant experience. Be careful with dog parks. You don't know all of those dogs and one might come up to your pup and frighten or hurt him which will ruin him. Try to find people with friendly dogs that you trust with your pup. A puppy kindergarten class with a reputable and gentle trainer who uses motivation and reward training methods, not scolding methods would be good for your pup as long as it's not overwhelming him.
Let us know how he comes along. Get him out and about even more, associating everything with as much fun as you can. His critical period for socialzation is about at it's end. Socialization must continue forever, but those first 3 or 4 months are vital to saturate him without freaking him out.
However, If he is terribly uneasy, frightened, freaking out, even in your yard, it is doubtful that getting out into the world to socialize is going to be very beneficial. If he can't get more comfortable in your yard very soon, I would speak to a vet about some kind of sedative. I don't promote drugs, especiall for such a small dog, but in some cases, if a dog can be a little calmer and relaxed, that enables him to soak in experiences and digest them properly and in a healthy way. If he's so terribly and extremely into his fight or flight drive all the time, he can't think. He's on auto pilot. His autonomic nervous system is what is taking over and his thinking brain isn't turned on. Phychological health is every bit as important or more so, IMO, as physical health. So for anyone who is paranoid about using some kind of beta blocker or sedative type thing temporarily, I would say to think again.
I do hope things can be turned around and he can be a well adjusted, brave little soul. Let us know how things go. I bet he's adorable. Any pictures?