Do NOT DILUTE the goat milk.
A decent puppy formula is goat milk with an egg yolk or two added, along with a tablespoon of white karo syrup and some baby vitamins.
At 2 weeks the puppy needs feeding every 3 hours or so around the clock.
You can speak to your vet about how to TUBE FEED the puppy if necessary. This is MUCH faster in the middle of the night, but must be done carefully to prevent the aspiration of the fluid into the lungs.
Stimulation of the elimination is also important.
by 3 to 3.5 weeks the puppy should be able to learn to eat gruel, so you only have about a week and a half of really hard work, but until 4 to 5 weeks it will still need to be fed a couple of times during the night.
I would not worry too much about socialization until 5 weeks or so. At that time if there are other puppies or good settled older bitches he can play with, that would be good.