First, I wouldn't start trying to make a 6 year old dog get used to a crate. Remember, he was undoubtedly packed up in a crate for his trip, which couldn't have been a good experience.
German Shepherds bond incredibly closely to their people, and although he is already bonding to you, he still won't have completely gotten over losing his former home and masters. He's going to need a lot of understanding and work in the next few weeks. And you've got a language barrier to overcome, too, if his former owners spoke German.
It would be a good idea to keep him confined to an area that's not carpeted when you aren't with him.
Remember, he's not just eliminating in the house, he's marking his new territory, and he's probably somewhat insecure right now, so reinforcing good behaviour is going to be much more effective than scolding bad behaviour. Sure, tell him "No, not in the house," if you catch him in the act and take him outside, but then praise him when he finishes outside. It will also help with the territory concept if you will walk him around the perimeter of his yard several times. Then he'll naturally want to do his marking around the edges of the yard.
Typically, a dog will need to eliminate 15 to 30 minutes after eating, especially if he's been active or playing afterward, and he'll likely need to go out after a long nap, first thing in the morning, etc. Try to get a schedule started. No food after a certain time in the evening and limited water at night, just like you would with a child learning to use the toilet. And you're going to have to teach him the language just like you would a child. Tap the object and say the name; he'll learn quickly. When you take him for a walk, for example, tell him "go for a walk." That kind of attention will help him learn quickly.
He's a beautiful creature. That face is so typically thoughtful and dignified. Personally, I can't imagine not having a German Shepherd in my life.