The people mentioned above are memorable in their roles, but in my opinon many of them are just playing a stereotypical role or just mimicking a public persception of what a German military man was like and not a "real" German soldier with any real depth to their personality or character.
Having lived long enough to have seen many films from the all decades of the 20th century and having grown up watching them every weekend, another notable is Erich von Stroheim who appeared as Rommel in "Five Graves to Cairo" and Otto Preminger, the director, who appeared in "Stalag 17" as Colonel von Scherbach.
With that said, one man I've seen in many excellent war movies in the role of a German officer and doing so with great ease and without pretention is Wolfgang Preiss. He played the Major Herren in "The Train" opposite Burt Lancaster, and the German officer that takes over the POW train in "von Ryan's Express" starring Frank Sinatra. Preis also played a SS Major in the "The Cardinal" tv miniseries, Maj. Gen. Max Pemsel in "The Longest Day", Major Linkmann in the 1959 German film "Hunde wollt ihr ewig leben?" (Dogs, do you want to live forever?) which was remade into the 1993 film "Stalingrad", General Rommel in "Raid on Rommel", and "War and Rememberance" tv miniseries as General Walter von Brauchitsch as well as many other films and tv series including a single appearance in the 60's tv series "The Rat Patrol".
I think that a classic performance of a German officer by a non-German was James Mason in the role of Erwin Rommel in at least two films, "The Desert Fox" and "The Desert Rats". His performance was smooth and natural and not a comic book representation of a German commanding officer. He also played General Count von Klugermann, the husband of Bruno Stachel's (George Peppard) love interest Countess Kaeti von Klugermann (Ursula Andress) in "The Blue Max."
Karl Michael Vogler did a excellent job as the hard pressed Hauptmann Otto Heidemann, in "The Blue Max."
Ralph Fiennes portrayed Amon Geoth, the camp commander, in "Schindler's List" and did a excellent job. His portrayal of a murderous character was much convincing than that of Peter O'Toole in "Night of the Generals". I never quite bought into the premise of O'Toole as German officer in his films.
Jurgen Prochnow did a wonderful job as the U-boat commander in "Das Boot". I felt he did a great job of making you feel like he was a U-boat commander and not just an actor playing a U-boat commander.
As for enlisted men or junior officers, I know someone or two out there are going to mention "Cross of Iron" and James Coburn as Rolf Steiner. Even though Coburn seemed to give it a good effort, I never really could ignore the horrible German accent he affected and the his stiff portryal of a disillusioned German soldier. It all seemed too scripted and predictable and Maximillian Schell's protrayal of the medal hungry officer was just lacking something to me.
The cast of "Stalingrad", made in 1993, was a much better "average german soldier" movie than "Cross of Iron" and more believable. I have never seen the two previous versions of this same movie that have been released since the end of WWII so I can't compare the 93 version to those.
"All Quiet on the Western Front" did a good job of presenting the German soldier of WWI. I've seen the early version made in the 1930s and the 1979 made for tv version with Richard Thomas and Ernest Borgnine. I tend to prefer the later version but I could never get past the sight of Thomas as main character, Paul Baumer, and kept seeing him as "John Boy" from "The Waltons". For me, Thomas is forever typecast in that role. Borgnine did a great job as the "old soldier", Stanislaus "Kat" Katczinsky, that took Baumer under his wing and later became his friend.
There are many films made in Germany by Germans which I haven't seen that probably do an admirable job or representing the lives of German soldiers. Two of these, "Das Boot" and "Stalingrad", managed to make it to the U.S.
These are my opinions. Your's may differ.