People trade historical items. It does not mean they sympathize with the people who owned them.
People own, and keep, and sell items which someone in their family got during a war (spoils of war) and the people who buy those things off them are collectors.
Collectors do not necessarily agree with the historical figures, laws, religions or people associated with the items that fascinate them. Seriously, MOST of the time a person is interested in a particular culture's weapons, battle dress etc. is not expressing that they adhere to those people's ideologies.
The real issues here are
a) people assuming the worst of others, which we can't help with
b) the idea that even knowing people do study and collect historical items, but you have a cut-off point which for you, personally, a line that "morally cancels out" historical atrocities, and the Nazis fall short of that line (say living memory, or 100 years or something) while eg: the ancient Greeks might have done stuff wrong but the physical items and emblems don't offend you (swastikas notwithstanding).
if it's b, that's totally understandable, as long as people understand that to someone interested in the history, that line may not exist or may be completely different to yours, and they may view it without the emotionally charged point of view you take.