I think that both extremes have negative consequences.
Contrast is a tool for making more/less prominent what it is more/less important. If you make everything high contrast, or use a very similar grey palette for all text it becomes hard to communicate what it is more important and help users to perceive the visual hierarchy. Thus, the perceived complexity is increased.
they also make controversial choices like pure white text on a pure black backgroundAgree. Another interesting link on why to avoid pure black: http://ianstormtaylor.com/design-tip-never-use-black/