Honestly I don't think that's the case. My average views and viewer retention actually *increased* when I started monetising videos. The thing is, almost every major channel monetises their videos, so as a result, 99% of the videos anyone watches are monetised. Now some people like myself just use adblock and never see ads regardless - while those who don't, expect to see ads on almost any video. If they don't see ads on a video, rather than thinking "Great this guy is nice and hasn't monetised" - they just think "Ah I've probably watched enough ads in the last 15 minutes that now YouTube has decided not to make one appear on this video".
You should monetise your channel from day 1 in my opinion. There is no reason not to, and in all honesty I suspect YouTube search algorithms favour monetised videos over ones that aren't - after all, it makes sense from a business standpoint that YouTube would favour the videos it makes more money from over those that don't show ads or make them the same money.
It was after I started monetising my channel that my first video suddenly shot up in search rankings and started climbing in views (now has 26k, was under 1k before monetisation).
Tl;dr do it already. Enable all 3 types of ads on your videos too. Like seriously, if someone is bothered enough about those ads, they'll just install adblock anyway - so it won't matter.