My speeds drop but nothing that severe. Normally my speeds are from 9-10 megabits, and when I'm uploading they drop down to like 6-8 sometimes, or occasionally a bit lower but honestly it's hard to tell if it's even the upload that's slowing it down or not.
I use a wireless adapter rather than having myself plugged straight into my router, and it's a tiny thing and struggles to get more than 3 bars of signal, so speeds on their own where my computer is positioned end up only being like 1-2 megabits. So I made a little makeshift tin foil antennae for it and that gets the speeds all the way up to 10 megabits, but occasionally I'll bump the antennae when plugging in my camera prior to uploading and so the speeds will drop again. So it's possible the uploads aren't even slowing it down at all and it's just me plugging in my camera and messing up the antennae that slows it down for me.
Even so either way the drops aren't as big as yours, so that's odd.
One thing I'd be curious about - are these videos uploading as fast as they should be? i.e. like 20 minutes or less for a 1gb file if you're on a 5 megabit upload? Because if not then perhaps it's your internet being throttled. Some ISPs try to detect when people are torrenting and the like and your speeds get limited, so perhaps some ISPs wrongly do that for YouTube uploads as well and drop your speeds down?