Hey, Holocaust jokes are heilarious Anne Frankley so are most offensive jokes. There is humor in the dark shit.
Really though you are pretty much right about Milo's problem. Though to expand on it his problem was he liked jumping into any sort of multicar pileup in the political world. Gamergate, Ghostbusters, the Orlando shooting, and the whole election. It both helped and hurt him, it got plenty of supporters but plenty of detractors too that jump on him for anything.
Watching the video (atleast one version), it doesn't seem like he's advocating for pedophilia or the teenage version if you take him at his word that when he said "boy" he was using the term euphemistically to refer to a young man in a daddy/son gay relationship. In the Rogan video he points out that adults can think physically mature teens are attractive even when underage, that age is just a number and doesn't factor into your body and mind responding to an attractive person.
But assuming he actually wasn't advocating for pedophilia, the fact he's pissed off so many people makes it easy to twist his words and attack him especially with how hsitty the media is (see Pewdiepie or George Zimmerman for evidence of media's willingness to take things out of context or edit shit).
One thing I find heilarious about this is this kinda shows why protesting and advocating against Milo speaking was so damn stupid in the first place. If you give someone enough rope to hang themselves they are damn sure going to be knotting that noose. Milo in his own words hurt himself, through media manipulation or him being a kiddy diddler enabler, in a way that would be impossible if people shut him up like they wanted to.
Everyone, especially public figures, are their own worse enemy and will wind up fucking themselves over far more then any protest or riot or attacks of their free speech.
Why I say let the fuckers speak. Let Milo speak, let Trump tweet, let Spencer stay on his feet, let them tie their own knots.