I think part of the reason Matrix is so weird is that it's trying to do two things at the same time that are exact opposites:

be an IRC replacement
be a Signal replacement

it kinda feels like being good at one necessarily makes it worse at the other? and the thing is I don't actually care about a chat system that's a Signal replacement, because Signal is, eh, it's fine; whatever

but the cryptographers who care about it being a Signal replacement are like "whoa; hang on! this thing isn't as good as Signal at being super state-actor-attack resistant!" and I'm like "you're probably right, because you're a mega-brain cryptography nerd, but on the other hand I don't actually care"