"you can't upload your own images DIRECTLY to the gallery"
Ok, if that's an issue to you (and was once possible), I see what you mean.
However, you can just use the visual editor's mass uploader and upload everything you want first, and then add it to the gallery the way shown above.
So the statement "it forces us to use source edit" is still wrong. Everything can be done directly in visual editor.
Nevertheless, I still think everything is more time consuming with ve than it is with source editor. Only exception I can think of is editing big and complicated tables. But that's maybe also me not liking visual editing at all, in general, and being used to do everything in source mode only.
Actually, in source editor you have even the possibility to add all uploaded images into a gallery right away, which seems not to be the case in source editor:
"if I type the EXACT filename in the source editor it literally just adds an image with a black background and the red link (which I typed) in the middle of it into the gallery, although I follow every step"
Never seen or experienced anything like this. If you can show me an example of this, I'm happy to look into that issue.
"I know that adding images somewhere else and adding it to the gallery isn't really a problem, but the bureaucrat in the wiki I'm active on considers it vandalism and deletes it."
What?? No matter how and where you upload images, they all end up in the same place, which is the File: namespace. There is zero difference whether you use Special:Upload, the mass uploaders of visual and source editors, or the "add new image" button on Special:NewFiles.
Why would an admin consider something as vandalism depending on the upload method? That doesn't make any sense, honestly.