After a quick search, I found out that the term for what you’re describing seems to be gender apathetic. I’ve met people who seemed a bit apathetic to gender before, so it makes sense that there’s a label for it.

Oh, and don’t forget that non-binary people that use ‘he’ and/or ‘she’ aren’t necessarily gender apathetic. They could, for example, be demigender.