To show you where it comes into play, I would say that your ability in a language roughly progresses as per the following graph.
To skip to the top of the skill chart, you've got the native level—which very few language learners ever obtain—and fluency. At the level of fluency, they may still be able to tell that you're not a native speaker, but you're language skills are so good that they assume that if you say something rude, you're actually being rude—as opposed to it just being a linguistic screw-up. And, since you're fluent, they're probably right.
Between proficiency and fluency, you've got what I've cornily designated "the danger zone". At this level, you can easily communicate just about whatever you want to communicate, and some fluent speakers will start to think that you are fluent as well. However, in reality, you're not, and some of the mistakes you're making—which may be considered to be rude—are merely just you mangling the language.
This is just where I find myself in certain areas of Japanese. In casual conversation, I might have punched through to fluency. However, in the formal business Japanese that I constantly need to use at work, I find myself smack in the danger zone; I'm probably just about good enough to fool some fluent speakers into thinking that I'm also a fluent speaker in this regard, but in fact I'm still learning to smoothly use the various polite forms that are necessary in these kinds of communications. I probably get some leeway simply because I'm clearly not Japanese, but I definitely need to be careful because once in a while I screw something up that would obviously be considered rude if said by a native speaker.
So that's the diagnosis, but what's the cure? Well, it's really no different than learning anything else in a language: exposure, exposure, exposure. I'm constantly hearing this language around me, so I'm pretty confident that I'll get it sooner or later, but for now I'll just have to struggle through and try not to offend anyone too horribly.