In your argument you intimated that the "friendly" use of the word in return should not be met with criticism. That it's use in black culture in this fashion leads to "confusion."
First, in the scenario you posed, nothing comes from the black guy saying "what's up my N" to the white guy. Nothing negative anyway. The same can't be said the other way around because that exchange doesn't exist in a bubble. That word holds weight that even your black friends approval can't shoulder.
Second, black people tried to take a word that de-humanized them and tried to pull off the mother of all re-branding efforts. Now the extent of their success or the continued use of the word in it's current state has nothing to do with being "part of the problem." (The problem being the racist undertones of even a friendly white guy using the N word in a passive manner i.e. Bill Maher)
...
btw, nothing against you, this is all in fair discussion.