You’re not going to the game, are you?

Correct, but in casual speech you might appear annoyingly pedantic.

As I'm not speaking English as a native I can't say exactly, but it sounds right to me. Would be the way I use to answer..

I'm afraid the exact opposite is true. If you are agreeing with a negative question, you should answer no. If you are agreeing witha positive question, you should answer yes. If you are disagreeing with a positive, you answer no. If you are disagreeing with a positive, you answer yes:

Are you going to eat that? Yes I am / No I'm not.

You're not going to eat that, are you? Yes I am / No I'm not.

The negative ellicits the same response, i.e., since the question is now negative, yes I am is disagreeing, No I'm not is agreeing.

Note, the French (and other languages) have a separate word for disagreeing with a negative. Oui is yes. Non is no. Si is yes, but only in an answer that contradicts a negative question.

Does the question demand only a yes or no answer?
If not, why worry about giving an answer at all that is so potentially misleading and subject to misinterpretation?
One thing to be clear--another to stumble into deliberate confusion with lazy shorthand.
Rather, be sensibly specific and avoid the trap.
I am not going to the game.
or
I am going to the game.
That's not so difficult, now, is it?

correct

I would answer NO, in response to the "are you?" at the end.

Well, I'd really say "Why do you assume so much about me? Maybe I AM going to the game."

If your friend has a problem with you giving ambiguous answers, perhaps your friend should stop asking ambiguous questions. My Mom goes even further, asking things like "You're not going to the game, are you?"

Here's a clip that deals with this issue: http://www.youtube.com/watch?v=wpptlT-E9_E

If I was your friend, I'd stop asking you anything.

Seriously, once you've established that the answer you're going to give is misleading, why do you insist on giving it? Tag questions are always reversed polarity; "You're going to the game, are you?" is actually a statement that "You may think so, but, no, no, you're not."

You should answer with "Yes" if your statement is positive, "No" if it's not, and consider pragmatics rather than logic.