I wish I could say 'cheer up' but that would be easier said than done, right?

I think it's time you become really rude. Never mind that you are in Japan and they are not usually confrontational, but desperate times call for desperate measures....And since they're being that blunt and blatant about it, give 'em a dose of your medicine.

This may not be the best advice but....get even.

And DON'T move out. :)