Is it normal to not speak up? Watch
I had the same problem with two of my friends I'd chilled with (brother and sister) and sometimes they'd start chatting in their own language while I'm literally in the middle of them ??? I finally told the sister about it and she stopped doing it, it was such a relief.
I'd mention it to him if it upsets you. Hopefully he will understand.