No. The upside down face 🙃 can add irony, awkward humor or a playful reaction to a mistake. It may express frustration without being aimed at you.
Someone writing “I put the milk in the cupboard again 🙃” is probably making fun of their own mistake. After an argument, it may feel sarcastic instead. The full message gives you more to go on than the emoji alone.
Related question: Is the slightly smiling face emoji always passive-aggressive?
Reference: Emojipedia: Upside-Down Face.
