I think the joke is the joke that it's not a joke. If the fish was a human it would could be the tragic truth, the person has well intentions but the fish just has a REALLY bad day and doesn't want to be bothered, yet the person continues to try to be nice and get his attention while the fish is purposely ignoring him until he breaks.

Edit: If you want cool emoji like the fishy: search 'kaomoji' + 'The word you want an emoji out of'