Nonbinary refers to any gender that is not exclusively male or female. A similar term is genderqueer.
Some common nonbinary gender identities include: agender, bigender, genderfluid, androgyne, and neutrois.
It is usually considered to be under the transgender umbrella, and some nonbinary people will also identify as trans.

Fallon: You're not a girl or a boy, are you?
Skylar: Nope, I'm not! I'm nonbinary.

Usually under the genderqueer or transgender umbrellas. when someone's sense of gender is not male, female, or something in-between (androgynous). Includes neutrois, agender, genderless, gender-neutral, and other senses of gender devoid of male or female aspects.

They don't identify themself as male nor female, perhaps they are non-binary?

A term used and almost exclusively coined by Tumblr frequents and Feminists, another entry in the 'I don't identify by your standards' blanket, meaning that asking whether someone's male or female will be met with overwhelming hostility due to the exclusive nature of being a modern internet Fascist.

Typically if a person feels the need to include this term in a description of themselves, you can be assured their attitude is dictated by the hypocritical menstrual cesspool that is Tumblr. Expect Daddy issues, self-diagnosed depression, borderline Psychotic defensiveness (known as being 'triggered') and a general underlying feeling of extreme unpleasantness if you happen to be a stereotypical white male.

'I'm a Non-Binary Transgendered Race Horse and offering me that bale of hay is essentially rape'

"So are you a boy or a girl?"
*Receive 50 anonymous messages telling you you're an awful person and part of the problem*