I remember going into the cinema to watch Rushmore with no expectations. I hadn't read any reviews, nor had I seen Bottle Rocket, director Wes Anderson's first film. But as soon as the opening scene started rolling, and we settled down to watch Max Fischer completing "probably the hardest geometry equation in the world" (before having that whipped from under our feet as we realise he's only dreaming), I knew I was going to love it.

For starters, there's the cast. Bill Murray. Here he's on magisterial form as self-made millionaire industrialist Herman Bloom ("Take dead aim on the rich boys. Get them in the crosshairs and take them down"). That's enough right there to know that this is a great film. He's mean, selfish and bullying, but also kind, warm and generous with his time and money. His comic timing is as brilliant as always – watch him fall over a fence, and you would think he had invented slapstick.

Jason Schwartzman – who has gone on to have a distinctly underwhelming career – had never appeared on screen before, but as Max he's perfect: a young boy we come to simultaneously adore and pity, run to and from. Max is obsessed with the eponymous Rushmore Academy ("I guess you've just gotta find something you love to do and then do it for the rest of your life. For me, it's going to Rushmore.").

He is captain of everything from the fencing team to the bee-keeping society, but is permanently on the point of expulsion because he is all endeavour and no grades. Even as his crazed passion for the school threatens to derail his life, we sympathise with him, and want him to come out on top.

Around these two leads orbits a cast that includes the preternaturally English Olivia Williams as Rushmore teacher Rosemary Cross, Seymour Cassel as Max's father and Brian Cox as the headmaster who comes to regret giving young Max a scholarship. There are also perfect miniature performances from Mason Gamble as Max's chapel partner (for a child actor, he deadpans lines like "with friends like you, who needs friends?" as if he had been born to play the straight man), and from Sara Tanaka as our hero's eventual love interest.

Rushmore is also something of a high-water mark for Anderson, who is the inadvertent godfather of an irritatingly quirky school of US cinema that grew out of the late 1990s and came to give us such annoying films as Napoleon Dynamite and I Heart Huckabees. I don't even really like his subsequent films, each seemingly more self-conscious and arch than the last. But here his direction is perfect. He doesn't force his characters to be weird. He just lets them be.

Rushmore is a film about obsession, and the relationship between Max and Bloom is the driving force behind the drama, going from love to hate, and back again. One is a 15-year-old boy, the other a 50-year-old man, but for the entirety of the film it's unclear who is the adult, particularly when they're competing for the affection of Ms Cross. At one point Max fills Bloom's hotel room with bees, to which Bloom responds by repeatedly driving over his love rival's bicycle, only to later find the brake cables on his Bentley cut. All this to win the heart of a recently widowed woman still struggling to cope with her loss. Incidentally, when you see Ms Cross teaching an art class, wearing an oversized man's shirt turned backwards and covered in flecks of paint, you understand the lengths to which they will go for her.

Rushmore is a film of these small, beautifully observed touches, with an unimprovable soundtrack arranged by Mark Mothersbaugh. The scene in which Bill Murray aimlessly throws golf balls into his swimming pool while drinking whiskey as he is forced to endure his wife flirting with her tennis coach, before performing a dive bomb off the high board, fag in mouth, tells us everything about his despair. This is all backed by Nothin' in the World Can Stop Me Worryin' 'Bout That Girl by The Kinks, which is just so right.

But nothing compares to the moment towards the end when Bloom learns that Max's father is not the brain surgeon he has constantly claimed him to be, but is instead a barber (it might be obligatory to preface that with humble, but there's no need here, for he is surely the humblest man to have had any job, anywhere, ever). Murray's face collapses for the barest of milliseconds, before recomposing itself. He immediately understands why Max has been lying, absorbs it, and loves him all the more for it. I've rewatched that scene numerous times, and I'm pretty sure it can't be beaten.

And then there are the set-piece school plays, which Max writes and directs like some kind of deluded Orson Welles. These are works of a quality surely never achieved in any actual school. His Serpico – complete with undercover nun – is a minor masterpiece. It is then followed by an excruciating scene in which Max drunkenly turns on a doctor friend of Ms Cross – played by Anderson regular Luke Wilson – who she has brought along to see the play ("I like your nurse's uniform, guy." "These are OR scrubs" "O R they?").

But the crowning glory is Max's Vietnam play, Heaven and Hell, which he makes in part as a eulogy to Bloom ("Were you in the shit?" "Yeah, I was in the shit.") Were it to be made for real it would surely rival Apocalypse Now for its insights into the minds of American soldiers. American soldiers faced by a schoolgirl sniper, obviously.

In truth, Rushmore is slight, and silly, and occasionally a little mannered. But few films are as generous to the geeks who never inherit the earth, and who struggle even to work out how to occupy their little corner of it. It ends perfectly for such an ensemble piece with the entire cast on stage, as the Faces sing, "I wish that I knew what I knew now, when I was younger". Amen to that.

• This article was amended on 15 November 2011. Due to an editing error, the film I Heart Huckabees was replaced with Little Miss Sunshine. This has now been corrected.