We are at an interesting point in the evolution of comics, comparable to the juncture reached by pop music in the mid-1960s, when academics started praising the Beatles' Aeolian cadences and comparing Bob Dylan to Keats. The comics scene, ignored by serious lit lovers for decades, has stockpiled a vast amount of fiction, much of it aimed at adults. Suddenly, that material seems to have reached critical mass, and even the snootiest readers have realised they've been missing something.

In this climate, reviewing new graphic work is both easier and more difficult. It's no longer necessary to convince people that comics can be more than Batman or the Beano. On the other hand, anything with any merit tends to get overpraised and is routinely spared the sort of critical scrutiny brought to bear on everything else, from a new Zadie Smith novel to the latest Star Wars flick. The mainstream press almost never measures a graphic novel's actual achievement against its unfulfilled potential. New converts, reluctant to show their cluelessness about the ninth art, merely parrot the publishers' hype.

Rutu Modan's Exit Wounds, published in 2007, is an emotional whodunit set in modern Israel, in which a taxi driver and a female ex-soldier try to discover the identity of a man killed in a bomb attack. It's a well-constructed tale, told with understatement and quiet insight - the graphic equivalent of a decent literary novel. I liked it but was uneasy about the extravagant accolades it garnered. Now, emboldened by those accolades, Jonathan Cape is issuing Modan's earlier work, a selection of shorter pieces that originally appeared in various anthologies as far back as 1998. The author's afterword, and interviews she's given elsewhere, make it clear that she regards these stories as steps towards artistic maturity, a gradual progression that ended in Exit Wounds. If that book was no masterpiece, does this mean that Jamilti and Other Stories is a ragbag of juvenilia?

Not at all. In my view, Jamilti is the more interesting book, and it raises important questions about current notions - held by both consumers and creators - of how comics for grownups ideally ought to function. The younger Modan is rougher at the edges, quirkier, more self-consciously arty. Apart from the most recent piece, "Your Number One Fan", there's nothing here that could easily be made into a movie. "The Panty Killer", a madcap thriller about a rash of underwear-enhanced murders in Tel Aviv, is wickedly witty, but the wit is in the artwork and its playful juxtapositions. Without the cinematic pacing that makes Exit Wounds a quick read, almost every panel invites a long, leisurely look. The fantastically dowdy dresses of the women and the blowsy wallpaper of their homes are riots of intricate design and colour.

The oldest of the pieces is "King of the Lillies", one of the first comics published by Actus Tragicus, the Israeli artists' collective of which Modan was a founder member. In this elegantly bizarre fable (set in a fantasised pre-war Sweden) the Lillies are a Sapphic troupe of women, surgically altered by a plastic surgeon in the image of his long-lost beloved. The influence of Edward Gorey is unmistakable, not just in the archaic style but also in the characters' eerie dignity in the face of grotesque calamity.

In time, Modan came to recognise that grotesque calamities - and the dark humour of denial - were in plentiful supply in her country. Her subsequent stories address Israel's mingled complacency and fear as the war with the Palestinians drags on. In the resonantly powerful title story, a nurse's chance meeting with a suicide bomber casts a shadow of ambivalence over her imminent marriage to her macho boyfriend. In "Homecoming", a senile father continues to believe that his son, shot down over Lebanon, will return. Then a plane starts to circle the kibbutz, and the family gather on the beach where the father has scrawled "welcome home" in the sand. Told in 30 full-page panels, "Homecoming" feels like a children's book that maintains a spirit of innocence in a terrifying world.

In "Bygone", Modan attempts to exorcise her own grief at losing both parents when young, through a queasily erotic fantasy about orphaned sisters running a spooky hotel. Drawn in monochrome, with up to nine panels per page to increase the sense of emotional claustrophobia, "Bygone" could have been an eye-straining mess, but Modan's highly developed skills in composition, and her gift for capturing psychological nuances with a few lines, make this not just readable, but involving and fun. The lost-parent theme turns up again in "Energy Blockage", a sad farce played out in the tacky world of quack medicine scams.

"Your Number One Fan", composed after the author's recent relocation from Tel Aviv to Sheffield, is a low-key, wry vignette about a wannabe rock star performing at a Jewish buffet in that city. Like Exit Wounds, it's drawn in a style Modan describes as "more realistic, more clean", reminiscent of Hergé's ligne claire; indeed, it has a somewhat Tintin-esque look. And, without the detective elements that enlivened the slowburn romance of Exit Wounds, this tale of mediocre misfits stays just on the right side of humdrum. "When I was young," Modan has said, "I wanted my work to be unique, so I made it surreal and grotesque. Now I find myself mostly doing the opposite: I make things more subtle than what really happened. I'm really not interested in exaggerating life any more." This mistrust of anything mystical or expressionist, and the aspiration to make one's drawing style ever more transparent and matter of fact, applies not just to Modan but to many other well-regarded comics creators of recent years, such as Chris Ware, Adrian Tomine, Alison Bechdel, Dan Clowes, Seth and others. Their cool objectivity and droll observation promote an almost documentary aesthetic. While this can yield impressive results, I can't help thinking that it represents a retreat from the limitless potential of handcrafted visual art. In her "mature" work, Modan produces indie movies on the page, just waiting to be discovered by Hollywood. In her earlier pieces, her art is bolder, stranger, more surprising. I hope that Jamilti enjoys enough success for Modan to reconnect with techniques she's supposedly outgrown, and be inspired to display the full depth and range of her extraordinary talent.