The music begins at the limit of audibility, the barest whisper of bow against strings. A voice enters, singing just about as high as the adult male voice can go. The syllables are so extended that it is impossible to tell what language is being sung, while the halo of strings around the words further obscures their identity. As other voices join in, then brass and percussion, the music continues its stately progress towards meaning. Eventually, the ear makes out a word, then another, eventually the whole prayer: "Oh Lord, of Thy heavenly bounties deprive me not."

This is Litany, a setting of 24 prayers, one for each hour of the day and night, and the composer of this austere work is Estonian-born Arvo Part. It is deeply religious music that is filled with awe, not to say dread, as if this vengeful God might indeed deprive the composer of "heavenly bounties". Before that terrible prospect, the music seems reluctant to break the surrounding silence, only doing so when faced with the absolute need to speak. In that sense, it resembles the composer, who talks about himself, and his music, with extreme reticence. "While I'm writing a work, it's important not to talk about it," he says. "Otherwise, I lose the impulse to write it. And when the work is written, there's nothing more to be said about it."

Yet, if he is a private man, that does not make him an ivory-tower composer. On the contrary, he enjoys working closely with performers. The solo voices in Litany are the Hilliard Ensemble, who have consistently championed Part's work over the past 15 years (they perform Litany on next week's Contemporary Music Network tour with other long-term Part champions, the conductor Tonu Kaljuste, the Estonian Philharmonic Chamber Choir and Tallinn Chamber Orchestra). The purity of the Hilliard voices is special, Part acknowledges: "If, when I work with musicians, the contact is creative and constructive, then I allow myself, on both a conscious and an unconscious level, to be influenced by the interpreters. When I am writing for the Hilliard Ensemble, of course I hear their individual voices, it's impossible not to. It's the nuances of their voices that give the piece its sound, and I absorb these influences, these radiations, on a physical level."

Talking to journalists comes less easily, but Part is not an obstructive interviewee. I have been fortunate enough to meet him three times: in Stockholm, attending recording sessions of his music; in Norwich Cathedral, when the choir was rehearsing the premiere of his I am the True Vine; and in Estonia, where Kaljuste and his choir were performing Part. On each occasion, he has been scrupulously careful in his choice of words, but also kind, warm, even playful. At the end of the Stockholm sessions, there was a celebratory meal at one of the city's finest restaurants. While those about him enjoyed the chef's specialities, all laid out in drizzle-on-a-plate splendour, he settled for soup and a salad, but at the end of the meal, acknowledging its artistry, he conceded: "A music student could learn more from this meal than from a bad teacher." This from a man who confesses that when the composing is going badly, he peels potatoes.

The verbal parsimony, the monastically abstemious meal, the peeling of potatoes: it is tempting, though not necessarily accurate, to read it all in biographical terms. Part was born in 1935 in Estonia, which became a Soviet Socialist Republic in 1940. He grew up under Stalinism, and his musical training bore the hallmarks of Soviet orthodoxy, with which his own ideas were at odds. Part has since denounced the "sterile democracy" of 12-tone music, but in 1960 he composed Nekrolog, a serial work dedicated to the victims of fascism, and one of the pieces recorded at the Stockholm sessions. Listening to the orchestra from the control booth, he made copious annotations in his copy of the score, occasionally passing comment. "There's too much piano there... This is nice... Why do young people like such loud music?" It is as if it is the work of a stranger, rather than his younger self. Later, I asked how he felt about such pieces: "I don't listen to my early music. It's difficult to talk about. Let me put it this way: it's my child, but as an adult. We are both adults."

It is not surprising that he seems distanced from the piece - his music has long left behind the anguished expressionism of Nekrolog. In the mid-Seventies, immersion in the world of Renaissance music acted, he says, like a "midwife" for what he calls his "tintinnabular" style, a devotional music built on back-to-basics tonality in which a few notes, stretched to the limit with little variation, might be made to speak volumes. Old music made new? New music made old? Part's tintinnabulations have been called both, but it is music only he could have written, and only after experiencing the polarity of serialism and Renaissance polyphony. His work has developed in the intervening decades, but a 1994 piece such as Litany is recognisably the music of the same composer who in 1977 wrote Tabula Rasa, which had a galvanising effect on a new generation of Estonian composers, and on audiences and composers much farther afield.

If Part's serialism found little favour with the Soviet authorities, his tintinnabulations, so overtly religious, were also denounced (shades of the "holy minimalism" tag that has dogged his work in the West). Increasingly frustrated, Part renounced Soviet citizenship in 1980, emigrating first to Vienna, then to Berlin, where he has lived since 1982. If he is reticent about his early music, it is impossible to get him to talk about his life in Estonia. Since the country reclaimed its independence in 1991, he has returned several times, and last autumn, in Tallinn for Tonu Kaljuste's performances of his music, he was greeted as a hero. National TV sent camera crews to his press conference, the results appearing on the evening's main news: can you imagine that happening here? Journalists from Britain, Sweden, Finland and Germany as well as Estonia were there. An Estonian translator enabled Part to speak in his native tongue, and he was noticeably more voluble than when painstakingly assembling answers in German or English: at one point he denounced the "paparazzismus" that drives adoring researchers to hunt out his unfinished juvenilia.

Yet still he would sometimes ask his wife Nora what it was he wanted to say. Talking to Part earlier this week, I could hear him occasionally asking questions of someone else in the room, helping him avoid this or that verbal trap. I asked if he felt himself to be an Estonian composer: "I am Estonian, about that I have no choice, but I don't know what the significance of that might be. I am who I am. I don't identify my music with any country. Maybe that's something specific to music, or at least to the way I write."

What about his influence on Estonian music, an influence many of the country's composers readily acknowledge? He is not sure. "I don't know whether my music is significant for Estonian composers, sadly I don't know enough of their work," he says. "I used to have a greater global view of Estonian music than I do now, when I'm farther removed from it, but I know that the situation is now more open, especially towards the West, and that's very fruitful. The old times have come back and I'm happy that my Estonian composer colleagues and their music are attracting greater interest from the public in the West."

Which brings us to next week's tour, when Part's Litany and Trisagion will be juxtaposed with works by Erkki Sven Tuur, perhaps the most interesting of the next generation of Estonian composers. The tour opens in Durham Cathedral, and Part has strong opinions about performing his music in church as opposed to concert halls. "In concert halls, the acoustic and architectural experience has become problematic, and if you write for a modern concert hall, you have to write differently because the acoustics are different. Cathedrals have a naturally beautiful acoustic, very human but with a touch of God."

You don't, of course, have to feel the "touch of God" to be moved by Part's music. Its purity transports believer and non-believer alike. As the fourth prayer in Litany has it, this is music that, whether in concert hall or cathedral, "free(s) me... from despondency and stony insensibility".

The Contemporary Music Network tour of music by Arvo Part and Erkki Sven Tuur opens on Tuesday, 7.30pm, Durham Cathedral (0191-374 3210), then Huddersfield Town Hall on Wednesday (01484 430528), and QEH, London on Thursday (0171-960 4242)