Who would have thought that lyrics would be so popular? Certainly not the music publishers or record labels. Although printed lyrics, along with a cover and sleeve notes, have typically been incorporated ("by permission") within the price of an album, the digital revolution has in effect terminated the marriage between words and music. Buy a 79p download from iTunes and you receive nothing more than a naked digital file, along with the licence to play it; music stripped to the bare bones - the equivalent of a blank silver disc housed a clear plastic wrapper.

To fill the gap, a host of "unauthorised" websites (ie "without permission") has flourished - user-generated databases where consumers can search, edit and copy amateur transcriptions from other fans. These services might be rough and ready, not mention overloaded with annoying popups (or worse), but they are very popular, according to online monitoring company Hitwise: in the first week of May, 12 of the UK's 25 most-visited music websites were unauthorised user-generated databases of lyrics or guitar tabulatures.

Significant undertaking

Quite clearly, consumer demand for this kind of information is huge. Which makes it odd that it was not until April this year that US music publishers, with the help of digital media giant Gracenote, launched a legal alternative. Hosted by Yahoo!, the service will allow users to view lyrics from approximately 400,000 songs, and return to publishers a share of advertising revenue. According to Ross Blanchard of Gracenote, the project was a significant undertaking. "It has taken us several years and several millions of dollars to build up this system," he says. "Everything with publishing is more complicated than you think it will be."

The service is an important step for many publishers, and one that will legitimise a legal campaign against the owners of the kind of commercial site that Nicholas Firth, president of BMG Publishing Worldwide, describes as "egregious thieves". Lyrics might often be "free" in the context of buying a physical album, but in isolation they have traditionally provided revenue in the shape of sheet music, music books, folios and other licences. Those song lyrics that used to appear in Smash Hits all had to be paid for. There are also complaints about accuracy.

"I think we're trying to do this a little more softly-gently than some other sectors of the music industry," explains Ralph Peer, chief executive of Peermusic, an independent music publisher. "It's very important that we provide a comparable service before being too hard-hitting against those sites making money off the composers' backs. [But] there's going to come a time when unlicensed sites are going to have to pay the piper."

Such actions stretch back to 1996, when the US-based Online Guitar Archive was dissolved, while in December 2005, Lauren Keiser, president of the US Music Publishers' Association (MPA), the body that represents sheet music companies, ramped up the pressure, by suggesting that those running unauthorised websites should be thrown into jail.

However, even if publishers dissociate themselves from the heavy-handed actions of their colleagues in the Recording Industry Association of America in their pursuit of filesharers, they have at least followed the recording industry's lead by placing limitations on what consumers can do with the Yahoo! service, including disabling the ability to cut, paste, copy or print lyrics. "We regret the inconvenience," runs Yahoo!'s disclaimer, "but this is the only way at present to offer this content legally."

Blanchard notes that Gracenote was only supplied with a "tiny percentage" of lyrics first-hand, and actually had to employ a team of up to 100 people to transcribe them over a period of two years.

He describes the Yahoo! service as "first steps" and promises more innovation to come, but as it stands this is a seriously compromised service.

It also raises questions about the wider digital world. Pressuring large-scale commercial lyric sites is one thing, but if publishers are too under-resourced to digitise their catalogues, then how can they police the sort of fragmented online ecosystem where lyrics are located on fansites, blogs and social networks? And how can they change consumer perceptions of what is now commonly considered to be "free", and essentially is free in the offline world?

Yearly licences

Firth reiterates that publishers are not concentrating on these smaller players - yet - but remains bullish that they can employ what he terms a "graphic rate". "In answer to the question of why aren't we giving away lyrics free now, the better question is, why were we giving them away for all those years? We've looked at the huge demand and decided that this is an untapped income stream."

Experience, however, suggests that this will be difficult. When Warner Chappell, Radiohead's publisher, contacted Adriaan Pels, webmaster of fansite AtEaseWeb.com, in 2003 and requested he remove lyrics, Pels refused, arguing that the site directly benefited the band. "I was shocked," says Pels. "Lyrics for Radiohead are hard to find and Thom Yorke's voice is not crystal clear, so I spent a lot of time working on them. I think a fansite like I have is something that contributes to the success of the band. Even at the moment, when Radiohead is doing nothing, we still have 20,000 people a day coming to the site."

After intervention from the band's management, Warner Chappell backed down and announced it would be issuing "yearly free licences" to allow similar fan websites to post lyrics. Pels claims he is still waiting for this licence, but the inconvenience of employing the letter of the law is illustrated well enough.

"This is the problem that the recorded industry is facing," says Tahir Basheer at media specialist law firm Sheridans. "If consumers build up a way of behaving, trying to change that becomes extremely difficult. You can fill that vacuum, but you have to do it in a clever way, and you have to ask what the public is prepared to do to get those lyrics."

And surely the most practical solution here would be to embed lyrics into digital tracks and reinstate the relationship between words and music. Given that Apple's AAC files already have a vacant lyrics tab, and Gracenote supplies iTunes with its album and song data, such a move is already on the cards, says Blanchard. "That's definitely part of our deals with the publishers, and it's part of the systems that we built here and we're taking that proposal out to anyone who will listen."

Rather than antagonising music fans, this seems a safer long-term bet. Certainly, by removing DRM and offering tracks with higher sound quality, EMI has already raised the prospect of premium downloads retailing for 99p. If that offer could also include lyrics, liner notes and sleeve, then everybody can potentially benefit.

It's the basic law of digital business: give the consumer the sort of attractive and compelling product they want, and try not to legislate against what they want to do.

· If you'd like to comment on any aspect of Technology Guardian, send your emails to tech@guardian.co.uk