The "wormy looking things" could be any number of things. There are hundreds of marine based worms that could be found in your system. Without photos, or at least a better description, any attempt at an ID would be a complete shot in the dark.

Can you be a bit more descriptive of the "changes" you are observing in your live rock? Generally, when you buy live rock, It is quite "dirty" looking. Once it's been introduced to your system, Fish, Snails, Crabs, Starfish, and various worms will begin to feed off of the "dirty" stuff (mostly dead hitchhikers and coral, plant life, and detritus). Over a relatively short period of time, the rocks will start to take on a cleaner appearance. If conditions are proper, you will also start to see coralline algae growth on your rocks. Eventually, you rocks will become well covered in nice purple and/or red coralline. It is possible to see hitchhiker corals emerge from the live rock, and providing they are kept in proper conditions, it is possible for them to thrive. The downside to this is, in most cases, the hitchhiker corals are plain and unattractive. The corals you see in most aquariums are usually either glued to their rocks, or strategically placed/mounted so that they can grow and take foothold to the rocks.

Corals are very delicate, sessile (in most cases), invertebrate animals (that's right, animals, not plants) that have special needs.
Corals require proper lighting, as most feed off of zooanthelle algae that grows within their bodies.
Corals require constant flow (current) within the tank to keep them clean, shed old "skin", and to bring planktonic food from within the water column to them.
Corals require proper levels of calcium, bicarbonate, magnesium, and other ions for continued growth and health.
Corals require impeccable husbandry as water changes are needed not only to remove nutrients, but also to replenish the major and minor ions in the saltwater.
Some corals even require spot feeding for continued survival.
And the thing that perhaps makes it seem difficult, is that different corals require different levels of all of the above.

But essentially; No, corals are not difficult to care for. They require more attention, and better understanding. They require a more devoted hobbyist, as Reef Tanks require daily attention.

sk-just checked out your tank photos, your salt tank is incredible, i love it. i am just working with a 29 gallon, but am working my way up to 55 or 75g. just taking it slow and learning everything i can. i also loved your cichlid tank, that is actually where i came from-we've kept cichlids for the past 5 years. i really had a hard time with the aggressiveness of the cichlids...but i loved their personalities at the same time-they are so interesting to watch.

As for the bristleworms, contrary to what you may have read, the common Bristleworms (pink front end, grayish-black tail end) are very beneficial to your system. If you still have the little guy, I'd toss 'em back into the tank. They are scavengers and often take care of dead snails, crabs, and fish that may otherwise have gone unnoticed.