Last night, one of our admins accidentally replicated the R8 pilot server address book to a R6.5.6 production server address book through his notes client.

This royally screwed up the 6.5.6 address book. Some of the servers rebooted themselves, mail wouldn’t route, dc.nsf appeared to be out of order, etc.

We fixed it by manually replacing the address books on the bad servers at the file system level with a good copy from another server that hadn’t replicated the bad design yet. Another option to keep open is to keep a local replica of the NAB on all of your admin clients, and set your admin clients only to replicate with the server ever hour, 2 hours, or 4 hours. This way, the local replica copy of the NAB on the admin clients is pretty much up to date, but if something happens, it gives you a bit a time to discover that something is wrong. It’s not a perfect solution, but good (possible) option to have in place when you notice that something is very very wrong. A scenario might be that you discover your NAB is bad, and your local admin client hadn’t replicated for 3 hours and does not have the bad design. This can save you.

Of course, there is always the backup from the night before, but then you’ve lost all of your changes from the morning or earlier in the day. In busy environments, this can be alot of work.

When we came in this morning, some users were complaining that the dc.nsf wasn’t accessible. Upon further investigation, what happened was the when the admin replaced the NAB at the file system level, they didn’t delete the names.ft (full text index directory files) and re-create the full text index of the NAB. This somehow interfered with the dircat process which runs on one of the servers and updates dc.nsf every 4 hours.

When we searched some of the NAB for a person’s name, the results would produce 10 or 15 names and not the name we searched for, and other oddities.

After we manually deleted the full text indexes and re-created them, things went back to normal.