This file can sometimes span up to 45k in entries for client records. My first issue was that I needed to figure out how to diff these files generated let's say hourly and only pick up the differences since it would take a long time to clear out the ldap database and re-add everything to keep the database fresh.

Aka: Generate 1.full then 2.incremental, diff the two files and create 1.diff. Then apply the diffs via ldapupdate. I was using ldifdiff.pl for the task. I believe it's a perl module.

So that was working for a period of time until we introduced hashed passwords. That screwed up everything since now each time the sync occurred a new SHA password would be computed for the incremental file.

Back to the drawing board...

To solve that issue, I added a field called "lastUpdate", this would print the time that a record that we were pulling from the database was updated.

My logic was now I can somehow just parse the two files and only print out the differences if this "lastUpdate" field was modified.

Well, since then I've been spinning my wheels on how to accomplish this. Some ideas I've been throwing around in my head are to:

1. Load everything into a hash and diff the values there? 2. Load the two files and extract the lastUpdate field between the two files and print out the blocks of records that have had their update times modified? 3. I also thought about modifying ldifdiff.pl and seeing if I could just add a conditional to check if lastupdate has been modified between the two files.

Still unsure so throwing it out here to see if anyone could offer some advice.