I have 20 or so linux servers and I want to sync all of their clocks to a single NTP server, which is in the same rack and switch as my servers. Nothing is virtualized.

Our admins are having trouble getting the clocks on the various machines synced closer than about 500 ms. I would have guessed, and this post implies that we should be able to get the linux boxes synced to within 2 ms of of the source and each other.

Are my expectations for NTP unreasonable? Any hints as to what the admins should be doing/checking?

On a network with consistent latency, and properly configured polling it should be able to be very accurate. The default polling values may be too large for you. I know the default settings on Debian/Ubuntu result in 2-6ms offsets in my VMs, and 1-3ms offsets on my physical boxes.
–
ZoredacheNov 20 '10 at 0:17

1

Also see the related question about how to monitor the time offset. If you require your time to be accurate to >2ms you may want to periodically monitor all your hosts so you can be sure. serverfault.com/questions/183298/…
–
ZoredacheNov 20 '10 at 0:24

3 Answers
3

I own a hosting company and we do exactly this. Here is how we accomplish this.

To start with, you need a NTP master source. So one of your Linux servers will become the master. I would create a DNS A record called time.example.com (assuming example.com is the domain). This way, if your master moves you need not update the other 19 servers.

On the master server you need to have an appropriately configured ntp.conf file.

Here is what one of our master /etc/ntp.conf files looks like. Note, this is a data center with a private address space (RFC1918) using 172.17.x.x so you'll need to adjust accordingly. If you want more than one master, create more than one DNS A record each with different IP to get a bit of fault tolerance if so desired.

Use the ntpq command to see the servers with which you are synchronized.
It provided you with a list of configured time servers and the delay,
offset and jitter that your server is experiencing with them.
For correct synchronization, the delay and offset values should be
non-zero and the jitter value should be under 100.

Also on our client nodes, we have a rc script (/etc/rc.d/rc.local) that synchronizes the clock before starting the NTPD daemon. Here are the important parts... They are order dependent.

Synchronize the client's clock with the master time source
/usr/sbin/ntpdate -b time.example.com

Start the NTPD daemon allowing for large time adjustments during start-up.
/usr/sbin/ntpd -g -x

Finally, depending on your set up, you'll need to punch a firewall rule to allow your time.example.com master to reach the Public Internet over UDP port. Here is a typical and appropriately placed IPTables rule

Do you do anything to monitor your systems using a tool not running on the system? What is your average time offset?
–
ZoredacheNov 20 '10 at 0:20

Yes, we monitor using a number of custom scripts. Some scripts run on the clients that just make sure the NTPD daemon, among other things, is running and if it is found not running will attempt to restart it and if that fails, thing are logged and emails are sent. Data center time is tracked by external systems - a combination of Nagios and Munin for the most part.
–
KiloNov 20 '10 at 0:30

What is it in this answer that hints the OP's admins on what to investigate? Is it the multicastclient and broadcastdelay that is the key, then say so.
–
MattBiancoApr 4 '11 at 11:56

@MattBianco: No. We use a custom script for that. It is an implementation detail specific to the environment. To shed some light on this, the script runs on our masters and requests the time from each client node out there. It knows of the clients as all our hosts use central LDAP. This script is part of many we run as a daily "morning report" we call it. Variables in the script allow us to set thresh holds which we report out through a nagios dashboard.
–
KiloApr 8 '11 at 13:30

FYI: newer versions of ntp can use restrict source blah blah instead of the four lines for each pool server.
–
dfcJan 19 '14 at 17:31

I am not sure wheather you can achieve so much less time sync but correct configuration of the ntp server will make the servers sync almost 10-20ms that I have done on my servers. Minimize the drift time. Its not impossible to get that, but after setting up the NTP server and points all the servers to that NTP server and sync up time for the first time manually will reduce the time diff b/w servers.

We have over 700 servers and VMs in our 3 data centers and none are off by more than 1 or 2 seconds. Anything beyond 1-2 seconds is almost always due to a recent reboot, etc... Normal daily operations for us is everything sub-second synchronized.
–
KiloNov 20 '10 at 0:32

" NTP provides accuracies generally in the range of 0.1 ms with fast LANs and computers and up to a few tens of milliseconds in the intercontinental Internet." cis.udel.edu/~mills/ntp.html
–
RedGrittyBrickNov 21 '10 at 13:18