DATEDIFF produces an error if the result is out of range for integer values. For milliseconds, the maximum number is 24 days, 20 hours, 31 minutes and 23.647 seconds. For seconds, the maximum number is 68 years.

To work around this issue you can do something like following

instead of calculating the difference in seconds, you can calculate it in minutes, then multiply it by 60