2008Q2 Grant Proposal - Make localtime() and gmtime() Work Past 2038

Synopsis: Prevent the Y2038 bug from effecting Perl users. This means make localtime() and gmtime() work for times beyond 2038. Do this regardless of the state of the system C libraries. A working solution, in C, already exists. This grant is to finish the portability issues, write additional tests and change perl to use the repaired C library.

Name:
Michael G Schwern

Project Title:
Make localtime() and gmtime() Work Past 2038

Synopsis:
Prevent the Y2038 bug from effecting Perl users. This means make localtime() and gmtime() work for times beyond 2038. Do this regardless of the state of the system C libraries.

A working solution, in C, already exists. This grant is to finish the portability issues, write additional tests and change perl to use the repaired C library.

Benefits to the Perl Community:
Outlined here in detail. http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-02/msg00040.html

Many applications and modules make use of gmtime() and localtime() directly or indirectly to display dates or do date math.

Many applications have to work on dates 30 years or more in the future.

The problem will only get worse with time.

The C time libraries are being very slow to fix this well known problem.

Not everyone can just upgrade to 64 bit hardware.

Even DateTime is effected by this when it works with Unix timestamps.

Perl has traditionally shielded programmers from the limitations of C. The idea that time ends at 2038 is a ridiculous notion to anyone but a Unix C programmer. When localtime() returns the wrong date programmers will blame Perl, not C, resulting in another ding against Perl.

We can instead turn this into a PR win for Perl. Similar to the publicity for being one of the safest Y2K languages, Perl can be one of the first Y2038 safe languages.

Deliverables:
64 bit localtime/gmtime

Write C versions of localtime() and gmtime() which work with times beyond 2038 regardless of the limits of the system C libraries.

Test for negative times.

If it can be made to work, make it work.

If not, provide a proper error message.

Fix any performance issues so the process is O(1) (currently parts of the code are O(n)).

Make gmtime64_r() properly report an EOVERFLOW error when the year is too large to be held by tm.tm_year.

Project Details:
Perl will have to bundle it's own 64 bit clean localtime() and gmtime() implementations.

There is one OS specific component to localtime() which will need to be duplicated or gotten by other means and that is how to adjust for the current time zone. The scalar value of localtime() and gmtime() is not effected by locales. This problem has been solved. http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-02/msg00259.html

The usual sorts of system compatibility problems will need to be resolved. A discussion of the issues has been started here. Much of the information is already gathered by Configure. Andy Dougherty has said he'll work on the rest. http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-02/msg00277.html

localtime() and gmtime() will still be limited by the size of Perl scalars. For 32 bit systems this means NV limits which is about 2**47 or about 4 million years in the future. With 64 bit integer scalars it should be able to go all the way out to the limit of a signed long long or 2**63 which is something like 300 billion years. However, since we're still using the 32 bit tm struct and tm.tm_year is a signed int so the practical limit on 32 bit systems is about 2 billion years in the future which I'm happy with.

Permission has been gotten from the original author of the 64 bit localtime code to use and modify it in Perl without a binary attribution clause.

Project Schedule
I will be available to work on the project after June 20th (post YAPC::NA).

The first, speculative phase is already complete. This involved writing the 64 bit clean localtime/gmtime C functions which work on 32 bit machines and shoving it into the perl core.

Write a 64 bit localtime/gmtime which works on 32 bit machines. [COMPLETE See http://svn.schwern.org/svn/y2038/]

First cut patch adapting pp_gmtime() and pp_localtime() [COMPLETE]

Second phase is cleanup of the patch with perl to make it play nice with Configure and not leak memory, etc. This will require the assistance of p5p and those who know Configure, perl and C.

Make the patch play nice with Configure.

Fix any known and revealed cross-platform issues.

Fix any known and revealed bugs.

Extend existing core time tests to check for dates beyond 2038 and fix any revealed bugs.

At this point it works well enough to be shipped in bleapderl. This is a long back and forth process requiring lots of input from many people on many different machines. A lot of time will be spent waiting for feedback. It is anticipated this will take at least one
month, probably two.

Third phase is to improve the C functions.

Test for negative times.

If it can be made to work, make it work.

If not, provide a proper error message.

Fix any performance issues so the process is O(1) (currently parts of the code are O(n)).

Make gmtime64_r() properly report an EOVERFLOW error when the year is too large to be held by tm.tm_year.

These parts are relatively straight forward and should be solved in two weeks. They can be done in paralell with the 2nd phase.

Finally, backport into 5.10 and 5.8. This should be trivial.

The project is considered complete when maintperl contains a 64 bit localtime.

The earliest date for completion is July 26th. Latest is August 18th. It is not a coincidence that these correspond with the end of OSCON and YAPC::Europe. They provide opportunities to get some intense hacking done on any hanging issues that might be holding up the process.

Bio
Although my C skills are far from ideal for solving this problem, I seem to be the guy who cared enough about this problem to push it through and write a complete implementation. I work as an independent contractor and will be working half time for the summer in anticipation of working on grant work.

Because my expertise is not up to snuff in some places, I would request another $500 to be dispersed to folks helping out, mostly with the Configure and porting issues and necessary cleanup, to ensure the work gets done. Should this money not be necessary it will be returned.