Nick Hodges

Fun With Testing DateUtils.pas #6

13 Apr

Okay, so when we last left off, IncMillisecond was still failing in certain circumstances.  Let’s take a look at that.  Note, too, that I have this crazy notion that if you have a function called IncMillisecond, then it should be able to, you know, increment a millisecond. :-)

Here is the IncMilliseconds that you very likely have on your computer:


function IncMilliSecond(const AValue: TDateTime;
  const ANumberOfMilliSeconds: Int64): TDateTime;
begin
  if AValue > 0 then
    Result := ((AValue * MSecsPerDay) + ANumberOfMilliSeconds) / MSecsPerDay
  else
    Result := ((AValue * MSecsPerDay) - ANumberOfMilliSeconds) / MSecsPerDay;
end;

Now that probably works just fine for you — as long as you don’t have a date that has a value less than the epoch. Below the epoch, and particularly in that magic "48 Hours" area right around the epoch itself, things go horribly awry. As we saw last time, this test will fail:


  TestDate := 0.0;
  TestResult := IncMillisecond(TestDate, -1);
  Expected := EncodeDateTime(1899, 12, 29, 23, 59, 59, 999);
  CheckTrue(SameDateTime(Expected, TestResult), 'IncMillisecond failed
    to subtract 1ms across the epoch');

It fails because of a number of reasons actually. The first is precision. The current implementation of IncMillisecond does division using a very small number in the denominator.  In the case of this test the numerator is a really big number multiplied by a really small number.  All of this cries out “precision error!”. (You should thank me – I almost used the <blink> tag there.  Phew!)  And that is basically what happens.  IncMillisecond isn’t precise enough to “see” the difference.

Plus, if you do things around the value of zero, it gets really weird.  For instance, check out the output of this console application:


program IncMillisecondTest;

{$APPTYPE CONSOLE}

uses
  SysUtils, DateUtils;

var
  TestDate: TDateTime;
  TestResult: TDateTime;
  DateStr: string;

begin
  TestDate := 0.0;
  TestResult := IncMilliSecond(TestDate, 1001);
  DateStr := FormatDateTime('dd mmmm, yyyy hh:mm:ss:zzz',  TestResult);
  WriteLn(DateStr);
  TestResult := IncMilliSecond(TestDate, -1001);
  DateStr := FormatDateTime('dd mmmm, yyyy hh:mm:ss:zzz',  TestResult);
  WriteLn(DateStr);
  ReadLn;
end.

I think it is safe to say that something is amiss.

So finally, it is time to rework IncMillisecond, because this pesky little routine is actually at the heart of a bunch of issues with DateUtils.pas. As it will turn out, if you call any of the IncXXXX routines, it all ends up as a call to IncMilliseconds, so this needs to be right.

Okay, so I started out writing this really cool implementation that checked for before and after the epoch, and divided large increments into years and months and days to make sure that their was no loss of precision.  I spent a lot of time on it, and had  whole bunch of tests written and passing with it.   But then it suddenly occurs to me that the trusty TTimeStamp data type and its accompanying conversion routines can once again come to the rescue:


function IncMilliSecond(const AValue: TDateTime;
  const ANumberOfMilliSeconds: Int64 = 1): TDateTime;
var
  TS: TTimeStamp;
  TempTime: Comp;
begin
  TS := DateTimeToTimeStamp(AValue);
  TempTime := TimeStampToMSecs(TS);
  TempTime := TempTime + ANumberOfMilliSeconds;
  TS := MSecsToTimeStamp(TempTime);
  Result := TimeStampToDateTime(TS);
end;

And here is the cool thing:  I was able to change from my sweet but overly complicated version to the new version above without worrying too much about it, because when I made the switch – all of the tests that I had written for my original version still passed.  This was so cool – I could make the change with confidence because of the large set of tests that I had that exercised all aspects on IncMillisecond.

Anywhow….  Again, the TTimeStamp type is precise, and easy. No need to do direct arithmetic on the TDateTime itself. Instead, we can deal with integers and get the exact answer every time no matter how many milliseconds you pass in. You can pass in 5000 years worth of milliseconds, and all will be well. For instance, this test passes just fine.


TestDate := EncodeDate(2010, 4, 8);
MSecsToAdd := Int64(5000) * DaysPerYear[False] * HoursPerDay * MinsPerHour *
    SecsPerMin *  MSecsPerSec; // 1.5768E14 or 157680000000000
TestResult := IncMilliSecond(TestDate, MSecsToAdd);
Expected := EncodeDate(7010, 4, 8);
ExtraLeapDays := LeapDaysBetweenDates(TestDate, Expected);
Expected := IncDay(Expected, -ExtraLeapDays);
CheckTrue(SameDate(Expected, TestResult), 'IncMillisecond failed to
   add 5000 years worth of milliseconds.');

And for you curious folks, here the implementation for the helper function LeapDaysBetweenDates:


function TDateUtilsTests.LeapDaysBetweenDates(aStartDate,
       aEndDate: TDateTime): Word;
var
  TempYear: Integer;
begin
  if aStartDate > aEndDate then
    raise Exception.Create('StartDate must be before EndDate.');
  Result := 0;
  for TempYear := YearOf(aStartDate) to YearOf(aEndDate) do
  begin
    if IsLeapYear(TempYear) then
      Inc(Result);
  end;
  if IsInLeapYear(aStartDate) and
     (aStartDate > EncodeDate(YearOf(aStartDate), 2, 29)) then
    Dec(Result);
  if IsInLeapYear(aEndDate) and
     (aEndDate < EncodeDate(YearOf(aEndDate), 2, 29)) then
    Dec(Result);
end;

From there, the rest of the IncXXXXX routines are simple –- they merely multiply by the next “level up” of time intervals, and call the previous one.  I’ve marked them all inline so that it all happens in one need function call.  Thus, we have:


function IncHour(const AValue: TDateTime;
  const ANumberOfHours: Int64 = 1): TDateTime;
begin
  Result := IncMinute(AValue, ANumberOfHours * MinsPerHour);
end;

function IncMinute(const AValue: TDateTime;
  const ANumberOfMinutes: Int64 = 1): TDateTime;
begin
  Result := IncSecond(AValue, ANumberOfMinutes * MinsPerHour);
end;

function IncSecond(const AValue: TDateTime;
  const ANumberOfSeconds: Int64 = 1): TDateTime;
begin
  Result := IncMilliSecond(Avalue, ANumberOfSeconds * MSecsPerSec);
end;

One thing to note: DateUtils.pas will only handle years from 1 to 9999. TDateTime won’t handle any date less than midnight on January 1, 0001 nor a date larger than December 31, 9999. So if you are using Delphi to track specific dates in dates before that (or if you plan on doing some time travel into the far future) you’ll have to use some other data type to keep track of dates.

Now, once you’ve done the above, it is tempting to say “Hey, for IncDay, I’ll just add the days to the value passed in.  I mean, that’s all you are really doing.  Well guess what!  You can’t do that!  If you have this for your IncDay:


function IncDay(const AValue: TDateTime;
  const ANumberOfDays: Integer = 1): TDateTime;
begin
  Result := AValue + ANumberOfDays;
end;

Then this test will not pass because of the strange “48 hour” deal we talked about last post:


  TestDate := EncodeDateTime(1899, 12, 30, 1, 43, 28, 400);
  TestResult := IncDay(TestDate, -1);
  Expected := EncodeDateTime(1899, 12, 29, 1, 43, 28, 400);
  CheckTrue(SameDate(Expected, TestResult), 'IncDay failed to
    decrement one day from the epoch');

Instead, you have to send it all the way back to milliseconds via IncHour, IncMinute, and IncSecond:


function IncDay(const AValue: TDateTime;
  const ANumberOfDays: Integer = 1): TDateTime;
begin
  Result := IncHour(AValue, ANumberOfDays * HoursPerDay);
end;

Once you put those changes in, well, things get a lot greener.  I have now written a very thorough set of unit test  for testing all of the IncXXXX routines, adding and subtracting dates for both before and after the epoch.  I also test very carefully incrementing and decrementing across the epoch and inside that crazy little 48 hour spot.  They are all passing.

I’ll create a unit with these new fixes in it that you can use if you want.  I’ll also publish the unit that includes these tests that I’ve written.  (When you look at it, be nice.  It’s not very pretty, but it gets the job done.)  As I continue through, I’ll update that file with any other fixes and changes that get made.

10 Responses to “Fun With Testing DateUtils.pas #6”

  1. 1
    I am new to blogging. How do I add a subscribe function to my site so new post will go to their email? | G2Roms.com Says:

    [...] Nick Hodges » Blog Archive » Fun With Testing DateUtils.pas #6 [...]

  2. 2
    Mason Wheeler Says:

    I believe you mean incrementing and *decrementing*, not deprecating?

  3. 3
    Miles W. Says:

    I hope you meant ‘decrementing’ instead of ‘deprecating’ :)

  4. 4
    Nick Hodges Says:

    Doh!

    Fixed. Thanks, guys.

  5. 5
    Jarrod Hollingworth Says:

    Hi Nick,

    Your new implementation for IncMilliSecond will break existing code. It needs to handle sub-millisecond date-times like the previous one by preserving the sub-millisecond component of the input.

    With current dates TDateTime can hold date-times accurate to around one microsecond.

    Granted, Now returns a value using whole milliseconds and some DateUtils calculation functions operate only to millisecond precision but there’s nothing wrong with adding half a millisecond to a TDateTime.

    e.g.
    TempDateTime := Now + OneMillisecond / 2;

  6. 6
    xsintill Says:

    Nick,

    Again this is unrelated to your article. 2 weeks ago I reported your videos being broken. You’ve asked me to tell you if after a week this was still the case since you are moving offices. I just checked again but I still see no videos of author Nick Hodges. Just 4 I’ve seen authored by David Intersimone but even those listed video’s are broken. Could someone look into this. Thank you.

  7. 7
    Xepol Says:

    Seems to me that you have clearly illustrated the true problem with TDateTime - trying to accurately encode decimal information in mantissa format.

    I understand why TDateTime is floating point - it made math easy, but it definitely comes with a price (several in fact, since editing the binary representation is dang ugly).

    Perhaps it is time to investigate a new Record or object/class based solution that can carry proper whole number fields as a step towards the future. A TNewDateTime as it were. I beleive there are enough tools in the language to keep the classical date math design working, and even provide simple interoperability with the current TDateTime format.

    Obviously, this isn’t a simple problem - or there would not be so many different ways to represent times in the various OSes and languages - but just maybe the current TDateTime method needs a reboot that learns from the years of experience we already have with it.

    Heck, representing dates isn’t even a simple solution in the real world. Anyone care to guess what date 3-4-5 represents? Or is that 3/4/5? I think you get my point. Temporal information is just plain ugly, even in its simplest forms.

  8. 8
    Dalija Prasnikar Says:

    Hi Nick,

    I agree with Xepol it is time for new DateTime concept. I understand why floating point is used before. That way you can have simple variable for DateTime and it is supported by VCL streaming system.
    If you declare TDateTime as object it will be supported by streaming system, but you will pay the price in space(for additional object pointer) and you have to worry about creation and destroying of that var.
    Record would be the right way to go, and with extended records you could pack all DateTime routines inside the record (I found that new feature extremely usefull).
    But, records are not supported by streaming system, and at this point I must wonder why not? I have tons of code that needs to be small and fast, and introducing class is not a way to go. So I use records. But when I want to publish those records (some can be very complex) I have publish every single variable separately.
    BTW, I loved reading your "Testing" series.

  9. 9
    Hoflix Says:

    Hi Nick,

    As I understand the float was/is a result of interaction with Windows (The offset was even corrected to that weird one in D2 to match that OLE stuff). And with the direction of Delphi into other OS (Mac and perhaps Linux later) it would be nice to shed the internal date structure to a better one (but keep it as OS specific conversions).

    Another point I get worried about is speed the simple IncMS is converted to a quite intensive one. I understand your strive to get it perfect but in general date are 99.9% well over the nineteen fifties. Perhaps if dates after 1950 (or some round integer around that and MS < 50 years then use old method???

    Love the series though, good way to put colleagues to the testing set of mind.

  10. 10
    Inverse Function…HELP MATH PEOPLE…Pre-Calculus? | islandpeoplesearch.com Says:

    [...] Nick Hodges » Blog Archive » Fun With Testing DateUtils.pas #6 [...]

© 2014 Nick Hodges | Entries (RSS) and Comments (RSS)

Your Index Web Directorywordpress logo
Close