A Walk Down Delphi’s Memory Lane: "Pascal Market Trends" circa 1993

As Nick mentioned in this post, we will soon be moving to a new building here in Scotts Valley.  We have been in our current building since 1993.  As Nick states:  "So when you move, you generally use that as an opportunity for cleaning up, clearing out and generally clearing away the build up of years and years of hard work.  As we are doing that, we are finding tons of cool old stuff from “back in the day”."

As I have been gradually cleaning things up in my office over the last few weeks, I came across a file folder in a drawer with some interesting old documents.  One, in particular, is an internal document, titled "Languages Two Year Plan, Second Draft - March 25, 1993".  It describes the market, opportunities, competitive landscape, and plans for Borland’s C++ and Pascal product lines.  Quoting from the "Executive Overview", it was "intended to be an overview of the development activities for Languages for the 24 months beginning January 1993".

Since the 15th anniversary of Delphi’s initial release is approaching, I though it might be fun (and interesting) to look back at some items in this document — though the document was produced nearly two years before Delphi’s initial release, and though it still focuses on the "Borland Pascal" product line, it clearly shows the trend towards what would become Delphi.  I’ll have a couple of posts over the next week or so, leading up to Delphi’s 15th anniversary on February 14th, 2010.  Note, since I only have a hard-copy of this document, I’ll be transcribing it manually, so it is likely that any typos, misspellings, etc. are my own and not part of the document.

To start, here are the first two sections of the "Pascal" part of the document:

Pascal Overview

In January 1994, we will release major new versions of Borland Pascal for Windows and DOS.  The Windows product will be built around easy-to-use visual tools that leverage existing Borland technology in order to bring our DOS customers into Windows.  The DOS product will provide incremental improvements so that we continue to generate upgrade revenues.  Six to nine months later, we will follow up with 32-bit versions of Pascal for both Windows and DOS.  This 32-bit compiler code base will enable us to continue to harvest upgrade revenues with new versions.

Pascal Market Trends

There are three important trends affecting the language market place in 1993.  These are:

1. Increasing demand for visual tools on Windows
2. Increasing awareness of 32 bit tools
3. Decreasing importance of DOS

Microsoft’s Visual Basic and imminent Visual C++ have changed the competitive landscape by shifting the emphasis from OOP to visual tools.  Visual tools have a greater impact on simplifying Windows development, so we must address this area if we are to compete successfully, just as other language companies had to supply integrated development environments and OOP features in response to innovations that first appeared in Turbo Pascal and Turbo C.

Visual Basic has significant weaknesses in language, performance and scalability, that limit its acceptance.  This gives us an opportunity to deliver products which simplify Windows programming and offer more power and performance.  Many Pascal customers are asking for a "Visual Pascal" that provides the ease of use of Visual Basic with the extensibility, power and performance of Turbo Pascal.

Just as Microsoft leveraged their domination of the Basic language market to create Visual Basic, we can leverage our domination of the Pascal market to make changes to the language, compiler, and libraries to make Windows development easy.  This is the best way to bring our large installed base of DOS customers into the Windows market.

There is also an increasing awareness of 32 bit development tools.  Shipment of Win32s and NT in 1993 will make "32 bit" a requirement of any language.  However, large volume deployment of 32 bit applications will probably not happen until Microsoft ships the "Chicago" version of 32 bit Windows on 32 bit DOS, in mid 1994.  Pascal customers continue to ask for 32 bit versions to overcome long standing limitations in Pascal’s global data space, data structure sizes, ability to link with standard OBJ files etc.

Although Windows development tools get the most press coverage, DOS products continue to bring in significant revenues.  Many DOS programmers will continue to upgrade as long as we offer incremental improvements in the language, libraries and tools.  But this revenue stream will decline over time if there is no clear commitment to Pascal on 32 bit DOS and Windows.

Up Next:  Excerpts from the "Pascal Strategy" section

Share This | Email this page to a friend

Posted by Chris Hesik on February 5th, 2010 under Delphi | 1 Comment »

Haiti Relief Auction: Signed Softball

One of the "one-of-a-kind" items that I donated to the Borland Memorabilia Auction for Haitian Relief is now listed and ready for bids.

It is a softball signed by the the members of the winning softball team (who were mostly members of the Delphi and C++Builder engineering team) from the 2000 Company Softball League. You can read about this item on Anders’ blog and you can view the auction here.

Incidentally, 2000 was the last year that we had a company softball league, so we are still the reigning league champions :-).

I also donated a couple of unworn league championship t-shirts that we received after winning, and have asked Anders to include one of those for the winner of the softball auction.

Share This | Email this page to a friend

Posted by Chris Hesik on February 3rd, 2010 under Uncategorized | Comment now »

RAD Studio 2007 informal, unofficial fix available: Windows 7 problem addressed

Two weeks ago, we released a hotfix for the 2009 release to address problems using the debugger under Windows 7 (primarily 64-bit versions).  You can read about that hotfix here:  http://blogs.embarcadero.com/chrishesik/2009/12/03/35132.

If you look at the comments on that post, you’ll see several people asking for a version of this hotfix which applies to the 2007 release.  While we are unable to provide an officially-supported fix for that release, I am able to provide an informal fix.  I took a look at the Windows 7-specific fixes that were applied to the 2009 codebase, and I merged the applicable fixes over to the 2007 codebase.  I then built an informal build on my machine and hand-tweaked the file version in the version info resource.  I checked exactly one test case on a Windows 7, 64-bit machine and it appears to work ok.

I’ve created a CodeCentral entry for it. You can find it here: http://cc.embarcadero.com/item/27521

Note the following disclaimer on the CodeCentral submission:

"This fix has not been tested at all, and it is not supported in any way by Embarcadero.  If it works for you, great… if not, you’ve been warned."

I’m not joking about not testing this at all.  Our internal QA engineers have not looked at this at all.  I spent a grand total of three minutes playing with the debugger with this fix in place.  I’m sure the informal patch referenced on Olaf’s blog has been tested much more thoroughly.  That said, the version I’ve uploaded contains a few extra Windows 7-specific fixes that aren’t included in the patched version from Olaf.  If you are running into problems debugging on Windows 7 with the 2007 release and want to give this fix a try, it’s now available.

Share This | Email this page to a friend

Posted by Chris Hesik on December 17th, 2009 under C++Builder, CodeGear Debugger, Debugger, Delphi, RAD Studio | 2 Comments »

RAD Studio 2010 Update 4: Report on Automatic Incident Reports

In an earlier post, I briefly mentioned the tool I wrote that helps us on the RAD Studio R&D team to analyze and fix problems that are reported via the Error Reporting mechanism built into the IDE.   In the past, we’ve taken an ad-hoc approach to analyzing the data and fixing reported problems.  More recently, however, I’ve started pushing for a more formalized approach to tracking, investigating, and addressing these issues.

For those who aren’t sure what I’m talking about here, the next time you get an error while working in the IDE, check to see if the error dialog has a "Details >>" button.  If if does, click that button and you will be shown a stack trace for the error.  From there, you can click the "Send" button to start the "Send Report" wizard that allows you to submit an "Automatic Incident Report" in QualityCentral.

Now that RAD Studio Update 4 is available, I want to talk about some of the specific bugs we fixed in the update as a direct result of analyzing the data.  As its most basic function, the tool I wrote takes the stack traces that are attached to the Incident reports, and provides a sorted view of the most common stacks.   It makes it very easy to see which errors are being reported most often.  For the purpose of this post, I’m going to concentrate on six of the most common issues that have been reported since RAD Studio 2010 has been released.

There are several types of issues that get reported by our customers.  Before looking at the different categories, it is important to note that the IDE will allow you to send in a report for any unhandled exception — this is important as there are couple of categories of issues reported that are not product bugs in the traditional sense.

The three main categories of issues I’ve seen reported are:

1. Actual bugs that slipped through our QA process and into the released product
2. Errors that occur due to incorrect configuration, user error, etc.
3. Errors caused by third-party controls or packages installed into the IDE

Category 1: Actual bugs that slipped through our QA process and into the released product

The first category is by far the most interesting, and it is on these types of issues that I spend most of my time when looking at these reports.  Out of the six issues we’re looking at, three fall into this category, and two of them have been fixed for Update 4.  They are:

1. An error with the message:  "Cannot create file "<some Welcome page javascript file>". The process cannot access the file because it is being used by another process".  A common thread among people running into this issue was that they were running the Kaspersky AntiVirus software.  It turns out the welcome page was unable to open the file because the virus scanner already had it open.  This basically boiled down to the welcome page code creating a TFileStream using "fmCreate" create mode when less stringent file permissions would allow it to do what it needed to.  The fix for Update 4 is for the welcome page to use a more share-friendly create mode.  So far, there have been 55 reports filed for this issue.
2. An error with the message:  "”false” is not a valid integer value." when either starting the IDE or when loading a project.  This error occurred when certain data existed in a project file (or in the default project file).  The project options file parser had a bug in it when reading the data for one particular compiler option.  For Update 4, the parser has been fixed to handle that data as well.  So far, there have been 33 reports for this issue.
3. An error with the message "The system cannot locate the object specified. Line: 0".  This error occurred when some unexpected data was being read from a project’s desktop file (the .dsk file).  Unfortunately, we weren’t able to provide a fix for this issue in this update.  I wanted to mention it here because there’s a fairly simple way to fix this if you are running into it — removing the offending .dsk file should make this problem go away.  If you don’t want to remove the entire .dsk file, you should be able to edit it to remove the unexpected data.  The unexpected data is an open module that has a ModuleType of "TBaseProject".  Search your .dsk file for "TBaseProject", and remove the section that contains it (the section will have a project file’s name).  Then you should also remove the corresponding entry in the [Modules] section.  After saving the .dsk file, you should be able to reopen the project.   So far, there have been 21 reports for this issue.

Category 2: Errors that occur due to incorrect configuration, user error, etc.

These errors show up because there are parts of the product that rely on exceptions for normal error handling.  This category can be broken down further, into two subcategories:  a) cases where proper error messages are given, providing all the information we know at the time of the error, and b) cases where more or better information can be given to the user.  For the first subcategory, there is nothing more to be done — it has been deemed that the information given in the error message is useful enough.  What we usually do in cases like this is to change the code to still show the error message without giving the user the opportunity to send the report in.  This is done in order to reduce the amount of "noise" in the reports.   We’ve done quite a bit of this in the last few years in order to reduce this noise.   For the second subcategory, we basically have a usability issue — something has gone wrong, but we haven’t given enough information to the user to allow them to solve the problem.   Of the six issues we’re looking at, there is one of each of type:

1. Design-time database errors.  Examples are error messages like:  "DBX Error:  Driver could not be properly initialized.  Client library may be missing, not installed properly, or of the wrong version.", "unavailable database", or "Access denied for user ‘xxxxx’ (using password: YYYY)."  Members of the database team looked at each report and determined that the error messages given are good enough and that there is no real value in gathering these reports.  Thus for Update 4, if you encounter this type of error, you will no longer be given the opportunity to send these in.  So far, there have been 69 reports submitted for valid database errors.
2. Errors that occur when asking for help if you do not have the help installed.  The most common report we’ve received for the 2010 release has the error message "Class not registered, ClassID: {314111F0-A502-11D2-BBCA-00C04F8EC294}".  This is a case where the error message given doesn’t give you enough information as to what is wrong and what you can do to solve the problem.  If you run into this, you probably have a good idea that it has something to do with the help, since it occurs in response to pressing F1 or clicking a Help button, or something similar.  Since it was determined that this error occurs if you do not have the help installed, we’ve added code to give you that extra bit of information when this error occurs.  In Update 4, in addition to the "Class not registered" text, the error message will also include the text "RAD Studio’s help namespace is not defined.  Please reinstall RAD Studio’s documentation".  So far, there have been 93 reports filed for the "Class not registered" issue, as well as several others that appear related to not having the help installed properly.

Category 3: Errors caused by third-party controls or packages installed into the IDE.

Since third-party packages (which include design time code for components and IDE addins) run within the IDE’s process space, any unhandled exceptions they cause can be reported to us via the error reporting mechanism.  In most cases that I’ve looked it, these reports represent problems with the third party packages.  There are, however a few cases where real problems in the IDE have shown up due to a slightly different way a third-party package does something.  In those cases, there are things we can do in the IDE to make it a more robust host process.  For update 4, there is only one highly-reported issue that can be attributed to a third-party addin.  That this the JVCL issue I mentioned in an earlier post.  Fortunately, the JVCL team fixed this issue really quickly after the 2010 release.  See the comments from Uwe and Olivier on that post for more information on how to get the fix.  So far, there have been 17 reports filed for the JVCL issue.  I can think of at least one bug that AIR reports have helped us fix in the past that only occurred when a certain third-party package is installed, but since that is not specifically related to Update 4, I’ll save that story for another blog post.

Conclusion

There are several things I want to mention in conclusion:

1. I talk about Update 4 in this post, but all of the issues mentioned were originally fixed for the recalled Update 2.  So if you were one of the few who successfully downloaded and installed Update 2 before it was recalled, you would have already had the benefit of these fixes.  Still, please heed Nick’s advice and install Update 4/5.
2. While I’ve only mentioned six specific issues in the post, there are many additional bugs that have been fixed in Update 4 as a direct result of analyzing the Automatic Incident Reports that have been filed. There are also many other bugs, reported via more conventional means, that have been fixed in this update.  The full bug lists are available on EDN.  See http://edn.embarcadero.com/article/40204/ for the Delphi and General bugs and http://edn.embarcadero.com/article/40205/ for the C++-specific bugs.
3. In looking at the data, it’s interesting to see how quickly the graph drops down to reports where there have been only one or two occurrences.  It seems that there are a lot of "one-off" type errors.
4. I also found it interesting how far down in the list I had to go before encountering an Access Violation.  There have been past releases where I remember Access Violations being much more prevalent.  I think this says something about the stability of the 2010 release.
5. I can’t stress enough how valuable it is to have data that shows the types of problems our customers are running into.  I’d like to thank anyone who has taken the time to file one of these reports. One of my main motivations in writing this post is to show that these reports are indeed valuable and to encourage people to keep sending them in.
6. The bits of information that some people provide on the "Description and Steps" page of the IDE’s Send Report wizard have been really helpful in several cases (and quite entertaining in other cases as the user’s frustration is quite evident).

Share This | Email this page to a friend

Posted by Chris Hesik on December 14th, 2009 under C++Builder, Delphi, IDE, RAD Studio | 5 Comments »

RAD Studio 2009 Debugger hotfix available: Windows 7 problems addressed

A Delphi 2009, C++Builder 2009 and RAD Studio 2009 hotfix is now available to address several problems with the debugger when running on Windows 7.

You can find the hotfix on CodeCentral at http://cc.embarcadero.com/item/27476.

The most glaring problem fixed is an assert that can occur when debugging multi-threaded applications on 64-bit versions of Windows 7.  The text of the assert is:


Assertion failure: "(!"SetThreadContext failed")"
in ..\win32src\thread32.cpp at line 434
Continue execution?


You can read about this problem on Olaf Monien’s blog.

In addition, there are several other Windows 7-specific problems addressed by this hotfix:

• A problem with Wait Chain Traversal not working on Windows 7
• A problem with a Windows Error Reporting dialog showing up when an application being debugged terminates
• A minor problem related to step-to-next-source functionality

All of the problems addressed had already been addressed in the 2010 release (or will be addressed in an upcoming patch), but we wanted to make them available as a hotfix to users of the 2009 release as well.

Share This | Email this page to a friend

Posted by Chris Hesik on December 3rd, 2009 under C++Builder, CodeGear Debugger, Debugger, Delphi, RAD Studio | 19 Comments »

Using Delphi "Unit Aliases" to aid in unit testing

• Resolving unit dependencies.  Think of a case where a unit uses another unit just so it can access one symbol from that unit.  Assume that the used unit iteself uses many other units.  In order to build a test for the code in this unit, the compiler must resolve all used units.  Due the nature of the IDE source code, it’s very easy to get into a situation where a unit test ends up pulling in a very large number of source units that aren’t really needed for the test.
• Resolving IDE services accessed by the unit being tested.  Think of a case where a unit being tested uses a core IDE service to, say, display status in the IDE splash screen during startup, or to output a message to the IDE’s message view (just two of many possible examples).  Because the IDE isn’t running, it’s unlikely that these services would be available to the unit test.

There have been different approaches to overcoming these challenges, that those of us on the IDE team have taken when writing unit tests:

• Refactoring the unit being tested has helped in some of these cases.
• Another approach has been to litter the code being tested with {$IFDEF} or {$IFNDEF} directives so that only the code being tested is visible to the test.  Yes, I use the term "litter" because of its negative connotation –  using IFDEFS can compromise code readability and maintainability.  Also, any time you change code (even if to only add/remove IFDEFS), you risk breaking something (yes, this may be less of an issue, since you are, after all, changing code so you can write unit tests).
• A third approach, which is the subject of this post, is to provide a unit with stubs for the dependencies and then use the Delphi compiler’s Unit Alias feature when building the unit test.  A major benefit of this is that it allows you to test existing code without having to modify that code.

Take the over-simplified unit shown in this screenshot:

Let’s assume we want to write a unit test for TMyClass.DoSomething and that the OutputAMessage procedure 1) lives in a unit (MessageServices.pas) that pulls in a lot of extra units, and 2) requires UI that is not available while unit testing.  A simple way to write this test is to provide a unit which provides an implementation of "OutputAMessage".  Let’s call the unit TestStub:

Yes, you could use an IFDEF in Unit1’s uses clause to conditionally use either the MessageServices or TestStub unit.  But what if the unit being tested is more complex?  What if there are many functions/symbols/etc. living in spearate units that you want to provide a stub for?  What if the code uses a unit name in order to qualify a symbol (i.e. what if the code in Unit1.pas referenced "MessageServices.OutputAMessage" instead of just "OutputAMessage" in order to resolve a symbol ambiguity)?

Unit aliases allow you to address those concerns.  When compiling the unit test, if you define a Unit Alias of "MessageServices=TestStub", it will use the stubbed version in TestStub.pas.  When you build your main project, it will continue to use the real version in MessageServices.pas.  If you wanted to provide a single stub unit for multiple "real" units, you would just provide addtional aliases:  "MessageServices=TestStub;AnotherUnit=TestStub".  And due to the way Unit Aliases are implemented by the compiler, it will properly alias unit names used as scope qualifiers in the code.  As mentioned earlier, a great benefit of this approach is that you can test your existing units without having to modify them at all.

You can define Unit Aliases in the IDE using the Project | Options dialog.  In Delphi 2010, this option is on the "Delphi Compiler" page.  When building from the command line, the appropriate compiler switch is -A (i.e. -AMessageServices=TestStub).

While the compiler’s support for Unit Aliases was originally added to increase backwards compatibility and to ease migration when a new version of a unit has a different name than an older version, I have found the feature to be quite useful in helping to unit-test code that might otherwise not be easily tested.

Share This | Email this page to a friend

Posted by Chris Hesik on November 24th, 2009 under Delphi, IDE, RAD Studio | 7 Comments »

RAD Studio 2010: Delay Load import exceptions

One of the things I do after a product release (or an update release, for that matter) is keep an eye on the Incident Reports that are submitted to QualityCentral via the Send Report wizard that is built into the IDE.  A few years ago, I wrote a tool that makes it really easy to see which reports are coming in at a high frequency.  The data gleaned from this tool is really helpful when looking for "big bang for the buck" type bug fixes to go after for an update (or future product release).  One issue that I’ve seen reported at a moderate level (about a dozen reports, so far) for the 2010 release is related to the new "delay-loaded import" feature.  You can learn more about this feature by reading the following posts by Allen Bauer and Christian Wimmer:

http://blogs.embarcadero.com/abauer/2009/08/25/38894

http://blogs.embarcadero.com/abauer/2009/08/29/38896

http://blog.delphi-jedi.net/2009/08/29/version-checking-for-delphi/

In Christian’s post and in Allen’s second post, there is discussion of the exception that is raised when a delay-loaded method can not be found at runtime.  It is this exception that is prompting D2010 users to submit QC reports.  In fact, every single one of these reports I’ve seen so far are coming from developers who are using the JVCL at design time in the IDE — in particular it appears that these developers are using a TJvCustomSpeedButton descendant.  Based on the exception, it would appear that each report was generated on a pre-Vista OS.

The error message associated with this exception is either "External Exception $C0FB007E" or "External Exception $C0FB007F."  Here is the top of the stack trace for one of these reports (this is what you’ll see if you click the "Details" button on the error message in the IDE):


[7C812A6B]{kernel32.dll} RaiseException + $52
[50011BDB]{rtl140.bpl  } System.@_delayLoadHelper (Line 302, "C:\Builds\TP\rtl\sys\delayhlp.c" + 158) + $D
[500B0681]{rtl140.bpl  } UxTheme.uxtheme.dll (Line 5112, "UxTheme.pas" + 0) + $5
[500B0712]{rtl140.bpl  } UxTheme.DrawThemeTextEx (Line 5138, "UxTheme.pas" + 1) + $20
[490EE508]{JvCoreD14R.bpl} JvThemes.DrawGlassableText + $C4
[490EE54C]{JvCoreD14R.bpl} JvThemes.DrawGlassableText + $108
[49ACF1B4]{JvStdCtrlsD14R.bpl} JvSpeedButton.TJvxButtonGlyph.DrawButtonText + $C4
[49ACF2A6]{JvStdCtrlsD14R.bpl} JvSpeedButton.TJvxButtonGlyph.DrawEx + $E2
[49ACDA81]{JvStdCtrlsD14R.bpl} JvSpeedButton.TJvImageSpeedButton.PaintImage + $C5
[49AE416C]{JvStdCtrlsD14R.bpl} JvToolEdit.TJvEditButton.PaintImage + $78
[49ACCC96]{JvStdCtrlsD14R.bpl} JvSpeedButton.TJvCustomSpeedButton.Paint + $336
[49AE4078]{JvStdCtrlsD14R.bpl} JvToolEdit.TJvEditButton.Paint + $98
[501CE436]{vcl140.bpl  } Graphics.TCanvas.SetPenPos (Line 4050, "Graphics.pas" + 1) + $9
[5029F685]{vcl140.bpl  } Controls.TGraphicControl.WMPaint (Line 13141, "Controls.pas" + 7) + $5
[49ACD58E]{JvStdCtrlsD14R.bpl} JvSpeedButton.TJvCustomSpeedButton.WMPaint + $B6
[50295DFC]{vcl140.bpl  } Controls.TControl.WndProc (Line 7062, "Controls.pas" + 91) + $6


 

The exception is raised because in Delphi 2010, the DrawThemeTextEx declaration in UxTheme.pas was changed from using the LoadLibrary/GetProcAddress mechanism mentioned by Allen to use the new delay-loaded mechanism. 

Below if the pre-D2010 version of the implementation of DrawThemeTextEx:
 


function DrawThemeTextEx(hTheme: HTHEME; hdc: HDC; iPartId: Integer;
  iStateId: Integer; pszText: LPCWSTR; cchText: Integer; dwTextFlags: DWORD;
  pRect: PRect; var pOptions: TDTTOpts): HResult;
begin
  if Assigned(_DrawThemeTextEx) then
    Result := _DrawThemeTextEx(hTheme, hdc, iPartId, iStateId, pszText, cchText,
      dwTextFlags, pRect, pOptions)
  else
  begin
    Result := E_NOTIMPL;
    if ThemeLibrary > 0 then
    begin
      _DrawThemeTextEx := GetProcAddress(ThemeLibrary, 'DrawThemeTextEx'); // Do not localize
      if Assigned(_DrawThemeTextEx) then
        Result := _DrawThemeTextEx(hTheme, hdc, iPartId, iStateId, pszText,
          cchText, dwTextFlags, pRect, pOptions);
    end;
  end;
end;


And here is the D2010 version:


function DrawThemeTextEx(hTheme: HTHEME; hdc: HDC; iPartId: Integer;
iStateId: Integer; pszText: LPCWSTR; cchText: Integer; dwTextFlags: DWORD;
pRect: PRect; var pOptions: TDTTOpts): HResult; stdcall; external themelib name 'DrawThemeTextEx' delayed;


Because of the change, existing code that calls this API on a pre-Vista system would now generate the aforementioned delay-load exception.  With the old mechanism, calling this method would have resulted in a "E_NOTIMPL" return value.   The JVCL code that calls this API, had error-handling code in place to properly handle the E_NOTIMPL return value, but this did not help with the new exception-raising mechanism.   I just checked, and it looks like the JVCL team has addressed this issue.  The sources contained in the current JVCL tip revision properly handle the situation by doing a Windows version check before calling the API (see revision 12491 of JVThemes.pas, committed on September 4th, 2009).  That said, if you are using the JVCL and have not yet updated to the most recent sources, you may still run into this exception in D2010 when running on Windows XP (or any pre-Vista version of Windows).  To avoid this, I would suggest updating your JVCL sources.  If that’s not practical, I would at least apply the changes made in the aforementioned revision.
 

On a related note, it would probably be a good idea for the IDE to give a better error message in this particular case.  I think that if the IDE used a solution similar to one presented by Allen in his second post, that would definitely make things much more user-friendly should an error like this occur again in the future.   Telling the user that a particular entry point is missing from a particular module is much nicer than giving an external exception with an obscure error code.   I’ll be investigating such a solution for a future release.

 UPDATE: I just noticed that Andreas Hausladen blogged on this exact issue a few months ago:  http://andy.jgknet.de/blog/?p=660

Share This | Email this page to a friend

Posted by Chris Hesik on November 2nd, 2009 under C++Builder, Delphi, IDE, RAD Studio | 9 Comments »

Debugger Tip: Ignoring non-user breakpoints

I just realized that a feature that was added for the 2007 releases is not mentioned on my page that outlines debugger additions that have been made since Delphi7/BCB6.  I’ve gone ahead and updated that page, but I think this feature is worth mentioning in a separate post.

The feature I’m talking about is the addition of the debugger option called "Ignore non-user breakpoints".  In the 2010 release, this option can be found on the "Embarcadero Debuggers" page on the Tools | Options dialog (or you can use IDE Insight to find it).  In previous releases the page would have been called either "CodeGear Debuggers" or "Borland Debuggers", depending on the company name at the time of the release.

This function of this option should be self-explanatory — the only thing that might not be clear is what exactly constitutes a "non-user breakpoint".  A non-user breakpoint is any breakpoint that you have not explicitly set in the IDE.  There are a couple of different things that cause a non-user breakpoint to be triggered.  Some common causes:

1. A hardcoded breakpoint using inline assembly.  The assembly instruction for a breakpoint is "int 3".  If you have any inline assembly that includes this instruction, a breakpoint will be triggered if you are running under the debugger when that instruction is executed.
2. A call to the WINAPI procedure "DebugBreak".  If you have code that calls this procedure, a breakpoint will be triggered if you are running under the debugger when that procedure is called.
3. Pressing F12 in an application that is running under the debugger.  This hotkey is controlled by the OS (on by default for XP, can be disabled; off by default in newer Windows versions, can be enabled).  It basically causes an app that is running under the debugger to break back into the debugger via a breakpoint.
4. Sometimes passing bad parameters or data to some WINAPI functions causes those functions to trigger breakpoints if you are running under the debugger.

There may be other situations that cause a non-user breakpoint to get hit.  You’ll know you’ve hit one by looking in the CPU view after your application unexpectedly breaks back into the debugger.  If you see an "int 3" instruction in the CPU view at the address immediately preceding the instruction at the address indicated by the current EIP, you can be pretty sure you’ve hit a non-user breakpoint.  All of the common causes mentioned above boil down to an "int 3" instruction getting executed by the app being debugged.

Using the "Ignore non-user breakpoint" debugger option, you can tell the debugger that you don’t want your process to stop when it encounters one of these.  When that happens, the debugger will respond to the breakpoint triggering by just setting the process running again — there will be no indication that a non-user breakpoint is hit.

This can be especially useful for case #3 above — if your app uses F12 as a hotkey, and you need to debug its handling of the F12 key on XP, but don’t want to disable it globally on your system.

Share This | Email this page to a friend

Posted by Chris Hesik on October 28th, 2009 under C++Builder, CodeGear Debugger, Debugger, Delphi | 7 Comments »

RAD Studio 2010: TThread support for naming a thread

In my post on Thread-Specific Breakpoints, I mentioned that the Delphi RTL’s TThread class has improved support for naming threads.

Before looking at that, let’s quickly review how naming threads for the debugger works.  The concept of naming threads in Delphi and C++Builder is not new (if I remember correctly the debugger support was added in the 6.0 versions), and it was modeled after other tools that supported that same mechanism.  Basically the debugger listens for a specific exception type (one with an exception code of $406D1388) within the application being debugged.  When this exception type is raised by the application, the debugger recognizes the exception type and cracks open the structure passed to the RaiseException call to extract the thread name.  It will then apply this name to the appropriate thread (which is also determined by looking at the structure that contains the name).  The most common case is to name the thread that raises the exception, but this mechanism also supports naming any thread running within the process from any thread running within the process.  For reference, this MSDN article shows how to do this from an application.

This mechanism is also supported by other Windows debugging tools (including Visual Studio and AQTime).

In previous Delphi and C++Builder versions, you would have to provide all the code yourself to name a thread.  If you created a thread using the "Thread Object" item that appeared under either the "Delphi Files" or "C++Builder Files" section of the File | New | Other dialog box, you were given the option to name the thread you were about to create.  If, however, after the fact, you wanted to go back and name your thread, or if you had a thread that you manually created,  you were on your own to copy/paste the appropriate code into your thread object.

Now, however, the Delphi RTL has built-in support for doing this.  You can now easily name any thread using a single function call.  The TThread class now has a new method:

    class procedure NameThreadForDebugging(AThreadName: AnsiString; AThreadID: LongWord = $FFFFFFFF); static;

The simple case is to call it passing in just a string representing the name for the thread.  In that case, the debugger will name whichever thread makes the call.  If you want to name a different thread, you can do that by also passing in the OS thread ID that corresponds to the thread you want to name.  Because this is a class method and because the second parameter can be the thread id of any thread in your application, you do not even need to be within a TThread object to call it, and in fact, you can call it to name a thread even if that thread is not a TThread descendant.  Two other things to note:  first, this new method does nothing if the application is not running under a debugger (IsDebuggerPresent returns false); and second, the code that raises the exception for the debugger is wrapped with an emtpy try/except block to ensure the exception doesn’t bubble out of the NameThreadForDebugging procedure.

Even though thread naming has been around for a while, it wasn’t until this small addition was made to the RTL, that we (the IDE team) made a concerted effort to name the threads created by the IDE.  As soon as it became dead-simple to name a thread, it was a simple matter of hunting down the threads created by the IDE and adding a single call to each thread.  For TThread descendants, we just added a single call as the first line of each thread’s overridden "Execute" method.

This alone has made debugging the multi-threaded parts of the IDE much simpler.  When debugging a crash or a thread-contention issue, you used to have to double-click the various threads in the Thread Status view and then look at the Call Stack view in order to find a particular thread.  Now, with most of the critical IDE threads named, it is a simple matter of locating the thread, by name, in the Thread Status view.  And naming all our threads, has made it possible for use to effectively use Thread-Specific Breakpoints, and the new Thread Freeze/Thaw mechanism that was also introduced in Delphi 2010 and C++Builder 2010.

Share This | Email this page to a friend

Posted by Chris Hesik on October 22nd, 2009 under C++Builder, CodeGear Debugger, Debugger, Delphi, IDE, RAD Studio | 6 Comments »

Debugger Tip: Another case for selective symbol table loading

A few weeks ago, I talked about a debugger feature, "Selective Symbol Table Loading", that can be used to increase performance of the debugger.  In addition to its performance benefit, this feature is also useful in other situations.  One situation, in particular, where it comes in handy, is when a single source file is shared by more than one PE Module loaded by the process you’re debugging.  It is especially useful when that shared source file contains IFDEFs so that the source is compiled differently for each PE Module.  Let’s take a look at a test case (which, I’ll admit, is a bit contrived) and see how this feature can help out in this case.

First a couple of IDE screenshots:

What we have here is a basic C++ project group that contains two DLL projects and one EXE project.  The two DLLs, both include the SharedUnit.cpp.  As you can see in Figure 1, that source file contains a single function, whose implementation is IFDEFd so that its return value depends on whether or not the conditional symbol "TEN" is defined.  The "RegDll.dll" project builds this unit without that define, while the "TenDll.dll" project builds it with "TEN" defined.  In Figure 2, you see the source for the EXE project.  All it does is dynamically load the two DLLs using the LoadLibrary Windows API call.

So, let’s say you build all three projects, and then use the debugger to step over the two LoadLibrary calls in the EXE.  If you then go and look at SharedUnit.cpp in the code editor, where would you expect the blue dots to appear?  Figure 3 shows where they appear.

As you can see, a blue dot only appears on the first return statement, even though both return statements are actually valid locations for breakpoints (they are both valid because both DLLs are loaded).   Due to some limitations in the way the debugger deals with source files that appear in more than one symbol table, when painting the blue dots in the code editor, the debugger will only see the symbols for the first DLL loaded which reports symbols for that source file.   Since TenDLL is loaded first, its symbol table wins and the blue dots for SharedUnit.cpp reflect the source for that DLL.  If RegDLL had been loaded first, then its symbol table would have won, and the blue dots would have reflected the source for that DLL.

So, what are you to do if you are running into this situation and it’s not practical to alter the PE module load order?  Let’s take a look at how Selective Symbol Table Loading can help.  Assuming that, for the moment, you only care about debugging into RegDll.dll, you can tell the debugger to temporarily ignore the symbols for TenDll.dll.  As explained in the previous post on this topic, you do this using the Symbol Tables page of the Project | Options dialog.  On that page, you can add the DLL you want ignored to the list of Modules, while leaving the "Symbol Table Path" blank (as shown in Figure 4).

Then, when you debug the EXE again, and step over the two Load Library calls, this time the blue dots shown in the SharedUnit.cpp source file reflect the symbols loaded for RegDll.dll (Figure 5).

While I’ve admitted that test case I’ve used to illustrate this feature is a bit contrived, this exact situation exists within the Delphi codebase itself.  The Delphi compiler sources (written mostly in C), can be used (with the help of many IFDEFs) to build different flavors of the compiler dll.  Back when the IDE hosted both the Delphi win32 and Delphi.net compilers, it would be possible to run into this situation when trying to debug one flavor of the compiler instead of the other (dcc100.dll vs. dcc100il.dll).  Selective Symbol Table Loading was quite handy in easing the debugging of the shared, ifdef’d source files, as you typically only cared about one compiler flavor at a time.  And although I haven’t needed this yet, I expect it to be just as handy when needing to debug the IDE-hosted versions of the MacOSX and Linux compilers that are being worked on for ProjectX.

Share This | Email this page to a friend

Posted by Chris Hesik on October 19th, 2009 under C++Builder, CodeGear Debugger, Debugger, Delphi, ednfront | 3 Comments »