RADStudio XE - Quality Review and Metrics
Quality is always on our mind. For XE, quality has always been a major focus, and I’d like to share some of the data we look at internally to KNOW we’re on the right track.
Senior VP of Development Tony De La Lama presented the company focus on quality at Delphi Live at the morning opening keynote. Tony presented quality metrics which I’ll add a bit of background and context.
1) Over 4200 fixes in RAD XE by the team compared to over 3000 in RAD 2010.
Number of fixes is a metric when on its own can mean several things. Take into context that the team has more unit testing than ever before, dramatically improved their develop and testing processes, as well as drove through quality central and focused on beta feedback, you’ll see this is a VERY positive and significant number. I’ll go into some of those numbers next…
2) Well over 200 000 automated tests run in XE. Looking back two releases, we had 50 000 in RAD 2009, 100 000 in RAD 2010 and now well over 200 000. These automated tests run over a 24 hour period, ensuring the product is stable and rapidly catches serious issues introduced into the build. Automated testing is done on all areas - I’ll aim to blog a bit more about this in the future to show you our reporting tools, and how we structure and manage tests.
3) Over 23 000 additional VCL/RTL API is documented, with over 100 public documentation bugs fixed.
As a sign of our commitment, we added several engineers to the doc team last year (even with a painful economy). Great progress, more to do – we’re on the right path. Dee Elling, our documentation manager, has been working constantly to raise the documentation experience to Delphi 7 and beyond.
4) A HUGE focus by the team on Quality Central, our public bug tracking system. We’re three years in to the team helping out in clearing the backlog of "Reported" issues. Some really impressive numbers this year – every single issue reported by a beta tester was reviewed and processed by the team. Including the public reports processed, a total of 7500 issues (wow) handled by a combination of sysops (love you guys) and QA. Overall we’ve caught up a net of 1200 issues since Jan of 2009. This is pretty amazing given the focuses on improved test automation and work completed on tools for testing cross platform compiler. Go sysops! Go QA team!
5) 87 generics issues fixed. The team internally is using generics in many places of the RAD Studio product. Between internal use and great beta testers the team made amazing improvements.
6) Compiler performance improvements - the compiler team conducted additional performance tuning after completing the generics fixes. First this was a result of beta tester feedback the that additional fixes to the generics feature caused a slowdown, which was fixed and then surpassed in the majority of cases. For example, compiling the JEDI JCL Code Library (57 units, 427319 lines of source code) is significantly faster in RAD Studio XE:
dcc32 -Q %OPT% bzip2 Jcl8087 JclAbstractContainers JclAlgorithms Jcl AnsiStrings JclArrayLists JclArraySets JclBase JclBinaryTrees JclCharsets JclCom pilerUtils JclComplex JclCompression JclContainerIntf JclCounter JclDateTime Jcl DevToolsResources JclExprEval JclFileUtils JclHashMaps JclHashSets JclIDEUtils J clIniFiles JclLinkedLists JclLogic JclMath JclMIDI JclMime JclNotify JclPCRE Jcl Queues JclResources JclRTTI JclSchedule JclSimpleXml JclSortedMaps JclStacks Jcl Statistics JclStreams JclStrHashMap JclStringConversions JclStringLists JclStrin gs JclSynch JclSysInfo JclSysUtils JclTrees JclUnicode JclUnitConv JclUnitVersio ning JclUnitVersioningProviders JclUsesUtils JclValidation JclVectors JclWideStr ings pcre zlibh
RAD Studio 2010:
OPT=-B : 2.75 seconds
OPT=-M : 2.54 seconds (1st time, delete *.dcu before compile)
OPT=-M : 2.15 seconds (2nd time, there is *.dcu compiled by previous run)
RAD Studio XE
OPT=-B : 1.37 seconds
OPT=-M : 1.36 seconds (1st time, delete *.dcu before compile)
OPT=-M : 1.40 seconds (2nd time, there is *.dcu compiled by previous run)
7) Possibly the most important - our beta testers love the release. Over 85% indicate "recommend to a friend" and a large amount of feedback that XE surpasses 2010 in terms of performance, stability, quality and functionality. This is an important bar for the team, and we also spent a huge amount of effort contacting beta testers directly, make sure we understand the issues, and improve as much as we can. This really helped polish off the release - I’d like to take this opportunity to say a big "Thank You!" to the XE Beta testers.
Can’t wait to release and see what the world thinks!
August 27th, 2010 at 9:49 am
Fine, fine
August 27th, 2010 at 10:13 am
Great job. Very encouraging, and just as important as new features. But I do feel for the people not on SA who won’t get the benefit of all this work without upgrading. And I’m really hoping XE won’t require yet another round of purchasing 3rd party component upgrades…
August 27th, 2010 at 11:11 am
Great news Chris,
Being quite unpatient I’ve posted about the DelphiLive "secrecy" and the lack of info from such an event…. so your post just started to rise the veil about this kind of info. 10x !
DelphiLive "secrecy" ? No footage, video, podcasts, blogs, interviews, demos … ?!
https://forums.embarcadero.com/thread.jspa?threadID=42448
In this regards, could you shed more light about how you’re using Hudson CI for RAD Studio testing ?
August 27th, 2010 at 1:34 pm
Hi Ciprian,
Here is a link you should appreciate for DelphiLive -
http://www.infoworld.com/d/developer-world/programming-embarcadero-unveils-big-plans-delphi-130
We use Hudson CI for continuous build, integration, and testing. RAD Studio has well over 100 000 unit tests run prior to QA even getting the build, which is a 6 hour process. QA tests then run within 24 hours, and we schedule daily builds.
I think we use a fairly mature process - each checkin has a series of basic tests applied to it prior to being promoted into the main build. We’ve reduced the build breakages as a result (tremendously - it’s quite rare to break a build now). Then we run a series of smoketests, which if pass the build is considered good for testing, and the remaining tests fire off.
We use Hudson for many of our projects - not just RAD. It’s great technology.
Regards,
Chris
August 27th, 2010 at 2:07 pm
Very encouraging, thanks for the post.
Can you give any details on the modifications to the compiler to improve performance?
August 27th, 2010 at 2:20 pm
Chris,
Can you help me parse this?
> 3) Over 23 000 additional VCL/RTL API is documented
Thanks,
Brad.
August 27th, 2010 at 2:58 pm
Hi,
Could you publish the list of bugs (QC#) which have been fixed in this release, please ?
Thanks.
August 27th, 2010 at 3:48 pm
[...] http://blogs.embarcadero.com/chrispattinson/2010/08/27/38971/ This entry was posted in General. Bookmark the permalink. ← 3rd Delphi XE sneak preview posted [...]
August 27th, 2010 at 10:07 pm
Hi Chris,
10x for the details and the link … I’m getting really excited now
I really like what I read ! …
Hopefully into the future you’ll post more details about your Hudson-CI integration. That’s a very powerful (and FREE) addition to every developer tool chest.
August 28th, 2010 at 1:09 am
I think it’s a good news that xe fixed a lot of bugs.
But, as to the compile speed, as we all know, the better the optimization is, the lower the compile speed will be.
So I want to know, is there any optimization improvement in xe? or just going backward?
This is very important.
August 28th, 2010 at 9:42 am
Nice post.
But there is only 1 bugfix that could make me upgrade from RAD Studio 2009 to XE:
My editor freezes while waiting for the code completion dropdown to appear. The freeze lasts for several seconds, sometimes longer and makes using Delphi 2009 very frustrating.
August 29th, 2010 at 4:43 am
"My editor freezes while waiting for the code completion dropdown to appear. "
Mine too. Has this issue been addressed?
August 29th, 2010 at 9:56 pm
How about this annoying (!) bug with DeHL:
http://qc.embarcadero.com/wc/qcmain.aspx?d=85788
DeHL has many generics, and code completions cannot cope with this: it totally hangs!
August 29th, 2010 at 11:25 pm
87 Generics + optimizations caught my attention, excellent!
Is the problem in the Properties Grid (where the divider repositions itself every time you change window width) solved?
August 30th, 2010 at 1:43 am
Please sync RAID and QC as soon as possible. Otherwise it is not clear for me what has been fixed.
August 30th, 2010 at 8:23 am
Hi André - QC 85788 is fixed in XE.
Rich and Gad - several issues around this fixed. Do you have specific QC numbers? I can check.
We’ll likely update QC in the next day or two to sync with all the fixes.
Tank - not sure I understand the question. The compiler speed was improved, and the team tests both project compile time and application run time (which is overall most important).
Regards,
Chris
August 30th, 2010 at 6:19 pm
Chris Pattinson :
optimization means that the compiled application has the smallest size, but has the fastest run speed.
For delphi, it seems that optimization haven’t been improved anymore since delphi 7.
But for vc/vs, their optimization is better than delphi.
August 31st, 2010 at 3:59 am
各位好，你们用的2011好用么，感觉怎么样？
September 1st, 2010 at 7:51 am
[...] http://blogs.embarcadero.com/chrispattinson/2010/08/27/38971/ [...]
September 2nd, 2010 at 3:44 am
Hi Chris,
Just want to bug you about this blog (they found some ugly bugs in XE !).
http://delphibugs.blogspot.com/
Maybe next time you/embarcadero might consider to add some "Delphi haters" to the beta-testers, to make sure that the next release will be flawless
September 6th, 2010 at 6:19 pm
Todos estamos haciendo mucha "fuerza mental", para que Delphi vuelva a ser la mejor herramienta de desarrollo; pero que además sea reconocida así por todos los desarroladores.
Ojalá que Embarcadero haga un buen trabajo, a ver si los que trabajamos con Delphi podemos caminar con la frente en alto.
Ánimo!
December 13th, 2010 at 10:38 am
[...] RADStudio XE - Quality Review and Metrics [...]
January 21st, 2011 at 2:17 pm
[...] RAD Studio XE development and beta testing, the team went through over seven thousand Quality Central reports. Besides the survey, Quality Central is a great place to record any issues or suggestions for the [...]
February 1st, 2011 at 10:11 pm
Too bad we have to pay over $1000 to get bugs fixed that never should have been there in the first place. And the documentation up until now has been a disgrace - anyone who bought 2010 should get a free upgrade to XE so maybe they get some documentation with the product…
February 9th, 2011 at 10:34 am
Hi!
How fix this problem in XE?: http://bcbjournal.org/forums/viewtopic.php?f=10&t=1131
Any ideas?
July 29th, 2011 at 8:44 pm
[...] RADStudio XE - Quality Review and Metrics [...]
November 17th, 2011 at 5:55 pm
Hi!
I want to know,if I have a dll file from Delphi6, how to work in Delphi xe?
For example, I write this in xe:
var
……
DllHandle: Integer;
begin
……
DllHandle := LoadLibrary(Pansichar(’RMV2dll.dll’));
……
It doesn’t work in xe, but it works in Delphi6 that is good.
Is it a bug?
March 29th, 2012 at 5:23 am
jewish dating…
[...]Chris Pattinson » Blog Archive » RADStudio XE - Quality Review and Metrics[...]…