Embarcadero Quality Review - 2009 to 2010

It’s been a while since I’ve blogged, mainly since I’ve been working with many of the QA managers at Embarcadero in the past year to define, implement and improve on software development practices across the organization - and with some really great successes! Now is a great time to highlight some of the teams accomplishments, and explain a little about how we did what we did - and why I know its "Good stuff".

A rough outline of steps and activities is the following:

1. Conducted internal audit and ranking of Embarcadero product teams in terms of quality processes. As part of this, I created a ‘Quality Maturity Model’ (QMM) to ranking products in 8 categories, and define basic standards around quality practices. I’ll likely blog more about this soon, as I think it really helped drive this improvement program.
2. Rallied with the product owners and business managers to sync up on top areas they wanted to see quality improvements in the product. Feedback ranged from functional quality, to performance improvements, to faster cycle times to market, to rock solid product stability. This feedback was layered in to the rankings in the QMM to help focus the teams on highest value QA activities.
3. Launched a Quality Leadership Team. A weekly meeting of quality leaders at Embarcadero to work together on defining the best practices, tools and processes. This was a great meeting to bring up challenges and issues, and come up with good solutions. For example - RAD team has a mature automated test framework we internally call ‘zombie’, a couple other teams needed an automated test framework and instead of building one from scratch,  they now share the Zombie codebase with RAD team.
4. Have QA leaders determine quarterly quality goals that work to improve their QMM rankings. Now that the team understood where we were, what the highest priorities were, and had a team to figure out the effort and costs to build up the systems how the business owners wanted, it was a matter of selecting a few goals each quarter, completing or making progress against them, and then move on to the next. There is literally several years of improvements defined that the teams know about now, it’s just a matter of doing the most important first, then the next and so on.
5. Built up the Embarcadero QA Roadmap - I’d take the feedback from the quality leaders (who also had the backing of their product teams) and designed up a QA Roadmap. This roadmap would graphically describe the QA activities that ranged from training, development, implementation and maintenance tasks at a fairly high level (IE: "Develop automated tests covering basic user functionality as defined by product management" or "Train the team on Zombie best practices in test development"). Success criteria were provided in more detail in a separate document used by quality leaders to communicate to me their quarterly goals. The roadmap was useful to rallying the teams around who was doing what, and what the major focus was each quarter.
6. Each quarter - review accomplishments of past, identify any roadblocks or areas that need support, and plan for the next quarter.
7. Conducted annual review. In the past two weeks we conducted a QMM assessment of each product team to see how we’re doing from a high level, identified any areas we might be weak in, and are now working up the next years high level plan.

That defines the process; I’ll give a sample of some of the results:

1. RAD Studio had a mature quality process entering 2009 (Marty Thompson is a very experienced manager and great quality champion). In 2010 the team focused on beefing up their testing and reporting systems, reviewed over 6000 various points of feedback from customers and internal staff, and built up a very well structured and prioritized list of areas to improve in RAD for 2010.  Marty and the RAD core management team do a great job managing quality in the RAD product which you’ll see more of in upcoming releases. 
2. DBArtisan is a product with many releases in a 12 month period to continually provide customers with high value. It’s a extremely powerful suite for database administration, and testing it’s huge set of features in a short period of time is challenging. To improve both the time to market and the quality of releases, the team spent a huge effort in 2009 to 2010 in building sophisticated automated tests that setup database servers, run tests while also checking server stability, report results with rich debugging and error messaging, and are able to run on many different DB and OS platforms in a short period of time. The first major milestone of these efforts was completed for the DBArtisan 8.7.4 release, and this had a major impact on the product quality, and will help with all future releases. In addition the team worked in automated performance tests and is now focusing on a release around optimizing the product performance - everyone loves best of class performance in their tools, right?
3. ER/Studio is one of my favorite tools. It’s not just a diagramming tool, you can use it for project planning, workflows, defining internal systems, architecture diagrams, and any sort of conceptual map – even a brainstorm chart. ER/Studio already a rock solid product, however the team had mainly manual testing practices and as a result the release cycle was quite long. In 2010 the team literally took off like a rocket with new processes in entering test cases into TestLink, and then automating the tests using the Zombie framework. They are only a few months into the effort, and already have a nightly test run of over 1000 automated functional tests, with hundreds more coming online each week. This is both going to keep the product in top quality shape, as well as help development to rapidly implement and release to market new features.

So lots of good news from the QA Organization this year, and I really have to thank the QA leaders in my teams - managers, engineers and architects. And really, not only them - quality is a truly team effort, and it’s the project teams at Embarcadero driving quality forward together.

What’s next for 2011? First ideas we’re looking at:

1. Adding business logic into our quality reporting systems to rapidly identify areas of regressing or concerning quality as well as rapidly evaluate what 100 000 automated tests "mean" to a product build.
2. Leveraging of test coverage analysis tools such as AQTime (we had a great pilot program with RAD on this several years ago. Wonderful tool…)
3. Ramping up our beta systems further. We’re currently using CenterCode "Connect" technology to drive out beta programs, and they really know their stuff. Stay tuned for more on this in the future.

Of course we’ll also continue our drive forward on automated testing and championing the voice of the customers. After all, we’re really doing all this effort for our great customers J

Chris

Share This | Email this page to a friend

posted @ Thu, 29 Jul 2010 22:29:32 +0000 by Chris Pattinson

An iPad for your thoughts?

posted @ Tue, 27 Jul 2010 13:26:00 +0000 by Kyle Hailey

Oracle 10053 SQL Trace Viewer

posted @ Mon, 26 Jul 2010 16:42:00 +0000 by Kyle Hailey

New VST graphics coming in DB Optimizer 2.5.1

posted @ Sun, 25 Jul 2010 21:29:00 +0000 by Kyle Hailey

Delphi, C++Builder, Delphi Prism and RAD Studio this week

posted @ Fri, 23 Jul 2010 16:24:00 +0000 by Tim

Speaking at the InSync10 event in Melbourne

posted @ Fri, 23 Jul 2010 12:05:00 +0000 by Kyle Hailey

"If I can't picture it, I can't understand it." - Albert Einstein

New book recommendation:

Refactoring SQL Applications by Stephan Faroult

I found the book easy to read, well thought out and covers efficiently designing code to access the database, efficiently designing SQL and how to refactor existing applications and testing the results for accuracy and performance. Excellent book. What's that have to do with pictures and visualization? Stephan Faroult includes some new and innovative ways to visually represent SQL statements in such a way as to see patterns and problems. Look to see these methods in a future version of DB Optimizer! In discussion DB Optimizer with Stephan, he mention a quote that rings so true for me:

"If I can't picture it, I can't understand it." - Albert Einstein:

Along the same lines

"It is impossible to even think without a mental picture." - Aristotle:

"Man's mind cannot understand thoughts without images of them." - Thomas Aqunias:

"The evolution of images is a kind of intermediate between that of the perceptions and that of the intelligence." - Jean Piaget:

Mathematics is cognitive process-thinking-that requires the dual coding of imagery and language. Imagery is fundamental to the process of thinking with numbers. Albert Einstein, whose theories of relativity helped explain our universe, used imagery as the base for his mental processing and problem solving. Perhaps he summarized the importance of imagery best when he said, "If I can't picture it, I can't understand it." - NANCI BELL AND KIMBERLY TULEY

One of my favorite examples of the power of graphics to easily, quickly and powerful display quantitative information is Anscombe's Quartet. From http://en.wikipedia.org/wiki/Anscombe's_quartet

Anscombe's Quartet

I		II		III		IV
x	y	x	y	x	y	x	y
10.0	8.04	10.0	9.14	10.0	7.46	8.0	6.58
8.0	6.95	8.0	8.14	8.0	6.77	8.0	5.76
13.0	7.58	13.0	8.74	13.0	12.74	8.0	7.71
9.0	8.81	9.0	8.77	9.0	7.11	8.0	8.84
11.0	8.33	11.0	9.26	11.0	7.81	8.0	8.47
14.0	9.96	14.0	8.10	14.0	8.84	8.0	7.04
6.0	7.24	6.0	6.13	6.0	6.08	8.0	5.25
4.0	4.26	4.0	3.10	4.0	5.39	19.0	12.50
12.0	10.84	12.0	9.13	12.0	8.15	8.0	5.56
7.0	4.82	7.0	7.26	7.0	6.42	8.0	7.91
5.0	5.68	5.0	4.74	5.0	5.73	8.0	6.89

Edward Tufte uses this example from Anscombe to show 4 datasets of x

and y that have the same mean, standard deviation, and regression

line, but which are qualitatively different. - http://matplotlib.sourceforge.net/examples/pylab_examples/anscombe.html

posted @ Thu, 22 Jul 2010 06:59:00 +0000 by Kyle Hailey

Embarcadero recognized as a Microsoft Launch Partner with Delphi Prism 2011

posted @ Tue, 20 Jul 2010 11:12:00 +0000 by Tim

Webinar: Parallel programming with Delphi and OmniThreadLibrary

I just read about this upcoming webinar (July 27) posted by Robert Love on the Delphi non-technical forum.  The webinar is hosted by the Virtual Delphi User Group. The topic is "Delphi parallel programming with OmniThreadLibrary".  The webinar will be presented by Primož Gabrijelčič and is sponsored by GurockSoftware, an Embarcadero Technology Partner.

Webinar Date/Time:

Tue, Jul 27, 2010 7:00 PM - 8:00 PM GMT/UTC
Tue, Jul 27, 2010 3:00 PM - 4:00 PM EDT (East Coast USA)
Tue, Jul 27, 2010 12:00 PM - 1:00 PM PDT (West Coast USA)

Visit TimeAndDate.com for conversions to your local timezones.

For additional information and to register, go to:

http://vdug.org/2010/07/parallel-programming-with-omnithreadlibrary/

This is a virtual user group meeting, using Citrix’s GoToMeeting, and is free for anyone to attend. All you have to do is just sign up.

You can find additional information about development using the open source OmniThreadLibrary by reading Primož’s blog posts at http://17slon.com/blogs/gabr/2008/06/omnithreadlibrary-example-1-beep-world.html. You can also find additional information, including FAQ and download link, at http://otl.17slon.com/index.htm.

Share This | Email this page to a friend

posted @ Mon, 19 Jul 2010 23:03:07 +0000 by David Intersimone

Debugging Initialization and Finalization sections

There was a discussion on the ADUG list recently about Initialization sections. During the discussion I mentioned a technique for stepping through all the Initialization and Finalization sections in your project.

Afterward, I got a few emails from people saying they had not heard of it before, so I thought I’d do up a short video to show how it works.

If the YouTube video above does not show up, here’s a link.

Download

mp4 - 16.8mb (Quicktime)

m4v – 15.4mb (iPhone, etc)

posted @ Sun, 18 Jul 2010 18:07:52 +0000 by Malcolm