.Net String vs StringBuilder
I’ve always been a little shady on the reason the StringBuilder class was created in dotnet. Obviously it has some advanced string formatting functionality which allow you to create complex composite strings easily, but what else does it provide. The answer to that question came to me when I was debugging our internal translation application written in C#.
We were receiving out of memory exceptions on and off from the app. After analyzing the code by hand for several hours I was still lacking an answer as to why this program swelled to 3 gigs of memory usage. It did store a massive size of string data in string classes, but since the database containing the strings was only 90 megs, it made no sense that string data would be the reason the program was eating memory.
However I had not taken one important thing into account, .Net strings are immutable, and in several places we we’re playing fun games with the strings values to properly format them to the output, calling string functions such as SubString and Trim liberaly. Turns out 2.7 gigs of memory we’re being used by all the intermediate string values as the string mutated to its final output.
The solution here was to use the StringBuilder class. After replacement memeory usage dropped down to 300 megs for the entire program, simply because we were not creating a bunch of invariant strings as we mutated our data.
Moral of the story, when handling string data that you expect to be changing alot in place, or strings that are quite hefty in size, StringBuilder is hands down a better solution, especially if memory use is a prime concern.
Share This | Email this page to a friend
Posted by David Lock on July 20th, 2007 under Uncategorized |
RSS Feed
July 20th, 2007 at 7:16 pm
Part of the lesson here is that VCL.Net’s dependance on treating win32 strings as dotNet Strings is a fundamental problem in the concept that you can just move win32 apps to dotNet apps.
True, a lot of large string manipulation in win32 delphi can be bad, but in dotNet’s world, large amounts of string manipulation of ANY size is bad.
Without realizing it, the move from win32 Strings to dotNet strings is more significant and potentially more damaging than the move from ShortStrings to LongStrings.
In nearly every dotNet presentation I have ever seen from Microsoft, this point about immuatable strings and allocation patterns is always a 20 minute presentation for exactly this reason.
July 21st, 2007 at 3:53 am
I do find it surprising that programmers have to worry about these sort of things – not exactly progress! A clever system could do this for you: i.e. if a function returns a string and includes code like Result := Result + ‘A’, then a stringbuilder is created behind the scenes (I’m sure there are lots of problems with this :-)). The big problem is when you first create a function which manipulates a string – as that function grows over time the performance degrades. Constantly having to think about whether one should use a string or a stringbuilder is not productive.
July 21st, 2007 at 11:06 am
That’s one of the little gotcha’s a lot of people learned when they first moved to .NET 1.0.
I was "lucky" enough to first learn .NET under ASP.NET (as I still use Delphi for our desktop apps) and in the ASP.NET camp everyone knows to always use a StringBuilder when outputting anything more than a few lines of text to a page.
It’s one of odd things about the split between web and desktop devs. In almost any 101 tutorial on ASP.NET they strongly advise you to always use a StringBuilder while on 101 desktop application .NET tutorials they seem to never mention it. Same language yet obviously two different types of devs writing the articles.
I’d guess it’s because web devs have always had to worry about how to best get string data down and were often advised to chain together Response.Write()’s instead of building a string and then sending it, so a StringBuilder seemed pretty natural.
July 21st, 2007 at 9:58 pm
David -> The problem isn’t a single string concatination, but rather a combination of automatic garbage collection and a high number of string changes causing a high number of abandoned string objects awaiting collection.
String builders are only useful for when you do a number of operations to a string, otherwise you have just added ANOTHER abandoned object to be GC’ed. The goal of a string builder is to reduce the total number of objects used instead of increasing it.
Any system requires you to have some basic knowledge of how it works to get best advantage. You can write bad code that works or you can write good code that works even better.
July 22nd, 2007 at 3:05 am
-> C Johnson - yes I was assuming a number of changes, but at what point one changes from a string to a stringbuilder seems less clear. Sometimes one might end up using a stringbuilder to start with ‘to be safe’ in case a function grows or is used with longer strings than first expected.
For your last point (’any system requires you…’) - thanks I didn’t realise that
July 22nd, 2007 at 3:21 am
To debug memory issues in .NET you should use WinDdbg and !dumpheap -stat command. It will scan all memory and print out how much memory is taken by different objects/classes. If you want to trace the source of the allocation, you should take a look at CLR profiler.
Here are some examples:
http://blogs.msdn.com/tess/archive/2007/03/15/asp-net-case-study-tracing-your-way-to-out-of-memory-exceptions.aspx
CLR profiler:
http://www.microsoft.com/downloads/details.aspx?familyid=86ce6052-d7f4-4aeb-9b7a-94635beebdda&displaylang=en
July 23rd, 2007 at 7:22 am
StringBuilder could/should also be used to create a buffer area when calling a Win32 DLL when needing to pass back a string. The size of the buffer area must also be passed to the DLL.
August 3rd, 2007 at 9:19 pm
Agreed that there are certain situations where the StringBuilder is useful, but in general I think it is not necessary and probably overused by most developers.
Check out my reasoning here:
http://codeslammer.wordpress.com/2007/07/07/do-not-use-the-stringbuilder/
August 3rd, 2007 at 9:19 pm
Agreed that there are certain situations where the StringBuilder is useful, but in general I think it is not necessary and probably overused by most developers.
Check out my reasoning here:
http://codeslammer.wordpress.com/2007/07/07/do-not-use-the-stringbuilder/
November 1st, 2007 at 9:45 am
Come on David, it’s time for a new post isn’t it? Show us some good news. =)