Coders vs Programmers: Big Ball of Mud

In a recent post, Scott Bellware rants about the Microsoft developer personas Mort, Elvis, and Einstein and the software development methodologies and tools Microsoft promotes through them; and also throws in a few punches at Rocky Lhotka's opinions on TDD and software development.

The post has spurred a lot of comments, one of them from a "programmer" that feels that all these efforts to define better methodologies and to build better tools to support them, is the search for the holy grail and, in fact, the real problem of the software industry:

"Some people actually have a life and the holy grail of software engineering is not it. Go home, relax, get a life. Let the rest of us go to work, do our thing and go back to living. Working a 40hr/wk job with real deadlines and with a real family life or life that has it own requirements and demands quite honestly doesn't leave much time to learn all this new methodolgy. Especially when it changes so quickly.

That my friend, is the reason why the industry is the way it is. PLAIN AND SIMPLE! Perhaps if we were all workaholics that had no other ambitions in life but to reach the holy grail of software development nirvana, then things may be the way you want. Your wishing against reality."

I have heard this kind of arguing before, it is not uncommon. In fact, it is part of a pattern called the Big Ball of Mud. I really recommend reading the whole BBoM article, it even contains a finding to support the "get-a-life/workaholic/nirvana/against reality" arguments above:

"it is inevitable that 'on average, average organizations will have average people'. One reason for the popularity and success of BIG BALL OF MUD approaches might be that this appoach doesn't require a hyperproductive virtuoso architect at every keyboard."

This leads me to my own little quest at work these days:

Coders vs Programmers.

A lot of developers are campers inside big companies, hiding in their cozy foxholes, doing their thing, blaming the everchanging tools and methodologies (and requirements) for not being able to produce anything but a BBoM. This kind of (demotivated) developer is what I call a "coder" (see also the related "code monkey - type 1").

Perhaps if they could make their code just a little better by using some methodology, they could go home knowing that their solution is built to last, and in addition, be proud of their workmanship ("Software Craftsman"). There is no conflict between having a life, working 40 hours a week, and producing quality code.

The real problem of the software industry is that there are way too many coders, and too few programmers. Programming is more than just emitting code that compiles and works; I state that coders produce BBoM even when not constrained by time or money.

I believe that the whole software quality issue revolves around being genuinely interested in software design issues, striving to be an architect rather than a "swamp guide" (from BBoM). The book "Zen and the Art of Motorcycle Maintenance" by Robert M.Pirzig contains a lot of good thoughts about quality, and "pride of workmanship" is a central theme in the book. I recommend reading ZAMM, it will give you some philosophical insight to the topics discussed in the BBoM article.

VSMDI file - The weak spot of the VSTS test system

The Visual Studio Test Metadata File (.VSMDI) has caused some grief in our ongoing project, in which we use Team Foundation Version Control (TFVC) also. We have a solution with six projects for the application layer, including one common test project.

What typically happens is that VS2005 automatically checks-out the metadata file even when a developer do not work on a unit test, or touch the test project or the test manager. As the check-out is silent, the developer does not notice this. After doing some coding, the developer will try to check in the pending changes. In the meantime, other developers will also have done some unit tests, coding and check-ins, causing a conflict on the .VSMDI file. At this point, VSMDI is a disaster waiting to happen...

Never ever use 'auto merge' to resolve check-in conflicts on the .VSMDI file. The chances are that the file will get corrupted. If you have a conflict, I recommend discarding your local changes, get the latest version of the .VSMDI file from TFVC (use 'force get...' and 'overwrite...' if necessary), then re-apply your changes to the tests and test lists, and check the file in immediately. Do not keep the .VSMDI file checked out longer than strictly necessary.

A sure sign of a corrupted VSMDI file is a Test Manager that never stops loading, the progress bar teasing you at "almost there" forever.

[UPDATE] I recommend that you reconfigure the .VSMDI file TFVC settings to not allow merging or multiple check-out. This is done in the 'Edit File Type' (see image above), which is available at 'Team-Team Foundation Server Settings-Source Control File Types' in VSTS.

VSTS will also check-out .VSMDI even when doing a get latest on the whole solution. I recommend my developers to immediately do a 'undo pending changes' on the file when this happens. The check-out seems to be quite unnecessary, why can't VSTS leave the file as-is until I touch the test system ? I know that VSTS monitors the disk for changes to relevant files, but I would prefer that the test metadata only got updated when building the solution.

Due to the VSTS test system's unreliable handling of the metadata file, we from time to time get "The test 'TestName' does not exist in the test list. It may have been moved, renamed or deleted". Yesterday, the 'Test Manager' showed all tests duplicated in the test lists. Today, another developer saw just some chinese characters when opening the metadata file. Sometimes we even get a second VSMDI file (e.g. Application2.vsmdi) in our solution, or no VSMDI file at all.

Some of my Objectware co-workers have given up TFS for unit testing due to this, and are using NUnit and CruiseControl.NET in their projects instead.

Published at MSDN - InfoPath Developer Home

My blog post 'Using XPath preceding-sibling in InfoPath rules' is published on the InfoPath Developer Portal. Nice to get noticed over at MSDN.

System.Transactions - Nesting Scopes, Enlist Connection in Transaction

In a previous post I decribed how the Lightweight Transaction Manager (LTM) of System.Transactions has the bad habit of "promoting" to full-blown MS DTC transactions even when using a single SQL Server 2005 database. You can use the .Suppress option to exclude some code from being part of a transaction and thus stay LTM even when using multiple open connections against the SS2K5 database.

To circumvent this limitation (feature), I have implemented a data session object used to pass an open SqlConnection object around my biz logic and data access logic. Last week, when implementing a unit test for some logic involving a method in the call stack using the .Suppress option, I noted that changes made to the data was not rolled back when the error I was debugging, occurred.

Some experimentation with nested transaction scopes lead me to conclude that I needed to open the common, shared connection within a TransactionScopeOption.Required using block before passing it around to get the correct behavior (rollback on error, etc). It seemed that opening the connection in a .Suppress block would suppress transactions even in nested 'required' scopes.

I posted my findings at forums.microsoft.com and got an "expected behavior" from Alazel Acheson of the ADO.NET team. The explanation is that the connection will enlist in a transaction only when opened, thus when opened in a suppress block, the connection will never use a transaction. To ensure that a connection gets enlisted in a transaction, you need to actually open the connection within each using TransactionScope block.

The need to stay LTM is a very common requirement in the System.Transactions community, and Acheson has implemented a DbConnectionScope class that simplifies the process of using a common connection in nested transaction scopes. The DbConnectionScope class has been further refined to include nesting options to provide detailed connection reuse control. Good work, Alazel!

I recommend anyone who has implemented their own LTM data session object to switch to the DbConnectionScope. I strongly advice never to roll your own transaction management mechanism, the chances are that you will not be able to make it 100% bulletproof.