Critical Bugs and Stop Ship Issues
After reading my opinions in the last couple of posts and comments on the issue of bugs and known issues some readers of this blog think that I have lost it. I now know why. It appears that my readers and I are not talking about the same thing, and maybe I am to blame for that.
Throughout this discussion I have used the word "crash" in conjunction with "bug" to signify that I am talking about bugs that cause a crash, not "this-tool-does-not-work-that-way or that-tool-does-not-work-this-way" kind of logical bugs. I am talking Crash! Boom! Bang! here.
As always, things are best explained by means of an example. So I will use one that I talked about on this blog earlier - the AutoCAD PolyFace Mesh issue. After I reported the problem to Autodesk (for the second time), they analyzed it, admitted it was a bug, limitation or whatever you want to call it, after which they arrived at the conclusion that it was not a "stop ship issue". Means it didn't cause a crash. Means there was no need to stop shipping the product. Surely reporting that a mesh has -1218 vertices and 0 faces (see image) is as big a bug as can be. But it didn't cause a crash. So they went ahead and released AutoCAD 2010 with this bug/limitation. And this kind of a bug is precisely what I have NOT been talking about.
Compare this to what Kevin Quigley said in a comment:
"I am talking about bugs that are logged in the system as bugs, that are perhaps not critical but cause features to fail or behave inconsistently. These are bugs, they are logged as bugs and they are released as known bugs - just look at the release notes of any application under 'known issues'. In your utopian programmers world no software should be released like this? Really? That is a recipe for bankruptcy."
Clearly we are talking two very different things here. Lets be clear. Bugs that cause crashes are critical bugs, stop ship issues or whatever fancy term that a developer many choose to use to describe them. My point is that no self respecting developer, whether it is a mammoth company like SolidWorks or a puny outfit like SYCODE, will (or rather should) ship a software with a critical bug that causes a crash. And if they have to release the software for whatever reason, they will (or rather should) convert it into a known issue by hard coding a check just before the crash occurs which make the operation abort gracefully. I don't think I can be more clear that that.
Now if any user (SolidWorks or otherwise) can show me a critical bug (which caused a crash) that was reported and which was not addressed in a future service pack or new version, I would appreciate it.
Of course, contrary to what some think, I happen to live in the real world and can understand if a few critical bugs slip though the cracks. But for users to come up in arms like the way they have, these issues have to be much more than just a "few".
Throughout this discussion I have used the word "crash" in conjunction with "bug" to signify that I am talking about bugs that cause a crash, not "this-tool-does-not-work-that-way or that-tool-does-not-work-this-way" kind of logical bugs. I am talking Crash! Boom! Bang! here.
As always, things are best explained by means of an example. So I will use one that I talked about on this blog earlier - the AutoCAD PolyFace Mesh issue. After I reported the problem to Autodesk (for the second time), they analyzed it, admitted it was a bug, limitation or whatever you want to call it, after which they arrived at the conclusion that it was not a "stop ship issue". Means it didn't cause a crash. Means there was no need to stop shipping the product. Surely reporting that a mesh has -1218 vertices and 0 faces (see image) is as big a bug as can be. But it didn't cause a crash. So they went ahead and released AutoCAD 2010 with this bug/limitation. And this kind of a bug is precisely what I have NOT been talking about.
Compare this to what Kevin Quigley said in a comment:
"I am talking about bugs that are logged in the system as bugs, that are perhaps not critical but cause features to fail or behave inconsistently. These are bugs, they are logged as bugs and they are released as known bugs - just look at the release notes of any application under 'known issues'. In your utopian programmers world no software should be released like this? Really? That is a recipe for bankruptcy."
Clearly we are talking two very different things here. Lets be clear. Bugs that cause crashes are critical bugs, stop ship issues or whatever fancy term that a developer many choose to use to describe them. My point is that no self respecting developer, whether it is a mammoth company like SolidWorks or a puny outfit like SYCODE, will (or rather should) ship a software with a critical bug that causes a crash. And if they have to release the software for whatever reason, they will (or rather should) convert it into a known issue by hard coding a check just before the crash occurs which make the operation abort gracefully. I don't think I can be more clear that that.
Now if any user (SolidWorks or otherwise) can show me a critical bug (which caused a crash) that was reported and which was not addressed in a future service pack or new version, I would appreciate it.
Of course, contrary to what some think, I happen to live in the real world and can understand if a few critical bugs slip though the cracks. But for users to come up in arms like the way they have, these issues have to be much more than just a "few".
posted by Deelip Menezes at
9:18 AM
6 Comments:
You've moved your "bug" definition a loooong way from the original statement from Devon, which was "Therefore, due to a lack of time, they knowing [sic] release software that has bugs and that has not been thoroughly tested."
You already moved from the simple "bugs" to "critical badly-behaved crash-causing bugs", and now you're providing another cop-out in that it doesn't count if it is later addressed in a service pack or new version! Come on, keep the goalposts still.
I have little doubt that the original statement is as accurate for SolidWorks as I know it is for Autodesk, and most likely every developer of highly complex applications. In fact, I believe that even your very much narrowed-down definition of the original accusation can be met by at least one company (which shall remain nameless). You're unlikely to get anybody giving you concrete examples, because that would involve breaking an NDA.
Anyway, I don't think there is such a thing as a stop-ship bug at Autodesk any more. The product will ship on time, ready or not. Is this acceptable? No. Is this any worse than other companies with 12-month cycles? I'm guessing, but I'd say probably not.
By
Steve Johnson, At
6:47 PM, June 15, 2009
Steve, I write software for a living. By virtue of being human, my software has bugs. That is a fact of life I cannot change. Do you think I am that daft to suggest that all bugs (all types, shapes and sizes) should be removed before shipping a piece of software? I wouldn't have made any money in all these years, my friend ;-)
This is what I said in my comments:
=============
"The difference between the two [bug and known issue] is that one crashes and the other does not."
"Bottom line. If you found a bug, reported it and it still shows up after a service pack or in a new version, without the programmers converting it into a known issue (preventing a crash) then it is a big problem and someone should be held accountable."
"For example take a modeling kernel like ACIS. If you push and pull stuff around in SpaceClaim, it will not always work and you get a decent "modeling failure error" and the operation aborts. The software will not complete the operation, but it will not crash."
==========
I have used the word "crash" over and over again. I was obviously talking about bugs which cause a crash, not logical, functionality-related bugs.
By
Deelip Menezes, At
7:30 PM, June 15, 2009
Now were are talking about the same thing Deelip :-)
I do a lot of beta testing and in all the year's I've done it I've not come across a known and verified crashing bug that ships with release software except in extreme situations (such as it being discovered after the DVDs have gone to production, or where it affects a tiny minority of users and there are steps to avoid it. Verified being the key element there as unless the bug can be verified it doesn't get fixed as it may be a local installation issue.
Maybe that's the problem? So many things can cause applications to bomb that have little to do with the application itself - from graphic display driver conflicts to power variations in the hardware. When it happens though the user just blames the crappy software!
On a personal note I have to say that ALL software I use is vastly more stable (in the crashing sense) than it was 5 years ago. How many recall visiting trade shows and watching the poor demo jocks struggle through a demo with software bombing every few minutes? I can think of one Alias one back in 1994. The poor guy was nearly in tears at the end!
By
Kevin Quigley, At
8:07 PM, June 15, 2009
Kevin,
Its nice to know that we are on the same page. Looking at it from where you stand, I guess you must have thought I was a real nut job, right? ;-)
By
Deelip Menezes, At
9:05 PM, June 15, 2009
Ha! Not at all. The real nut job is the one I look at in the mirror in the morning :-)
By
Kevin Quigley, At
12:53 AM, June 16, 2009
paranóia hein fih... pensei que o negócio tava mais organizado porra...
By
Unknown, At
1:12 AM, February 01, 2010
Post a Comment
Subscribe to Post Comments [Atom]
<< Home