Bugs and Known Issues
In the comments to my last post "Users and Non-Users", a couple of SolidWorks users leveled what I believe to be a pretty serious accusation against SolidWorks. They claim that SolidWorks finds bugs in their software, does not fix them due to lack of time and goes ahead and releases the software. As a software developer I find this pretty hard to believe. But then maybe that's because I draw a clear distinction between "bugs" and "known issues".
Known issues are problems in the software that programmers have come to know of and are unable to fix for a variety of reasons, some of which may be beyond their control. For example, the compiler used to create the software may have a bug that prevents a fix. The data being sent to the software for processing may be just too large. In such cases, programmers hard code in a check just before a problem can occur so that the software aborts gracefully and does not crash. This information and the behavior related to it is then passed on to the people writing the documentation and they add it to "Known Issues" section of the product documentation. This is perfectly fine with me and should be fine with any reasonable user as well.
A bug, on the other hand, is something that the programmers have come across or users have reported, the cause of which is yet to be ascertained or the solution to which is yet to be implemented. Personally I consider releasing software with such unfixed bugs, without converting them to known issues, is sacrilege. It is unpardonable. It is like Boeing shipping a plane to a customer comforting itself with the hope that the chance of it crashing is fairly low. OK, maybe I am exaggerating a little here, but I think you get my point.
Earlier I mentioned that programmers may not be able to fix bugs for a variety of reasons. Time should not one of them. That's why, in principle, I am against this annual release cycle thing that some CAD software vendors have got going. Now we all know that it makes good business sense to do that. But having said that, I would like to share something which an extremely successful businessman, Bob McNeel, said to me at COFES 2009 when I asked him when Rhinoceros 5.0 would be released. He replied, "I don't know". I almost spat out the beer that I was drinking. He went on to explain, "Of course, we do have a date in mind, but it all depends upon how the software looks like at that point in time. We have never had an annual release cycle. That would just makes a mess of everything."
Great minds think alike, eh?
Known issues are problems in the software that programmers have come to know of and are unable to fix for a variety of reasons, some of which may be beyond their control. For example, the compiler used to create the software may have a bug that prevents a fix. The data being sent to the software for processing may be just too large. In such cases, programmers hard code in a check just before a problem can occur so that the software aborts gracefully and does not crash. This information and the behavior related to it is then passed on to the people writing the documentation and they add it to "Known Issues" section of the product documentation. This is perfectly fine with me and should be fine with any reasonable user as well.
A bug, on the other hand, is something that the programmers have come across or users have reported, the cause of which is yet to be ascertained or the solution to which is yet to be implemented. Personally I consider releasing software with such unfixed bugs, without converting them to known issues, is sacrilege. It is unpardonable. It is like Boeing shipping a plane to a customer comforting itself with the hope that the chance of it crashing is fairly low. OK, maybe I am exaggerating a little here, but I think you get my point.
Earlier I mentioned that programmers may not be able to fix bugs for a variety of reasons. Time should not one of them. That's why, in principle, I am against this annual release cycle thing that some CAD software vendors have got going. Now we all know that it makes good business sense to do that. But having said that, I would like to share something which an extremely successful businessman, Bob McNeel, said to me at COFES 2009 when I asked him when Rhinoceros 5.0 would be released. He replied, "I don't know". I almost spat out the beer that I was drinking. He went on to explain, "Of course, we do have a date in mind, but it all depends upon how the software looks like at that point in time. We have never had an annual release cycle. That would just makes a mess of everything."
Great minds think alike, eh?
posted by Deelip Menezes at
9:22 PM
8 Comments:
So it is not a bug if they tell you it is there? That doesn't sound right.
Beware anyone saying they are shipping bug free software. Its because they stopped testing not because they fixed them all.
By
Burhop, At
11:25 PM, June 14, 2009
Deelip,
It is absolutely true that SW ships software with bugs that were pointed out in beta testing. I can't make sense of the semantic difference between bugs and known issues, but some of the bugs get fixed in a service pack, so they aren't unfixable.
Also, not that it excuses SW from shipping shitty software with sp0, but the size of the SW code is somewhat different from the size of Sycode code, isn't it?
This bug issue is really at the heart of some user's discontent. Well, that along with the fact that some people continue to pay support, volunteer to find bugs in beta, and still have to live with the same bugs in production software.
By
Matt, At
4:35 AM, June 15, 2009
I've responded in the previous post in general but like Matt - as a user - there is no difference between a bug or known issue. All it means is that the bug is further down the stages perhaps, or that the engineering effort required to resolve it is at odds with the release dates.
Everybody works to deadlines. I do. My customers do. Even Boeing do. There always comes a time when you have to stop and release it. When you say it is good enough and we can make further improvements later when we have had more feedback. If we didn't have this mentality nothing would ever get built. Ever.
By
Kevin Quigley, At
5:00 AM, June 15, 2009
Burhop: "So it is not a bug if they tell you it is there? That doesn't sound right."
Correction. It is not a bug if they tell you it is there "and stop it from crashing".
Matt: "I can't make sense of the semantic difference between bugs and known issues."
Same as above. The difference between the two is the one crashes and the other does not.
Matt: "but the size of the SW code is somewhat different from the size of Sycode code, isn't it?"
So are the number of programmers, testers and users involved. ;-)
Bottom line. If you found a bug, reported it and it still shows up after a service pack or in a new version, without the programmers converting it into a known issue (preventing a crash) then it is a big problem and someone should be held accountable.
By
Deelip Menezes, At
6:20 AM, June 15, 2009
Kevin: "There always comes a time when you have to stop and release it."
I understand that completely. We work the same way. And that's the reason why this annual release cycle is a bad idea. So even if you have to release something you have enough time to code in enough checks so that the software stops crashing.
For example take a modeling kernel like ACIS. If you push and pull stuff around in SpaceClaim, it will not always work and you get a decent "modeling failure error" and the operation aborts. The software will not complete the operation, but it will not crash. That's my whole point.
I also understand that there will also be times when you cannot code in checks for each and everything. But the situation should not be as bad as you guys have been pointing out.
By
Deelip Menezes, At
6:35 AM, June 15, 2009
Deelip, I think you're way off with your idea that a publicly documented problem that aborts gracefully is somehow not a bug. However, I couldn't agree more with the 12-month cycle being the root cause of lots of bugs/issues/whatever making it into shipping software. I'll go into more detail on my own blog.
By
Steve Johnson, At
8:34 AM, June 15, 2009
Steve,
Take a look at this: "Critical Bugs and Stop Ship Issues" - http://www.deelip.com/2009/06/critical-bugs-and-stop-ship-issues.html
By
Deelip Menezes, At
10:23 AM, June 15, 2009
I'm a TurboCAD user, each new version comes out in May sometime. I don't touch it for a month, just keep an eye on all the forums and by the time I pick it up I have a pretty good idea what I’ve got and where snags might lurk.
By
Unknown, At
4:45 PM, June 15, 2009
Post a Comment
Subscribe to Post Comments [Atom]
<< Home