The PolyFace Mesh
Two years ago I posted an article on this blog ("The long and short of AutoCAD's PolyFace Mesh") reporting a problem in the way AutoCAD stores a PolyFace Mesh object in a DWG file. I ended the article with:
"The irony of all this is that over the years Autodesk has revised the DWG file format a number of times, more recently to make it more difficult for people like ODA to reverse engineer it. I only wish they had changed the short to a long and added real value to it. There are supposedly billions of DWG files out there. Too bad not a single one can store a mesh with more than 32,767 vertices."
Two years ago I registered a formal service request with Autodesk regarding this issue and at that time I was told that this situation would be remedied only when they next change the DWG format, which was in AutoCAD 2010.
Today, with great disappointment, I have to report that AutoCAD 2010 will not be able to store a PolyFace Mesh containing more than 32,767 vertices. Why? Because they still did not change the blessed short to a long. Why? There were not enough votes for my service request.
I am trying to understand how these big companies work. From this experience I have to assume that they receive so many service requests and bug reports that they simply cannot fix all of them. I can understand that. So things get prioritized based on how many "votes" each request gets. This line of thought leads me to believe that there is nobody in Autodesk who is entrusted with the job of using a little common sense to figure out for himself whether a service request is worth attending to, without looking at the number of votes it has received.
All other CAD systems I know can deal with large meshes. All mesh formats I know use long's instead of short's when storing mesh data. Any high resolution terrain mesh will exceed this limit. Almost all dental scans with exceed the limit a few times over. And considering that Autodesk is laying special focus on 3D in AutoCAD 2010, a little common sense should have been enough to conclude that AutoCAD 2010 should support large meshes as well.
Now lets consider how AutoCAD behaves with large meshes. I open a DWG file containing a single large PolyFace Mesh in AutoCAD 2010. The drawing window comes up empty, but if I run the "list" command type "all" at the "Select objects" prompt, AutoCAD reports that it found one object and proceeds to list it as a PolyFace Mesh. So I know that the mesh is there somewhere. Now let's try and view its properties. Since I cannot select an object that I cannot see, I run the "properties" command and do an "Edit->Select All". This is what I get.
As you can see the PolyFace Mesh has -1218 vertices (yes, that's a negative number) and no faces. Let me explain. In programming parlance when a variable goes out of range, it loops around and goes negative. So you can imagine that this is no small problem. Firstly the mesh is not visible and secondly it has an invalid definition. OK, now lets try and fix it. I run the "audit" command and choose to fix errors. Audit did mention a problem with the mesh (vertex count less than zero) but then reported no errors and fixed nothing. It could not fix it because the PolyFace Mesh data structure is not capable of holding that kind of data. I guess it could have exploded the mesh into individual 3D Face objects, but with the mesh definition in the DWG file corrupted, that does not seem like a possibility either.
And so I still have a DWG file that does not show the PolyFace Mesh and which still has invalid geometry. Why? Because someone at Autodesk didn't find my service request had enough votes. So why didn't customers complain about this? I guess because they probably don't know that the problem exists. You cannot find fault in something that you cannot see or don't know about. The sad thing is that this very serious problem has a very simple no-brainer solution which I provided two years ago to Autodesk. But since they use this weird fire fighting approach of putting out only the largest fires and ignoring the small ones, without employing common sense to think for themselves, the problem still remains.
So for all these years AutoCAD users have been unknowingly working with invisible and invalid PolyFace Meshes, and if Autodesk keeps up with their three year cycle, this problem may be solved only in AutoCAD 2013, that too only if enough customers make a noise about it.
I find it very hard to believe that someone at Autodesk analysed my service request and decided not to fix the problem. I get the feeling that my service request lost on votes and just did not come up for hearing.
"The irony of all this is that over the years Autodesk has revised the DWG file format a number of times, more recently to make it more difficult for people like ODA to reverse engineer it. I only wish they had changed the short to a long and added real value to it. There are supposedly billions of DWG files out there. Too bad not a single one can store a mesh with more than 32,767 vertices."
Two years ago I registered a formal service request with Autodesk regarding this issue and at that time I was told that this situation would be remedied only when they next change the DWG format, which was in AutoCAD 2010.
Today, with great disappointment, I have to report that AutoCAD 2010 will not be able to store a PolyFace Mesh containing more than 32,767 vertices. Why? Because they still did not change the blessed short to a long. Why? There were not enough votes for my service request.
I am trying to understand how these big companies work. From this experience I have to assume that they receive so many service requests and bug reports that they simply cannot fix all of them. I can understand that. So things get prioritized based on how many "votes" each request gets. This line of thought leads me to believe that there is nobody in Autodesk who is entrusted with the job of using a little common sense to figure out for himself whether a service request is worth attending to, without looking at the number of votes it has received.
All other CAD systems I know can deal with large meshes. All mesh formats I know use long's instead of short's when storing mesh data. Any high resolution terrain mesh will exceed this limit. Almost all dental scans with exceed the limit a few times over. And considering that Autodesk is laying special focus on 3D in AutoCAD 2010, a little common sense should have been enough to conclude that AutoCAD 2010 should support large meshes as well.
Now lets consider how AutoCAD behaves with large meshes. I open a DWG file containing a single large PolyFace Mesh in AutoCAD 2010. The drawing window comes up empty, but if I run the "list" command type "all" at the "Select objects" prompt, AutoCAD reports that it found one object and proceeds to list it as a PolyFace Mesh. So I know that the mesh is there somewhere. Now let's try and view its properties. Since I cannot select an object that I cannot see, I run the "properties" command and do an "Edit->Select All". This is what I get.
As you can see the PolyFace Mesh has -1218 vertices (yes, that's a negative number) and no faces. Let me explain. In programming parlance when a variable goes out of range, it loops around and goes negative. So you can imagine that this is no small problem. Firstly the mesh is not visible and secondly it has an invalid definition. OK, now lets try and fix it. I run the "audit" command and choose to fix errors. Audit did mention a problem with the mesh (vertex count less than zero) but then reported no errors and fixed nothing. It could not fix it because the PolyFace Mesh data structure is not capable of holding that kind of data. I guess it could have exploded the mesh into individual 3D Face objects, but with the mesh definition in the DWG file corrupted, that does not seem like a possibility either.
And so I still have a DWG file that does not show the PolyFace Mesh and which still has invalid geometry. Why? Because someone at Autodesk didn't find my service request had enough votes. So why didn't customers complain about this? I guess because they probably don't know that the problem exists. You cannot find fault in something that you cannot see or don't know about. The sad thing is that this very serious problem has a very simple no-brainer solution which I provided two years ago to Autodesk. But since they use this weird fire fighting approach of putting out only the largest fires and ignoring the small ones, without employing common sense to think for themselves, the problem still remains.
So for all these years AutoCAD users have been unknowingly working with invisible and invalid PolyFace Meshes, and if Autodesk keeps up with their three year cycle, this problem may be solved only in AutoCAD 2013, that too only if enough customers make a noise about it.
I find it very hard to believe that someone at Autodesk analysed my service request and decided not to fix the problem. I get the feeling that my service request lost on votes and just did not come up for hearing.
posted by Deelip Menezes at
6:12 PM
7 Comments:
It might have been too complex or time consuming for them to fix it in relation to the number of users complaining about it. But I doubt that this only lost on votes.
They at least fixed some of the problems related to showing, having and working with geometry at locations far away from 0,0. Something I've reported and complained about for a long time.
By
JTB World, At
9:13 PM, February 21, 2009
Things don't get done by programmers, even when the request comes from near the top.
At a press event about 5 years ago, I asked why AutocAD's Clipboard supported the old WMF but EMF.
In front of a roomfull of Autodesk staff and CAD media, the vp in charge of AutoCAD told his people to make sure it got done.
It never happened.
Subsequently, Autodesk did add a bunch of technical-publishing-friendly features, such as gradients, PDF export, and solid fills -- none of which I found useful.
I think this is what happens: rather than fix a small problem, they'd rather throw in a whole new feature set, because it's easier.
Some mesh commands go as far back as version 2.6, which came out in 1987. I think Autodesk programmers don't want to touch that code, because it is so old that no one knows how to fix it.
By
Anonymous, At
10:20 PM, February 21, 2009
"I am trying to understand how these big companies work. From this experience I have to assume that they receive so many service requests and bug reports that they simply cannot fix all of them."
The "Law of Low Hanging Fruit" says "fix the stuff that is easiest to fix first" thus the term low hanging fruit is the easiest to grab from the tree.
Your issue may affect so much within the source code that to try and up the short to a long could break so much within AutoCAD that the return on that particular programming investment may not be worth the resulting investment in time.
However, it wouldn't seem to me to be a big part of the code base as it is a somewhat isolated command.
But then their are lots of those fancy add-on Autodesk applications downstream with their "Custom Objects" (read: Proxy Nightmare Objects) that could be broken by the change...
Perhaps you should get the Open Design Alliance to take the lead in defining this into DWG? Perhaps it is time that the ODA (a non profit organization owned by no one but it's members) take the lead in defining DWG?
As you can imagine, the end users would be better off if a consortium of vendors jointly took over DWG and drove it forward then allowing Autodesk to continue to use DWG as their shield against all competition...
By
Anonymous, At
1:35 AM, February 22, 2009
Autodesk is like GM: their business isn't making cars or software. They forgot that years ago, sold their souls and became banks the year(s) they listed themselves on stock exchanges, signed contracts for revolving credit in exchange for controlling interest from (other) banks.
By
Anonymous, At
6:26 AM, February 22, 2009
Deelip, a quick look at Steve's cadnauseam.com (posting Feb. 6 & 28) and you can view three wise men sitting on a couch explaining exactly how "does Autodesk listen".
This issue is the same as many others, as far as Autodesk goes; they 'say' they are addressing 3D in AutoCAD but of course they simply don't want it used by a single sole!
Just like their handling of MDT: the staff of Autodesk,and most dealers, failed to understand how to use it,and what a good foundation it was, so they adopt an approach that said no one else should and actively work towards making it more difficult by ignoring the functions and or stopping development.
By
Anonymous, At
3:12 AM, February 23, 2009
Jimmy: "It might have been too complex or time consuming for them to fix it in relation to the number of users complaining about it. But I doubt that this only lost on votes."
AutoCAD IntelliCAD: "Your issue may affect so much within the source code that to try and up the short to a long could break so much within AutoCAD that the return on that particular programming investment may not be worth the resulting investment in time."
The only issue I can imagine is saving large meshes to earlier versions that have the "short" limitation. In that case Autodesk could simply split the meshes into chunks, which is something that we do at SYCODE. Or explode it into 3D Faces, which is precisely what AutoCAD does when saving to previous versions that do not support newer entities. Apart from that, I see no reason why someone competent enough to be hired by a company like Autodesk would consider my request unworthy of immediate action.
The fact that AutoCAD's own "audit" command is unable to fix something is reason enough to act. Don't tell me invisible, unselectable geometry is fine with Autodesk.
I am convinced that this definitely lost on votes.
By
Deelip Menezes, At
1:29 PM, February 23, 2009
Deelip,
A little on how this works from Autodesk's side of things. I'll let you decide whether you "lost on votes" or not.
My team (Developer Technical Services - the staff providing the various technical services via ADN) receives bug reports and enhancement requests from ADN members, which we then validate and submit to our various engineering teams.
Once we've logged a Change Request (CR) in our CRM system, we report back to the person submitting the issue with an ID that can be used in future to refer to that CR. We also make sure we ask the person submitting the issue to justify the priority of the issue, so that we can help prioritise dealing with it.
That's our standard procedure... now for how timing can affect things.
You submitted this particular limitation in early December 2006. Now of course you're going to say "but that's given you two years to deal with this!", but let me explain why this unfortunately isn't quite how it works from our side. You submitted the issue at what was unfortunately a very late stage of our development cycle for AutoCAD 2008, and without some justification of external urgency/priority, we didn't have much choice but to defer the handling of the issue for that release. Early on in the development cycle we are much more open to such requests, but as the cycle proceeds we get very conservative - which I know can feel frustrating, but is also a pragmatic approach for maintaining the quality of the AutoCAD product/platform.
OK, your next question is probably "so why wasn't this carried over automatically for handling in the next release?" Once the CR is in the system and gets deferred for a particular release, we wait for one of two events to occur: either the developer submitting the issue gets back to us with a "business case" so that we can help engineering decide where to prioritise, or we have other developers submit the same issue, which then get linked into the same CR and cause us to ask engineering to take another look into it. In this case we unfortunately received neither a further justification from your side for making the change nor any subsequent report from a developer complaining about the issue.
When I say a business case, here's what we actually asked for in our email to you:
"If you would like to provide a business case towards implementing this Change Request, please append it to this current case and we will forward it to our development team. Your providing a business case can increase the urgency of your request with our engineering team. Please provide as much information as possible, including the impact of this issue on your application, the number of users affected, and the expected revenue impact for both you and Autodesk. "
To be clear about this... we don't necessarily need a lengthy, detailed or even a financially impressive justification for us to prioritise handling an issue, but we do need to hear from someone (whether you or someone else) that this issue is worthy of resource investment. If we had received some kind of justification, even something like "as models gets bigger, it's going to be increasingly important for customers to deal with huge meshes", then that would probably have been enough to at least have someone from engineering evaluate the amount of work required to address this.
So, a couple of suggestions for getting changes made to Autodesk software: firstly, always let us know when an issue is important to you. I know that might sound silly, but when you have software that is used by millions of end-users with a development community numbering in the thousands of companies, squeaky wheels get more attention. So be vocal.
Also, keep track of your pending CR IDs and don't be shy about sending them into my team to get status updates (and to give us a prod). We can tell you whether issues are basically in a holding pattern (i.e. a black hole) or whether something is likely to happen. As a matter of course my team now sends status updates to developers when the status of their issues changes, but that's only a process we've had in place for a year or two (and issues submitted prior to that won't be covered).
I hope this helps explain some of what went on behind the handling of this issue.
Kean Walmsley
Senior Manager
Developer Technical Services
Autodesk
P.S. I would hope that the work done on sub-division surfaces in AutoCAD 2010 would also mitigate the need for this change, but as I'm not myself an expert on 3D modeling please feel free to correct me.
By
Anonymous, At
12:01 AM, February 24, 2009
Post a Comment
Subscribe to Post Comments [Atom]
<< Home