[osg-users] Slow optimization and OpenFlight

[Andreas Ekstrand]

Hi Robert,

Thanks for looking into this!

Yes, you can argue that the dataset is awful, but it's actually just 
storing 150k triangles on one graph level, as separate OpenFlight face 
records, which is what the OpenFlight file format offers if you don't 
use its specific mesh nodes that aren't supported by all tools and don't 
offer full control of each triangle. OpenFlight is a modeling file 
format which is surely not visualization-friendly. I think the 
OpenFlight plugin simply uses the Optimizer to convert this to a dataset 
for effective visualization instead of converting to a more optimal 
scene graph to start with - a choice that I can't really say if it's 
good or bad, but the Optimizer should of course handle bad data in the 
best way possible.

The main issue for me though, is that OSG handles this dataset in a much 
slower way than before, hence my re-introduction of the previous 
optimization. But you're completely right, and I realize now, that other 
nodes than drawables aren't handle correctly. I haven't gotten used to 
drawables being nodes yet...

So the fact that the OpenFlight plugin creates a non-optimal scene graph 
and then uses the Optimizer itself to fix this is probably a separate 
and larger problem - if it indeed is a problem. But I believe it's 
necessary to at least get MergeGeometryVisitor to behave as fast as 
before, either by modifying my fixes or by rewriting parts of it. I hope 
you will find the time to decide on this soon, and please let me know if 
I can help!

Regards,
Andreas


On 2017-11-27 14:27, Robert Osfield wrote:
> HI Andreas,
>
> I gave up the valgrind tool=callgrind because it was taking too long
> to complete.  What I did gleen from it was that the run confirms that
> removeChildren() is a bottleneck.
>
> To get some context to why it's a bottlenck I put in some stats output
> from the MergeGeometryVisitor::mergeGroup(..) and got the following
> output:
>
> $ osgconv house_150k.flt test3.osgb
> Optimizer::MergeGeometryVisitor::mergeGroup() 148976
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=3333
>      duplicateList.size()=2324
> Data written to 'test3.osgb'.
>
> What this tells us that this dataset generated by the OpenFlight
> loader is truly dreadful.  It's just appalling how it's handling the
> house_150k.flt database.  I don't know if this can be traced back to
> what the OSG's OpenFlight plugin is doing or whether the tool that
> generated the .flt file is just has a completely awful export tool.
> What I can say is that the fact that the OSG can handle at all such a
> brain dead messed up dataset is a miracle.
>
> The best way to resolve this issue is to refactor the
> MergeGeometryVisitor so it handles these really bad datasets with more
> grace, avoiding the pitfalls of an O(N^2) algorithm.
>
> Having the core OSG "fix" really bad data is not a proper resolution.
> it doesn't fix the fact the the OpenFlight loader is generating such a
> dreadful scene graph.  It might be that the OpenFlight loader is just
> doing what the source database is tell it to do and isn't to blame,
> but like the change MergeGeometryVisitor, it may be worth catching
> this dreadful datacase on load and never generating the broken scene
> graph.  It's better to fix problems like this early rather than let it
> get all the way to the osgUtil::Optimizer to patch fix things up, the
> early it is done the less memory will be used and less memory will be
> fragmented and better the performance will be.
>
> I am afraid I just don't have the time to go chasing fixes to awful
> 3rd party data, I already have enough core OSG issues to do that
> unpaid without taking on more.  I will resolve the
> MergeGeometryVisitor O(N^2) issue but improving the OpenFlight loader
> to handle dreadful 3rd party data better I'll have to defer to the
> community.
>
> Cheers,
> Robert.
> _______________________________________________
> osg-users mailing list
> osg-users at lists.openscenegraph.org
> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openscenegraph.org/pipermail/osg-users-openscenegraph.org/attachments/20171127/5250d545/attachment.html>