[osg-users] LineSegmentIntersector NaN warnings on zero-scaled nodes.

[Jannik Heller]

Hi,

The LineSegmentIntersector creates NaN warnings when applied to a node with zero scale. I've attached a 3 line change to osgintersection.cpp to reproduce the issue.

The NaN's first pop up when OSG tries to invert a zero matrix here: https://github.com/openscenegraph/osg/blob/master/src/osgUtil/LineSegmentIntersector.cpp#L294

Check with the following code snippet:

Code:

        osg::Matrix zero(0,0,0,0, 0,0,0,0, 0,0,0,0, 0,0,0,1);

        if (zero.invert(zero))
            std::cout << "inverted " << zero << std::endl;



invert() returns true but the inverted matrix is 

inverted {
	-nan -nan -nan 0 
	-nan -nan -nan 0 
	-nan -nan -nan 0 
	nan nan nan 1 
}

This looks like a bug in the Matrix invert function.

However even if the matrix was still zero instead of NaN, there is another issue here:

    osg::ref_ptr<LineSegmentIntersector> lsi = new LineSegmentIntersector(_start * matrix, _end * matrix);

Multiplying a vec3 with a zero matrix will cause NaN's, because there is an 1.0 / (mat * vec ...) in the Matrix::preMult(Vec3) function.

Not sure on the best way to fix this. Should we add explicit checks for zero scaled matrices in the LineSegmentIntersector, or fix the Matrix and Vec classes so they don't produce NaN's under any circumstances for (borderline) valid input? The latter seems the cleaner fix but would have a performance impact all over OSG.

Thank you!

Cheers,
Jannik

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=65720#65720



-------------- next part --------------
A non-text attachment was scrubbed...
Name: osgintersection.cpp
Type: text/x-c++src
Size: 11140 bytes
Desc: not available
URL: <http://lists.openscenegraph.org/pipermail/osg-users-openscenegraph.org/attachments/20151121/1a2aa49c/attachment-0002.cpp>