[osg-users] RayIntersector with KdTree causes a crash

[Ale Maro]

Hi Robert,


I attached the stack trace for the modified osgPick example.

We analyzed the LineSegmentIntersector::intersect code (see below) and we saw that if we build KDTree AND the transofmation matrix has negative scale then "geometry->getVertexArray()" returns an empty array and "settings._vertices" becomes empty.
In debug an assert stops the execution (see stack trace), in release the execution continue but the geometry is not shown


Code:

void LineSegmentIntersector::intersect(osgUtil::IntersectionVisitor& iv, osg::Drawable* drawable,
                                       const osg::Vec3d& s, const osg::Vec3d& e)
{
    if (reachedLimit()) return;

    LineSegmentIntersectorUtils::Settings settings;
    settings._lineSegIntersector = this;
    settings._iv = &iv;
    settings._drawable = drawable;
    settings._limitOneIntersection = (_intersectionLimit == LIMIT_ONE_PER_DRAWABLE || _intersectionLimit == LIMIT_ONE);

    osg::Geometry* geometry = drawable->asGeometry();
    if (geometry)
    {
        settings._vertices = dynamic_cast<osg::Vec3Array*>(geometry->getVertexArray());
    }






Cheers,
Ale

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74635#74635




Attachments: 
http://forum.openscenegraph.org//files/stacktrace_370.txt