[osg-users] VBO Bug with 3.6.1 and Normal Arrays

[Daniel Emminizer, Code 5773]

Hi Robert,

Oops -- I sent this earlier today but apparently to the bounces list; that explains the confusion on GitHub -- my mistake.  This was supposedly sent right before I posted the PR.  Here's the original text:


I think I found a bug in 3.6.1.  I am loading a FLT model and it's causing a crash in my application to draw it.  The same model does not crash in OSG 3.4.  I think I've finally tracked down the cause and have a candidate solution too.

A few weeks back I saw a similar crash in my own code, and figured it was due to incorrect usage of the binding flags on osg::Array.  Much of our code was using osg::Geometry::setNormalBinding() (and related methods).  During debugging, I was able to determine my normals were crashing in 3.6, and the problem went away when I used the osg::Array(osg::Array::Binding) signature -- i.e. assigning a binding on construction.  At the time I thought it was something I was doing wrong and moved on.

The problem showed up again earlier this week but not in my code, and not manifesting in exactly the same way.  Here's the run-down of what's going on:

- Loading FLT model
- FLT model loads a face, which has vertices, textures, and normals
- FLT uses getOrCreateNormalArray(), which allocates an array (BIND_UNDEFINED) and sets it on the geometry
- Geometry::addVertexBufferObjectIfRequired() is called on the normals, but nothing done due to BIND_UNDEFINED
- FLT later sets normals to BIND_PER_VERTEX appropriately, which is a direct set and does not do anything to the Geometry's VBOs
- First frame starts to render
- Geometry::drawVAImpl calls vas->setNormalArray()
- VertexArrayState::setArray() calls new_array->getOrCreateGLBufferObject(), which returns 0.  This is the first major problem.
- Because vbo is NULL, unbindindVertexBufferObject() is called, leading to GL_ARRAY_BUFFER to go to 0
- vad->enable_and_dispatch() gets called and does glVertexAttribPointer() with a non-NULL data ptr, which is a GL error because array buffer is 0

Unwinding the error:
- enable_and_dispatch() shouldn't be called if ptr is non-NULL and no GL_ARRAY_BUFFER is 0
- GL_ARRAY_BUFFER is set to 0 because there is no VBO set up for the normal array
- There is no normal array because the only place it seems to be created is in setNormalArray(), which fails because at that time, it is BIND_UNDEFINED
- Binding gets swapped from BIND_UNDEFINED to BIND_PER_VERTEX after setNormalArray(), leading to the error

There are several possible solutions I can see.  You can probably see more:
1) Change FLT plugin to assign array binding per vertex on construction of array.  Seems poor because invariably this bug is crashing other code -- maybe it's the cause of the DXF that Brian Hutchison reported earlier this week?
2) Update Array::setBinding() to create the VBO if needed.  I do not know if this is possible nor how to do it.
3) "Lazily" detect this issue somewhere in the rendering calls and create VBO there if necessary



PR 554 was an attempt at approach #3 but I agree with your assessment on GitHub.  It does not solve the problem in all cases.

Attached is a demo of the problem that generates a console warning.  More complex scenes can cause crashes.  The red triangle has the problem, but the green one does not.

 - Dan

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: CMakeLists.txt
URL: <http://lists.openscenegraph.org/pipermail/osg-users-openscenegraph.org/attachments/20180601/f58ef8af/attachment.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: main.cpp
URL: <http://lists.openscenegraph.org/pipermail/osg-users-openscenegraph.org/attachments/20180601/f58ef8af/attachment.ksh>