Virtual Planet Builder
How to compress an existing database
- Details
- Category: Virtual Planet Builder
- Published: 14 February 2013
- Written by openscenengraph
- Hits: 13195
Versions of OSG since around November 2008 support the compression of .ive files using the -O "compressed" writer option. I've tested the procedure on a 22GB database and the resulting database was reduced to 13GB.
Overview
The idea behind this example is to take an existing database that was not built using compression and use osgconv to compress each .ive file into a new database. This procedure is not needed if one creates the database with compression from the start by passing -O "compressed" to vpbmaster.
Assumptions
* Let old_db = the root of the old database, i.e. the directory where the top-level .ive file is located.
* Assume that the parent dir of old_db is writable.
* Let new_db = the root of the new database. It would be a sibling of the old_db directory.
* Using bash on Linux.
Procedure
Turn off optimization
export OSG_OPTIMIZER=OFF
Create a script to convert all .ive files in a given directory (reason later)
cd old_db emacs conv_all_in_dir.sh
Put the following in the script, edit as needed, comments inline
#!/bin/bash # used with e.g.: #find . -type d -exec ./conv_all_in_dir.sh {} \; # the output root, edit as needed OUT_ROOT=`pwd`/../new_db #echo $OUT_ROOT # 1st parameter is the directory we should process IN_DIR=$1; echo Changing to $IN_DIR cd $IN_DIR # find: find all ive files in this directory only, not subdirs # sed: remove ./ in front of names # xargs: run for every input file, start multiple processes (-P) # osgconv -O "compressed": make compressed ive files find . -maxdepth 1 -name "*.ive" | sed -e s/"\.\/"// | xargs -P 2 -I {} osgconv {} $OUT_ROOT/$IN_DIR/{} -O "compressed"
The script is passed a subdirectory of old_db, it goes through that directory and converts all the .ive files and places them in the correct new_db directory.
If you have a multi-core machine, adjust the -P parameter of xargs to suit.
Make the script executable
chmod u+x ./conv_all_in_dir.sh
Recreate the directory structure of the input database. osgconv does not like to create directories.
find . -type d -exec mkdir -p ../new_db/{} \;
Now, go to each directory of old_db and convert all the .ive files into the proper directory of new_db.
find . -type d -exec ./conv_all_in_dir.sh {} \;
Wait...
Reason for cd before osgconv
For some reason osgconv must be run with only the input file name, i.e. from the directory where the file is located. If it is run by passing not only the filename, but a whole path to the file, the links to files in other directories get corrupted somehow. When the links are broken, one can only open the database properly from a single directory, e.g.
cd new_db osgviewer terrain.ive
would work, but
cd .. osgviewer new_db/terrain.ive
or execution from any other path would not.