Transcription
COLLADA CTS TutorialBy Paula Berinsteinpaula@mechnicality.com July 2011OverviewThe COLLADA intermediate languagespecification is intended to standardize 3Ddata, but there’s a wide variation in the wayapplications work with COLLADA files. To helpdevelopers adhere to the standard and create3D tools that work together seamlessly, theKhronos Group created the COLLADAConformance Test Suite (CTS).COLLADA documents. You can test packagesthat support only some of these functions,but they can only earn conformance within thecontext of a conformant implementationenvironment (i.e., one that has already passedthe test suite) that provides the missingfunctionality.The suite tests:The objective of the tests is to insure thatconformant products can import from andexport to other conformant products so thatthe entire 3D tool chain performs seamlessly.The tests may be used on one targetapplication alone for both conformance andregression testing, or to verify interoperabilitybetween one application and another, e.g.,Google SketchUp , Autodesk 3ds Max andAutodesk Maya .The COLLADA CTS comprises 613 tests. Thereare three possible levels of conformance(badges)--Baseline, Superior, and Exemplary—but regardless of the application’s capabilities,all the tests must be run. Based on theresults, the framework determines which, ifany, badge to award.The conformance tests can be used on twotypes of applications: import only andimport/export. Import-only applicationsinclude 3D document viewers, game engines,ray tracers, and rendering packages.Import/export applications include mosttraditional content creation tools that domodeling and animation. Both classes ofapplications must be able to render COLLADAdata as an image because that is the primarymethod the CTS uses to determine whether aCOLLADA document has been correctlyunderstood. For a complete test, the softwaremust also be able to import and export Whether the target application handlesCOLLADA input and output properly,and whether files are renderedconsistently using files created underdifferent conditions (Import, Render,Export) Whether the target applicationoutputs COLLADA documents thatconform to the COLLADA schema(Validate).Some of the issues tested are: How complete the support is for eachfeatureHow robust the target application iswhen bad data is encounteredWhether images and movie clips lookas expectedWhether all features are preservedduring a load/save cycle. 2011 Khronos Group.Group All product names are trademarks or registered trademarks of their respective holders.
Types of tests include: Minimal unit tests for specificfeatures, like document referencingand skinningSystem tests that exercise desirablecommon scenarios, such as skinned ragdoll export/import with COLLADA FXmaterials on different body partsStress tests for very large scenesHandling of local or temporary testdataHandling of invalid or corrupt data.All importers must be able to read any validCOLLADA document without failure, even ifthat document contains features notsupported by the importer. Features beyondthe importer’s level of support are not requiredto be understood, but simply encounteringthem should not cause the importer to fail.Unusual or unexpected ordering or quantity ofelements should not cause an importer to failas long as the document validates to schema.For example, library elements may occur in anyorder and libraries of the same type canappear multiple times. Importers must notrequire their own COLLADA extensions in orderto understand valid documents.Working with the testsUsing the tests is a matter of installing,configuring, and running them. The broadsteps are:1. Prerequisites (download and install thesuite and some helper applications)2. Integration (write and test a Pythonscript)3. Set up the tests (create a testprocedure, add tests)4. Run the tests5. Interpret the results6. Submit results to Khronos.This tutorial covers all the steps.All exporters must create documents thatvalidate to the COLLADA schema. Documentsexported by the implementation shouldsupport the same feature set as thatimplementation's importer in order to providea lossless (roundtrip) content pipeline.The CTS works through a GUI. You can also usea command line interface, and certainfunctions can be performed through theWindows file viewer. (The suite currentlyworks only on computers running theMicrosoft Windows operating system.) 2011 Khronos Group.Group All product names are trademarks or registered trademarks of their respective holders.
Things you need to knowabout the testsBlessed imagesThe CTS incorporates the idea of “blessing.”This has nothing to do with religion, but refersto results that have been officially “blessed”by Khronos, i.e., considered correct andconformant by the COLLADA work group. A setof blessed images and animations comes withthe test suite; you can compare these withyour results to see how close you are and howyou differ. Other than that, you don’t have toworry about the idea of blessing. (There aresome legacy menus and sections in thedocumentation that refer to blessing; ignorethem.)target application must generate its ownimages so that an “internally consistent”comparison can be made by the framework.Image comparison tools also allow theintegrator or developer to easily compareagainst “blessed” images as an aid todebugging. Exact pixel-for-pixel matches arenot required in order to pass a test.Types of testsThere are three types of tests: simple,medium, and complex:1. Simple.Simple Tests that pass/fail based onsuccessful importing, or importing andthen exporting a COLLADA document.Examples include tests that check forproper handling of URIs onimport/export, checking for unwantedadding/removing of elements from thedocument, testing sensitivity toelement ordering, and checking for theability to process any valid COLLADAdocument without crashing.2. Medium.Medium Tests that require asuccessful import, export, andrendering of the exported document inthe target application. These tests aregenerally used to confirm thatprograms are correctly processing andre-exporting a document withoutcorrupting the geometry and otherelements in it.Figure 1. A blessed image.Because COLLADA is not a rendering standard,not all applications will render images thesame way. Differences can include variationsin lighting, cameras, geometry, shading, colorspace, anti-aliasing, and so on. Nevertheless,the tests have been designed such that the3. Complex.Complex Tests that require asuccessful import, export, andrendering of the exported document inthe target application. This type of testinsures that the application is correctlyinterpreting the contents of thedocument. It is used to check how wellthe application handles materials,textures, cameras, lights, etc. 2011 Khronos Group.Group All product names are trademarks or registered trademarks of their respective holders.
Two cyclesEach test runs through two cycles: The first cycle involves importing thetest case's COLLADA document,rendering it, and exporting it. This cycletests whether the target applicationcan import and work with a validCOLLADA document (that waspossibly) created in another tool. In the second cycle,cycle the COLLADAdocument that was exported in thefirst cycle is reimported into the targetapplication, re-rendered, and validatedagainst the COLLADA schema. Thiscycle tests whether the documentexported in the first cycle imports andcompares properly and outputs validCOLLADA that can be imported byother conformant tools. The Validatestep is needed to verify that theexported document meets criteria nottested in the Export step.As you can see, a complete test includes twoimport and render operations. The first time,you import the test document furnished byKhronos; the second time, you import thedocument you just exported. This workflow isshown in Figure 2.Figure 2. The steps in a test. Understandingthis workflow is key to passing the tests.Datasets for input and outputA collection of datasets come with the testsuite (the StandardDataSets directory). Thesedatasets hold the input to the tests. Someinput files come with the suite; others aregenerated by the tests. The paths to the inputare specified in the integration script. 2011 Khronos Group.Group All product names are trademarks or registered trademarks of their respective holders.
The output files are written to subdirectoriesin the TestProcedures directory.PrerequisitesPythonPython - related toolsFigure 3. The StandardDataSets directory inthe CTS hierarchy.Getting StartedIn order to run the COLLADA conformancetests, your application must be able to: Read and write COLLADA filesRender a PNG image (at 512 x 512!)Include scriptable viewpoints andscriptable lights.The tests run on Windows machines only rightnow. The conformance test framework onlyofficially supports 32-bit Windows, but youcan run it on a 64-bit machine. Make sure yoursystem has enough juice; the tests take up alot of memory.Before you can run the suite, you will need toinstall some Python-related tools, which youcan find in the CTS\Prerequisites directory.(The directories referred to in this tutorial arepart of the test framework, not areas of theKhronos site.) These include Python itself, theWin32 extensions, wxPython, and pyOpenGL.Even though the suite comes with the correctversion of Python, you may have to installmore than one version because some applications, such as Blender, need them.It is a good idea to check the "add software"panel on your computer before installinganything to see if more recent versions ofthese tools are already installed. You must usethe version numbers shown in this directory orhigher. If your copy of Python isn’t installed inthe standard location, you may need to set itspath in the config file.On Windows 7 and Vista, you have to installthe Python tools as an administrator.NETCheck to make sure the Microsoft .NETframework is installed. If you don’t have it, theframework cannot validate your COLLADAdocuments (MSXML is the default validator),and you can’t enable crash detection. You candisable crash detection or specify the timeoutvalue in the config file. See the Integrationsection of this tutorial.Downloading and installing the CTSThe test suite uses long file names that cancause problems on Windows systems byexceeding the 255-character path name limit.To make sure your installation goes smoothly,unzip the package in the root directory of yourdisk volume or some other directory with ashort path name. 2011 Khronos Group.Group All product names are trademarks or registered trademarks of their respective holders.
IntegrationIntegration involves sending information back and forth between the test suite and the targetapplication. It’s accomplished by writing and running a Python script and modifying a config file thatcomes with the suite. The Python script generates and executes other scripts for the targetapplication.Integrating a target application with the test suite requires the following steps:1. Create the Python script in the Scripts directory to map your application. Usually you will copyan existing script and modify it.2. Update config.txt in the top-level directory to set up your application path.3. Debug by running a small sample of the suite.The config fileThe config file is held in the root of the CTS directory.Each line in the config file is a setting. On the left is the description; on the right, separated from thedescription by tabs, is the value.You will need to update the config file with the path for yourapplication. Make sure you use a tab delimiter between the key and value and that your editordoesn't convert tabs to spaces. Be sure, too, that there isn’t a blank line at the end of the file. Ifthere is a blank line at the end, you will get an error message like this:C:\CTS colladatestsuite
include 3D document viewers, game engines, ray tracers, and rendering packages. Import/export applications include most traditional content creation tools that do modeling and animation. Both classes of applications must be able to render COLLADA data as an image because that is the primary method the CTS uses to determine whether a
