What is new in Qt Quick 3D 6.0

It has been awhile since I've posted any updates regarding Qt Quick 3D, but not because there has been no progress. In fact quite the opposite: my team and I have been so busy getting Qt Quick 3D ready for 6.0 we haven't had time to talk about all the cool new features we have added. So today I would like to talk about some of the things we have done so far.

Qt Quick 3D Can Render Everywhere

The largest development effort for the Qt 6 release was porting the rendering engine to the Qt Rendering Hardware Interface, and this effort will be well worth it because now it is possible to use Qt Quick and Qt Quick 3D with the optimal graphics API for each platform. The list of supported graphics APIs is:
  • OpenGL (version 3 and higher)
  • OpenGL ES (version 2 and higher)
  • Vulkan (1.0 and higher)
  • Direct3D 11 (11.1 and higher)
  • Metal
What this means is that whatever platform and graphics API combination you choose to use, Qt Quick 3D will work as expected without you needing to need to know the details how each graphics API works. Any place where graphics API specific knowledge is needed, for example when it comes to coordinate systems, we just use OpenGL semantics and translate them for you to the native API transparently to the application. Graphics development may have gotten harder with the further fragmentation of graphics API's in the last several years, but with Qt Quick and Qt Quick 3D this is not something you will need to worry about.

Deeper Integration of 2D Content in 3D Scenes

In previous releases of Qt Quick 3D, all 2D content had to be rendered to an offscreen surface, such as a texture, before it could be used in 3D. This worked very similar to how Layers work in Qt Quick already. In Qt 6 it is now possible to render 2D Qt Quick Items directly into 3D scenes. So if you have ever been frustrated by the lack of perspective transformations in Qt Quick scenes, this is now a possibility.

In addition to providing a more direct and efficient rendering path for 2D items in 3D, it should be easier than ever to integrate that 2D content in your scene. Here is an example that demonstrates integrating 2D content into a 3D scene:


import QtQuick
import QtQuick3D

Window {
    visible: true
    width: 640
    height: 480
    title: qsTr("2D in 3D")
    color: "black"

    View3D {
        anchors.fill: parent

        PerspectiveCamera {
            z: 200
        }

        DirectionalLight {

        }

        Model {
            source: "#Cube"
            materials: DefaultMaterial {}
            eulerRotation.y: 20
            Node {
                // Empty spatial Node to give 2D item
                // a position in 3D space
                y: 100
                Text {
                    // 2D content in 3D
                    anchors.centerIn: parent
                    text: "Cube Label"
                    color: "white"
                }
            }
        }
    }
}

This the above example the Text component is a 2D item that is a child of the 3D Cube, and if that Model were to move around, the 2D item's position would be tied to the Model.
 

If you wanted to render Qt Quick content to a texture instead, that is still possible.  An example of that would be:

import QtQuick
import QtQuick3D

Window {
    visible: true
    width: 640
    height: 480

    View3D {
        anchors.fill: parent

        PerspectiveCamera {
            z: 200
        }

        DirectionalLight {

        }

        Texture {
            id: dynamicQMLTexture
            sourceItem: Rectangle {
                width: 256
                height: 256
                color: "pink"

                Text {
                    anchors.centerIn: parent
                    text: "Dynamic Texture!"
                    color: "white"
                    font.pointSize: 24
                }
            }
        }

        Model {
            source: "#Cube"
            eulerRotation.y: 20
            materials: DefaultMaterial {
                diffuseMap: dynamicQMLTexture
            }
        }
    }
}
In the above example the Texture components content comes from whatever Qt Quick Component is set to set the the sourceItem property. This internally renders that component hierarchy to a QSGTexture that can be used for texture data in the 3D renderer, which in this case is mapped the the cube Model.

Improved glTF2 Support

Qt Quick 3D really works well with glTF2 content, and now it works even better that ever.  With the 6.0 release we have tried to support as much of the base glTF2 spec as possible. One of the biggest areas of improvement has been around supporting the animation capabilities of glTF2 assets. It is now possible to import content that contains rigged animations.

brainstem-dance

One of the additional improvements we did to enable more efficient animation support was to add support binary keyframe support to QtQuickTimeline. This means that components that have complex animations don't need to be represented by thousands of lines of QML.

One thing that is notably missing from the 6.0 release though is glTF2 Morph animations. The work for this is very far along but unfortunately did not get completed in time to make the 6.0 release. You can expect this feature in a near future release.

In addition to the animation work, we have put a lot of effort into making sure imported glTF2 assets are rendered the same way they are in your 3D content creation tools. Our goal is to have 100% support for glTF and as many extensions as we can, so moving forward expect more improvements in this area.

Redesigned Custom Material and Post-Processing Effects system

One of the compromises we had to make with the Qt 5 version of Qt Quick 3D was how Custom Materials and Post Processing Effects were defined. This system was largely the same as what was in Qt 3D Studio, and was unfortunately hard to use and created a lot of complexity in the rendering engine. With Qt 6 and the port to the Rendering Hardware Interface we took a hard look at these systems and determined we could do better.

Custom Materials

The objective of Custom Materials is to provide users with a way to define their own materials for shading Models in a scene. This traditionally means writing shader code. In Qt Quick we provide an API called ShaderEffect that gives the user a way to apply shader effects to 2D items and a powerful way to bind QML properties as shader uniforms. What we wanted to do was have that same kind of API for 3D Materials, but unlike in 2D there is a lot more state that needs to be considered when writing shader code. For example 3D items need to be aware of things like lighting, shadows, and blending.  A shader for rendering a 2D item only needs to work about the source texture as an input, so the shaders can be fairly self contained, but in 3D unless writing very simple materials it isn't possible to just write all the shader code needed for rendering. This means that for most users, creating your own material can be too difficult.

That is why we decided to create more of a framework for defining materials. Checkout the new custom material API details here.

Post-Processing Effects

The post-processing effects API is quite similar to how it was in previous releases, but has been updated to follow the same patterns as the new CustomMaterial API. Post-processing effects by their nature are already quite similar to ShaderEffect in the sense that the majority of the state is that there is an input texture and an output texture. It is not however exactly the same in such a way that we could just use the ShaderEffect API directly, because in addition to the color texture, in 3D you would also have access to a meaningful depth buffer.  It is also possible that, unlike ShaderEffect, you need to perform multiple passes to achieve the desired effect, so the Post-Processing Effect API enables that. Checkout the post-processing effects API details here.

The Custom Material and Post-Processing Effect APIs are very powerful and make possible a lot of interesting use cases and really can't be described thoroughly here, so expect another blog post soon just about these features.  If you can't wait there is a comprehensive overview available here with all the details.

The Power of QQuickRenderControl with Qt Quick 3D

One of the most powerful features of Qt Quick is QQuickRenderControl. QQuickRenderControl allows for users to define exactly how and where Qt Quick content is rendered, for cases where just using a QQuickWindow doesn't make sense. Improvements made to QQuickRenderControl in Qt 6 via the RHI mean that it is now possible to render 3D content with any graphics API in a variety of scenarios. An example of this is that the possibility of rendering to the head mounted displayed used for Virtual Reality and Augmented Reality directly is now possible without having to take inefficient rendering paths.


This is only a taste of what is to come, so expect more news in this area in the future.

Major Rendering Fidelity Improvements

With this release we decided to go through the rendering code with a fine toothed comb and make sure that everything we do is as correct as possible. With the PrincipledMaterial we wanted to provide an easy to use materials that could provide a solution for Physically Based Rendering (PBR), but to do so meant that we need to be very vigilant about how we are rendering. With realtime rendering engines, it is necessary to make compromises and approximations that non-real-time rendering engines don't need to worry about, but that comes at the cost of fidelity. The more corners you cut, the more potential differences are introduced between what the designer of an asset intends vs. what actually gets rendered. So the objective was to try and fix up many of these compromises to make sure that what is rendered is as close as possible to what the asset designer intended.

The details of what has been done so far is a bit too detailed to be in scope for this article, but check out a few examples of how assets look now in Qt 6:

Tonemapping Support

One of the areas that Qt Quick 3D previously didn't do a very good job of is the handling of colors and tonemapping. In general, rendering operations are conducted in linear color space, and then mapped back to sRGB color space before rendering to the screen, but Qt Quick 3D was not very disciplined about not mixing these color space for inputs to the renderer. In the Qt 6.0 version of Qt Quick 3D, not only has this been rectified, but a new API has been added to configure how colors are tonemapped. In addition this default tonemapping can be disabled in case you want to perform your own tonemapping as the result of a post processing effect (which in the case of some effects is necessary).

Texture Filtering

An outstanding bug that was in 5.15 was that all texture filtering was limited to bilinear filtering.  Now not only is it possible to use nearest neighbor filtering, it is also possible to generate mipmaps and use mipmap filtering if desired.
 

Offline Shader Baking

Now that we are using the Qt Rendering Hardware Interface for rendering, the preferred way of using shaders is to pre-generate them. For 3D scenes this is problematic because generally shaders are generated on the fly based on a combination of the properties set on a material, and the content of a scene.  For example, different shaders are used based on how many lights are in a scene, or if shadows are enabled.

With the new shader generation tool it is possible to give the command-line tool a scene, and it can figure out what shaders would be created at run time, based on the content of the scene. With this list of shaders, it can pre-generate a cache with all the different shader variants that the materials in the specified scene require at runtime. The limitation of this is that if you dynamically change the scene at runtime, it is possible that new combinations get created, and then we would still need to generate new shaders at runtime.
 
We will continue to improve this functionality going forward, but the current tool should already be useful for generating the majority of shaders needed for a scene at build time.

iOS Support

In theory Qt Quick 3D should have worked on iOS in 5.15, but it was never considered a supported platform and
was not tested. In Qt 6.0 not only does Qt Quick 3D work on iOS, but it will use Metal for optimized native rendering.

Improved Resource Management

Previously Qt Quick 3D only unloaded any GPU resources when a View3D was destroyed. That meant that even though Models and Textures were removed from a scene, the GPU resources were still kept in memory in case they were used again. In the 6.0 release we now have improved resource management so that when GPU resources are no longer referenced, we make sure to release them. 

Dynamic Geometry

An API for procedurally generated geometry data from C++ was present in 5.15. In 6.0 this API has been improved to be easier to use. Normally Models expect to be given a static mesh file to render their content. For cases where you won't necessarily know the mesh data up front, it's possible to use the Dynamic Geometry API to procedurally generate geometry data by subclassing QQuick3DGeometry in C++, and setting an instance of that component to the geometry property of a Model component.

This makes many advanced cases possible, since you can create geometry from data. The Dynamic Geometry data API also makes it possible to use primitives other than triangles for rendering like points or lines.

Dynamic Textures

While it is already possible to generate dynamic textures by using 2D items as a texture source, sometimes it is useful to be able to load texture data directly from a file or other source at runtime. The dynamic texture API provides a C++ API for loading texture data at runtime similar to how the Dynamic Geometry API works.  To do this you just need to subclass QQuick3DTextureData and set an instance of that component to the textureData property of Texture.

Documentation and Examples

A big part of what makes Qt such an awesome product to work with is its documentation and examples, so when developing Qt Quick 3D this is something we have tried to take very seriously.  Much work has been done to provide comprehensive documentation and examples to make using Qt Quick 3D as smooth of an experience as any other part of Qt.  You can check out the current snapshot of the work in progress Qt 6.0 documentation pages here.

Conclusion

So as you can see, we have been very busy over the last year making the 6.0 release of Qt Quick 3D as awesome as possible. The scope of 3D rendering engines and their features are huge, and so even with all this, I know there is a lot more that you would probably like to see. So please let us know what features you think are missing but important for your project. We have a lot of ideas on what to do next, but I would rather we work on the features that would be most useful for our users, so don't hesitate to speak up.
 
If video tutorials and live coding are your thing, I recommend you check out the talk I gave at the virtual Qt World Summit here as it gives a great introduction into most of the features and concepts of Qt Quick 3D in 6.0.
 

Blog Topics:

Comments