Microsoft Trellis is an incredible AI tool that can create intricate 3D models from a description or from an image. This has incredible potential when paired with traditional AI art generation, as I demonstrate in the video below. You can try it for yourself here.
This is impressive technology, and at first glance, it seems like a holy grail for game development. But how good is it really?
The Experiment
In my Microsoft Trellis tutorial, I generated a 3D model of a wiener dog. The resulting model has roughly 8,500 faces and about 6,000 verts.
These number’s aren’t terrible. On the surface this really does look incredible. But for game developoment or rigging, it would be hard to use in its current state due to some hidden issues.
Our first issue is that the model isn’t one complete model — there are duplicate edges that exist on top of each other. This means if we apply a decimate modifier to make the model less complex, gaps will appear. The technical term for this is “non-manifold”.
To see this for yourself. Enter Edit Mode in Blender, and make sure nothing is selected. Press F3, type “non-manifold”, then choose the option to select non-manifold geometry.
There is an easy fix for this. In Edit Mode, Press A to select all geometry. Then press M to open the Merge menu. Select Merge by Distance. Now test if there is any remaining non-manifold geometry and adjust the merge distance as needed. Sometimes a vert or two escapes the default merging distance. This operation ends up saving us a free 2,000 verts, or 30% of the original amount.
Now we can decimate the model without issue. Here I was able to decimate down to 1,000 faces and the model + texture still look decent. The decimation doesn’t cause any unsightly gaps.
You may think to yourself, couldn’t I just resculpt the topology myself? Yes, you could, but you wouldn’t be able to use the default textures. The UV mapping and texture image for the model is non-sensical to a human.
Directly attempting a remesh with this texture leads to strange seam artifacts on the non-manifold edges we just fixed.
To be fair, changing the topology would mess up any textured mesh. But the especially non-sensical look of the texture complicates things. Especially if you don’t know much sculpting.
What else can it do?
On the Trellis showcase page, there is a variant of a pick up truck that uses voxel graphics. The toplogy of organic material doesn’t look perfect, but editing a voxel model is far easier when you don’t have sculpting experience.
Unfortunately, only the image to 3D model is available right now.
Final Thoughts
Technology like this is very exciting. I can’t wait to get my hands on the text to model and variant features. I’ll be updating this article if I find other tips, tricks, or new strategies to share.