Meshes simply don't know how to behave

It's been a while since I posted last time, but I've been quite busy with new developments for Biped Blocks. While I explore those a bit more, I want to share some of the underlying ideas. So today, I want write about why I think that the inside of a 3d model is really important, and how that makes me want to explore modeling it at the same time as the surface.

Now, let me start by saying that I have almost zero experience with 3d modeling (at least by any conventional means). Even moreso with animation. I do have quite a bit of experience with 3d graphics programming, both some writing software 3d engines back in the day, and with OpenGL and Direct3D. This post is about the kinds of data that I wish were embedded in 3d models, and what I'd want to do with it.

What I wish is that modeling the inside of a 3d object would be a natural part of modeling the object itself. I can't imagine how long it would take for standard tools to support this (if ever), but I'm not going to wait for any of that! So I've started to develop some tools to explore the space myself (which is what I want to write about next time). There's a little teaser video about it at the end. But for today, let's focus on why.

It's not just the outside (of a 3d model) that counts

If you just want to render an opaque object, it's enough to model the surface. You can't see through it anyway. But let's assume that your goal is not just to render the object, but to make the viewer believe in that the portrayed object is there.

To make an object believable, behavior is just as important as appearance. In games (and animated movies), characters are the 3d models that generally have the most behavior. Animators invest loads of effort to make them behave in a believable way. A very important aspect of this, which I want to focus on today, is physicality:

• An object's volume should usually be more or less preserved even when it is deformed. If it becomes squashed in one direction, it should be stretched in the others, and vice versa. 
• Inertia and momentum are very important to give a good feel for weight and weight distribution.
• Loosely attached parts should lag behind the character's motion, which makes it feel more fluid.
• The way that different parts oscillate after a disturbance and how quickly the oscillation is dampened out is also important to convey a sense of an object.

The animation of a character conveys not only physicality, but also internal driving forces such as personality, mood, intention, reflexes, etc. That kind of inside definitely counts as well, but that's beyond the scope of this post. For an inanimate object on the other hand, physicality is usually the whole behavior.

The way that an object moves and deforms under external forces depends on what it is made of, and how the parts fit together. Watching an object move and deform tells us a lot about what's on the inside, and when it doesn't behave in accordance with what it looks like, it can be immersion breaking. This issue is compounded in VR, where you interact up close with many objects - often ones that you can move straight through!

Deep modeling

I think that software could handle/help with a lot of behavior that relates to physicality. But to e.g. preserve an object's volume, the program first needs to know what it is!

What if we would model the inside of a 3d object at the same time as the surface? Let's call this deep modeling. The surface representation, such as a mesh, would still be derived from it. But you could also assign materials to different parts of the volume, with physical properties such as weight, compressibility, stiffness, damping, etc. Character models would still have a skeleton, but it would probably be coupled to the volume instead of the surface.

Information about the inside of a 3d model could be useful both to create more believable responses in inanimate objects, and to aid animators in establishing physicality. There could also be other benefits to having an inside from the start:

• The modeling tool could let you deform the object as you're working with it, simplifying many kinds of adjustments.
• As you model, you get a feel for the object that you're working with (and can adjust that feel).
• Let's say that you build a base model of a tree, to be used many times in a scene. You could then quickly adjust the shape in believable ways by physically deforming each copy, both to lessen repetition and fit it to the particular use.

How would this best be implemented in practice? I think it's too early to say. One could use volumetric meshes instead of surface meshes (maybe borrow things from the Finite Element Method?), or connect primitives with volume together, or other ways that I haven't thought of yet...

Teaser

My experiments so far have focused on building objects out of simple primitives (spheres, boxes) by overlapping them, because I found this to be a simple and fruitful place to start. At this stage, I'm focusing on exploring the physical side of modeling, while keeping the visuals simple. I leave you with a short teaser video about some things that I have been able to accomplish in this space so far. More about that next time!