Embracing a "3D math / physics state of mind" in the immersive media economy
On how having a "3D math state of mind" gives us a common vocabulary to create immersive worlds from scratch
image source: Teamlab Shanghai
One of the most common job description requirements you see for many VR/AR developer jobs is the ability to handle 3D math problems, and have a basic understanding of linear algebra, shaders, and basic graphics programming.
These topics are very hard to teach well, even prompting some developers to request a Netflix for shaders:
I’ve had a lot of thought on this question - what are some way to study, or get better at 3D math in the context of real world spatial computing problems?
For context, I took physics & linear algebra in university, but the theory that I was taught is way different from the real world applications you’ll bump into.
Things like taking into account physics for AR gesture interactions for medical training purposes, or VFX animated triggers for enterprise AR interactions for remote work. Or, simulating particle physics for VR data visualization.
There’s layers of layers of 3D math involved, and I’ve found that it takes more than just reading several 3D tutorials online to get good.
On cultivating a culture of curiosity between the merging of architecture, physics, and mathematics
Like a spiritual practice or meditation, having a “3D math state of mind” is something that can be cultivated via quiet study and deep reading into the theory of natural phenomena.
It’s a practice that doesn’t come overnight. In fact, like yoga, it is a science that requests daily observation and focus.
One of my favorite websites to model these natural phenomena is The Nature of Code via Processing.js, which has great exercises ranging from coding physics collisions to studying cellular automata.
The Coding Train also has very similar exercises, even going into topics like Fluid Simulations:
Observing nature via math: a spiritual practice that takes you to a 3D math state of mind
At your 9-5, we will always be constrained by the business requirements that your company will set on you.
Fortunately, cultivating that garden of curiosity outside of your day job will give you that vehicle you need to explore these natural phenomena.
Like a telescope inside an astronomy observation center, it’s how you observe nature that will give you the theoretical framework needed to simulate 3D worlds.
Without the theoretical background toward natural phenomena, your scale and reach of impact will always be limited.
Your ability to study and mimic these natural phenomena via 3D math will give you keys to a language that allows you to contribute to the rewarding process known as world-building.
The fortunate thing is that in the World Wide Web, there’s never been a more friendlier canvas to share and get feedback via frameworks or libraries such as WebGL three.js, processing.js, Open Frameworks, etc.
Studying 3D math via nature is more metaphysical, but it’ll give you a vocabulary to tackle tougher problems
I realize my answer to this question isn’t the typical game developer, “3D shooter game projectile response”. Given the amount of tutorials toward that genre, I do think that route is a very valid route to pick up 3D math.
But, why observing nature to get good at 3D math?
I believe the genre of immersive world building benefits from the study of 3D mathematics from observing the messiness of nature - fluids, flora, particles, light, acoustics, etc.
They’re very difficult topics, but they shape elements of successful spaces in landscape architecture or interior design and are critical assets to our sensory experiences.
Having a 3D mathematical understanding of these topics not only gives us a mental model to make sense of these phenomena, but also gives us the tools needed to build it.
These aren’t topics to be feared, but rather expertise that we can embrace and contribute to the culture of product development of spatial computing.
