Houdini 18 Review: Pyro, Guided Destruction, Adaptive Solver

With the recent release of Houdini 18, there’s a lot of great stuff coming for the procedural toolset. The feature list is quite extensive, including a huge amount of improvements in Pyro and FLIP Fluids simulations, USD infrastructure, RBD Destruction, and game dev tools (now called SideFX Labs).

We’ve talked with some technical artists, who were actually working on the amazing Houdini 18 reel, discussing some of the interesting changes in the new version and how they’ve made their work easier.

Pyro Simulations

Filippo Robino, Head of VFX at Alps VFX

Filippo started his career as a 3D Artist after studying Graphic & Virtual Design at Polytechnic University of Turin in Italy. He co-founded Alps VFX and now works there as Head of VFX.

Filippo: I quickly became fascinated with the visual effects. From an artistic viewpoint, it’s such interesting and challenging work. I soon decided to pursue a career in this field and continued my studies, specializing in FX and lighting. After working for several big studios around the world, such as Weta Digital, Method Studios, Scanline VFX, and Trixter, I decided to move back to Turin, Italy, and co-found Alps VFX together with Alessio Bertotti. While we specialize in FX, environments, and compositing, we also enjoy working on entire projects from start to finish. We’re a new company and as such are always looking for the most modern and innovative way to approach our work and solve problems, supported by an always-growing infrastructure and workflow.

Challenges in Pyro Simulations

In my experience, there are 3 main issues in Pyro Simulations. Firstly, some artists are afraid to experiment and push boundaries. The second issue is “data pollution” and the third is the emission phase. 

When I teach Houdini and Pyro, or when I show my techniques to my colleagues, they often tell me that my code, tools, and micro-solvers are too complicated and that my VEX/VOP skills are too advanced. However, they don't realize that I am just writing simple math, the one that people normally use when they go shopping or when splitting the bill at the restaurant. 

Now about the issue of “data pollution”. I often see users write out billions of voxels, hoping to get amazing simulations by just increasing the division size. Within a production environment, creating such huge files can increase I/O and slow down everyone’s productivity. An artist should spend more time trying to extract as much detail as possible from a simulation and write only necessary data on disk by optimizing the number and precision of fields needed. This leads to higher efficiency by having faster sims, less I/O, faster render times and less storage needed. 

The last issue is the emitters. Having a good animation and a good workflow for your Pyro emission usually solves most issues with your sim. If you don’t invest enough time in it, you could have the best Pyro simulation setup in the history of computer graphics, but it won’t work if timing and quality of emission are not right. I really think that SideFX tackled and helped out a lot in those areas with this new release. Some technical aspects of Pyro sims have been simplified significantly and give the artist the possibility to invest more time on those critical issues I mentioned. 

Sparse Solver

The main thing that we wanted to show is the power of the new sparse solver and its workflow. We intentionally picked the worst-case scenario: a trailing Pyro sim with a curved trajectory. For reference, we used this video of a rocket called Proton M that had a problem during liftoff and crashed.

Here are some reference videos used:

We wanted to show how well the new solver handles this kind of situation, without having to use camera cheats and various tricks to save memory. And of course, we wanted to make it as cool as possible in the given time. 

The workflow is actually a very standard one. The emission and the combustion workflows are a bit different from the previous version of Houdini, but they actually make everything easier and faster. We started with the rigid bodies sim. I enlisted Francesco Giugliano to help out with this part. He’s a great senior FX TD who’s frequently collaborating with Alps VFX and he did a great job! We had to make sure that the timing was working correctly and that the simulation was as smooth as in the references.

With the rigid body sim done, I was able to focus entirely on the emitter creation and simulation setup. While working, I started to get used to the new solver and its new features. I have to thank Omar Zarifi from SideFX for all his support in the process. He explained the new features to me and implemented some of my suggestions in the latest version of the software. Once I had wrapped my head around it all, I started setting up the entire workflow, creating all the different emitters for the trail simulation and the explosion sim. I kept refining the sims, from their behavior to optimizations for the duration of the project. We tried to use all available resources at our disposal without wasting any opportunities.

In the end, we had just three simulations in total, which is kind of amazing. In the past, we would probably have needed at least 20 simulations to create all of this. Managing 20 simulations is a complex task, takes a lot of resources, time and makes it overall harder, less efficient and less productive. 

The biggest improvement of this new solver is that I didn't have to care about the trajectory of the emitters by having to split it up into several containers to optimize the space used by the fields. This is normally one of the biggest challenges in this kind of scenario. I just had to focus on the emitter structure and animation, the behavior of the three different simulations and the overall lighting and shading. Omar also showed me the new disturbance micro-solver, which is pretty awesome. It is optimized, faster and has some nice new features. The key was to extract as many details as possible from all the sims. Everything was achieved in the usual way, combining different levels of turbulence, adjusting the forces and adding features such as customs micro-solvers. Then all of it was, of course, rendered with Mantra and comped together with COPs (Houdini Compositing Operators). I also made extensive use of the Procedural Dependency Graph functionality through TOPs (Houdini Task Operators). It helped me a lot during this project to optimize and use all of the resources at my disposal.

I am very much looking forward to adopting this new version of Houdini in production. It is such a big step for productivity and flexibility. It solves so many workflow issues and lets you focus on creating nice, well-balanced and good-looking pictures. I would like to thank again Omar Zarifi and Fianna Wong for the support and help, SideFX for this release and for this collaboration opportunity. It was such a pleasure working with them!

Art Directable Destruction

Artem Smirnov, Senior FX TD at Digital Domain

Artem Smirnov has built an amazing career, coming from low poly modeling in 3ds Max and getting the opportunity to do some outstanding work with Digital Domain.

Artem: I started out in the industry as a low-poly modeler in 3ds Max, back in 2008. Later, I moved to the Marketing Department and that is where I started to play with FX. I worked with Thinking Particles, FumeFX, Krakatoa, and other tools until I saw Houdini. It took me 3 failed attempts throughout several months to get into it. My university engineering background turned out to be quite helpful with Houdini.

Then, I was hired as an FX Artist at Melnitsa Animation (in Saint Petersburg). The FX Department there was small and purely Maya-oriented, and it took me quite an effort to switch the whole work to Houdini. After some time, I became Head of the Department. Melnitsa was a nice place to work because it was quite tolerant of employees' personal projects and let us use the farm and other resources for self-education.

After four years and two consecutive second-place awards in the Houdini Splashscreen contest, MrX (Toronto) reached out to me with an offer. At MrX, I learned a lot about the studio pipeline, working with supervisors. Then, Weta reached out and asked if I would like to work on Avengers. The answer was obviously yes, so I moved to Wellington and participated in some dope projects (Avengers, Mortal Engines, GoT) that definitely level-upped my skill set. At some point, it felt like it was time to have an extended vacation, so I took off for 6 months. I finally assembled my demo-reel (that was my first reel after 11 years in the industry) and got my next gig at Digital Domain.

Art Directed Destruction in Houdini

The thing about Houdini is that it is only as good as the user is. The user is given full control over data and complete responsibility for the result. No "make it cool" buttons. For some people, it is hard to deal with. For others, it is the only way to do things. Destruction is not different in that sense. 

Native SideFX bullet implementation went a long way, and now it's fast, stable and full of awesome features, while still offering the same level of total control over simulations. It has become the standard in feature film production. One of the key reasons behind it is that if you don’t have a feature you need, you can implement it yourself, and it's pretty straightforward. For instance, I needed shape-dependant air drag for one project and it's not available in Houdini currently (out of the box). So, it took me only roughly 200 lines of code to implement it and that is the power of Houdini in production. Guided simulation workflow also was done for production needs. I had a train crash sequence for a TV show at MrX and was asked to develop a solution to art direct simulations based on anim department input. That's when I came up with the idea of virtual animation constraints to drive simulations.

80lv: One of the key features we’ve seen is guided destruction simulation. It does seem absolutely extraordinary. But we were wondering how approachable these simulations are.

The basic case (art direction) is, of course, very approachable. Thanks to the SideFX team and Nick Petit specifically, the tool is now quite user-friendly. The user simply needs to plug animation guide geometry into corresponding input and get the result. Other cases, however, were not considered at this time. Semi-independent distributed simulations, for example, still will require some setup work and understanding of underlying processes. Same with infinite mass collisions problem. My personal version of the tool is a bit different as it works outside of bullet framework and it makes it easier for me to play around and test things or tweak it for different results. I do welcome people to get inside and tweak the tool to get what they want. It's fairly straightforward implementation and shouldn't be too challenging to understand for a professional Houdini TD.

Probably one of the coolest things is that you’ve got different layers of geometry here, and they interact in different ways. I really look forward to what kind of cool things the Houdini community will come up with using these new geometry tools - e.g. the new Bend tool. I’d love to see some motion graphics artists doing their crazy voodoo with it. Personally, I use it for a very limited number of things in production, probably because in feature films, destruction tasks are almost always similar and that is getting quite boring. But yeah, the craziest thing for me is that technically you can destroy a city in high detail within a reasonable sim time, as long as you have enough farm blades and disk space available. Furthermore, if you don't like some part of the simulation - just re-sim the corresponding cluster, no need to re-run the whole thing. That turned out to be a very powerful concept, which is surprising considering how simple it actually is.

I'm not sure how a game engine would benefit from this workflow, other than making some highly directed precached simulations. Of course, when we’re talking about real-time, there is no technical reason why it can’t work there. The math is very straightforward and computational cost is minimal. So, if somebody would love to try it, I would be interested to see the result. For cinematics, it makes more sense. Generally, cinematic production is not fundamentally different from feature films or TV shows. As I mentioned before, the reason this tool exists is for control of massive scale simulations. Those are not very common things in games. But I could be wrong as I'm not playing games at all these days.

Fluid Simulations

Igor Žanić, VFX Artist

Igor is a co-founder of Rebel CO and specializes in doing incredible water VFX.

Igor: I am a freelance FX artist from Serbia, mostly working on the water VFX. I like to challenge myself and fluid simulations are something that I really like. In the past, you could see amazing water effects made only by big FX studios, but nowadays, if you have great software and a fast computer, you also can do something interesting.

Doing Wedges

Previously, I used different software for liquid simulations like RealFlow, Naiad, Bifrost, but now I mostly use Houdini. At the moment, Houdini is the most complete software on the market for liquid simulations, thanks to its in-house tools. Houdini provides huge control over your simulations, and you can always make a custom setup if you need it.

I personally have my own way of working that involves doing a lot of comparison tests (or wedges) to see how my sim will work with different parameters. It helps me to find the best result as well as learn new things. With all these tests, you can create your own library that you can use in future works. With time, you become faster and more efficient. Over the years, I did more then 3000+ wedges, and I learned a lot from them.

Speeding Up Simulations

Flip Fluid simulation is a really demanding and slow process. When I just started doing simulation work, I didn't have a powerful computer, so optimization was a big part of my process. Now, with upgraded hardware, people think less about such things. However, trust me - even with powerful software some simulations can still be very slow. When doing wedges, I see how different stuff works, what I need for my sims and what range of values I can use. You might spend a few days on such tests, but it's going to save you time later.

In Houdini 18, Adaptive Solver helps with the tank simulation setup and can make it 1.5-3 times faster. But even then, you can find other solutions to speed up the process: caching all assets that you use for simulation, balancing collision details, etc. A few minutes here, a few minutes there, and in the end can, you can save up to 1-3 hours.

Adaptive FLIP Pressure Solver

To explain Houdini's Adaptive Solver better, I'd like to use an excerpt from SideFX documentation: 

"The pressure field is solved on an adaptive (octree) background grid to significantly improve performance over solving with a uniform grid. In order to maintain high-resolution details in the simulation, a band of fine grid cells is constructed along the boundaries of the fluid.

The grid is then progressively coarsened into the interior of the fluid. To minimize the difference between the adaptive and uniform grid pressure solves, the adaptive pressure is up-sampled and smoothed back to the uniform grid before updating the velocity field".

Adaptive Solver helps with FLIP tank setups, but to gain most out of it, you should have the proper depth in your simulation. Things like shallow rivers and beaches don't get that much benefit from Adaptive Solver compared to a large ocean with a 5-20m depth tank. In shallow sims, there's no room for Adaptive Solver.

I did a lot of nonAdaptive/Adaptive tests to see where each workflow works better, you can check them out here.

Think Simple

As FX artists, we always want to speed up our simulations, and Houdini 18 provided a faster workflow with new Adaptive Solver which also means we can run bigger simulations safely.

Every artist approaches his work differently, but I personally think that it's really important to know how to optimize fluid simulations. Sometimes, artists use a lot of custom tools, VEX, a bunch of nodes, etc. that really slow down the sim. They think that if a solution is too simple, it will not work how they want it, so they go for more intricate and demanding techniques. But simple setups give faster results, and if you combine a few simple solutions, they can give a really complex outcome.