Back in 2019 I worked on a project called The Living Reef. As part of a small team (2 programmers, 1 artist, 1 animator, 1 tech artist, 1 designer, 1 producer)  we were tasked with replacing the existing reef interactive that had been displayed on The Cube with a reef-freshed version. The Cube is a composite display of two projectors and 20 55inch touch panels networked together. Within a year we hit our first release milestone and then over the following year expanded the content on it.

One of the goals for the Living Reef was to surpass the former interactive by having the fish swim freely and exhibit characteristics you would expect to see in a reef, such as schooling, collision avoidance and interaction between predators and prey. This is where I spent my time.

Boids

The starting point for anything schooling related is Boids. Craig Reynolds developed an artificial life program called Boids back in 1986 which simulated birds flocking. His algorithm is comprised of three main components.

• Cohesion
• Alignment
• Seperation

Cohesion is the force required to move towards the centroid of the boids neighbours. Separation is the normalized sum of the forces to move away from neighbours that are too close. Alignment is setting the boids facing direction to be the averaged sum of its neighbours. Each of these components can have a weighting to change the behaviour.

This simple algorithm results in complex behaviours. It isn’t my first foray using boids but it was good to be using the algorithm again. A simple but nice implementation of boids can be found here https://processing.org/examples/flocking.html

To make the reef look like a real reef we were going to need a lot of fish and that this would require optimization. Unity’s burst and job system seemed like a good fit to squeeze out performance and I had previously gained experience with these systems on the project Water Tank. I did start to investigate implementing this all in DOTs but the support for animations was not there yet as the animation’s required blending on multiple layers.

Physics

A decision I made early on was to not use the inbuilt PhysX physics engine. I didn’t think that it would be feasible to integrate the Parallel Job system I was making with PhysX. So I wrote a very basic euler physics integration step that I added to the system I was creating.

This led onto where the bulk of my time was spent. Making the fish physics behave like fish. The basic boid algorithm makes things look like they are flocking, but the behaviour of schooling fish and flocking birds is different. So through a lot of trial and error and experimenting I developed a fairly complex set of rules for constraining the fish to feel more like fish. Drift, turning rates for different speeds etc really change how a fish feels. Each species of fish had its own custom settings. There were also some edge case fish such as Cuttlefish which had entirely different locomotion and required custom behaviours.

Animation

The 3D movement of the fish is the first part of the puzzle to make them look real. The second part is animation. We were fortunate to have the talented animator Michael Baxter who painstakingly crafted the fish animations. I worked with him to create an animation system that would blend between the various animations he had made based on the speed of the fish as well as natural variations and special case animations. It took a lot of tweaking to get this to feel right but the end results were worth it.

To enhance the animations some dynamicness was added. For example the shark tail bones I modified in LateUpdate to follow the path that the shark was swimming in while additively adding the animation that Michael had created. This worked by recording the position of the shark and then mapping the bones to this path. Daniel Fisher, the other programmer on the project, developed some tech for the dorsal fins on some of the fish to be affected by the path movement. He did this by having each bone lerp towards its parent bone a little bit like the Follow The Leader algorithm. It’s a simple system but effective and performant.

Collision Avoidance

As previously mentioned I made a simple custom Euler physics step instead of using the inbuilt Unity physics engine. This meant there were no colliders on the fish so they would of course swim through the terrain and each other. 

Raycasting for each fish would be too expensive, however in a previous project I had used signed distance functions for terrain collisions. So I thought maybe I could do something similar here. I created a small editor tool that created a 3D grid. For each element in the grid it would raycast to find a vector that was the vector to the nearest terrain surface. A secondary pass then smoothed out these values. This was all baked offline into a data structure that was feed into the job system. Each fish would then look up which grid cell it was in, check to see if the distance of the terrain was too close, and if this was true then swim in the opposite director of the vector. For fish to avoid other fish I integrated a flee behaviour into the boids. It doesn’t guarantee they won’t swim through each other but in practice it works well.

Predator Prey

Finally for the predator and prey, I added support for this by leveraging the boid system. Different fish have different tags. Sharks > Large Fish > Medium Fish > Small Fish. During the Boid calculations the closest predator is found for the fish (if any) and then this is added into the resulting force that is fed into the custom physics system. In The Living Reef some of the sharks are actively pursuing the snapper who in turn are fleeing and really helps makes the scene feel alive.

Making it look good

Ryan Bargiel, the artist on the team, had the job of making all of these fish models (we have around 30 fish species!) as well as creating the scene and environment. He also created some awesome bioluminescent shaders in ShaderForge which you can see here.

Helping Ryan was Lucas Milner who was procedurally generating various coral species in Houdini which were then exported out. Cool stuff!

The caustics were implemented by using a projector on a light with an animated texture.

Everything else

It goes without saying that the first time we ran this project the performance wasn’t great. The machines we had were beefy but both projectors were displaying at 4K each which was very demanding. Ryan ended up billboarding a lot of the background corals and tweaking various quality settings.

The other difficulty that I haven’t touched on was that this was all networked on a Local Area Nework. Each of the panels and projectors are independent executables. We had around 700 fish in the scene. To get this amount of fish to sync across the network I sent just the position and rotation data. To network the animations just the animation triggers were networked and it was up to the local machine to then animate. Generally everything stays synced as the animations are constantly changing which prevents drift.

There are other bits and pieces to the project such as mini games and information panels but these are outside of the scope of this post. I was really proud of what we put together at the end. If you are ever in Brisbane, Queensland, Australia, come check out The Living Reef. It’s open to the public free of charge.