Virtual Texture Performance Testing
Virtual Textures have been around for a while in Unreal, but Epic has recently been pushing them hard alongside Nanite and Lumen, and new UE5 projects created now have them turned on by default in the project settings.
​
Part of my job includes investigating the feasibility of new features, so I wanted to test if these Virtual Textures really were better than traditional mipmap textures, or if they had some downsides. We're gonna take a look at what they are, how they compare to traditional textures, and what situations they might be useful in.
Traditional vs Virtual Textures
Traditional textures load every part of the texture into memory and chooses the appropriate mipmap for it. Even if only a small part of the texture is seen on screen, the full texture is loaded, which can cause wasted memory in the memory pool. For traditional textures, the memory pool fluctuates up and down based on which objects are on screen and how far away.
​​
Virtual textures cut up each texture into a grid of small squares. Then it streams in the relevant grid tiles that are on screen, meaning that only some of the texture is loaded, saving memory in some cases. Some extra processing is done here to determine which tiles are visible, which makes it slightly less performant.
​​
Virtual Textures support UDIM textures, but traditional textures do not.
​
Epic mentions that things like VFX and noise textures used in shader effects should not be virtual textures, since virtual textures can sometimes cause visual artifacts that are more noticeable in these situations.​ So I'm excluding them from testing.​​​
Performance Costs
Sampling from a virtual texture is more expensive than a traditional texture. A virtual texture is normally two texture fetches and some math instructions, but “stacking” the virtual textures can reduce the overhead cost. So when using 2 virtual textures with the same UVs/Sampler, we have 2 texture lookups and 1 stack lookup. So the best use-case here is to opt into multiple virtual textures on a material instance: if the Base Color/Normal/MRA all have the same UVs, then they can all be part of the same virtual stack and reduce overhead costs.
​
Virtual textures must be a power of 2, and are more efficient if they are square resolution. Anti-aliasing can cause some noise on virtual textures. You can increase the Tile Border Size (default is 4) for better anti-aliasing, but that also increases the total memory usage. Virtual textures can cause some visible pop-in if the camera moves too fast when viewing them.
Testing
When testing the cost differences between Traditional and Virtual textures, I created an empty first-person level and assigned 12 unique MIs to 48 cubes. Each MI had 3 textures on it (color, normal, mra) so we had 36 unique textures total, and the cubes were given different orientations to show different parts of the textures.
I played the game in Standalone mode, with a static viewing angle of the cubes that fully maxed out our memory, and used these stat commands to see the memory pool information:
​
-
stat streaming
-
stat virtualtexturing
-
stat virtualtexturememory
From there, I wrote down the following information under various conditions:
Understanding the Data
When looking at the virtual textures, I noticed that moving the camera around the map had no impact on the memory pool. The virtual textures didn't care if the camera was on the other side of the map, or facing the opposite direction with no textures on screen. It was a permanent, static memory cost that stayed in the memory pool, similar to classic non-streamed textures.
​
When increasing the border of VTs for better anti aliasing, we went from 469mb to 513mb total disk size. The total physical memory increased from 127mb to 252mb. Everything else stayed the same.
​
Using virtual textures costs about 0.4ms (on 4k textures) compared to 0.07ms not using VT. This is probably the extra rendering cost that the documentation mentions.
​
Virtual textures seem to have a flat physical memory cost of 127mb, for both 512 and 4k textures, regardless of where the camera is located on the map. Dividing by 36 unique textures, we have ~3.5mb of memory permanently allocated for a virtual texture. And when we increase the Tile Border Size from 4 to 16, that memory increases to 7mb permanently allocated for a virtual texture.
​
Regarding traditional textures: with Default Compression and no Alpha, we typically allocate 11mb to a 4k texture, 2.7mb to a 2k texture, 0.7mb to a 1k texture, and 0.17mb to a 512 texture.
​
This means that a virtual 4k (3.5mb) is cheaper than a traditional 4k (11mb). When that traditional 4k swaps to a 2k mipmap however, the 2k only costs 2.7mb and becomes cheaper than the permanent 3.5mb virtual texture cost. Since most textures quickly mip out of their 4k resolution once the camera is a few feet away, this makes virtual textures less efficient for most environment situations.
​
Virtual Textures might be better to use in a few situations though:
-
Character and weapon textures for singleplayer games. These "hero asset" textures are constantly near the camera and will probably be full 4k resolution the entire time, so it might make more sense to make them virtual and cheapen the cost. But if you are creating a multiplayer game, this might not be a good idea because only the local player's textures will get a discount. The permanent memory cost of all of the other characters will probably outweigh the discount of the local player.
-
Non-streamed textures. Things like skybox textures and UI textures, which typically aren't mipped and often take up a large chunk of your non-streaming memory anyway. Since these textures already have a permanent cost, converting them to virtual textures might give you a good discount provided there are no visual artifacts.
Overall though, it's hard to justify using them when traditional mipmapping is already incredibly helpful for reducing memory costs. And there aren't many situations where you would need an 8k texture for anything, especially if you use trim sheets and micro normals to maintain high fidelity. So this feature seems fairly niche despite Epic's push to standardize it.