Rendering BIM in AR: Technical Architecture for Real-Time Visualization

October 8, 2020


The challenge: Take BIM data — vertices, indices, materials — and render it convincingly in an AR environment. On mobile hardware. At 60fps. With filtering, selection, and real-time updates.

BIM models aren't built for real-time rendering. They're optimized for accuracy and information, not performance. A single pipe might have hundreds of triangles when a dozen would suffice for visual representation.

The rendering pipeline bridges this gap. Import BIM geometry, construct Unity meshes, apply materials, organize for filtering, optimize for mobile, blend with camera feed.

Pipeline Overview

Five stages transform BIM data into rendered AR:

Each stage has specific responsibilities. Mesh generation handles geometry. Materials handle appearance. Hierarchy enables filtering. Rendering integrates with AR.

Mesh Generation from BIM Vertices

BIM elements provide vertices and submeshes. Unity needs Mesh objects. The conversion processes each element type once during import:

public class ElementType
{
    public float[] vertices { get; set; }
    public Submesh[] submeshes { get; set; }
    public int id { get; set; }
    public string guid { get; set; }
    public string name { get; set; }
    public string category { get; set; }
    public object[] parameters { get; set; }
}

public class Submesh
{
    public int[] indices { get; set; }
    public Color color { get; set; }
}

The float array contains packed vertex data — typically position, normal, UV in sequence. Submeshes define triangle lists with associated colors. Multiple submeshes per element enable multi-material rendering.

Unity's mesh construction:

Mesh mesh = new Mesh();
mesh.vertices = ConvertToVector3Array(elementType.vertices);

mesh.subMeshCount = elementType.submeshes.Length;

for (int i = 0; i < elementType.submeshes.Length; i++)
{
    mesh.SetTriangles(elementType.submeshes[i].indices, i);
}

mesh.RecalculateNormals();
mesh.RecalculateBounds();

Vertices map directly from BIM's coordinate space. The handedness correction transform handles any left/right-handed conversion.

Submeshes enable efficient multi-material rendering. Each submesh references a portion of the vertex buffer with its own index list and material.

Normal recalculation happens once during import. BIM data sometimes lacks normals or has incorrect ones. Recalculating ensures consistent lighting.

Material Handling Strategy

BIM elements carry material information through colors and categories. The system maps these to Unity materials using templates:

Material baseMaterial = Resources.Load<Material>("Materials/Template_Mtl");

foreach (Submesh submesh in elementType.submeshes)
{
    Material instanceMaterial = new Material(baseMaterial);
    instanceMaterial.color = submesh.color;
    materials.Add(instanceMaterial);
}

MeshRenderer renderer = prefab.AddComponent<MeshRenderer>();
renderer.materials = materials.ToArray();

Template materials define shader and rendering properties. Instance materials inherit these settings and apply element-specific colors.

This approach balances flexibility and performance. Shared shaders batch effectively. Per-element colors enable visual distinction.

Category-Based Organization

The importer handles 23 distinct BIM categories:

public enum Category
{
    None = 0,
    Area = 1 << 1,
    Ceiling = 1 << 2,
    Covering = 1 << 3,
    CurtainWall = 1 << 4,
    Door = 1 << 5,
    Fascia = 1 << 6,
    Floor = 1 << 7,
    Generic = 1 << 8,
    Gutter = 1 << 9,
    InstallationElectrical = 1 << 10,
    InstallationMechanical = 1 << 11,
    Inventory = 1 << 12,
    Mass = 1 << 13,
    Railing = 1 << 14,
    Ramp = 1 << 15,
    Roof = 1 << 16,
    Room = 1 << 17,
    Site = 1 << 18,
    Stairs = 1 << 19,
    Structure = 1 << 20,
    Unknown = 1 << 21,
    Wall = 1 << 22,
    Window = 1 << 23
}

Flags enum enables efficient filtering. Check multiple categories with bitwise operations. Toggle visibility without traversing entire hierarchy:

public void SetCategoryVisibility(Category categories, bool visible)
{
    foreach (Transform categoryTransform in categoryTransforms.Values)
    {
        Category cat = (Category)Enum.Parse(typeof(Category), 
            categoryTransform.name);
        
        if ((categories & cat) != 0)
        {
            categoryTransform.gameObject.SetActive(visible);
        }
    }
}

This enables use cases like:

Storey-Based Spatial Organization

Beyond categories, storeys provide spatial filtering. Focus on specific floors without loading unnecessary geometry:

List<(long id, string name)> storeys = new List<(long, string)>
{
    (1, "Ground Floor"),
    (2, "First Floor"),
    (3, "Second Floor"),
    (4, "Roof")
};

Each storey loads independently. UI controls enable toggling visibility per floor. This reduces rendering cost when examining specific areas.

Combined category and storey filtering provides precise control. "Show mechanical systems on ground floor" becomes a simple filter operation rather than complex spatial query.

Layer Management for Rendering Control

Unity's layer system separates objects for rendering and raycasting purposes:

private void AssignLayers(GameObject obj, int layer)
{
    obj.layer = layer;
    foreach (Transform child in obj.transform)
    {
        AssignLayers(child.gameObject, layer);
    }
}

Layers enable:

Selective rendering — Different cameras see different layers. Main camera shows all geometry. Depth camera renders only specific elements.

Raycast filtering — Selection raycasts ignore certain categories. Touch input hits only interactive elements.

Shader replacement — Depth rendering uses simplified shaders on specific layers for performance.

The depth-aware edge blending system relies on layer separation. Shell objects (walls, floors) render to one depth buffer. Inside objects (pipes, equipment) render to another. Composition shader compares both.

Performance Optimization Strategies

Mobile AR demands efficiency. Several techniques maintain 60fps:

GPU Instancing
Similar elements batch automatically when using identical materials and shaders. The type library pattern maximizes this benefit.

Occlusion Culling
Unity's occlusion system prevents rendering hidden geometry. BIM models have many occluded elements — walls hide interior systems. Culling recovers significant performance.

LOD Management
Complex geometry receives level-of-detail variants. Near camera: full detail. Far from camera: simplified mesh. Transition is imperceptible but performance impact is substantial.

Bounds Optimization
Accurate bounds enable effective frustum culling. Unity's auto-calculated bounds sometimes overestimate. Manual bounds calculation improves culling effectiveness:

Bounds CalculatePreciseBounds(Mesh mesh)
{
    Bounds bounds = new Bounds(mesh.vertices[0], Vector3.zero);
    
    for (int i = 1; i < mesh.vertices.Length; i++)
    {
        bounds.Encapsulate(mesh.vertices[i]);
    }
    
    return bounds;
}

Draw Call Batching
Shared materials enable dynamic batching for small meshes and GPU instancing for larger ones. Material instances use the same shader, maximizing batch opportunities.

Scale Transformation Handling

BIM models use real-world units — millimeters, meters, feet. Unity defaults to meters but scale flexibility is essential:

elem.position = element.transform.ExtractPosition() * 
    ApplicationConfiguration.Configuration.Model.Scale;

The scale factor adapts imported geometry to Unity's coordinate system. This configuration enables working with BIM data in any unit system without modifying source files.

Scale affects:

Centralizing scale conversion prevents unit mismatches throughout the application.

Import Reporting and Debugging

Complex imports require visibility into progress and failures. The import report system tracks operations:

ImportReport report = new ImportReport(
    $"[{nameof(ModelImporter)}] Import Report.", 
    ~ImportReport.MsgType.None, 
    true);

report.StartTask("Importing (all)", "import started", 
    ImportReport.MsgType.Info);

// ... import operations ...

report.EndTask("Importing (all)", "");

report.ReportAll();

Task nesting provides structure:

Importing (all)
  TypeImport
    StoreyTyped: Ground Floor
    StoreyTyped: First Floor
  ElementImport
    StoreyElem: Ground Floor
    StoreyElem: First Floor

This enables debugging import failures. Which storey failed? Which type caused issues? The hierarchical report pinpoints problems.

Model Controller Integration

The ModelController component manages the imported model:

ModelController mc = parent.gameObject.AddComponent<ModelController>();
mc.Categories = handednessCorrector.gameObject
    .AddComponent<CategoryController>();

This provides API for:

Other systems reference the ModelController to interact with the BIM model without knowing internal structure.

Mobile AR Considerations

Mobile hardware has constraints. The rendering pipeline addresses these:

Memory pressure — Type library reduces geometry duplication. Texture atlasing combines multiple materials. Mesh compression reduces storage.

Fill rate limits — Depth prepass renders geometry front-to-back. Overdraw minimized through proper sorting and culling.

Bandwidth constraints — Vertex data packing reduces memory bandwidth. Index buffers use 16-bit indices where possible.

Thermal throttling — Frame budget monitoring prevents sustained high load. Quality settings adapt to device capabilities.

These optimizations maintain performance across device generations — from flagship phones to mid-range tablets.

AR Camera Integration

The BIM model renders to texture, not directly to screen. This enables post-processing and blending:

Camera bimCamera;
bimCamera.targetTexture = new RenderTexture(
    Screen.width, Screen.height, 24, RenderTextureFormat.ARGB32);

The BIM render texture feeds into composition shaders. These blend virtual geometry with camera feed, apply edge detection, handle depth-based occlusion.

Separating BIM rendering from camera rendering provides control. Different cameras can have different field-of-view, different culling masks, different post-processing stacks.

Why This Architecture Works

The pipeline separates concerns effectively:

Import handles data conversion. Mesh generation handles geometry. Materials handle appearance. Hierarchy handles organization. Rendering handles display.

Each stage operates independently. Change import format? Mesh generation unchanged. Adjust materials? Import unaffected. Modify hierarchy? Rendering continues working.

This separation enables:

The architecture accommodates BIM's complexity while maintaining real-time rendering performance. That combination enables practical AR-BIM applications on mobile devices.