qr-based ar anchoring in construction

date: April 15, 2020

The problem: GPS doesn't work indoors. Construction sites need AR overlays positioned accurately, within centimeters, not meters. Walk into a building under construction, point your device at a wall, and the virtual BIM model needs to align perfectly with physical elements.

Traditional AR solutions use feature tracking or SLAM. These drift over time and struggle in sparse environments. We needed something deterministic. Something that works on day one of a project when there's nothing but bare concrete.

Solution: QR markers with known positions. Scan a code, instantly anchor the entire BIM model to physical space.

The Challenge

Indoor AR positioning presents three core problems:

No GPS signal. Satellite positioning fails inside buildings. Alternative methods needed.

Precision requirements. Construction tolerances demand accuracy. A pipe off by 10cm is a problem. AR overlays must match that precision.

Dynamic environments. Construction sites change daily. Today's landmark is tomorrow's demolished wall. The positioning system can't rely on persistent features.

Architecture Overview

The solution uses three coordinated systems:

Multi-threaded QR detection. OpenCV running on background thread, processing camera frames continuously without blocking the main thread.

Marker management system. Handles marker lifecycle, spatial relationships, and intelligent neighbor activation to maintain tracking coverage.

Transformation math. Converts marker-relative positions to world-space coordinates, applying inverse transforms to anchor the BIM model correctly.

Threading Strategy

QR detection is expensive. Processing frames on the main thread would drop framerate to unacceptable levels. The solution: dedicated detection thread using concurrent queues.

private Thread processThread;
private readonly ConcurrentQueue<string> detectedCodes = new ConcurrentQueue<string>();
private byte[] pixels;
private bool bufferAvailable;

private void Update()
{
    if (detectedCodes.TryDequeue(out string newCode)) 
        SetLastDetectedCode(newCode);

    if (bufferAvailable) return;

    if (cameraImage == null || cameraImage.PixelBufferPtr == IntPtr.Zero)
    {
        cameraImage = VuforiaBehaviour.Instance.CameraDevice
            .GetCameraImage(PixelFormat.GRAYSCALE);
        return;
    }

    if (pixels == null)
        InitializeBuffer(cameraImage.BufferWidth, cameraImage.BufferHeight);

    Marshal.Copy(cameraImage.PixelBufferPtr, pixels, 0, 
        cameraImage.BufferWidth * cameraImage.BufferHeight);

    bufferAvailable = true;
}

private void DetectorThread()
{
    while (isRunning)
    {
        if (!bufferAvailable) continue;

        string detectResult = DetectFunction();

        if (!string.IsNullOrWhiteSpace(detectResult))
            detectedCodes.Enqueue(detectResult);

        bufferAvailable = false;
        GC.Collect();
        Thread.Sleep((int)(detectionTimeInterval * 1000));
    }
}

Main thread grabs camera frame, copies pixels to shared buffer, sets flag.

Detection thread waits for flag, processes with OpenCV, queues result, sleeps.

Concurrent queue safely passes detected codes back to main thread for processing.

This pattern maintains 60fps AR rendering while continuously scanning for markers. No locks. No blocking. Clean separation.

Marker Management

Each physical marker has stored metadata: position, rotation, neighboring markers. The system doesn't track every marker simultaneously. That's wasteful. Instead, intelligent activation based on detection and proximity.

private void OnQrCodeDetection(string obj)
{
    Match match = idRegex.Match(obj);

    if (!match.Success || !int.TryParse(match.Value, out int id)) return;

    if (activeMarkers.ContainsKey(id) && activeMarkers[id].enabled) return;

    DisableAllActiveMarkers();

    ActivateMarker(id, null, true);
}

private void ActivateMarker(int markerID, StoreyData storeyData = null, 
    bool activateNeighbours = false)
{
    if (activeMarkers.ContainsKey(markerID))
    {
        activeMarkers[markerID].gameObject.SetActive(true);
        activeMarkers[markerID].enabled = true;
        return;
    }

    storeyData ??= storeyDataMap[activeStorey];

    Texture2D texture = textureDBMap[storeyData.markerDatabaseID][markerID];

    if (!markerInstanceMap.ContainsKey(texture))
        markerInstanceMap[texture] = BuildMarker(texture, 
            MARKER_WIDTH_IN_METERS, storeyData.markers[markerID]);

    MagicMarker magicMarker = markerInstanceMap[texture];

    activeMarkers.Add(markerID, magicMarker);
    activeMarkers[markerID].enabled = true;
    activeMarkers[markerID].gameObject.SetActive(true);

    if (!activateNeighbours) return;

    foreach (MarkerData neighbourData in storeyData.markers.Values
        .OrderBy(x => Vector3.Distance(x.position, magicMarker.Data.position))
        .Take(closestNeighboursCount))
    {
        ActivateMarker(neighbourData.id, storeyData);
    }
}

When a QR code is detected:

1. Extract ID using regex pattern matching
2. Disable all currently active markers
3. Activate the detected marker
4. Activate the N closest neighboring markers

This creates tracking redundancy. If the user moves and loses sight of the first marker, neighbors are already active and tracking continues seamlessly.

Position Calculation

The critical piece: converting a marker's stored position into world coordinates that correctly position the entire BIM model.

Each marker stores its position relative to the BIM model's origin. When tracked, we need the inverse: where should the model be, relative to this marker?

public void UpdateModelPosition()
{
    if (!ModelController.Instance) return;

    ModelController.Instance.transform.SetParent(transform);

    ModelController.Instance.transform.localPosition = 
        Vector3.Scale(Data.Matrix.inverse.ExtractPosition(), 
        new Vector3(-1, 1, 1));
    
    ModelController.Instance.transform.localRotation = 
        Data.Matrix.ExtractRotation();
}

Parent the model to the marker. This makes the marker the reference frame.

Apply inverse transform. If the marker is at position (5, 0, 3) in the model, the model must be at (-5, 0, -3) relative to the marker.

Handle coordinate handedness. Unity uses left-handed coordinates, BIM data often uses right-handed. The scale operation flips the X axis.

Vuforia handles marker tracking. We handle everything else: managing which markers are active, storing their spatial relationships, calculating transforms.

Regex-Based ID Extraction

QR codes contain structured data. We only need the numeric ID. Regex extracts it cleanly:

private static readonly Regex idRegex = new Regex("[0-9]{3}");

Match match = idRegex.Match(qrCodeContent);
if (match.Success && int.TryParse(match.Value, out int id))
{
    // Process marker ID
}

Pattern [0-9]{3} matches exactly three digits. The QR code can contain other data (project identifiers, metadata, URLs). We ignore it. Three-digit ID is all we need.

This loose coupling means QR codes can evolve. Add project codes, embed URLs, include validation data. The detection system doesn't care. It extracts the ID and continues.

Performance Characteristics

Frame processing: 60fps maintained during active scanning
Detection latency: ~1 second from code appearance to anchor update
Memory footprint: ~50MB for marker textures and instances
Activation time: Less than 100ms to switch primary markers

The system handles typical construction site scenarios:

• Poor lighting conditions
• Dusty or partially obscured markers
• Markers at various distances and angles
• Multiple markers in frame simultaneously

Thread isolation prevents detection cost from impacting rendering. The concurrent queue handles bursts: multiple markers detected in quick succession queue up and process in order.

Tracking State Management

Not all tracking is equal. Vuforia reports status: NO_POSE, LIMITED, DETECTED, TRACKED, EXTENDED_TRACKED. The system aggregates across all active markers:

public int GetTrackingState()
{
    int output = -1;

    foreach (MagicMarker magicMarker in activeMarkers.Values)
    {
        output = magicMarker.CurrentStatus.Status switch
        {
            Status.NO_POSE => Mathf.Max(output, 0),
            Status.LIMITED => Mathf.Max(output, 1),
            Status.DETECTED => Mathf.Max(output, 2),
            Status.TRACKED => Mathf.Max(output, 3),
            Status.EXTENDED_TRACKED => Mathf.Max(output, 2),
            _ => output
        };
    }

    return output;
}

This tells the UI whether to trust the AR overlay. If all markers report NO_POSE, show a warning. If any marker is TRACKED, the positioning is reliable.

World Center Switching

When a new marker gains tracking, it can become the world center (the reference point for all positioning):

private void OnTrackingChanged(ObserverBehaviour observer, TargetStatus newStatus)
{
    if (observer != observerBehaviour) return;
    if (VuforiaBehaviour.Instance.WorldCenter == observerBehaviour) return;
    if (newStatus.Status != Status.TRACKED) return;

    CurrentTrackedMarker = this;

    if (VuforiaBehaviour.Instance.WorldCenterMode == WorldCenterMode.SPECIFIC_TARGET)
        VuforiaBehaviour.Instance.SetWorldCenter(WorldCenterMode.SPECIFIC_TARGET, 
            observerBehaviour);

    UpdateModelPosition();
}

This enables smooth handoff between markers. Walk around a corner, lose sight of marker A, marker B takes over. The transition is imperceptible because the model's transform is recalculated relative to the new reference.

Why This Works

QR codes are deterministic. They don't drift. They don't require feature-rich environments. They work in empty spaces, finished buildings, outdoor sites.

The marker database stores surveyed positions. Place markers during site survey, measure their locations, upload to database. From that point forward, any device can scan any marker and instantly know exactly where it is in the building.

This matters for construction:

Day-one accuracy. No training period. No environment mapping. Scan a marker, get positioned.

Persistent across sessions. Turn off the device, come back tomorrow, same accuracy.

Multi-user consistency. Different users scanning different markers see the same aligned model.

Verifiable positioning. Need to check accuracy? Measure distances between physical elements and their virtual counterparts.

The threading strategy maintains performance. The marker management system maintains coverage. The transform math maintains accuracy.

GPS doesn't work indoors. QR markers do.