Skip to main content

DivideTiles - Tile Processing

Overview​

The DivideTiles interface is used to divide scenes into tiles before reconstruction. This interface supports 2D regular grid, 3D regular grid, and memory-based adaptive tiling methods, suitable for processing large-scale datasets.

Applicable Scenarios
  • Ultra-large-scale datasets (thousands of images)
  • Memory-constrained processing environments
  • Projects requiring parallel processing
  • Distributed computing scenarios

How It Works​

Interface Invocation​

Command Line Invocation​

# 方式一：通过主引擎
reconstruct_full_engine(.exe) -reconstruct_type 4 -task_json divide_config.json

# 方式二：独立分块引擎
divide_engine.exe divide_config.json

Parameter Description​

  • reconstruct_type: Fixed as 4 (represents DivideTiles)
  • task_json: Configuration file path

Configuration Parameters​

Required Parameters​

Parameter NameTypeDescription
working_dirstringWorking directory (must contain aerotriangulation results)
gdal_folderstringGDAL data path
resolution_levelintReconstruction accuracy (1=High, 2=Medium, 3=Low)
divide_parametersJSONTile parameters
roi_for_3dJSONROI for 3D reconstruction

Optional Parameters​

Parameter NameTypeDefault ValueDescription
roi_coordinate_systemJSONWGS84ROI coordinate system

Tile Parameters (divide_parameters)​

Parameter NameTypeDefault ValueDescription
divide_modeint-Tile mode: 0=2D grid, 1=3D grid, 2=Memory adaptive
tile_sizedouble-Grid size (required when mode 0/1)
target_memorydouble0Target memory limit (GB)
reconstruct_whole_tileboolfalseWhether to reconstruct complete tile (no boundary cropping)
tile_systemJSONAerotriangulation coordinate systemTile coordinate system

Tile Mode Details​

1. 2D Regular Grid (divide_mode = 0)​

Suitable for areas with little terrain variation:

{
  "divide_mode": 0,
  "tile_size": 500.0,        // 500m x 500m grid
  "tile_system": {
    "type": 3,
    "epsg_code": 32650       // UTM projected coordinate system
  }
}

2. 3D Regular Grid (divide_mode = 1)​

Suitable for areas with large elevation changes:

{
  "divide_mode": 1,
  "tile_size": 300.0,        // 300m x 300m x 300m cube
  "tile_system": {
    "type": 1                // Local coordinate system
  }
}

3. Memory Adaptive Tiling (divide_mode = 2)​

Automatically tiles based on memory limits:

{
  "divide_mode": 2,
  "target_memory": 16.0,     // Maximum 16GB memory per tile
  "reconstruct_whole_tile": false
}

Complete Configuration Examples​

Urban Large-Scale 2D Tiling​

{
  "working_dir": "C:/Projects/CityScale",
  "gdal_folder": "C:/MipMap/SDK/data",
  "resolution_level": 2,
  "divide_parameters": {
    "divide_mode": 0,
    "tile_size": 1000.0,     // 1km grid
    "tile_system": {
      "type": 3,
      "epsg_code": 32650
    },
    "reconstruct_whole_tile": true
  },
  "roi_for_3d": {
    "boundary": [
      [500000, 2500000],
      [510000, 2500000],
      [510000, 2510000],
      [500000, 2510000]
    ],
    "min_z": 0,
    "max_z": 500
  },
  "roi_coordinate_system": {
    "type": 3,
    "epsg_code": 32650
  }
}

Mountain Terrain 3D Tiling​

{
  "working_dir": "C:/Projects/Mountain",
  "gdal_folder": "C:/MipMap/SDK/data",
  "resolution_level": 2,
  "divide_parameters": {
    "divide_mode": 1,
    "tile_size": 500.0,      // 500m cube
    "tile_system": {
      "type": 0,             // LocalENU
      "origin_point": [114.123, 22.123, 100.0]
    }
  },
  "roi_for_3d": {
    "boundary": [...],
    "min_z": 100,
    "max_z": 2000           // Large elevation difference
  }
}

Memory-Constrained Environment​

{
  "working_dir": "C:/Projects/LimitedMemory",
  "gdal_folder": "C:/MipMap/SDK/data",
  "resolution_level": 2,
  "divide_parameters": {
    "divide_mode": 2,
    "target_memory": 8.0,    // Only 8GB available memory
    "reconstruct_whole_tile": false
  },
  "roi_for_3d": {
    "boundary": [...]
  }
}

Output Results​

tiles.json File Structure​

The tiles.json generated after tiling is complete:

{
  "tiles": [
    {
      "name": "tile_0_0",
      "empty": false,
      "max_memory": 12.5,    // GB
      "roi": {
        "boundary": [...],
        "min_z": 100,
        "max_z": 200
      }
    },
    {
      "name": "tile_0_1",
      "empty": false,
      "max_memory": 15.3,
      "roi": {...}
    }
  ],
  "divide_mode": 0,
  "tile_system": {...},
  "tile_grid": {             // Grid information (mode 0/1)
    "origin": [500000, 2500000],
    "rows": 10,
    "cols": 10
  },
  "tile_size": 1000.0
}

Using Tile Results​

Reconstruct3D will automatically read tiles.json and execute tile reconstruction:

{
  "working_dir": "C:/Projects/CityScale",  // Contains tiles.json
  // ... Other Reconstruct3D parameters
}

Tile Strategy Selection​

Scene Type Comparison​

Scene TypeRecommended ModeGrid Size RecommendationDescription
Plain city2D grid500-2000mFlat terrain, small height variation
Mountain terrain3D grid300-1000mLarge elevation difference, needs vertical layering
Mixed sceneMemory adaptive-Automatically optimizes tile plan
Linear engineering2D gridCustomSet long strip along route direction

Memory Estimation​

Memory requirement per tile ≈ (Number of images in tile × Image resolution × 3) / Compression ratio

Reference values:

  • 100 images of 20MP ≈ 8-12 GB
  • 200 images of 20MP ≈ 16-24 GB

Parallel Processing​

Local Parallel​

import subprocess
import json
import concurrent.futures

# Read tile results
with open("tiles.json", "r") as f:
    tiles_info = json.load(f)

def process_tile(tile):
    if tile["empty"]:
        return
    
    config = {
        "working_dir": f"./output/{tile['name']}",
        "tile_name": tile["name"],
        # ... Other parameters
    }
    
    with open(f"{tile['name']}.json", "w") as f:
        json.dump(config, f)
    
    subprocess.run([
        "reconstruct_full_engine.exe",
        "-reconstruct_type", "2",
        "-task_json", f"{tile['name']}.json"
    ])

# Parallel processing
with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
    executor.map(process_tile, tiles_info["tiles"])

Distributed Processing​

Tile results can be distributed to multiple machines for processing:

  1. Master node performs tiling
  2. Distribute tiles.json and data
  3. Each node independently processes assigned tiles
  4. Aggregate results

Best Practices​

1. Tile Size Selection​

  • Sufficient memory: Use larger tiles to reduce boundary effects
  • Memory constrained: Use smaller tiles to ensure stability
  • Parallel processing: Balance tile count and single tile processing time

2. Overlap Area Processing​

{
  "reconstruct_whole_tile": true,  // Generate complete tile
  // Merge overlap areas during post-processing
}

3. Coordinate System Selection​

  • 2D tiling: Use projected coordinate system (metric units)
  • 3D tiling: Local or LocalENU
  • Avoid 3D tiling in geographic coordinate system

4. ROI Optimization​

{
  "roi_for_3d": {
    "boundary": [
      // Use Smart ROI generated by aerotriangulation
      // Appropriately expand boundaries to avoid cropping
    ]
  }
}

Performance Optimization​

1. Estimate Processing Time​

Total time = Tiling time + max(Processing time per tile) + Merge time

2. Load Balancing​

  • Check max_memory distribution
  • Adjust parameters so each tile has similar load
  • Consider dynamic task allocation

3. Storage Optimization​

  • Use SSD storage for working directory
  • Tile results can be distributed to multiple disks
  • Reserve sufficient temporary space

Common Questions​

Q: Cracks at boundaries after tiling?​

A:

  • Set reconstruct_whole_tile: true
  • Increase inter-tile overlap (by expanding ROI)
  • Post-processing merge optimization

Q: Some tiles failed to process?​

A:

  • Check if the tile's max_memory exceeds limit
  • View image distribution of specific tile
  • Consider re-tiling or adjusting parameters

Q: Too many tiles?​

A:

  • Increase tile_size or target_memory
  • Use different tiling mode
  • Consider layered processing strategy

Advanced Applications​

Custom Tile Plan​

Based on tiles.json, you can:

  1. Manually adjust tile boundaries
  2. Merge small tiles
  3. Split large tiles
  4. Set priorities

Incremental Updates​

# Only process changed areas
changed_tiles = identify_changed_areas()
for tile in changed_tiles:
    process_tile(tile)

Next Steps​

Tip: A reasonable tiling strategy is key to processing large-scale data. It is recommended to test with small data first, find optimal parameters, then process complete data.