FluxSpace

Fluxspace Core Pipeline

A complete workflow for processing magnetic field measurements from sensor data to actionable visualizations

Pipeline Workflow

Step 1Data Collection

mag_to_csv.py

Collect magnetic field measurements from MMC5983MA magnetometer sensor and save to CSV

Input

MMC5983MA sensor via I2C

Output

data/raw/mag_data.csv

Key Features

  • •Auto-grid mode for systematic measurement points
  • •Multiple samples per point with averaging
  • •Records Bx, By, Bz components and B_total
  • •UTC timestamps for all measurements
  • •Configurable grid settings (NX, NY, DX, DY, X0, Y0)
Step 2Validation & Cleaning

validate_and_diagnosticsV1.py

Validate, clean, and generate diagnostics for magnetometer CSV data

Input

data/raw/mag_data.csv

Output

data/processed/mag_data_clean.csv

Key Features

  • •Validates CSV structure and required columns
  • •Cleans missing/invalid data
  • •Detects outliers using robust z-score statistics
  • •Detects spikes (sudden changes between measurements)
  • •Generates diagnostic plots and reports
Step 3Anomaly Detection

compute_local_anomaly_v2.py

Detect local magnetic anomalies by comparing each point to its neighborhood

Input

data/processed/mag_data_clean.csv

Output

data/processed/mag_data_anomaly.csv

Key Features

  • •Local anomaly computation (point vs. neighborhood)
  • •Configurable neighborhood radius
  • •Respects quality flags from validation step
  • •Adds local_anomaly, local_anomaly_abs, and local_anomaly_norm columns
  • •Optional scatter plot visualization
Step 4Visualization

interpolate_to_heatmapV1.py

Interpolate scattered measurement points onto a regular grid and generate heatmap visualizations

Input

data/processed/mag_data_anomaly.csv

Output

data/exports/mag_data_grid.csv + heatmap.png

Key Features

  • •IDW (Inverse Distance Weighting) interpolation
  • •Configurable grid resolution
  • •Tunable interpolation power parameter
  • •Exports grid data as CSV
  • •Generates heatmap PNG visualization

Data Flow

Raw Data

data/raw/

Processed Data

data/processed/

Exports

data/exports/

Key Concepts

Auto-Grid Mode

Systematic measurement collection with configurable grid parameters (NX, NY, DX, DY, X0, Y0). The script automatically calculates each grid point and prompts you to move the sensor there.

Quality Flags

Flag columns identify problematic data: outliers (extreme values), spikes (sudden jumps), and combined flags. These can be used to filter data in subsequent processing steps.

Local Anomalies

Unlike global anomalies, local anomalies compare each point to nearby neighbors. This helps detect small-scale variations, regional differences, and localized sources of magnetic disturbance.

Ready to dive deeper?

View detailed documentation for each script in the pipeline

See Real Examples

Explore actual pipeline results with real data from a complete workflow run