Introduction to Sonification with Mandala Synth

Welcome to an exploration of sonification - the practice of using non-speech audio to convey information or perceptualize data. Today, I’m excited to share an interactive project that bridges the gap between visual art and auditory experience: the Mandala Synth.

This project represents a convergence of several passions: digital art, music synthesis, data visualization, and accessibility. By transforming the visual patterns of mandalas into musical compositions, we create a multi-sensory experience that demonstrates the fundamental principles of sonification.

What is Sonification?

Sonification is the auditory equivalent of data visualization. While charts and graphs help us see patterns in data, sonification helps us hear them. This approach is particularly valuable for:

• Accessibility: Making visual information available to visually impaired users
• Pattern Recognition: Our auditory system excels at detecting patterns, rhythms, and anomalies
• Multi-modal Analysis: Combining visual and auditory channels for richer data exploration
• Artistic Expression: Creating aesthetic experiences from data and visual patterns

The Mandala Synth Project

The Mandala Synth transforms circular mandala patterns into real-time musical compositions. Here’s how it works:

Core Concepts

1. Color-to-Frequency Mapping: Each color in the mandala corresponds to a specific musical frequency
2. Spatial Navigation: A magnifying lens scans across the image, sampling colors as it moves
3. Real-time Synthesis: Colors are instantly converted to musical notes using Web Audio API
4. Interactive Control: Users can control rotation speed, audio density, and instrument selection

Technical Implementation

The system uses several key technologies:

• Web Audio API via Tone.js for real-time audio synthesis
• Color Thief for extracting dominant colors from mandala images
• Canvas API for pixel sampling and visual effects
• HSV Color Space for intuitive color-to-pitch mapping

Interactive Demo: Spiral Sound Algorithm

This minimal demo demonstrates the core concepts from my 2012 audio experiments during art therapy studies:

Core Concepts from 2012 Experiments

This minimal demo teaches three fundamental concepts I explored during my art therapy studies:

1. Simple Spiral Algorithm

The spiral moves through the mandala in a mathematical pattern, visiting different color regions in sequence. This creates a predictable yet organic path through the visual space:

// Basic spiral movement
spiralAngle += 0.02 * speed;
for (let i = 0; i < 8; i++) {
  const angle = (i * Math.PI * 2) / 8 + spiralAngle;
  const x = centerX + Math.cos(angle) * radius * 0.7;
  const y = centerY + Math.sin(angle) * radius * 0.7;
}

See how it works in practice:

2. Sound-to-Note Theory

Colors map to musical frequencies using established harmonic relationships. Each color ring corresponds to different notes in the selected scale:

• Pentatonic: Traditional 5-note scale (C-D-E-G-A)
• Major: Happy, bright 7-note scale
• Minor: Melancholic, deeper 7-note scale
• Chromatic: All 12 semitones for complex harmonies

3. Mandala Harmonic Patterns

The alternating colors in concentric rings create natural harmonic progressions. As the spiral moves through different color combinations, it generates musical relationships that mirror the visual symmetry of the mandala.

4. Distributed Scale Interpolation

Each mandala example uses scale progression - different musical scales are applied to different rings as the angle moves outward:

// Example: Cosmic mandala progression
scaleProgression: ['chromatic', 'pentatonic', 'minor']

// Inner rings (0-33%) use chromatic (complex, all 12 notes)
// Middle rings (33-66%) use pentatonic (harmonious, 5 notes)  
// Outer rings (66-100%) use minor (melancholic, 7 notes)

This creates evolving harmonic textures as the spiral moves through different radial zones, mimicking how mandalas often have different symbolic meanings from center to edge.

Mathematical Foundations of Music

Understanding the math behind the harmony helps appreciate why certain combinations sound pleasing:

Frequency Relationships

Musical notes are based on frequency ratios. Each octave doubles the frequency:

// Mathematical relationship: f₂ = f₁ × 2^(n/12)
// Where n = number of semitones
const frequency = baseFreq * Math.pow(2, semitones / 12);

// Examples:
// C4 = 261.63 Hz
// C5 = 261.63 × 2¹ = 523.26 Hz (one octave higher)
// G4 = 261.63 × 2^(7/12) = 392.00 Hz (perfect fifth)

Scale Mathematics

Different scales use specific interval patterns (measured in semitones):

• Pentatonic: [0, 2, 4, 7, 9] - avoids dissonant half-steps
• Major: [0, 2, 4, 5, 7, 9, 11] - follows whole-whole-half pattern
• Minor: [0, 2, 3, 5, 7, 8, 10] - lowered 3rd, 6th, and 7th degrees
• Chromatic: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] - all semitones

Harmonic Ratios

Pleasant-sounding intervals have simple mathematical ratios:

• Octave: 2:1 ratio
• Perfect Fifth: 3:2 ratio
• Perfect Fourth: 4:3 ratio
• Major Third: 5:4 ratio

Rotation & Phase Mathematics

The mandala rotation creates phase relationships between visual and auditory patterns:

// Rotation creates circular motion
x = centerX + radius × cos(angle + spiralAngle)
y = centerY + radius × sin(angle + spiralAngle)

// Where spiralAngle increases linearly with time
spiralAngle += angularVelocity × deltaTime

How to Use the Mathematical Demo

1. Enable Audio: Click the audio notice to activate Web Audio
2. Select Mandala Themes: Click thumbnails to explore different mathematical progressions:
   • 🔷 Geometric: Bright colors • pentatonic → major → pentatonic
   • 🌿 Natural: Earth tones • minor → major → chromatic
   • 🌌 Cosmic: Cool blues • chromatic → pentatonic → minor
   • 🔥 Fire: Warm colors • major → chromatic → major
3. Understanding Rotation: Adjust the Rotation Speed slider to see mathematical rotation in action
   • Watch how cos(θ) and sin(θ) create circular motion
   • RPM (Rotations Per Minute) controls angular velocity
4. Color Mathematics: Click the extracted color circles to hear frequency calculations
5. Spiral Algorithm: Control Spiral Speed to see how patterns traverse the mandala
6. Combined Motion: Start the spiral to hear both rotation and spiral mathematics working together

The Magic of Color-Sound Relationships

The demo uses a simple but effective mapping:

• Color position determines the base note
• Ring position adds harmonic intervals
• Combined patterns create the musical phrases you hear

This approach mirrors how mandalas work visually - simple elements combining to create complex, harmonious patterns.

Spiral Mandala Builder

Watch how the spiral algorithm can “steal” colors from the original mandala to create a new spiral representation:

How the Spiral Builder Works

The spiral builder demonstrates a key concept in algorithmic art - how simple mathematical patterns can transform and reinterpret visual information:

// 1. Calculate spiral position (Archimedean spiral)
const progress = currentStep / totalSteps;
const angle = progress * Math.PI * 6;        // 3 full rotations
const radius = progress * maxRadius;          // Linear radius growth

// 2. Sample color from source mandala
const sourceX = centerX + radius * Math.cos(angle);
const sourceY = centerY + radius * Math.sin(angle);
const color = sampleColor(sourceX, sourceY);

// 3. Paint sampled color at spiral position
const spiralX = centerX + radius * Math.cos(angle);
const spiralY = centerY + radius * Math.sin(angle);
spiralCtx.fillStyle = color;
spiralCtx.arc(spiralX, spiralY, 2, 0, Math.PI * 2);

Key Concepts:

• Pixel Sampling: Using getImageData() to read color values from canvas
• Coordinate Transformation: Converting polar (r, θ) to cartesian (x, y) coordinates
• Algorithmic Reconstruction: Rebuilding images using mathematical patterns
• Progressive Animation: Building spirals step-by-step to show the process

This technique forms the foundation of many generative art and data visualization approaches, where algorithms traverse and reinterpret existing visual data.

Advanced Aspiral Mandala Algorithm

Let’s explore a more sophisticated algorithm that creates an aspiral (Archimedean spiral) trace through a real mandala image, sampling and inheriting the original pixel colors:

Mathematical Foundation of Aspiral Algorithm

The aspiral (Archimedean spiral) follows the mathematical formula:

// Archimedean spiral: r = a + b*θ
const theta = progress * Math.PI * 100; // 50 full rotations (laps)
const radius = progress * maxRadius;    // Linear radius growth

// Polar to Cartesian conversion
const x = centerX + radius * Math.cos(theta);
const y = centerY + radius * Math.sin(theta);

// High-density sampling with 1-pixel precision
reconstructionCtx.arc(x, y, 1, 0, Math.PI * 2); // 1-pixel dots

Key Algorithmic Concepts:

1. High-Density Progressive Sampling: 5000 steps trace through 50 complete spiral laps
2. Pixel-Perfect Color Inheritance: Each pixel’s RGBA values are preserved at 1-pixel precision
3. Temporal Optimization: Complete trace runs in exactly 15 seconds for optimal viewing
4. Mathematical Precision: 50 rotations provide comprehensive mandala coverage

Applications:

• Image Analysis: Understanding pixel distribution patterns
• Data Visualization: Converting 2D images to 1D sequential data
• Artistic Reconstruction: Creating new interpretations of existing images
• Compression Studies: Analyzing how much visual information is preserved

Color Extraction and Musical Scales

Let’s start by extracting a 12-color palette from the mandala and understanding how colors map to musical scales:

Understanding Musical Scales in Sonification

Musical scales provide the harmonic foundation for our color-to-sound mapping:

// Musical scales for color-to-note mapping
const scales = {
  major: [0, 2, 4, 5, 7, 9, 11],        // Happy, bright sound
  minor: [0, 2, 3, 5, 7, 8, 10],        // Melancholic, deeper sound
  pentatonic: [0, 2, 4, 7, 9],          // Harmonious, no dissonance
  chromatic: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] // All semitones
};

// Convert color index to musical frequency
function colorToFrequency(colorIndex) {
  const currentScale = scales[selectedScale];
  const baseFreq = 261.63; // C4
  const noteIndex = colorIndex % currentScale.length;
  const semitones = currentScale[noteIndex];
  
  return baseFreq * Math.pow(2, semitones / 12);
}

Key Concepts:

• Color Extraction: Uses ColorThief library to find 7 dominant colors from mandala
• Scale Mapping: Each color maps to a note in the selected musical scale
• Interactive Exploration: Click colors to hear their corresponding notes
• Scale Comparison: Switch between different scales to hear harmonic variations

Interactive Aspiral Reconstruction

Now let’s combine the color extraction with aspiral reconstruction and real-time sonification:

This Interactive Aspiral Reconstruction demonstrates the complete sonification pipeline:

1. Color Extraction: Analyzes mandala to create 12-color palette
2. Pixel Sampling: Traces through image using aspiral algorithm
3. Color Matching: Maps each pixel to closest palette color
4. Musical Mapping: Converts palette colors to pentatonic scale notes
5. Real-time Synthesis: Plays corresponding notes as pixels are drawn

The result is a live musical interpretation of the mandala’s visual structure!

Full Mandala Playground

Experience the complete sonification system with real mandala images from my art therapy collection:

The full playground includes:

• Real mandala images from my 2012 art therapy collection
• Advanced synthesis controls with multiple instruments
• Full-screen immersive experience
• Complete magnifier system for pixel-level exploration
• Professional audio controls with knobs and real-time feedback
• Keyboard shortcuts for easy navigation

Navigation:

• Space - Play/Stop
• ←/→ - Navigate mandalas
• R - Random mandala
• Esc - Close playground

From Art Therapy to Code

This project represents a personal journey combining:

• Creative exploration from my art therapy studies
• Technical curiosity about audio programming
• Pattern recognition in both visual and auditory domains

The beauty of mandalas lies in their mathematical harmony - the same principles that make them visually pleasing also create musical consonance when translated to sound.

Evolution as Components

This minimal demo can progressively grow into:

1. Advanced spiral algorithms (fibonacci spirals, golden ratio patterns)
2. Complex harmonic relationships (just intonation, microtonal scales)
3. Interactive drawing tools (paint your own sonifying mandalas)
4. Real-time collaboration (shared mandala creation with live audio)

Educational Applications

Perfect for teaching:

• Mathematical patterns in art and music
• Cross-sensory experiences for accessibility
• Creative coding fundamentals
• Audio programming concepts

The simple interface allows focus on the core concepts without overwhelming complexity.

Conclusion

Sometimes the most profound explorations come from combining different passions. This sonification experiment bridges art therapy, mathematics, and music technology - showing how creative coding can be both educational and meditative.

Try it: Start with the pentatonic scale for harmonic comfort, then explore chromatic for more complex relationships. Notice how the visual symmetry translates to musical patterns.

Resources and Further Reading

• Tone.js Documentation - Web Audio framework used in this project
• Color Thief - Color extraction library
• International Community for Auditory Display - Academic sonification research
• Web Audio API Specification - Browser audio capabilities
• Sonification Design Guidelines - Best practices for audio data representation

This project builds upon my previous work in sonification, including the Data Synthesizer concept and explores practical applications of audio-based data exploration.