Not sure if anybody would be interested, but I have created an audio engine that can control an existing WLED ESP32 with a separate esp32S3_CAM board.
A summary of what it can do is below , let me know if you might be interested and I may develop it further and publicly release-
- The Core Functions
-
Dynamic Audio Playback: The system reads high-quality audio files directly from an onboard SD card, decoding them via a software-based MP3 engine (
minimp3). It streams the decoded data to a high-efficiency digital-to-analog amplifier (MAX98357A) using the I2S protocol to manage the background atmosphere, sound effects (like battle sequences and storms), and narrative vocal tracks. -
Synchronized WLED Lighting: The engine acts as a master controller for a complex addressable LED setup running WLED. It reads a timeline file and flashes lighting presets over the network using low-latency UDP packets to perfectly match real-time events.
-
Deterministic Event Timeline: Instead of relying on approximate delays, the system parses a master timeline file (
Ship_Effects.csv) exported straight from a Digital Audio Workstation (Reaper). The ESP32 tracks its playback clock down to the exact millisecond, executing lighting triggers within an incredibly tight, imperceptible 15–22ms window of the audio track.
2. The Timeline Structure (The Journey)
The software is designed to execute a fully automated, theatrical sequence through a narrative journey. Based on the calibrated seconds-based timeline, the following sequence processes is an example of what is possible:
-
The Voyage Begins: Gentle atmospheric transitions from a quiet dawn morning (Sunrise) into a beautiful, bright midday environment (Sunny).
-
The Elements Close In: The atmosphere darkens into heavy cloud cover (Cloudy), followed by an ethereal night display (Aurora), before dropping into a dense, tense ocean fog.
-
The Battle Sequence: A rapid-fire combat phase where the engine coordinates precise, rhythmic Cannon Fire triggers punctuated by immediate lighting resets to simulate muzzle flashes and clearing smoke.
-
The Tempest: A chaotic meteorological phase where the ship fights through a violent storm, synchronizing massive, randomized Lightning flashes and rumbling thunderclaps.
-
The Calm After: The storm breaks into a vivid Rainbow, transitioning into steady Rain, and easing into physical ship animations (Rocking).
-
The Supernatural & Final Horizon: Mystical maritime phenomena (Mystery Lights and Saint Elmo’s Fire) culminating in a deep-space Comet display, a final dramatic Fire sequence, and a clean automation termination (END).
3. Hardware Architecture & Electrical Design
To ensure reliability in a dedicated workshop environment, the hardware design relies on industrial, serviceable principles rather than fragile, disposable consumer electronics:
-
The Brain: An ESP32-S3 microcontroller, utilizing a dual-core architecture. One core focuses heavily on maintaining the timing-critical audio decoding tasks and DMA stream buffers, while the other handles background logic, SD card data fetching, and Wi-Fi transmission.
-
The PCB Philosophy: A custom board designed in KiCad 9 and fabricated professionally. It leans intentionally toward “old school” Through-Hole Technology (DIP packages) rather than tiny surface-mount (SMD) components. This ensures that every chip, level shifter, and terminal block can be easily inspected, maintained, and repaired right at your workbench.
-
Signal Integrity: Integrated hardware level shifters ensure that the ESP32’s 3.3V logic cleanly communicates with external components without signal corruption, alongside robust grounding planes to shield the sensitive I2S audio signals from digital Wi-Fi noise.
In short, ShipEffects S3 bridges the gap between digital audio engineering and physical scale-modeling, turning a static historical replica into a living, breathing piece of interactive theater.
