From 141487e3df66e9a5e7a5723ecd713e43f6ee04c4 Mon Sep 17 00:00:00 2001 From: Jens Mohrmann Date: Sun, 13 Sep 2026 19:16:54 +0200 Subject: [PATCH] Update docs: detection/matching pipeline, new settings, testing --- README.md | 10 ++++++---- TECHNICAL_REALIZATION.md | 25 ++++++++++++++++++------- 2 files changed, 24 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index c9cae0d..eae358c 100644 --- a/README.md +++ b/README.md @@ -7,11 +7,13 @@ A web-based application designed to help musicians practice sight-reading and ea ## Features - **Sight Reading Mode**: Read notes from the interactive staff and play them on your instrument. The app listens and confirms when you hit the correct note. -- **Ear Training Mode**: Listen to a reference note and try to reproduce it. -- **Real-time Pitch Detection**: Uses your device's microphone to detect notes instantly. -- **Instrument Support**: Optimized for Guitar and Piano, with configurable tunings for guitar. +- **Ear Training Mode**: Listen to a reference note and try to reproduce it. The reference note duration is configurable, and it can optionally wait for your instrument to go quiet before the next one plays. +- **Real-time Pitch Detection**: Uses your device's microphone to detect notes instantly, with an analysis window and detector adapted to your instrument. +- **Chromatic Tuner**: Tunable needle with cents display, always-on when enabled. +- **Instrument Support**: Optimized for Guitar, Bass, Piano, Voice and Whistle, with configurable tunings for guitar and bass. - **Virtual Instruments**: On-screen interactive guitar fretboard and piano keys for visual reference or touch input. -- **Customizable**: Adjust difficulty (range, accidentals), key signatures, rhythm/metronome settings, and more. +- **Customizable**: Difficulty (range, accidentals), key signatures, rhythm/metronome settings, reference note duration, microphone sensitivity, and more. +- **Dark & Light Themes**: Full theme support including native form controls. - **Zen Mode**: Distraction-free practice interface. ## Getting Started diff --git a/TECHNICAL_REALIZATION.md b/TECHNICAL_REALIZATION.md index 09aaa43..faca27f 100644 --- a/TECHNICAL_REALIZATION.md +++ b/TECHNICAL_REALIZATION.md @@ -11,8 +11,10 @@ The **Sheet Music Trainer** is a client-side Single Page Application (SPA) built - State management is primarily handled via React's `useState` and `useReducer` at the `App` component level, with props drilling for simpler hierarchies. 2. **Audio Processing**: - - **Pitch Detection**: Utilizes the `pitchfinder` library (YIN algorithm) to detect pitch from the user's microphone in real-time. This processing happens in a dedicated hook/worker to keep the main thread responsive. - - **Audio Synthesis**: Uses standard Web Audio API for generating simple tones (sine/triangle waves) for playback and feedback. + - **Pitch Detection** (`src/audio/PitchAnalyzer.ts`): Uses the `pitchfinder` library with a detector chosen per instrument range — ACF2PLUS for normal instruments (robust down to ~40 Hz; pitchfinder's YIN returns ~20 kHz garbage below ~100 Hz and on quiet input), YIN only for high registers (whistle). The analysis window adapts to the instrument's lowest note (~2 periods, power of two): roughly half the latency and a quarter of the CPU of the previous fixed 4096-sample window for guitar/piano/voice, while bass keeps the full window for E1. Readings implausibly below the instrument's floor (e.g. mains hum) are rejected. Detection runs on the requestAnimationFrame loop inside `usePitchDetector`, with display-only damping (EMA for cents/level, short note-name hysteresis) applied in the producer. + - **Note Matching** (`src/game/MatchTracker.ts`): A note counts as matched when held for `NOTE_MATCH_THRESHOLD_MS` (50 ms), tolerated across brief detection dropouts up to `NOTE_MATCH_GRACE_MS` (150 ms). Pitch detectors flicker during note attacks and decays; a naive consecutive-frame timer resets on every flicker. + - **Audio Synthesis** (`src/audio/AudioEngine.ts`): All speaker output goes through a single shared AudioContext (triangle waves). The engine tracks when output may still be audible so the app can ignore its own speaker: while a reference note is sounding (plus a 0.4 s decay tail), microphone-based matching and live feedback are gated — otherwise ear training would score the reference note it just played through the speaker. A suspended AudioContext (autoplay policy) is treated as inaudible so the gate can never deadlock. + - **Ear training auto-play**: The next reference note can optionally wait for the microphone to detect silence ("Wait for Silence" setting, 8 s safety cap), and its duration is configurable ("Reference Note Duration", 0.5–4 s). 3. **Music Rendering**: - **VexFlow**: The standard library for rendering music notation on the web. It is used in the `SheetMusic` component to draw the staff, notes, clefs, and key signatures dynamically based on the current state. @@ -34,15 +36,24 @@ The **Sheet Music Trainer** is a client-side Single Page Application (SPA) built ### UI & Icons * **[Lucide React](https://lucide.dev/)**: A clean and consistent icon library for the interface. +## Testing + +* **Unit/integration** ([Vitest](https://vitest.dev/)): `src/game/MatchTracker.test.ts` covers the flicker-tolerance matching logic; `src/audio/pitchDetection.test.ts` feeds synthetic instrument-like signals (guitar/bass/piano/whistle, quiet plucks, mains hum) through the exact pipeline helpers used by `PitchAnalyzer`; `src/music/NoteUtils.test.ts` covers note math and key signatures. +* **End-to-end**: the microphone can be stubbed in a real browser by replacing `navigator.mediaDevices.getUserMedia` with a `MediaStreamDestination` fed by a scripted oscillator — frequency ramps simulate tuning, envelopes simulate plucks. This setup (puppeteer-core + system Chrome, headless) reproduced and verified several real bugs: YIN's low-frequency garbage mode, noise-floor amplification freezing the tuner on a "note nobody played", and WebKit/Safari leaving constantly-interrupted CSS transitions visually stuck. + ## Directory Structure ``` src/ -├── components/ # Reusable UI components (SheetMusic, Fretboard, etc.) -├── hooks/ # Custom React hooks (usePitchDetector, useAudioPlayer, etc.) -├── music/ # Music logic, tuning definitions, and note utilities -├── styles/ # Global styles and component-specific CSS -├── assets/ # Static assets +├── audio/ # PitchAnalyzer (detection pipeline), AudioEngine (shared playback + mic gating) +├── components/ # Reusable UI components (SheetMusic, Fretboard, TuningMeter, etc.) +├── context/ # Settings context (split into context/Provider/hook files for fast-refresh) +├── game/ # MatchTracker: flicker-tolerant note-hold matching (unit-tested) +├── hooks/ # Custom React hooks (usePitchDetector, useGameLogic, useAudioPlayer, etc.) +├── music/ # Music logic, tuning definitions, and note utilities +├── styles/ # Global styles and component-specific CSS +├── types/ # Shared TypeScript types + pitchfinder module declarations +├── assets/ # Static assets ├── App.tsx # Main application logic and layout └── main.tsx # Entry point ```