jens/sheetmusictrainer
1
0
Fork 0
mirror of https://github.com/9x/sheetmusictrainer.git synced 2026-11-03 22:03:54 +01:00
Code Issues Packages Projects Releases Wiki Activity

Update docs: detection/matching pipeline, new settings, testing

Browse Source
This commit is contained in:
Jens Mohrmann
2026-09-13 19:16:54 +02:00
parent 3cc9a0c43d
commit 141487e3df
2 changed files with 24 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
TECHNICAL_REALIZATION.md
View File

@@ -11,8 +11,10 @@ The **Sheet Music Trainer** is a client-side Single Page Application (SPA) built
- State management is primarily handled via React's `useState` and `useReducer` at the `App` component level, with props drilling for simpler hierarchies.
2. **Audio Processing**:
- **Pitch Detection**: Utilizes the `pitchfinder` library (YIN algorithm) to detect pitch from the user's microphone in real-time. This processing happens in a dedicated hook/worker to keep the main thread responsive.
- **Audio Synthesis**: Uses standard Web Audio API for generating simple tones (sine/triangle waves) for playback and feedback.
- **Pitch Detection** (`src/audio/PitchAnalyzer.ts`): Uses the `pitchfinder` library with a detector chosen per instrument range — ACF2PLUS for normal instruments (robust down to ~40 Hz; pitchfinder's YIN returns ~20 kHz garbage below ~100 Hz and on quiet input), YIN only for high registers (whistle). The analysis window adapts to the instrument's lowest note (~2 periods, power of two): roughly half the latency and a quarter of the CPU of the previous fixed 4096-sample window for guitar/piano/voice, while bass keeps the full window for E1. Readings implausibly below the instrument's floor (e.g. mains hum) are rejected. Detection runs on the requestAnimationFrame loop inside `usePitchDetector`, with display-only damping (EMA for cents/level, short note-name hysteresis) applied in the producer.
- **Note Matching** (`src/game/MatchTracker.ts`): A note counts as matched when held for `NOTE_MATCH_THRESHOLD_MS` (50 ms), tolerated across brief detection dropouts up to `NOTE_MATCH_GRACE_MS` (150 ms). Pitch detectors flicker during note attacks and decays; a naive consecutive-frame timer resets on every flicker.
- **Audio Synthesis** (`src/audio/AudioEngine.ts`): All speaker output goes through a single shared AudioContext (triangle waves). The engine tracks when output may still be audible so the app can ignore its own speaker: while a reference note is sounding (plus a 0.4 s decay tail), microphone-based matching and live feedback are gated — otherwise ear training would score the reference note it just played through the speaker. A suspended AudioContext (autoplay policy) is treated as inaudible so the gate can never deadlock.
- **Ear training auto-play**: The next reference note can optionally wait for the microphone to detect silence ("Wait for Silence" setting, 8 s safety cap), and its duration is configurable ("Reference Note Duration", 0.5–4 s).
3. **Music Rendering**:
- **VexFlow**: The standard library for rendering music notation on the web. It is used in the `SheetMusic` component to draw the staff, notes, clefs, and key signatures dynamically based on the current state.
@@ -34,15 +36,24 @@ The **Sheet Music Trainer** is a client-side Single Page Application (SPA) built
### UI & Icons
* **[Lucide React](https://lucide.dev/)**: A clean and consistent icon library for the interface.
## Testing
* **Unit/integration** ([Vitest](https://vitest.dev/)): `src/game/MatchTracker.test.ts` covers the flicker-tolerance matching logic; `src/audio/pitchDetection.test.ts` feeds synthetic instrument-like signals (guitar/bass/piano/whistle, quiet plucks, mains hum) through the exact pipeline helpers used by `PitchAnalyzer`; `src/music/NoteUtils.test.ts` covers note math and key signatures.
* **End-to-end**: the microphone can be stubbed in a real browser by replacing `navigator.mediaDevices.getUserMedia` with a `MediaStreamDestination` fed by a scripted oscillator — frequency ramps simulate tuning, envelopes simulate plucks. This setup (puppeteer-core + system Chrome, headless) reproduced and verified several real bugs: YIN's low-frequency garbage mode, noise-floor amplification freezing the tuner on a "note nobody played", and WebKit/Safari leaving constantly-interrupted CSS transitions visually stuck.
## Directory Structure
```
src/
├── components/ # Reusable UI components (SheetMusic, Fretboard, etc.)
├── hooks/ # Custom React hooks (usePitchDetector, useAudioPlayer, etc.)
├── music/ # Music logic, tuning definitions, and note utilities
├── styles/ # Global styles and component-specific CSS
├── assets/ # Static assets
├── audio/ # PitchAnalyzer (detection pipeline), AudioEngine (shared playback + mic gating)
├── components/ # Reusable UI components (SheetMusic, Fretboard, TuningMeter, etc.)
├── context/ # Settings context (split into context/Provider/hook files for fast-refresh)
├── game/ # MatchTracker: flicker-tolerant note-hold matching (unit-tested)
├── hooks/ # Custom React hooks (usePitchDetector, useGameLogic, useAudioPlayer, etc.)
├── music/ # Music logic, tuning definitions, and note utilities
├── styles/ # Global styles and component-specific CSS
├── types/ # Shared TypeScript types + pitchfinder module declarations
├── assets/ # Static assets
├── App.tsx # Main application logic and layout
└── main.tsx # Entry point
```