# Technical Realization ## Architecture Overview The **Sheet Music Trainer** is a client-side Single Page Application (SPA) built with **React** and **TypeScript**, powered by **Vite**. It is designed to run entirely in the browser without a backend server, allowing for low-latency audio processing and interactivity. ### Key Architectural Choices 1. **Component-Based UI**: - The application is structured into reusable components (e.g., `SheetMusic`, `Fretboard`, `PianoKeys`, `Controls`) to maintain separation of concerns. - State management is primarily handled via React's `useState` and `useReducer` at the `App` component level, with props drilling for simpler hierarchies. 2. **Audio Processing**: - **Pitch Detection** (`src/audio/PitchAnalyzer.ts`): Uses the `pitchfinder` library with a detector chosen per instrument range — ACF2PLUS for normal instruments (robust down to ~40 Hz; pitchfinder's YIN returns ~20 kHz garbage below ~100 Hz and on quiet input), YIN only for high registers (whistle). The analysis window adapts to the instrument's lowest note (~2 periods, power of two): roughly half the latency and a quarter of the CPU of the previous fixed 4096-sample window for guitar/piano/voice, while bass keeps the full window for E1. Readings implausibly below the instrument's floor (e.g. mains hum) are rejected. Detection runs on the requestAnimationFrame loop inside `usePitchDetector`, with display-only damping (EMA for cents/level, short note-name hysteresis) applied in the producer. - **Note Matching** (`src/game/MatchTracker.ts`): A note counts as matched when held for `NOTE_MATCH_THRESHOLD_MS` (50 ms), tolerated across brief detection dropouts up to `NOTE_MATCH_GRACE_MS` (150 ms). Pitch detectors flicker during note attacks and decays; a naive consecutive-frame timer resets on every flicker. - **Audio Synthesis** (`src/audio/AudioEngine.ts`): All speaker output goes through a single shared AudioContext (triangle waves). The engine tracks when output may still be audible so the app can ignore its own speaker: while a reference note is sounding (plus a 0.4 s decay tail), microphone-based matching and live feedback are gated — otherwise ear training would score the reference note it just played through the speaker. A suspended AudioContext (autoplay policy) is treated as inaudible so the gate can never deadlock. - **Ear training auto-play**: The next reference note can optionally wait for the microphone to detect silence ("Wait for Silence" setting, 8 s safety cap), and its duration is configurable ("Reference Note Duration", 0.5–4 s). 3. **Music Rendering**: - **VexFlow**: The standard library for rendering music notation on the web. It is used in the `SheetMusic` component to draw the staff, notes, clefs, and key signatures dynamically based on the current state. 4. **Responsiveness**: - The application uses CSS variables and media queries to adapt to different screen sizes, with specific optimizations for mobile landscape mode to support instrument practice on tablets and phones. ## Frameworks and Libraries ### Core * **[React](https://react.dev/)**: The library for web and native user interfaces. * **[TypeScript](https://www.typescriptlang.org/)**: Strongly typed JavaScript for safer development. * **[Vite](https://vitejs.dev/)**: Next Generation Frontend Tooling for fast development and building. ### Audio & Music * **[VexFlow](https://www.vexflow.com/)**: A JavaScript library for rendering music notation and guitar tablature. * **[Pitchfinder](https://github.com/peterkhayes/pitchfinder)**: A collection of pitch detection algorithms for Javascript. used for detecting the note played by the user. ### UI & Icons * **[Lucide React](https://lucide.dev/)**: A clean and consistent icon library for the interface. ### Phrase Mode (sequential practice) **VexFlow 5 stave placement quirk**: `new Stave(x, y, w)` does not put the first stave line at `y` — the lines draw ~40.5px lower (default `space_above_staff_ln` = 4 line-spaces). The single-note `SheetMusic` absorbed this inside generous heights; the compact live-note staff and phrase staff had to compensate explicitly (negative `Stave` y / shifted row offset) or low ledger notes clipped. The third game mode renders and scores multi-bar single-voice phrases from a versioned normalized score (`src/score/model.ts`, PPQ ticks, ties pre-merged into "logical notes"). Generators (`src/music/melodyGenerator.ts`, `scaleDrills.ts`) and the ABC subset adapter (`src/exercises/abcParser.ts`) both produce it; `PhraseSheetMusic` renders it with measure-aware accidental display and a current-note caret. Matching (`src/game/PhraseMatcher.ts`) latches per-note results and blocks a sustained pitch from re-crediting a later note on the same pitch (release evidence, stable-pitch frames, at-your-pace timeout re-arm). Tempo transport uses audio-clock boundaries (schedule in `src/game/PhraseTransport.ts`) — cursor state follows the clock, never the click scheduler. Raw detection frames flow through `src/hooks/rawFrameBus.ts` (one per analysis tick, silence included); display smoothing is untouched. Metronome clicks are short noise bursts that blank the mic for 30 ms instead of entering the 400 ms pitched-audio gate. See `docs/PHRASE_MODE_IMPLEMENTATION.md` (contracts) and `docs/PHRASE_DELIVERY.md` (verified behavior + limitations). ## Testing * **Unit/integration** ([Vitest](https://vitest.dev/)): `src/game/MatchTracker.test.ts` covers the flicker-tolerance matching logic; `src/audio/pitchDetection.test.ts` feeds synthetic instrument-like signals (guitar/bass/piano/whistle, quiet plucks, mains hum) through the exact pipeline helpers used by `PitchAnalyzer`; `src/music/NoteUtils.test.ts` covers note math and key signatures. * **End-to-end**: the microphone can be stubbed in a real browser by replacing `navigator.mediaDevices.getUserMedia` with a `MediaStreamDestination` fed by a scripted oscillator — frequency ramps simulate tuning, envelopes simulate plucks. This setup (puppeteer-core + system Chrome, headless) reproduced and verified several real bugs: YIN's low-frequency garbage mode, noise-floor amplification freezing the tuner on a "note nobody played", and WebKit/Safari leaving constantly-interrupted CSS transitions visually stuck. ## Directory Structure ``` src/ ├── audio/ # PitchAnalyzer (detection pipeline), AudioEngine (shared playback + mic gating) ├── components/ # Reusable UI components (SheetMusic, Fretboard, TuningMeter, etc.) ├── context/ # Settings context (split into context/Provider/hook files for fast-refresh) ├── game/ # MatchTracker: flicker-tolerant note-hold matching (unit-tested) ├── hooks/ # Custom React hooks (usePitchDetector, useGameLogic, useAudioPlayer, etc.) ├── music/ # Music logic, tuning definitions, and note utilities ├── styles/ # Global styles and component-specific CSS ├── types/ # Shared TypeScript types + pitchfinder module declarations ├── assets/ # Static assets ├── App.tsx # Main application logic and layout └── main.tsx # Entry point ```