Implementation update: analysis v6

Built and deployed; fresh live upload remains unverified. The CPU-first upgrade now feeds both upload analysis and the manual/Preview pack pipeline: bar-guided tonal boundaries, capped neighbour/key support for chords with confidence still derived from original audio scores, one shared named-section/label pass, sustained energy-and-timbre transition candidates, grid provenance, and uncertain chord/key display. The source beat grid and export cuts are untouched.

Older v5 results remain readable during a bounded free v6 refresh. The scout now polls while that refresh is pending and redraws existing structure when the new result lands, without resetting beat/cut edits. Missing downbeats use an explicitly estimated four-beat bar position, not a false claim of verified downbeats. Heavy tagging, transcription and exact instrumentation detection are not included.

Focused generated-audio checks cover chromatic preservation, weak-vote suppression, immutable inputs, shared-feature reuse, full chord tail and both production orchestration paths. Upload analysis checks passed 4/4, label checks 13/13, scout/manifest checks 20/20, resolver/DOM/label checks 14/14 + 8/8 + 24/24, and scout polling checks include legacy-to-v6 replacement. The isolated-fill test initially exposed a false boundary; requiring stability on both sides fixed it. These are regression results, not real-music accuracy claims. Delivery: source commit a873eeb5 landed through 2671778f; studio Pages deployment 944effb9; public deployment-skew guard confirmed gateway running 2671778f. Both changed display scripts match live bytes. Fresh Chromium using the deployed SongStructure script at 1280 and 375 px rendered uncertain keys/chords, estimated bar position and working chapter seek with no overflow or page errors. These rendered supplied fixture data, not a newly processed upload. Live upload verification stopped: stored QA login returned 401; an existing authorized session read succeeded but upload returned 403. No upload/processing success is claimed. Database read showed a ready Hetzner worker, but direct code inspection stopped at a changed SSH host-key check; trust/credentials were not changed. Existing workers should auto-pull via the normal mechanism, but a particular worker’s loaded revision and completed v6 upload were not proven.

Decision

Yes: add confidence-weighted agreement between audio evidence, chords and local key; make one SANCT-grid structure the source of section timing; then add lightweight timbre, energy and vocal evidence. Do not force all chords into the first guessed key. Do not add transcription or stem separation to every free Preview pack.

A C-major region containing C♯ minor is suspicious, not automatically impossible. The chord could be wrong, the key could be wrong, or the music could genuinely use chromatic harmony or modulation. The system should expose uncertainty and require sustained evidence, not cosmetically rewrite the chord.

• Not MIDI: the audited key/chord path works from audio chroma (pitch-class energy), not detected MIDI. Bad transcription is not the demonstrated cause. Dense mixes, overtones, tuning, bass dominance and short windows are plausible failure sources, not measured diagnoses of the founder’s track.
• Key regions are time-based: afcut_engine/harmonic.py:detect_per_section_keys finds novelty on overlapping time cells, maps cuts to window-centre midpoints, scores aggregated chroma and merges matching/short spans. It accepts no beat/downbeat input. Its boundaries can be mid-bar. These are tonal regions, not verse/chorus structure.
• Chords use beats: chord_timeline reads beat times, with time-window fallback and grouping for long grids. afcut_engine/chord_sequence.py:decode_chord_sequence already uses Viterbi with a 0.12 switch penalty and a silence state. It receives scores, energies, bounds and chord names—not section keys. Raising the penalty alone risks erasing real changes.
• Named sections use bars: shared/section_segmenter_v4.py compares bars using 12 chroma features plus RMS, onset density and centroid; repetition and novelty guide boundaries. Explicit downbeats define bars; without them the segmenter groups four beats from the first supplied beat. That fallback does not establish the correct musical bar phase or meter.
• Both processing paths are wired: webapp/app.py:_build_bpm_key_analysis calls the v4 labels and song-section helpers; manual pack processing also calls them. shared/song_sections.py names/consolidates sections and attaches parts. The old report’s “not wired” statements no longer describe current source. This audit is not proof that a particular cached live track has fresh output.
• Two timing layers reach the display: webapp/static/js/song-structure.js normalizes per_section_keys, section_labels_v4 and song_sections. Named blocks prefer song_sections, while tonal regions remain separate. A mid-bar key region is therefore possible alongside bar-aligned section blocks. A wrongly phased downbeat grid, stale results or missing additive output are other possibilities; none was established for a specific track here.
• Some requested features already exist: shared/song_section_parts.py has fill, buildup, riser, drop-hit, break/stop, drum-dropout and turnaround heuristics from band flux, density, energy and brightness. These are not validated instrument detectors. Both audited callers pass vocal_activity=None; the parts fallback uses mid-band energy with percentile hysteresis. Guitar/synth energy can look “vocal”.
• Avoid duplicated work: the labels helper runs segmenter analysis, the song-section builder runs it again, then recomputes bar features; the parts layer makes additional spectral features. Share a feature result before adding more passes.

1. Keep evidence, not just winners. Retain top chord/key scores, audio quality, harmonic/percussive balance and the margin between candidates. Estimate tuning once. Beat-synchronous harmonic chroma is primary evidence; bass chroma is supporting evidence, never an automatic root.
2. Robust votes. Aggregate beat votes over bars and repeated phrases, weighted by usable duration, tonal clarity and score margin. Cap any one loud window’s influence. Skip silence, unpitched drums and weak evidence. Agreement among derived features is correlated evidence, not multiple independent witnesses.
3. Use a soft local-key prior. For candidate chord c in window t, use audio_score(t,c) + λ × key_confidence × compatibility(c,local_key) − transition_cost. Learn/calibrate scales and λ on held-out music; no numeric weight is justified yet. Keep non-diatonic chords available and turn the prior down when the key is uncertain.
4. Do not let the first key win forever. Start with audio-only key candidates; decode chords with a bounded prior; permit at most one re-score of the local key from reliable chord evidence. Cap the feedback so a wrong key cannot manufacture its own supporting chords. Sustained chromatic evidence may create a tonal change, without automatically creating a new song section.
5. Abstain honestly. Preserve N for silence/no chord. Add an uncertain low-evidence outcome for noisy pitched audio, rather than treating all audible windows as a confident triad. Show alternative keys or “uncertain” where needed. Relative major/minor ambiguity is not the same as a remote chromatic key.
6. Separate musical levels. Chords may change on beats, subdivisions or anticipations; key changes need sustained context; named section transitions normally use existing bar edges. Never force every chord or real pickup onto a downbeat. An exceptional off-bar event must be labelled as an event/tonal change, not silently called a new verse.

C-major → C♯-minor example: a single weak C♯m vote surrounded by strong C/G/Am/F evidence loses to audio-supported alternatives or becomes uncertain. A clearly audible sustained C♯m passage survives the prior and triggers local-key reassessment. No rule simply renames C♯m to C.

Cache grid-independent frame features once; aggregate them over the current SANCT beats/bars. Score each existing candidate bar edge with robust, locally normalized changes in harmony, timbre, energy, drum density, vocal activity and repeat structure. Require persistent change or repeat-edge support; a crash/fill alone is usually a part marker, not a section.

• Use 4/8/16-bar phrase lengths as soft preferences, never mandatory lengths. Support unusual meter, pickups and short sections explicitly; report uncertainty when the bar phase is unknown.
• Evaluate loudness/energy rise, bass/drum re-entry and a preceding sparse passage jointly for a drop. A downbeat hit alone does not mean “chorus”. Loudness is measurable; “excitement” is a heuristic label and must not be sold as an objective property.
• Compare repeated sections both harmonically and by timbre. Same chords with added drums/vocals may be a new section or a variation; identical key is not grounds to merge verse and chorus.
• After beat confirmation/inpainting, invalidate grid-dependent results with (audio_hash, feature_version, grid_revision_hash, analyzer_version). Preserve reusable frame features. Do not show an old-grid section result as fresh against corrected beats.
• One additive analysis contract should distinguish song_sections, tonal_regions, chords, parts and evidence/provenance. Preserve legacy reads. Never mix timestamps from different grid revisions.

SANCT constraint: beat/downbeat arrays remain read-only. Select section metadata from existing grid edges; never move beat cuts, re-detect them with a library, or zero-crossing-snap loops. Exact export sample boundaries remain unchanged. Metadata may round for display; sample-sensitive identity must retain grid indices/exact source timing.

• First choice: existing NumPy/SciPy + librosa features. Add compact MFCC/log-mel timbre summaries, spectral flux, harmonic/percussive features and beat/bar pooling to the current repetition system. Librosa’s documented Laplacian segmentation combines beat-synchronous recurrence and MFCC continuity. Reuse the idea with SANCT input, not its example’s beat tracker. Full eigendecomposition is a comparison candidate, not necessary for the first upgrade.
• pyloudnorm: BS.1770-4 metering backed by NumPy/SciPy. Useful for loudness descriptors; begin with existing RMS/crest/relative changes. Integrated whole-song loudness does not locate drops. Use appropriately sized windows, avoid unsupported tiny-window LUFS and never normalize the source audio for this task.
• YAMNet: strongest lightweight tagging experiment. Official tutorial describes MobileNet-v1, 521 event classes and 16 kHz input, returning time-varying scores and embeddings. Test singing/vocal and broad instrument-family evidence, not exact instrument inventories. A compact runtime/export needs separate compatibility and model-licence review; the tutorial alone does not establish deployable weight terms or worker speed. Prefer broad “likely vocals/drums/strings” with uncertainty.
• PANNs: comparison, not default. Provides clip tagging and framewise sound-event detection with PyTorch. Standard Cnn14 inference is not automatically a tiny model. Compare only after the lightweight tagger; validate mix-domain singing and instrument accuracy, checkpoint terms, memory and cold-start cost.
• WebRTC VAD: fast speech/non-speech decisions on 10/20/30 ms PCM frames, not a validated singing detector. Silero-style speech VAD would have the same domain question and was not researched here. Neither should be treated as ground truth for vocals in music.
• No throwaway transcription. Words are unnecessary for vocal presence. Cheap ASR can hallucinate on instruments and miss singing. Use a music-aware tagger; if a paid stem result already exists, aligned vocal-stem activity can add evidence, with bleed checks. Do not create stems solely for free Preview analysis.
• Essentia: rich tonal, temporal and high-level descriptors, but its repository states AGPLv3. Treat commercial licensing/obligations and separate model terms as an adoption gate, not an innocuous dependency swap.
• allin1: useful external structure comparison; its default is an eight-model ensemble with demixed inputs, plus PyTorch/NATTEN/madmom dependencies. The published RTX4090 speed is not our CPU/T4 speed. Do not replace SANCT beats with its output or assume existing stems drop in without an adapter. Model/dependency terms remain a gate.

Adoption order: shared features and timing contract → harmonic agreement → energy/timbre fusion → optional music-aware tagger → heavier reference models only if the measured improvement warrants them.

No new model was installed or run; no production jobs or costs were measured for this report. No speedup, accuracy gain or price is claimed.

• Keep the current 512-bar guard. A 512×512 float32 similarity matrix is 1 MiB (arithmetic only); several matrices, audio buffers and spectral/model workspaces cost more.
• Illustrative cache sizing: 300 s at a 0.1 s hop × 48 float32 features ≈ 576,000 bytes (0.55 MiB), excluding metadata. This is a proposed compact feature layout, not the current format or a measured memory footprint.
• Stream frames; share STFT/chroma/bar computations; bound tagger batches; avoid loading a full model per section. Run CPU work on queued auto-scaled workers; any heavy model/GPU work belongs on Modal, never the gateway.
• Before enabling a new default, measure warm/cold wall time, process CPU, incremental peak memory, model download/startup, cache bytes and preview completion latency for baseline and candidate on matched workers. Proposed initial budget: ≤10% median preview latency increase and ≤128 MiB incremental peak memory for the CPU feature-only upgrade; these are design ceilings, not measured achievements.
• Cost formula: incremental worker occupancy seconds / 3600 × actual worker hourly rate, plus any extra billed worker-hour, model/GPU time and storage/egress. The older skill’s approximate €0.025/hour is not a current provider quote. No credible €/track can be stated without the measurements. No new charges or plan changes proposed.

1. Reconcile timing first: one read-only grid context and shared feature bundle; label key regions separately from song sections; attach grid revision/provenance. Add diagnostics for missing/downbeat-fallback grids and mismatched versions. Check both free analysis and Preview pack paths.
2. Harmonic consistency: confidence-capped votes, soft key compatibility and bounded re-scoring behind a flag. Regressions: held chord with isolated chromatic noise; genuine chromatic/borrowed chord; sustained modulation; relative-major/minor ambiguity; detuning; silence and percussion-only audio. Require fewer false flips without suppressing real changes.
3. Section fusion: share timbre/energy/onset features; distinguish fills from transitions; preserve a same-key verse/chorus change. Test pickup, wrong bar phase, missing downbeats, 3/4, tempo drift and corrected-grid cache invalidation. Exact beat arrays and export cuts must compare unchanged.
4. Tagger experiment only: optional singing/instrument evidence; compare against the current mid-band proxy. Test instrumental synth/guitar false positives, quiet vocals, rap, choir and vocal effects. No lyric generation or default stem processing.
5. Release evidence: a small held-out rights-safe set spanning electronic, acoustic, dense, sparse and live material, with human section/key/chord/event annotations and permitted use recorded. Use mir_eval boundary precision/recall/F at both 0.5 s and one-beat tolerance, plus bar-index error, label/repetition quality, duration-weighted chord accuracy, key/modulation accuracy and abstention coverage. Report event precision separately and vocal false positives/recall. Compare feature ablations and runtime against baseline; choose thresholds before the held-out run. Existing synthetic results are not real-music proof.

Important limit: this task did not inspect the founder’s actual track, reproduce its wrong boundary, benchmark models, reconcile the old synthetic gate, or exercise a fresh live processing flow. The report identifies architectural causes that can produce the complaint—not a proven diagnosis of that track. Existing acceptance failures in earlier tabs are not waived.

Repository symbols above were read directly at the audited revision; line numbers are intentionally omitted because they drift. External pages read 20 October 2026:

• Librosa Laplacian segmentation — recurrence, MFCC continuity, beat pooling; example code ISC.
• Official YAMNet tutorial — architecture, input and outputs; not a full checkpoint licence audit.
• PANNs inference — tagging and framewise events.
• pyloudnorm — BS.1770-4 and dependencies.
• WebRTC VAD interface — speech framing/input contract.
• Essentia — capabilities and AGPLv3.
• allin1 — dependencies, ensemble, demixed inputs and vendor hardware benchmark.

Related historical work: the v3/v4 tabs here, docs/reports/song-structure/README.md, and docs/work-dispositions/section-tiling.md. This report supersedes only the wiring inventory where explicitly stated, not their recorded measurements.