RF-301 Week 12 — Capstone Preparation

"At Stage 5 you have a hypothesis. At the capstone you have a claim. The difference is evidence you can reproduce." -- RF-301 course doctrine

Lecture (60 min)

12.1 Capstone Mission Restatement

The capstone is not a test of whether you can correctly identify your target's protocol. It is a test of whether you can apply the full SIGINT and analysis pipeline -- Survey, Classify, Characterize, Hypothesize, Verify -- with professional discipline, and package your work so that a colleague can reproduce your results from scratch.

The three deliverables:

1. SIGINT hypothesis document. A confidence-assessed five-stage analysis of your chosen target signal. Every claim has a confidence level: CONFIRMED, INFERRED, or HYPOTHESIZED. No field is blank.

2. GNU Radio demodulator (make demod). A working GNU Radio flowgraph or Python script that reads your IQ archive and produces demodulated output -- PDUs, bits, or audio, depending on the signal. The Tier 1 gate requires this to run.

3. Reproducibility package. Everything a colleague needs to reproduce your results: IQ archive (or a pointer to it), Makefile with make demod and make verify, configuration parameters, and your hypothesis document. verify.py must pass.

Three grading dimensions (from CAPSTONE.md):

12.2 Tier 1 Gate

The Tier 1 gate is a hard prerequisite: make demod must run and produce output before you receive any credit for Dimension A. The gate exists because a SIGINT hypothesis document written after a non-functional demodulator typically describes the system the student hoped to build, not the system they built.

Tier 1 check-in (Week 12 lab session, see §12.4):

# The instructor will run this during Week 12 lab
cd your-capstone-dir
make demod

# Expected: output PDUs / bits / audio to stdout or a file
# Failure: any non-zero exit code or empty output

# Then:
python3 verify.py

# Expected: "PASS" for ≥50% of verification checks
# Failure: "FAIL" or import error

Minimum Tier 1 bar for verify.py:

#!/usr/bin/env python3
"""
verify.py -- capstone Tier 1 verification
Run: python3 verify.py
Pass criteria: ≥ 3 of the 5 checks pass
"""
import os, sys, subprocess, numpy as np

checks_passed = 0
CHECKS_REQUIRED = 3

# Check 1: IQ archive present and non-empty
iq_path = 'data/target_signal.cf32'
if os.path.exists(iq_path) and os.path.getsize(iq_path) > 1e6:
    print("CHECK 1: IQ archive present [PASS]")
    checks_passed += 1
else:
    print(f"CHECK 1: IQ archive missing or empty: {iq_path} [FAIL]")

# Check 2: make demod exits cleanly
result = subprocess.run(['make', 'demod'], capture_output=True, timeout=120)
if result.returncode == 0:
    print("CHECK 2: make demod returns 0 [PASS]")
    checks_passed += 1
else:
    print(f"CHECK 2: make demod failed (exit code {result.returncode}) [FAIL]")

# Check 3: demod output exists
demod_output = 'output/demod_frames.bin'
if os.path.exists(demod_output) and os.path.getsize(demod_output) > 0:
    print("CHECK 3: demod output file non-empty [PASS]")
    checks_passed += 1
else:
    print(f"CHECK 3: demod output missing or empty [FAIL]")

# Check 4: hypothesis document present with confidence levels
hyp_path = 'SIGINT-HYPOTHESIS.md'
if os.path.exists(hyp_path):
    content = open(hyp_path).read()
    has_confirmed = 'CONFIRMED' in content
    has_hypothesized = 'HYPOTHESIZED' in content
    if has_confirmed and has_hypothesized:
        print("CHECK 4: hypothesis doc has CONFIRMED + HYPOTHESIZED entries [PASS]")
        checks_passed += 1
    else:
        print("CHECK 4: hypothesis doc missing confidence labels [FAIL]")
else:
    print(f"CHECK 4: {hyp_path} not found [FAIL]")

# Check 5: reproducibility (README exists with run instructions)
if os.path.exists('README.md') and 'make demod' in open('README.md').read():
    print("CHECK 5: README.md documents 'make demod' [PASS]")
    checks_passed += 1
else:
    print("CHECK 5: README.md missing or lacks run instructions [FAIL]")

print(f"\n{checks_passed}/{5} checks passed. {'TIER 1: PASS' if checks_passed >= CHECKS_REQUIRED else 'TIER 1: FAIL'}")
sys.exit(0 if checks_passed >= CHECKS_REQUIRED else 1)

12.3 Capstone Project Structure

capstone-{your-name}/
├── Makefile                      # make demod, make verify, make clean
├── README.md                     # how to reproduce; hardware/software requirements
├── SIGINT-HYPOTHESIS.md          # five-stage analysis with confidence levels
│
├── data/
│   └── target_signal.cf32        # IQ archive (or symlink + download script)
│
├── output/
│   └── demod_frames.bin          # demodulated output (auto-created by make demod)
│
├── src/
│   ├── demod.py                  # demodulator (or .grc flowgraph)
│   └── verify.py                 # Tier 1 verification script
│
├── plots/
│   ├── stage1_spectrum_survey.png
│   ├── stage2_modulation_features.png
│   ├── stage3_symbol_rate.png
│   └── stage4_preamble.png
│
└── notebook/
    └── analysis.ipynb            # optional working notebook (not graded directly)

Makefile template:

.PHONY: demod verify clean

demod:
	@echo "Running demodulator..."
	python3 src/demod.py data/target_signal.cf32 output/demod_frames.bin
	@echo "Demodulation complete."

verify:
	python3 src/verify.py

clean:
	rm -f output/demod_frames.bin

12.4 Week 12 Lab Session Agenda

This is the Tier 1 gate check-in session. The instructor will meet with each student individually for approximately 10 minutes. The following is the sequence for each student:

If make demod does not run at check-in: The student receives a Tier 1 FAIL for Week 12 and may attempt re-check-in once during Week 13. A second failure closes Dimension B at zero. This is not a penalty -- it is the professional consequence of presenting an incomplete system.

12.5 Weeks 13-14 Open Lab Schedule

The final two weeks are self-directed with instructor office hours. The targets below are checkpoints, not graded submissions.

Week 13 targets (end-of-week self-check):

• Stage 3 complete: symbol rate estimated and documented with evidence
• Stage 4 complete: preamble identified (or documented absence with reasoning)
• GNU Radio demodulator functional: receives IQ input, emits PDUs or bits
• Lab notebook draft started: at least Stage 1-3 observations written up

Week 14 targets (capstone submission deadline):

• Full five-stage pipeline working and documented
• verify.py passing all 5 checks
• Reproducibility package complete: README.md reviewed, make clean && make demod confirmed
• Submission: zip -r csa301-capstone-{name}.zip capstone-{name}/

12.6 Capstone Rubric Walkthrough

Dimension A: SIGINT Rigor (40 points)

Dimension B: Demodulator Functionality (35 points)

Dimension C: Reproducibility (25 points)

12.7 Common Capstone Failure Modes

Failure Mode 1: Modulation hypothesis wrong and uncaught before Week 13.

A student spends Week 9-11 assuming the signal is FSK, builds an FM-demodulator, and discovers in Week 13 that the signal is OOK. This costs 6-10 lab days and often results in incomplete Dimension B. The fix: by the end of Week 11, run Stage 2 analysis twice -- once manually (Week 7 heuristics) and once with the RadioML CNN (Lab 11). If they disagree, investigate before building the demodulator.

Failure Mode 2: make demod passes but verify.py fails on output content.

The demodulator runs and produces output, but the output contains no recoverable signal structure -- it is effectively noise. This passes Check 2 (exit code 0) but fails Check 3 (empty-or-garbage output). The diagnostic: inspect output/demod_frames.bin visually. If every frame looks random and the bit distribution is 50/50, the demodulator is running but not demodulating. Review the symbol timing recovery and threshold logic.

Failure Mode 3: SIGINT hypothesis document with zero HYPOTHESIZED claims.

A student submits a hypothesis document where every field is labeled CONFIRMED. This is almost certainly wrong -- there will always be residual unknowns in a proprietary protocol. Graders will challenge CONFIRMED labels with evidence questions: "How did you verify this independently?" A claim is CONFIRMED only if it is verifiable by a second independent method. Mark uncertain claims HYPOTHESIZED; this is not weakness, it is professional calibration.

Failure Mode 4: Report written before demodulator is validated.

A student writes a detailed report describing the protocol architecture in Week 12, then discovers in Week 13 that the demodulator does not work. The report describes a system that does not exist. Fix: write the Stage 1-2 sections of the hypothesis document early, but hold Stages 3-5 until the demodulator is producing interpretable output.

Failure Mode 5: Reproducibility package missing the IQ archive path.

The README.md says "place your IQ file in data/" but does not specify the filename or format. When the grader runs make demod against a fresh checkout, the demodulator fails because it looks for data/target_signal.cf32 but the student's file is named data/capture1.cf32. Fix: hardcode the expected filename in both demod.py and verify.py, and document it explicitly in README.md.

Lab Preview

There is no separate lab file for Week 12. The lab session is the Tier 1 gate check-in (see §12.4). Bring your IQ archive, your make demod command working, and your Stage 1 spectrum survey plots.

What to have ready by the start of Week 12:

• IQ archive loaded (data/target_signal.cf32)
• make demod produces non-empty output
• plots/stage1_spectrum_survey.png exists with all 4 plots and labeled axes
• Hypothesis document §Stage 1-2 filled in with at least preliminary entries

Toolchain Diary Prompt

This week: no new tools. Use the Week 12 diary entry to reflect on the full toolchain you have assembled across the course, from scipy.signal.welch in Week 7 Stage 1 through crcmod in Week 9 and hackrf_transfer in Week 10. Which tool was most surprising? Which one has the steepest learning curve that a future RF-301 student should prioritize? Include your assessment in your lab notebook.