Certificate Eligibility Discovery Pipeline

Students routinely complete the coursework for certificates they never receive — they simply don't know they qualify, and no system tells them. At Coastline College I built an end-to-end discovery pipeline: scrape hundreds of public program and course pages, structure the certificate requirements, join them against student records in SQL, and apply business-rule matching. It identified hundreds of potentially eligible students and fed a real outreach effort.

Certificate requirements lived in one world (public catalog and program webpages, written for humans) and student completion records lived in another (the student information system). No process connected them. The result: earned-but-unawarded certificates — a completion metric the college was leaving on the table, and credentials students could have been putting on résumés.

The pipeline had three phases. Discovery: Python and R scrapers collected hundreds of public program and course pages and extracted requirement structures — required courses, unit thresholds, and elective groups. Matching: requirements were normalized into rule sets and joined against student course-completion records in SQL. Action: matched students were scored against the business rules and packaged into lists that student-services staff could actually work for outreach.

-- For each student x certificate: count satisfied requirements
SELECT s.student_id,
       r.certificate_id,
       COUNT(*)                                    AS reqs_met,
       MAX(req.total_required)                     AS reqs_total
FROM student_completions s
JOIN requirement_courses r
  ON r.course_id = s.course_id AND s.grade_points >= r.min_grade_points
JOIN requirement_totals req
  ON req.certificate_id = r.certificate_id
GROUP BY s.student_id, r.certificate_id
HAVING COUNT(*) >= MAX(req.total_required);

The hard part wasn't scraping — it was encoding how requirements actually work: course equivalencies and renumbering over time, 'choose N from this group' electives, unit minimums, and grade thresholds. Each rule type was implemented explicitly and conservatively: when a requirement couldn't be evaluated cleanly, the student was flagged for human review rather than auto-included or silently excluded.

Hundreds of potentially eligible students identified and handed to an operational outreach process — analysis that ended in action, not in a report. The project is a compact demonstration of the full arc I care about: ambiguous question → data acquisition → structure → rules → join → validated list → operational use.

• 'Potentially eligible' is the honest phrasing: final award decisions belong to evaluators applying catalog rights and petition rules the pipeline intentionally did not adjudicate.
• Scraped requirements drift as pages change; the pipeline is a snapshot process that needs re-runs, not a live system.

• Python and R web scraping and extraction from hundreds of public pages
• SQL joins and business-rule matching against student records
• ETL design, data normalization, and conservative edge-case handling
• Operationalization — turning analysis output into a worked outreach process