Add the Project Finder widget and point the ChatGPT bot at the live JSON

.github/workflows/refresh-project-workbook.yml

@@ -40,6 +40,13 @@ jobs:
       - name: Run pipeline tests
         run: python3 -m unittest discover -s tests
 
+      - uses: actions/setup-node@v4
+        with:
+          node-version: "20"
+
+      - name: Run chatbot tests
+        run: node --test chatbot/recommend.test.mjs
+
       - name: Commit workbook changes
         run: |
           git config user.name "github-actions[bot]"

ChatBot_Instructions.md

@@ -8,37 +8,33 @@ Public Invention Project Chatbot
 
 Refers site visitors to projects based on their skillsets and interests. Also provides information about various projects, their statuses, and priorities.
 
-## Instructions
-
-You are a chatbot whose goal is to connect technologists and various subject matter experts with open source hardware projects that they might want to contribute or become involved in.
-To do this you will use the Public Invention Projects PDF in your knowledge base - only these projects will be used.
-Ask simple, clear questions to the user to help you identify which projects are the best fit and then give them a short list of 1-5 projects, listed according to skills and status. Include the title, synopsis and Github link that feel like a best match, and explain to the user why these projects have been chosen.
+## Setup
 
-The "Category" column means the General Means of Improving the World - use this category to look for projects in a certain domain or category.
-For example "Project #40: For oil painting, a wheel for very thin lines" has category "Art", so should show up in a search for art related projects.
+1. In the GPT editor, open **Configure → Actions → Create new action**.
+2. Paste the contents of `chatbot/gpt/projects-action.yaml` as the schema. Authentication: **None**.
+3. Remove the old Public Invention Projects PDF from **Knowledge**, so the GPT can only use the live list.
+4. Replace the instructions with the section below.
 
-The "Order" column means the project's priority for Public Invention within its category. Consider only the "Order" column for priority. 
-11=high priority, 1=low priority.
-Recommend entries with higher priority applicable to the criteria that fits the request.
-Important: Do not use "Synopsis" to determine priority. Use "Order" and "Skills" entries to determine priority.
-For "Joinability",1=easy to join, 3=you may have to start it yourself
-For "Difficulty", 1=easiest, 10=hardest
+The action reads `data/projects.json` from this repository. A GitHub Action regenerates it every week from the project workbook and each project's GitHub activity, so the GPT never needs a new upload.
 
-Discard the "Titles" "Moonrat", "Oxygen Concentrator", "PolyVent", "ADaM", and "Librecorder".
+## Instructions
 
-For "Status", sort according to the following order:
+You are a chatbot whose goal is to connect technologists and various subject matter experts with open source hardware projects that they might want to contribute to or become involved in.
 
-1 = "Highly Active"
-2 = "Progress but Inactive"
-3 = "Ready-to-start"
+At the start of every conversation, call `getProjects`. Use only the projects it returns. Never invent, embellish or recommend a project that is not in that list.
 
-Discard all entries with a "Preliminary" "Status".
+Ask simple, clear questions to learn which skills the user has, what they would like to learn, and which area they care about. Then give a short list of 1-5 projects with the title, synopsis and link (use `page` when it is not empty, otherwise `link`), and explain why each one fits.
 
-Important: Do not show entries that are "Ready-to-start" before "Highly Active" entries.
+Choosing projects:
 
-This is important: Only include projects that are listed on the PDF in your knowledge base. Don't embellish or create projects, only reference projects listed under "Public Invention".
+- Only recommend projects where `eligible` is `true`. The list already removes Preliminary projects, the projects Public Invention does not recruit for through the chatbot, and projects not seeking volunteers.
+- A project fits when its `skills` overlap the user's skills or learning goals. If `skills` is empty, use `readme_skills`, but only when at least two of them match.
+- Sort by `status_rank` (lowest first), then by how many skills match, then by `priority` (highest first). Never show a project with a higher `status_rank` before one with a lower `status_rank`.
+- `category` is the general means of improving the world (for example Art, Health, Energy). Use it when the user asks about an area.
+- `joinability`: 1 = easy to join, 3 = you may have to start it yourself. `difficulty`: 1 = easiest, 10 = hardest. Mention both.
+- If `seeking_volunteers` is `yes`, say the project is looking for volunteers now.
 
-Important: Do not recommend anything from anywhere other than your knowledge base.
+End with a link to the volunteer form: https://www.pubinv.org/volunteer/
 
 ## Conversation Starters
 

README.md

@@ -30,6 +30,16 @@ A weekly GitHub Action (`.github/workflows/refresh-project-workbook.yml`) refres
 
 Skills come from the workbook's Skills column, mapped through `data/skill_aliases.json`, plus keywords found in each project's README or idea file (`pipeline/skill_rules.py`). To run it locally: `GITHUB_TOKEN=$(gh auth token) python3 pipeline/build_json.py`, and `python3 -m unittest discover -s tests` for the tests.
 
+## Project Finder (the chatbot)
+
+`chatbot/widget.js` is a guided chat for the website: it asks a visitor's skills, what they want to learn and which area they care about, then recommends projects from `data/projects.json`. Public Invention's rules decide the order, in code (`pipeline/ranking.py`, mirrored in `chatbot/recommend.js`):
+
+- never recommend Moonrat, Oxygen Concentrator, PolyVent, ADaM or Librecorder, Preliminary projects, or projects not seeking volunteers;
+- Highly Active first, then Progress but Inactive, then Ready-to-start;
+- within a status, more matching skills first, then the sheet's Order priority.
+
+`tests/ranking_cases.json` is shared by the Python and JavaScript tests, so both rank the same way. Embedding it on pubinv.org: `docs/wordpress-embed.md`. The ChatGPT version reads the same JSON through an Action: `ChatBot_Instructions.md`. How the volunteer form works today: `docs/volunteer-form.md`.
+
 ## Resources
 
 The current [Public Invention Project Chatbot](https://chatgpt.com/g/g-67c0c512437c8191844d6afd21abfb21-public-invention-project-chatbot?model=gpt-4o) is live and can provide details about projects currently on the project spreadsheet included in this repository.

chatbot/gpt/projects-action.yaml

@@ -0,0 +1,38 @@
+openapi: 3.1.0
+info:
+  title: Public Invention projects
+  description: The live project list the Public Invention chatbot recommends from, regenerated weekly from the project workbook and GitHub.
+  version: 1.0.0
+servers:
+  - url: https://raw.githubusercontent.com/PubInv/Project-ChatBot/main
+paths:
+  /data/projects.json:
+    get:
+      operationId: getProjects
+      summary: Every Public Invention project with status, priority, skills and whether it may be recommended.
+      responses:
+        "200":
+          description: Array of projects.
+          content:
+            application/json:
+              schema:
+                type: array
+                items:
+                  type: object
+                  properties:
+                    project: {type: string}
+                    status: {type: string, description: "Highly Active, Progress but Inactive, Ready-to-start or Preliminary"}
+                    status_rank: {type: integer, description: "1 is shown first"}
+                    eligible: {type: boolean, description: "false means never recommend it"}
+                    priority: {type: number, description: "The sheet's Order column; 11 is highest"}
+                    synopsis: {type: string}
+                    summary: {type: string}
+                    link: {type: string, description: "GitHub repo or idea file"}
+                    page: {type: string, description: "The project's page on pubinv.org, when it has one; prefer it over link"}
+                    category: {type: string}
+                    joinability: {type: number, description: "1 easy to join, 3 you may have to start it yourself"}
+                    difficulty: {type: number, description: "1 easiest, 10 hardest"}
+                    skills: {type: array, items: {type: string}, description: "Skills from the project sheet"}
+                    readme_skills: {type: array, items: {type: string}, description: "Skills guessed from the README; weaker evidence"}
+                    seeking_volunteers: {type: string, description: "yes, maybe, no or unknown"}
+                    activity_status: {type: string, description: "GitHub activity: Active, Stale, Dormant or Unknown"}

chatbot/index.html

@@ -0,0 +1,14 @@
+<!doctype html>
+<html lang="en">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>Public Invention Project Finder</title>
+  <style>body { margin: 24px 16px; background: #f6f8fb; }</style>
+</head>
+<body>
+  <!-- Local preview: run `python3 -m http.server` from the repo root, then open /chatbot/. -->
+  <div id="pubinv-project-finder" data-src="../data/"></div>
+  <script type="module" src="./widget.js"></script>
+</body>
+</html>

chatbot/recommend.js

@@ -0,0 +1,60 @@
+// Public Invention's recommendation rules for the browser.
+// Mirrors pipeline/ranking.py; tests/ranking_cases.json keeps them in step.
+
+export const EXCLUDED_TITLES = ["Moonrat", "Oxygen Concentrator", "PolyVent", "ADaM", "Librecorder"];
+export const STATUS_ORDER = { "Highly Active": 1, "Progress but Inactive": 2, "Ready-to-start": 3 };
+const HIDDEN_STATUSES = new Set(["Preliminary"]);
+const MIN_README_MATCHES = 2;
+
+export function isEligible(project) {
+  const title = (project.project || "").toLowerCase();
+  if (EXCLUDED_TITLES.some((name) => title.includes(name.toLowerCase()))) return false;
+  if (HIDDEN_STATUSES.has(project.status || "")) return false;
+  return project.seeking_volunteers !== "no";
+}
+
+export function statusRank(status) {
+  return STATUS_ORDER[status] ?? Object.keys(STATUS_ORDER).length + 1;
+}
+
+const lower = (list) => new Set((list || []).map((s) => s.toLowerCase()));
+
+/** Rank eligible projects: status first, then skill matches, then the sheet's Order priority. */
+export function recommend(projects, { have = [], want = [], category = "", limit = 5 } = {}) {
+  const haveSet = lower(have);
+  const wantSet = lower(want);
+  const ranked = [];
+  for (const project of projects) {
+    if (!(project.eligible ?? isEligible(project))) continue;
+    if (category && (project.category || "").toLowerCase() !== category.toLowerCase()) continue;
+    const sheet = project.skills || [];
+    const skills = sheet.length ? sheet : project.readme_skills || [];
+    const matchedHave = skills.filter((s) => haveSet.has(s.toLowerCase()));
+    const matchedWant = skills.filter((s) => wantSet.has(s.toLowerCase()));
+    const matches = matchedHave.length + matchedWant.length;
+    if ((haveSet.size || wantSet.size) && matches < (sheet.length ? 1 : MIN_README_MATCHES)) continue;
+    ranked.push({ project, matchedHave, matchedWant, key: [statusRank(project.status), -matches, -(project.priority || 0), project.project] });
+  }
+  ranked.sort((a, b) => {
+    for (let i = 0; i < a.key.length; i++) {
+      if (a.key[i] < b.key[i]) return -1;
+      if (a.key[i] > b.key[i]) return 1;
+    }
+    return 0;
+  });
+  return ranked.slice(0, limit);
+}
+
+/** Pick known skills out of free text like "I do Arduino and some 3D printing". */
+export function skillsFromText(text, skills, aliases = {}) {
+  const low = ` ${(text || "").toLowerCase()} `;
+  const found = new Set();
+  for (const skill of skills) {
+    if (low.includes(skill.toLowerCase())) found.add(skill);
+  }
+  for (const [phrase, mapped] of Object.entries(aliases)) {
+    if (phrase.startsWith("_") || phrase.length < 4) continue;
+    if (low.includes(phrase)) mapped.filter((s) => skills.includes(s)).forEach((s) => found.add(s));
+  }
+  return [...found].sort();
+}

chatbot/recommend.test.mjs

@@ -0,0 +1,19 @@
+import assert from "node:assert/strict";
+import { readFileSync } from "node:fs";
+import { test } from "node:test";
+import { recommend, skillsFromText } from "./recommend.js";
+
+const cases = JSON.parse(readFileSync(new URL("../tests/ranking_cases.json", import.meta.url)));
+
+for (const c of cases.cases) {
+  test(c.name, () => {
+    const got = recommend(cases.projects, { have: c.have, want: c.want, category: c.category });
+    assert.deepEqual(got.map((r) => r.project.project), c.expected);
+  });
+}
+
+test("free text maps to known skills through aliases", () => {
+  const found = skillsFromText("I do arduino and javascript programming", ["Arduino", "JavaScript", "Programming"],
+    { "javascript programming": ["JavaScript"] });
+  assert.deepEqual(found, ["Arduino", "JavaScript", "Programming"]);
+});