Sample secret shopper report

Top 5 Shippable Fixes (Next 90 Days)

1. Create a “First Contribution” hub page.
   Content: setup checklist, label definitions, 5–10 curated starter issues.
   Where: /docs/contributing/getting-started.md and linked in README.
   Effort: ~1–2 days.
   Impact: reduces onboarding time and repeated Q&A.
2. Update CONTRIBUTING and label definitions to match current practice.
   Actions: review CONTRIBUTING line-by-line, remove outdated steps, link label wiki in issue templates.
   Effort: ~0.5–1 day.
   Impact: fewer “but the docs say…” disagreements, smoother PRs.
3. Introduce a “New Contributors” recognition ritual each release.
   Actions: add a “Thanks to new contributors” section naming first-timers.
   Effort: 30 minutes per release.
   Impact: higher return rate from first-time contributors.
4. Add a one-line setup checklist to README.
   Actions: include Node version, npm, global tool, and “run tests” command as a bullet list at top.
   Effort: < 1 hour.
   Impact: quicker success for drive-by contributors.
5. Schedule a governance / CoC review every 6–12 months.
   Actions: add a calendar reminder and a “last reviewed” date to governance docs.
   Effort: 1–2 hours twice a year.
   Impact: keeps safety and stewardship ahead of growth.

1. ACCESS (Entry and onboarding)

Observations

• README is friendly and short, but key setup steps live only in a separate docs site that is not clearly linked.
• Install section assumes familiarity with Node and npm, without a simple one line checklist for newcomers.
• There is a "good first issue" label, but only two issues are marked and both are more than three months old.

Actions taken

• Followed setup instructions exactly as written on README, then followed the first linked docs page.
• Setup time from zero to running test suite, 35 minutes.
• Hit a missing step, needed to install an additional global tool that was only mentioned in a closed issue.

Notes

Once the missing global dependency was installed, everything worked smoothly. The biggest friction for a first time contributor is the scattered information. Most of the right steps exist, but a newcomer has to jump between README, docs site, and old issues to piece them together.

Recommended actions

• Add a single “First contribution” doc that links from README, consolidates setup, labels, and starter issues.
• Add a “Quick start checklist” to the top of README with explicit global dependency.
• Refresh “good first issue” labels quarterly and keep at least 3–5 current.

2. INTERACTION (Human experience)

Observations

• Issue threads show a consistent pattern of maintainers thanking people for reports and ideas.
• Tone is relaxed and respectful, with some humor, but no sarcasm at the expense of newcomers.
• Inclusivity signals are present, including explicit pronouns in maintainer profiles and a visible Code of Conduct link in the issue template.

Actions taken

• Filed an issue titled "Docs, missing global dependency during setup" that described the error in two paragraphs.
• Response time, two hours, from a maintainer in a European time zone.
• Reply acknowledged the gap, suggested a fix, and added a heart reaction on the issue.

Notes

The interaction experience is a strength. Even when maintainers are busy, they keep responses human and specific. The main risk is that this level of care may not scale if issue volume grows without more maintainers joining.

Recommended actions

• Document a soft target for first response to new contributors (for example, “aim for < 48 hours”).
• Create a canned reply snippet for setup issues pointing to the new “First contribution” hub.

3. PROCESS (Workflow and predictability)

Observations

• Issues have labels, but definitions of labels live only in a wiki page that is not linked from templates.
• There is a CONTRIBUTING file, but it has not been updated in over a year, and some details disagree with current review practice.
• PR reviews appear thoughtful but happen in bursts, suggesting review happens on a weekly block rather than daily.

Actions taken

• Submitted a small docs PR that adds the missing global dependency to the setup instructions and adds a short troubleshooting note.
• PR reference, internal for this report, not included here.
• Time to first review, 3 days, with a single maintainer leaving feedback.
• Feedback was clear and kind, requested a smaller change to wording and a link to the official Node installer.

Notes

The process feels like a real project that is run by humans with limited time, not a chaotic one. The main friction for a newcomer is that some written rules and real practice have drifted apart. A short review and refresh of CONTRIBUTING and label definitions would reduce confusion and help others help maintainers.

Recommended actions

• Update CONTRIBUTING.md to align with current review cadence and expectations.
• Move label definitions from hidden wiki to a linked “Labels explained” doc and reference it in issue templates.

4. RECOGNITION (Acknowledgment and support)

Observations

• Maintainers often say "thank you" in comments, but there is no structured way of celebrating new contributors.
• Release notes mention features, but rarely name individual contributors except core maintainers.
• There is no visible "hall of fame" or contributor highlights page on the docs site.

Actions taken

• The docs PR received a thank you comment and a thumbs up reaction after merge.
• No follow up mention in release notes or social channels for this type of contribution.

Notes

You can feel that maintainers appreciate help, but they do not yet turn that appreciation into visible signals. A few low effort rituals, for example naming first time contributors in release notes or a monthly "new contributors" thread, would make it easier for people to feel invited back.

Recommended actions

• Add a “New contributors” section to release notes naming people, not just features.
• Start a monthly “Thank you” issue or discussion thread that highlights small but impactful contributions.

5. SAFETY and GOVERNANCE (Stability and stewardship)

Observations

• Code of Conduct is linked from README and from the issue template, and hosted in a central docs folder.
• Last release was within 30 days, and commit activity is steady, not bursty.
• There is a "Maintainers" page listing three primary maintainers and their responsibilities.

Actions taken

• Scanned for explicit moderation examples, found one closed thread where a maintainer asked to move a heated debate to a different channel and cited the CoC.
• Looked for roadmap or long term vision, found a short "Project goals" page that is a year old but still mostly accurate.

Notes

Safety and governance feel intentional. The risk is mostly around keeping governance documents fresh. At current size, the project is stable. If growth accelerates, you will likely need clearer escalation paths and explicit expectations for new maintainers.

Recommended actions

• Add an “Escalation path” bullet list to the CoC or governance doc.
• Add a “last reviewed” date to governance pages so it’s clear they are alive.