Commit Confidential logo
Commit Confidential Signal. Feedback. Improvement.

Sample secret shopper report

This is a fictional example of what you receive from a Commit Confidential secret shopper audit. It is based on our v1.0 field instructions and the standard agent report template that agents use in the field.

Every mission runs through five pillars of contributor health access, interaction, process, recognition, and safety and governance with anchored scores from 1 to 5, concrete observations, and a final reflection.

Download blank report template Ask about this sample

Note, all names and details below are invented for illustration only. Real client reports stay private.

Agent ID: CC-017
Project: ExampleDocs JS
Date: 2025-11-15
Overall rating 3.4 / 5 Anchored average across five pillars

1. ACCESS (Entry and onboarding)

Score 3 / 5

Anchored meaning: mostly functional but with gaps, delays, or unclear elements.

Observations

  • README is friendly and short, but key setup steps live only in a separate docs site that is not clearly linked.
  • Install section assumes familiarity with Node and npm, without a simple one line checklist for newcomers.
  • There is a "good first issue" label, but only two issues are marked and both are more than three months old.

Actions taken

  • Followed setup instructions exactly as written on README, then followed the first linked docs page.
  • Setup time from zero to running test suite, 35 minutes.
  • Hit a missing step, needed to install an additional global tool that was only mentioned in a closed issue.

Notes

Once the missing global dependency was installed, everything worked smoothly. The biggest friction for a first time contributor is the scattered information. Most of the right steps exist, but a newcomer has to jump between README, docs site, and old issues to piece them together.

2. INTERACTION (Human experience)

Score 4 / 5

Anchored meaning: mostly warm and responsive, with clear good will toward newcomers.

Observations

  • Issue threads show a consistent pattern of maintainers thanking people for reports and ideas.
  • Tone is relaxed and respectful, with some humor, but no sarcasm at the expense of newcomers.
  • Inclusivity signals are present, including explicit pronouns in maintainer profiles and a visible Code of Conduct link in the issue template.

Actions taken

  • Filed an issue titled "Docs, missing global dependency during setup" that described the error in two paragraphs.
  • Response time, two hours, from a maintainer in a European time zone.
  • Reply acknowledged the gap, suggested a fix, and added a heart reaction on the issue.

Notes

The interaction experience is a strength. Even when maintainers are busy, they keep responses human and specific. The main risk is that this level of care may not scale if issue volume grows without more maintainers joining.

3. PROCESS (Workflow and predictability)

Score 3 / 5

Anchored meaning: partially structured but uneven and not fully predictable for newcomers.

Observations

  • Issues have labels, but definitions of labels live only in a wiki page that is not linked from templates.
  • There is a CONTRIBUTING file, but it has not been updated in over a year, and some details disagree with current review practice.
  • PR reviews appear thoughtful but happen in bursts, suggesting review happens on a weekly block rather than daily.

Actions taken

  • Submitted a small docs PR that adds the missing global dependency to the setup instructions and adds a short troubleshooting note.
  • PR reference, internal for this report, not included here.
  • Time to first review, 3 days, with a single maintainer leaving feedback.
  • Feedback was clear and kind, requested a smaller change to wording and a link to the official Node installer.

Notes

The process feels like a real project that is run by humans with limited time, not a chaotic one. The main friction for a newcomer is that some written rules and real practice have drifted apart. A short review and refresh of CONTRIBUTING and label definitions would reduce confusion and help others help maintainers.

4. RECOGNITION (Acknowledgment and support)

Score 3 / 5

Anchored meaning: basic acknowledgment is present, but recognition is not yet a deliberate practice.

Observations

  • Maintainers often say "thank you" in comments, but there is no structured way of celebrating new contributors.
  • Release notes mention features, but rarely name individual contributors except core maintainers.
  • There is no visible "hall of fame" or contributor highlights page on the docs site.

Actions taken

  • The docs PR received a thank you comment and a thumbs up reaction after merge.
  • No follow up mention in release notes or social channels for this type of contribution.

Notes

You can feel that maintainers appreciate help, but they do not yet turn that appreciation into visible signals. A few low effort rituals, for example naming first time contributors in release notes or a monthly "new contributors" thread, would make it easier for people to feel invited back.

5. SAFETY and GOVERNANCE (Stability and stewardship)

Score 4 / 5

Anchored meaning: governance feels real, project health is good, and expectations are mostly clear.

Observations

  • Code of Conduct is linked from README and from the issue template, and hosted in a central docs folder.
  • Last release was within 30 days, and commit activity is steady, not bursty.
  • There is a "Maintainers" page listing three primary maintainers and their responsibilities.

Actions taken

  • Scanned for explicit moderation examples, found one closed thread where a maintainer asked to move a heated debate to a different channel and cited the CoC.
  • Looked for roadmap or long term vision, found a short "Project goals" page that is a year old but still mostly accurate.

Notes

Safety and governance feel intentional. The risk is mostly around keeping governance documents fresh. At current size, the project is stable. If growth accelerates, you will likely need clearer escalation paths and explicit expectations for new maintainers.

Final reflection

1. What surprised you most?

The biggest surprise was how quickly maintainers responded, given that documentation and process feel slightly out of date. Human behavior is ahead of the written process, which is a better problem to have than the reverse.

2. What single improvement would have helped most?

A single, up to date "First contribution" page that links README, setup, label definitions, and a curated set of current starter issues would remove most of the guesswork. It would also give maintainers a stable link they can drop into replies.

3. Would you return to this community, and why?

Yes. As a newcomer I felt heard and supported. The remaining friction is mostly about scattered information, not behavior. With a small push on docs consolidation and contributor recognition, this could easily move from a good experience to an excellent one.

Pillar scores, at a glance

  • Access3 / 5
  • Interaction4 / 5
  • Process3 / 5
  • Recognition3 / 5
  • Safety and governance4 / 5

Overall rating

Average score, 3.4 out of 5. This project is healthy and welcoming, with most friction concentrated in scattered documentation and ad hoc recognition. A short, focused contributor experience project over one or two quarters would likely raise this to a 4 or higher.