First Pass AI Analysis - features in detail

AI Summary

The automated AI response summary is deliberately descriptive rather than interpretive. The AI is instructed to state exactly what the respondent selected and wrote, in 3 to 4 sentences, and it is explicitly constrained on the point most likely to go wrong: intensity.

Because the AI can see the question wording and options alongside the answers, the summary reads the way an analyst would describe the response, referring to what was asked and what position was taken, rather than echoing raw form data. Demographic details are kept out of the summary entirely as they are already captured in the response record.

Sentiment and rationale

Sentiment is a forced choice between exactly 4 values: Positive, Negative, Neutral or Mixed. The AI cannot return anything outside that list, because the structured form only accepts those specific options.

The assessment considers both written comments and selections, interpreted against the actual question options. A respondent who selects 'Strongly oppose' on the central question has expressed negative sentiment even if they wrote nothing at all. Equally, 'Neither agree nor disagree' is treated as a genuine, clear expression of neutrality rather than as an absence of signal. A response that supports one part of a proposal and opposes another comes out as Mixed.

Alongside the sentiment, the AI writes a 1 to 2 sentence rationale explaining why it made that call. This is written for the analyst: rather than trusting only the AI selected sentiment label, they can see the reasoning and judge for themselves whether they agree with it.

Confidence

Confidence answers one specific question: how clear was the signal? It deliberately does not measure how much the respondent wrote. The AI scores clarity from 0 to 100 against a defined scale: an unambiguous position such as a 'Strongly oppose' selection or a clear direct statement scores 90 to 100, hedged or mildly ambiguous responses sit in the middle, and responses that contradict themselves, for example selections that say one thing while the written comments say another, score low.

That score is then converted to the value your analysts see: High at 70 and above, Medium from 40 to 69, and Low below 40. The practical effect is that a short response with a clear position is rated High, while a long, rambling response with no discernible position is rated Low. Analysts can use Low confidence as a cue that a response needs a closer human read.

Keywords

Keywords are extractive, not generative. The AI is only allowed to return words and phrases that literally appear in the respondent’s written answers, and it draws them only from substantive written comments. Quantitative radio selections and checkbox choices, as well as single line textbox answers are excluded, because a keyword list built from the activity's own answer options would tell you nothing about the respondent.

The keyword list is capped at 10 and deduplicated on the way through: if a respondent writes about 'bike shelter', 'bike shelters' and 'shelters', only one form is kept, preferring whichever is most specific or most used. If a response contains no substantive written text, the keywords field simply records that none were identified.

Tags

Tags are the feature clients ask about most, so here is the full mechanism.

The AI can only choose from your list.

The available tags are read from the tag list your team has set up on the saved question once added to the activity. They are built into the structured form as the only permitted values, so the AI is mechanically incapable of inventing a tag of its own.

Matching is by topic, not keyword.

A tag is applied when the respondent’s written comments substantively discuss that tag’s subject, even if the tag word itself never appears. For example, a comment that 'cars speed dangerously through the junction' is a match for a 'Safety' tag despite never using the word 'safety'. A passing mention or a speculative connection is not enough; the instruction is substantive discussion or nothing.

Every tag must come with proof.

For each tag it applies, the AI must supply a short quote, copied word for word from the respondent’s written comments, demonstrating that the topic really was discussed. First Pass then independently verifies that quote: the software checks that the quoted words genuinely appear in the respondent’s text. If the quote cannot be found, the tag is discarded. This check is done by plain software, not by the AI, so the AI cannot talk its way past it.

Quotes are evidence, not output.

Once a quote has done its job of validating a tag, it is discarded. It is never written into Citizen Space or stored anywhere.

No written text, no tags.

Tags are grounded exclusively in written comments. If a response consists only of quantitative selections, tagging is switched off entirely for that response rather than letting tags be inferred from radio buttons or checkboxes.

The result is that a tag on a response is always traceable to something the respondent actually wrote, while still catching the many respondents who discuss a topic without using your tag’s exact wording.

Content flags

Content flags exist to triage, not to censor.

There are exactly 2 flags, each with a narrow definition:

• Objectionable content means material that could not appropriately be published: threats, intimidation, abuse or harassment directed at a person or group, hate speech and the like.
• Strong language means actual swear words, and nothing else.

The instructions are explicit about what does not qualify. Strong disagreement with a policy is not objectionable. Forceful wording, words in capital letters, 'disgrace', 'appalling', blunt criticism: none of these trigger a flag. Criticism is the expected currency of surveys and consultations, and the test applied is whether the response could be published, not whether it is polite.

When a flag is applied, the AI must also write a short reason quoting the specific words that triggered it. That reason is written to the response so a reviewer can verify the flag in seconds and make their own judgement, rather than re-reading the whole response to find out why it was flagged. When nothing is flagged, the reason field is left blank.