topic-modeling

Name: topic-modeling
Author: scdenney

34stars0forks

How to install topic-modeling?

scdenney/open-science-skills/topic-modeling

$npx -y skills add scdenney/open-science-skills --skill topic-modeling

Installs into the current project.

Summary

Design and diagnose structural topic models. Use for STM specification, topic-count selection, coherence-exclusivity tradeoffs, validation, and interpretation.

Files · 1

View on GitHub

SKILL.md

1	# Topic Modeling for Survey and Experimental Text Data
2
3	## Instructions
4
5	### 1. Model Selection
6
7	- Default to Structural Topic Models (STM) when analyzing text from surveys or experiments. STM incorporates document-level metadata — treatment conditions, respondent demographics, country — directly into estimation, allowing prevalence and content to vary with covariates (Roberts et al. 2014).
8	- Use standard LDA only when no document-level covariates are needed and the corpus is large enough for unsupervised discovery (Blei, Ng & Jordan 2003).
9	- Consider BERTopic when working with short texts where word co-occurrence is sparse, or when multilingual embedding-based similarity is required (Grootendorst 2022). BERTopic clusters document embeddings via HDBSCAN, which yields hard single-topic-per-document assignments (unlike STM/LDA mixed membership), c-TF-IDF topic words that can be unstable on small corpora, and no native covariate framework. Use it when embedding-based semantic similarity matters more than mixed-membership structure or covariate estimation.
10	- Recognize the distinction between topic categorization and attitude inference: topic models reveal what respondents discussed, not what they believe. If the goal is inferring latent attitudes rather than categorizing surface content, complement topic modeling with an embeddings-based scaling of contextually common words or a separate supervised classifier (Hobbs & Green 2025; see also the companion `text-classification` skill).
11
12	### 2. Preprocessing
13
14	- Make preprocessing decisions explicit and justify each choice. Preprocessing is not neutral — stemming, stopword removal, and term-frequency thresholds all affect which topics emerge (Denny & Spirling 2018).
15	- Lowercase all text. Remove punctuation and numbers unless they carry substantive meaning in the domain.
16	- Remove stopwords using a standard list, but inspect the list for domain-relevant terms that should be retained (e.g., "foreign" in immigration research).
17	- Apply stemming only after checking that it does not collapse substantively distinct terms. Compare results with and without stemming as a robustness check (Denny & Spirling 2018).
18	- Set a lower-frequency threshold to prune very rare terms, and express it as a fraction of documents rather than an absolute count so it scales with corpus size. For small open-ended corpora a fixed count of 2–5 documents is often appropriate; for larger corpora, pruning terms that appear in fewer than roughly 0.5%–1% of documents is a common starting point (Grimmer & Stewart 2013). In `stm`, use `plotRemoved()` to visualize how many documents and terms are dropped across candidate thresholds before committing to one, then pass it to `prepDocuments()`. Report the threshold chosen and the counts of terms and documents retained.
19	- For translated text, preprocess after translation. Note that translation artifacts (e.g., inconsistent phrasing across translators) may affect topic coherence — document any translation pipeline.
20
21	### 3. Model Specification
22
23	- Specify the prevalence formula to include theoretically relevant covariates. For survey experiments, include treatment conditions; for cross-national data, include country. Example: `prevalence = ~ treatment + country` (Roberts et al. 2014). For experimental applications, pre-register the prevalence-covariate hypotheses (see the companion `pre-registration-writing` skill) so that `estimateEffect()` outputs are confirmatory rather than exploratory by default (Nosek et al. 2018).
24	- Optionally specify a content formula when you expect the words associated with a topic (not just its prevalence) to vary by covariate — e.g., different framings of the same topic across countries or treatment arms. Content covariates parameterize word distributions via SAGE-style deviations from a baseline, which complicates direct comparison of β across groups; inspect group-specific vocabulary with `sageLabels()` and interpret with care (Roberts, Stewart & Tingley 2014; 2019).
25	- Use spectral initialization (`init.type = "Spectral"`) for reproducibility. Spectral initialization is deterministic on a given machine/BLAS configuration, unlike random initialization which requires multiple runs; cross-machine numerical precision can still produce minor differences, so record the hardware/OS when sharing replication materials (Roberts, Stewart & Tingley 2019).
26	- Set a random seed and record it regardless of initialization method. Also record the `stm` package version and R session info for DA-RT-compliant replication.
27
28	### 4. Selecting the Number of Topics
29
30	- Do not rely on a single metric. Evaluate candidate models across a range of K values (e.g., K = 5 to 30) using multiple diagnostics: semantic coherence, exclusivity, held-out likelihood, a

People who liked this also viewed

6 picks

Type

find-skillsHelps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skSkillsJul 20262.6M27k
grill-meA relentless interview to sharpen a plan or design.SkillsJul 2026633k182k
vercel-react-best-practicesVercel's official collection of agent skillsSkillsJul 2026573k29k
improve-codebase-architectureScan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.SkillsJul 2026522k182k
setup-matt-pocock-skillsConfigure this repo for the engineering skills — set up its issue tracker, triage label vocabulary, and domain doc layout. Run once before first use of the other engineering skills.SkillsJul 2026454k182k
triageMove issues and external PRs through a state machine of triage roles — categorise, verify, grill if needed, and write agent-ready briefs.SkillsJul 2026424k182k

topic-modeling

34stars0forks

View on GitHub

How to install topic-modeling?

scdenney/open-science-skills/topic-modeling

$npx -y skills add scdenney/open-science-skills --skill topic-modeling

Installs into the current project.

Summary

Design and diagnose structural topic models. Use for STM specification, topic-count selection, coherence-exclusivity tradeoffs, validation, and interpretation.

Files · 1

View on GitHub

SKILL.md

1	# Topic Modeling for Survey and Experimental Text Data
2
3	## Instructions
4
5	### 1. Model Selection
6
7	- Default to Structural Topic Models (STM) when analyzing text from surveys or experiments. STM incorporates document-level metadata — treatment conditions, respondent demographics, country — directly into estimation, allowing prevalence and content to vary with covariates (Roberts et al. 2014).
8	- Use standard LDA only when no document-level covariates are needed and the corpus is large enough for unsupervised discovery (Blei, Ng & Jordan 2003).
9	- Consider BERTopic when working with short texts where word co-occurrence is sparse, or when multilingual embedding-based similarity is required (Grootendorst 2022). BERTopic clusters document embeddings via HDBSCAN, which yields hard single-topic-per-document assignments (unlike STM/LDA mixed membership), c-TF-IDF topic words that can be unstable on small corpora, and no native covariate framework. Use it when embedding-based semantic similarity matters more than mixed-membership structure or covariate estimation.
10	- Recognize the distinction between topic categorization and attitude inference: topic models reveal what respondents discussed, not what they believe. If the goal is inferring latent attitudes rather than categorizing surface content, complement topic modeling with an embeddings-based scaling of contextually common words or a separate supervised classifier (Hobbs & Green 2025; see also the companion `text-classification` skill).
11
12	### 2. Preprocessing
13
14	- Make preprocessing decisions explicit and justify each choice. Preprocessing is not neutral — stemming, stopword removal, and term-frequency thresholds all affect which topics emerge (Denny & Spirling 2018).
15	- Lowercase all text. Remove punctuation and numbers unless they carry substantive meaning in the domain.
16	- Remove stopwords using a standard list, but inspect the list for domain-relevant terms that should be retained (e.g., "foreign" in immigration research).
17	- Apply stemming only after checking that it does not collapse substantively distinct terms. Compare results with and without stemming as a robustness check (Denny & Spirling 2018).
18	- Set a lower-frequency threshold to prune very rare terms, and express it as a fraction of documents rather than an absolute count so it scales with corpus size. For small open-ended corpora a fixed count of 2–5 documents is often appropriate; for larger corpora, pruning terms that appear in fewer than roughly 0.5%–1% of documents is a common starting point (Grimmer & Stewart 2013). In `stm`, use `plotRemoved()` to visualize how many documents and terms are dropped across candidate thresholds before committing to one, then pass it to `prepDocuments()`. Report the threshold chosen and the counts of terms and documents retained.
19	- For translated text, preprocess after translation. Note that translation artifacts (e.g., inconsistent phrasing across translators) may affect topic coherence — document any translation pipeline.
20
21	### 3. Model Specification
22
23	- Specify the prevalence formula to include theoretically relevant covariates. For survey experiments, include treatment conditions; for cross-national data, include country. Example: `prevalence = ~ treatment + country` (Roberts et al. 2014). For experimental applications, pre-register the prevalence-covariate hypotheses (see the companion `pre-registration-writing` skill) so that `estimateEffect()` outputs are confirmatory rather than exploratory by default (Nosek et al. 2018).
24	- Optionally specify a content formula when you expect the words associated with a topic (not just its prevalence) to vary by covariate — e.g., different framings of the same topic across countries or treatment arms. Content covariates parameterize word distributions via SAGE-style deviations from a baseline, which complicates direct comparison of β across groups; inspect group-specific vocabulary with `sageLabels()` and interpret with care (Roberts, Stewart & Tingley 2014; 2019).
25	- Use spectral initialization (`init.type = "Spectral"`) for reproducibility. Spectral initialization is deterministic on a given machine/BLAS configuration, unlike random initialization which requires multiple runs; cross-machine numerical precision can still produce minor differences, so record the hardware/OS when sharing replication materials (Roberts, Stewart & Tingley 2019).
26	- Set a random seed and record it regardless of initialization method. Also record the `stm` package version and R session info for DA-RT-compliant replication.
27
28	### 4. Selecting the Number of Topics
29
30	- Do not rely on a single metric. Evaluate candidate models across a range of K values (e.g., K = 5 to 30) using multiple diagnostics: semantic coherence, exclusivity, held-out likelihood, a

People who liked this also viewed

6 picks

Type