$npx -y skills add GPTomics/bioSkills --skill pileup-generationGenerate pileup data for variant calling using samtools mpileup and pysam. Use when preparing data for variant calling, analyzing per-position read data, or calculating allele frequencies.
| 1 | ## Version Compatibility |
| 2 | |
| 3 | Reference examples tested with: bcftools 1.19+, pysam 0.22+, samtools 1.19+ |
| 4 | |
| 5 | Before using code patterns, verify installed versions match. If versions differ: |
| 6 | - Python: `pip show <package>` then `help(module.function)` to check signatures |
| 7 | - CLI: `<tool> --version` then `<tool> --help` to confirm flags |
| 8 | |
| 9 | If code throws ImportError, AttributeError, or TypeError, introspect the installed |
| 10 | package and adapt the example to match the actual API rather than retrying. |
| 11 | |
| 12 | # Pileup Generation |
| 13 | |
| 14 | Generate pileup data for variant calling and position-level analysis. |
| 15 | |
| 16 | **"Generate pileup from BAM"** -> Produce per-position read summaries showing depth, bases, and qualities. |
| 17 | - CLI: `samtools mpileup -f ref.fa input.bam` |
| 18 | - Python: `bam.pileup(chrom, start, end)` (pysam) |
| 19 | |
| 20 | **"Count alleles at a position"** -> Extract per-base read support at a specific genomic coordinate. |
| 21 | - Python: iterate `pileup_column.pileups` and count bases (pysam) |
| 22 | |
| 23 | ## What is Pileup? |
| 24 | |
| 25 | Pileup shows all reads covering each position in the reference, used for: |
| 26 | - Variant calling (with bcftools) |
| 27 | - Coverage analysis |
| 28 | - Allele frequency calculation |
| 29 | - SNP/indel detection |
| 30 | |
| 31 | ## samtools mpileup vs bcftools mpileup (Deprecation) |
| 32 | |
| 33 | `samtools mpileup -g/-u` (BCF output for variant calling) was **deprecated in samtools 1.9 and removed in 1.15** (the option no longer exists; the usage/manpage directs users to `bcftools mpileup`) -- the genotype-likelihood code now lives in `bcftools mpileup`, which keeps mpileup logic versioned alongside `bcftools call` and avoids version-skew bugs. |
| 34 | |
| 35 | | Use case | Recommended tool | |
| 36 | |----------|------------------| |
| 37 | | Quick allele counts at known sites | `samtools mpileup` or pysam pileup | |
| 38 | | Germline variant calling (small genomes, simple cohorts) | `bcftools mpileup` -> `bcftools call` | |
| 39 | | Germline WGS / WES production | DeepVariant or HaplotypeCaller (not mpileup) | |
| 40 | | Somatic SNV/indel | Mutect2 / VarDict / VarScan2 (direct from BAM) | |
| 41 | | Long-read small variants | clair3 / DeepVariant ONT (direct from BAM) | |
| 42 | | Long-read SV | Sniffles / cuteSV (direct from BAM) | |
| 43 | | Ultra-low-frequency (ctDNA / MRD) | fgbio consensus -> `bcftools call` or hot-spot Mutect2 | |
| 44 | | Per-position allele counts (custom) | pysam pileup | |
| 45 | |
| 46 | `samtools mpileup` (without `-g`) is still the standard tool for human-readable per-position read summaries. |
| 47 | |
| 48 | ### Basic Pileup |
| 49 | ```bash |
| 50 | samtools mpileup -f reference.fa input.bam > pileup.txt |
| 51 | ``` |
| 52 | |
| 53 | ### Pileup Specific Region |
| 54 | ```bash |
| 55 | samtools mpileup -f reference.fa -r chr1:1000000-2000000 input.bam |
| 56 | ``` |
| 57 | |
| 58 | ### Regions from BED |
| 59 | ```bash |
| 60 | samtools mpileup -f reference.fa -l targets.bed input.bam |
| 61 | ``` |
| 62 | |
| 63 | ### Multiple BAM Files |
| 64 | ```bash |
| 65 | samtools mpileup -f reference.fa sample1.bam sample2.bam sample3.bam > pileup.txt |
| 66 | ``` |
| 67 | |
| 68 | ## Output Format |
| 69 | |
| 70 | Text pileup format (6 columns per sample): |
| 71 | ``` |
| 72 | chr1 1000 A 15 ............... FFFFFFFFFFF |
| 73 | chr1 1001 T 12 ............ FFFFFFFFFFFF |
| 74 | ``` |
| 75 | |
| 76 | | Column | Description | |
| 77 | |--------|-------------| |
| 78 | | 1 | Chromosome | |
| 79 | | 2 | Position (1-based) | |
| 80 | | 3 | Reference base | |
| 81 | | 4 | Read depth | |
| 82 | | 5 | Read bases | |
| 83 | | 6 | Base qualities | |
| 84 | |
| 85 | ### Read Bases Encoding |
| 86 | |
| 87 | | Symbol | Meaning | |
| 88 | |--------|---------| |
| 89 | | `.` | Match on forward strand | |
| 90 | | `,` | Match on reverse strand | |
| 91 | | `ACGT` | Mismatch (uppercase = forward) | |
| 92 | | `acgt` | Mismatch (lowercase = reverse) | |
| 93 | | `^Q` | Start of read (Q = MAPQ as ASCII) | |
| 94 | | `$` | End of read | |
| 95 | | `+NNN` | Insertion of N bases | |
| 96 | | `-NNN` | Deletion of N bases | |
| 97 | | `*` | Deleted base | |
| 98 | | `>` / `<` | Reference skip (intron) | |
| 99 | |
| 100 | ## Quality Filtering Options |
| 101 | |
| 102 | ### Minimum Mapping Quality |
| 103 | ```bash |
| 104 | samtools mpileup -f reference.fa -q 20 input.bam |
| 105 | ``` |
| 106 | |
| 107 | ### Minimum Base Quality |
| 108 | ```bash |
| 109 | samtools mpileup -f reference.fa -Q 20 input.bam |
| 110 | ``` |
| 111 | |
| 112 | ### Combined Quality Filters |
| 113 | ```bash |
| 114 | samtools mpileup -f reference.fa -q 20 -Q 20 input.bam |
| 115 | ``` |
| 116 | |
| 117 | ### Maximum Depth (Critical Trap) |
| 118 | ```bash |
| 119 | # samtools mpileup default -d 8000 silently truncates targeted / mt-DNA / amplicon / UMI-deduped data |
| 120 | # bcftools mpileup default -d 250 is far lower; both must be set explicitly when piping |
| 121 | samtools mpileup -f reference.fa -d 0 input.bam # no cap |
| 122 | samtools mpileup -f reference.fa -d 1000000 input.bam # explicit high cap |
| 123 | |
| 124 | # WRONG -- samtools 8000 cap, then bcftools 250 cap re-applied |
| 125 | samtools mpileup -f ref.fa in.bam | bcftools call -mv |
| 126 | |
| 127 | # RIGHT -- single tool, explicit -d |
| 128 | bcftools mpileup -d 1000000 -f ref.fa in.bam | bcftools call -mv |
| 129 | ``` |
| 130 | |
| 131 | ## BAQ: Base Alignment Quality (Critical Default) |
| 132 | |
| 133 | When `-f ref.fa` is passed, BAQ is enabled by default. BAQ Phred-scales the probability that a base is misaligned (HMM realignment over a small window) and reduces base quality near indels. Tradeoffs: ~30% slower; suppresses FP SNVs |