pyarrow-format-conversion-starter

Name: pyarrow-format-conversion-starter
Author: ma-compbio-lab

36stars5forks

How to install pyarrow-format-conversion-starter?

ma-compbio-lab/skillfoundry/pyarrow-format-conversion-starter

$npx -y skills add ma-compbio-lab/SkillFoundry --skill pyarrow-format-conversion-starter

Installs into the current project.

Summary

Use this skill to convert a small tabular file into Parquet with PyArrow and inspect a deterministic round-trip summary.

Files · 1

SKILL.md

1	# PyArrow Format Conversion Starter
2
3	Use this skill to convert a small tabular file into Parquet with PyArrow and inspect a deterministic round-trip summary.
4
5	## What This Skill Does
6
7	- reads a tiny tabular input with typed columns
8	- builds an Arrow table in memory
9	- writes a local Parquet file
10	- reads the Parquet file back and verifies the round trip
11
12	## When To Use It
13
14	- when you need a starter for `format-conversion`
15	- when you want a small Arrow-to-Parquet example without a larger data stack
16	- when you need a deterministic schema summary for tests or demos
17
18	## Run
19
20	```bash
21	./slurm/envs/data-tools/bin/python skills/data-acquisition-and-dataset-handling/pyarrow-format-conversion-starter/scripts/run_pyarrow_format_conversion.py --input skills/data-acquisition-and-dataset-handling/pyarrow-format-conversion-starter/examples/toy_matrix.tsv --parquet-out scratch/pyarrow/toy_table.parquet --summary-out scratch/pyarrow/toy_table_summary.json
22	```
23
24	## Notes
25
26	- The example uses tab-separated input to keep the fixture readable in the repository.
27	- The summary records typed schema information so downstream checks do not need to parse Parquet directly.

6 picks

Type