lamindb

Name: lamindb
Author: LeonChaoX

698stars61forks

How to install lamindb?

leonchaox/qinyan-academic-skills/lamindb

$npx -y skills add LeonChaoX/qinyan-academic-skills --skill lamindb

Installs into the current project.

Summary

This skill should be used when working with LaminDB, an open-source data framework for biology that makes data queryable, traceable, reproducible, and FAIR. Use when managing biological datasets (scRNA-seq, spatial, flow cytometry, etc.), tracking computational workflows, curatin

Files · 1

View on GitHub

SKILL.md

1	# LaminDB
2
3	## Overview
4
5	LaminDB is an open-source data framework for biology designed to make data queryable, traceable, reproducible, and FAIR (Findable, Accessible, Interoperable, Reusable). It provides a unified platform that combines lakehouse architecture, lineage tracking, feature stores, biological ontologies, LIMS (Laboratory Information Management System), and ELN (Electronic Lab Notebook) capabilities through a single Python API.
6
7	Core Value Proposition:
8	- Queryability: Search and filter datasets by metadata, features, and ontology terms
9	- Traceability: Automatic lineage tracking from raw data through analysis to results
10	- Reproducibility: Version control for data, code, and environment
11	- FAIR Compliance: Standardized annotations using biological ontologies
12
13	## When to Use This Skill
14
15	Use this skill when:
16
17	- Managing biological datasets: scRNA-seq, bulk RNA-seq, spatial transcriptomics, flow cytometry, multi-modal data, EHR data
18	- Tracking computational workflows: Notebooks, scripts, pipeline execution (Nextflow, Snakemake, Redun)
19	- Curating and validating data: Schema validation, standardization, ontology-based annotation
20	- Working with biological ontologies: Genes, proteins, cell types, tissues, diseases, pathways (via Bionty)
21	- Building data lakehouses: Unified query interface across multiple datasets
22	- Ensuring reproducibility: Automatic versioning, lineage tracking, environment capture
23	- Integrating ML pipelines: Connecting with Weights & Biases, MLflow, HuggingFace, scVI-tools
24	- Deploying data infrastructure: Setting up local or cloud-based data management systems
25	- Collaborating on datasets: Sharing curated, annotated data with standardized metadata
26
27	## Core Capabilities
28
29	LaminDB provides six interconnected capability areas, each documented in detail in the references folder.
30
31	### 1. Core Concepts and Data Lineage
32
33	Core entities:
34	- Artifacts: Versioned datasets (DataFrame, AnnData, Parquet, Zarr, etc.)
35	- Records: Experimental entities (samples, perturbations, instruments)
36	- Runs & Transforms: Computational lineage tracking (what code produced what data)
37	- Features: Typed metadata fields for annotation and querying
38
39	Key workflows:
40	- Create and version artifacts from files or Python objects
41	- Track notebook/script execution with `ln.track()` and `ln.finish()`
42	- Annotate artifacts with typed features
43	- Visualize data lineage graphs with `artifact.view_lineage()`
44	- Query by provenance (find all outputs from specific code/inputs)
45
46	Reference: `references/core-concepts.md` - Read this for detailed information on artifacts, records, runs, transforms, features, versioning, and lineage tracking.
47
48	### 2. Data Management and Querying
49
50	Query capabilities:
51	- Registry exploration and lookup with auto-complete
52	- Single record retrieval with `get()`, `one()`, `one_or_none()`
53	- Filtering with comparison operators (`__gt`, `__lte`, `__contains`, `__startswith`)
54	- Feature-based queries (query by annotated metadata)
55	- Cross-registry traversal with double-underscore syntax
56	- Full-text search across registries
57	- Advanced logical queries with Q objects (AND, OR, NOT)
58	- Streaming large datasets without loading into memory
59
60	Key workflows:
61	- Browse artifacts with filters and ordering
62	- Query by features, creation date, creator, size, etc.
63	- Stream large files in chunks or with array slicing
64	- Organize data with hierarchical keys
65	- Group artifacts into collections
66
67	Reference: `references/data-management.md` - Read this for comprehensive query patterns, filtering examples, streaming strategies, and data organization best practices.
68
69	### 3. Annotation and Validation
70
71	Curation process:
72	1. Validation: Confirm datasets match desired schemas
73	2. Standardization: Fix typos, map synonyms to canonical terms
74	3. Annotation: Link datasets to metadata entities for queryability
75
76	Schema types:
77	- Flexible schemas: Validate only known columns, allow additional metadata
78	- Minimal required schemas: Specify essential columns, permit extras
79	- Strict schemas: Complete control over structure and values
80
81	Supported data types:
82	- DataFrames (Parquet, CSV)
83	- AnnData (single-

People who liked this also viewed

6 picks

Type

lamindb

698stars61forks

View on GitHub

How to install lamindb?

leonchaox/qinyan-academic-skills/lamindb

$npx -y skills add LeonChaoX/qinyan-academic-skills --skill lamindb

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# LaminDB
2
3	## Overview
4
5	LaminDB is an open-source data framework for biology designed to make data queryable, traceable, reproducible, and FAIR (Findable, Accessible, Interoperable, Reusable). It provides a unified platform that combines lakehouse architecture, lineage tracking, feature stores, biological ontologies, LIMS (Laboratory Information Management System), and ELN (Electronic Lab Notebook) capabilities through a single Python API.
6
7	Core Value Proposition:
8	- Queryability: Search and filter datasets by metadata, features, and ontology terms
9	- Traceability: Automatic lineage tracking from raw data through analysis to results
10	- Reproducibility: Version control for data, code, and environment
11	- FAIR Compliance: Standardized annotations using biological ontologies
12
13	## When to Use This Skill
14
15	Use this skill when:
16
17	- Managing biological datasets: scRNA-seq, bulk RNA-seq, spatial transcriptomics, flow cytometry, multi-modal data, EHR data
18	- Tracking computational workflows: Notebooks, scripts, pipeline execution (Nextflow, Snakemake, Redun)
19	- Curating and validating data: Schema validation, standardization, ontology-based annotation
20	- Working with biological ontologies: Genes, proteins, cell types, tissues, diseases, pathways (via Bionty)
21	- Building data lakehouses: Unified query interface across multiple datasets
22	- Ensuring reproducibility: Automatic versioning, lineage tracking, environment capture
23	- Integrating ML pipelines: Connecting with Weights & Biases, MLflow, HuggingFace, scVI-tools
24	- Deploying data infrastructure: Setting up local or cloud-based data management systems
25	- Collaborating on datasets: Sharing curated, annotated data with standardized metadata
26
27	## Core Capabilities
28
29	LaminDB provides six interconnected capability areas, each documented in detail in the references folder.
30
31	### 1. Core Concepts and Data Lineage
32
33	Core entities:
34	- Artifacts: Versioned datasets (DataFrame, AnnData, Parquet, Zarr, etc.)
35	- Records: Experimental entities (samples, perturbations, instruments)
36	- Runs & Transforms: Computational lineage tracking (what code produced what data)
37	- Features: Typed metadata fields for annotation and querying
38
39	Key workflows:
40	- Create and version artifacts from files or Python objects
41	- Track notebook/script execution with `ln.track()` and `ln.finish()`
42	- Annotate artifacts with typed features
43	- Visualize data lineage graphs with `artifact.view_lineage()`
44	- Query by provenance (find all outputs from specific code/inputs)
45
46	Reference: `references/core-concepts.md` - Read this for detailed information on artifacts, records, runs, transforms, features, versioning, and lineage tracking.
47
48	### 2. Data Management and Querying
49
50	Query capabilities:
51	- Registry exploration and lookup with auto-complete
52	- Single record retrieval with `get()`, `one()`, `one_or_none()`
53	- Filtering with comparison operators (`__gt`, `__lte`, `__contains`, `__startswith`)
54	- Feature-based queries (query by annotated metadata)
55	- Cross-registry traversal with double-underscore syntax
56	- Full-text search across registries
57	- Advanced logical queries with Q objects (AND, OR, NOT)
58	- Streaming large datasets without loading into memory
59
60	Key workflows:
61	- Browse artifacts with filters and ordering
62	- Query by features, creation date, creator, size, etc.
63	- Stream large files in chunks or with array slicing
64	- Organize data with hierarchical keys
65	- Group artifacts into collections
66
67	Reference: `references/data-management.md` - Read this for comprehensive query patterns, filtering examples, streaming strategies, and data organization best practices.
68
69	### 3. Annotation and Validation
70
71	Curation process:
72	1. Validation: Confirm datasets match desired schemas
73	2. Standardization: Fix typos, map synonyms to canonical terms
74	3. Annotation: Link datasets to metadata entities for queryability
75
76	Schema types:
77	- Flexible schemas: Validate only known columns, allow additional metadata
78	- Minimal required schemas: Specify essential columns, permit extras
79	- Strict schemas: Complete control over structure and values
80
81	Supported data types:
82	- DataFrames (Parquet, CSV)
83	- AnnData (single-

People who liked this also viewed

6 picks

Type