apple-on-device-ai

Name: apple-on-device-ai
Author: dpearson2699

915stars45forks

How to install apple-on-device-ai?

dpearson2699/swift-ios-skills/apple-on-device-ai

$npx -y skills add dpearson2699/swift-ios-skills --skill apple-on-device-ai

Installs into the current project.

Summary

Build private, on-device AI features on iPhone, iPad, and Mac with Foundation Models, Core ML, MLX Swift, or llama.cpp. Use when choosing an Apple-local model runtime, building an Apple Intelligence chatbot or tool-calling feature, running an LLM on Apple Silicon, converting or c

Files · 1

View on GitHub

SKILL.md

1	# On-Device AI for Apple Platforms
2
3	Guide for selecting, deploying, and optimizing on-device ML models. Covers Apple
4	Foundation Models, Core ML, MLX Swift, and llama.cpp.
5
6	## Contents
7
8	- [Framework Selection Router](#framework-selection-router)
9	- [Apple Foundation Models Overview](#apple-foundation-models-overview)
10	- [Core ML Overview](#core-ml-overview)
11	- [MLX Swift Overview](#mlx-swift-overview)
12	- [Multi-Backend Architecture](#multi-backend-architecture)
13	- [Performance Best Practices](#performance-best-practices)
14	- [Common Mistakes](#common-mistakes)
15	- [Review Checklist](#review-checklist)
16	- [References](#references)
17
18	## Framework Selection Router
19
20	Use this decision tree to pick the right framework for your use case.
21
22	### Apple Foundation Models
23
24	When to use: Text generation, summarization, entity extraction, structured
25	output, and short dialog on iOS 26+ / macOS 26+ devices with Apple Intelligence
26	enabled. No app-managed API key, network round trip, or model hosting; still
27	handle system model asset readiness.
28
29	Best for:
30	- Generating text or structured data with `@Generable` types
31	- Summarization, classification, content tagging
32	- Tool-augmented generation with the `Tool` protocol
33	- Apps that need guaranteed on-device privacy
34
35	Not suited for: Complex math, code generation, factual accuracy tasks,
36	or apps targeting pre-iOS 26 devices.
37
38	### Core ML
39
40	When to use: Deploying custom trained models (vision, NLP, audio) across all
41	Apple platforms. Converting models from PyTorch, TensorFlow, or scikit-learn
42	with coremltools.
43
44	Best for:
45	- Image classification, object detection, segmentation
46	- Custom NLP classifiers, sentiment analysis models
47	- Audio/speech models via SoundAnalysis integration
48	- Any scenario needing Neural Engine optimization
49	- Models requiring quantization, palettization, or pruning
50
51	### MLX Swift
52
53	When to use: Running specific open-source LLMs (Llama, Mistral, Qwen, Gemma)
54	on Apple Silicon with maximum throughput. Research and prototyping.
55
56	Best for:
57	- Highest sustained token generation on Apple Silicon
58	- Running Hugging Face models from `mlx-community`
59	- Research requiring automatic differentiation
60	- Fine-tuning workflows on Mac
61
62	### llama.cpp
63
64	When to use: Cross-platform LLM inference using GGUF model format. Production
65	deployments needing broad device support.
66
67	Best for:
68	- GGUF quantized models (Q4_K_M, Q5_K_M, Q8_0)
69	- Cross-platform apps (iOS + Android + desktop)
70	- Maximum compatibility with open-source model ecosystem
71
72	### Quick Reference
73
74	\| Scenario \| Framework \|
75	\|---\|---\|
76	\| Text generation on Apple Intelligence devices (iOS 26+) \| Foundation Models \|
77	\| Structured output from on-device LLM \| Foundation Models (`@Generable`) \|
78	\| Image classification, object detection \| Core ML \|
79	\| Custom model from PyTorch/TensorFlow \| Core ML + coremltools \|
80	\| Running specific open-source LLMs \| MLX Swift or llama.cpp \|
81	\| Maximum throughput on Apple Silicon \| MLX Swift \|
82	\| Cross-platform LLM inference \| llama.cpp \|
83	\| OCR and text recognition \| Vision framework \|
84	\| Sentiment analysis, NER, tokenization \| Natural Language framework \|
85	\| Training custom classifiers on device \| Create ML \|
86
87	## Apple Foundation Models Overview
88
89	Use the system language model for short generation, summarization, tagging,
90	structured output, and tool-augmented tasks on Apple Intelligence devices. Gate
91	every entry point before creating a session:
92
93	```swift
94	import FoundationModels
95
96	switch SystemLanguageModel.default.availability {
97	case .available:
98	guard SystemLanguageModel.default.supportsLocale(Locale.current) else {
99	// Use locale fallback before generating
100	break
101	}
102	// Proceed with model usage
103	case .unavailable(.appleIntelligenceNotEnabled):
104	// Guide user to enable Apple Intelligence in Settings
105	case .unavailable(.modelNotReady):
106	// System model assets are not ready; show loading state
107	case .unavailable(.deviceNotEligible):
108	// Device cannot run Apple Intelligence; use fallback
109	case .unavailable(let reason):
110	// Unknown or future unavailable reason; use fallback and log reason
111	}
112	```
113
114	Then create a session and keep its shared context budget small:
115
116	```swift
117	let session = LanguageModelSession {
118	"You are a helpful cooking assistant."
119	}
120	session.prewarm()
121	let response = try await session.respond(to: "Suggest a quick pasta recipe")
122	```
123
124	Required guardrails:
125
126	- Sessions are stateful and accept one request at a time; serialize access and
127	check `isResponding` before issuing another response.

People who liked this also viewed

6 picks

Type

apple-on-device-ai

915stars45forks

View on GitHub

How to install apple-on-device-ai?

dpearson2699/swift-ios-skills/apple-on-device-ai

$npx -y skills add dpearson2699/swift-ios-skills --skill apple-on-device-ai

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# On-Device AI for Apple Platforms
2
3	Guide for selecting, deploying, and optimizing on-device ML models. Covers Apple
4	Foundation Models, Core ML, MLX Swift, and llama.cpp.
5
6	## Contents
7
8	- [Framework Selection Router](#framework-selection-router)
9	- [Apple Foundation Models Overview](#apple-foundation-models-overview)
10	- [Core ML Overview](#core-ml-overview)
11	- [MLX Swift Overview](#mlx-swift-overview)
12	- [Multi-Backend Architecture](#multi-backend-architecture)
13	- [Performance Best Practices](#performance-best-practices)
14	- [Common Mistakes](#common-mistakes)
15	- [Review Checklist](#review-checklist)
16	- [References](#references)
17
18	## Framework Selection Router
19
20	Use this decision tree to pick the right framework for your use case.
21
22	### Apple Foundation Models
23
24	When to use: Text generation, summarization, entity extraction, structured
25	output, and short dialog on iOS 26+ / macOS 26+ devices with Apple Intelligence
26	enabled. No app-managed API key, network round trip, or model hosting; still
27	handle system model asset readiness.
28
29	Best for:
30	- Generating text or structured data with `@Generable` types
31	- Summarization, classification, content tagging
32	- Tool-augmented generation with the `Tool` protocol
33	- Apps that need guaranteed on-device privacy
34
35	Not suited for: Complex math, code generation, factual accuracy tasks,
36	or apps targeting pre-iOS 26 devices.
37
38	### Core ML
39
40	When to use: Deploying custom trained models (vision, NLP, audio) across all
41	Apple platforms. Converting models from PyTorch, TensorFlow, or scikit-learn
42	with coremltools.
43
44	Best for:
45	- Image classification, object detection, segmentation
46	- Custom NLP classifiers, sentiment analysis models
47	- Audio/speech models via SoundAnalysis integration
48	- Any scenario needing Neural Engine optimization
49	- Models requiring quantization, palettization, or pruning
50
51	### MLX Swift
52
53	When to use: Running specific open-source LLMs (Llama, Mistral, Qwen, Gemma)
54	on Apple Silicon with maximum throughput. Research and prototyping.
55
56	Best for:
57	- Highest sustained token generation on Apple Silicon
58	- Running Hugging Face models from `mlx-community`
59	- Research requiring automatic differentiation
60	- Fine-tuning workflows on Mac
61
62	### llama.cpp
63
64	When to use: Cross-platform LLM inference using GGUF model format. Production
65	deployments needing broad device support.
66
67	Best for:
68	- GGUF quantized models (Q4_K_M, Q5_K_M, Q8_0)
69	- Cross-platform apps (iOS + Android + desktop)
70	- Maximum compatibility with open-source model ecosystem
71
72	### Quick Reference
73
74	\| Scenario \| Framework \|
75	\|---\|---\|
76	\| Text generation on Apple Intelligence devices (iOS 26+) \| Foundation Models \|
77	\| Structured output from on-device LLM \| Foundation Models (`@Generable`) \|
78	\| Image classification, object detection \| Core ML \|
79	\| Custom model from PyTorch/TensorFlow \| Core ML + coremltools \|
80	\| Running specific open-source LLMs \| MLX Swift or llama.cpp \|
81	\| Maximum throughput on Apple Silicon \| MLX Swift \|
82	\| Cross-platform LLM inference \| llama.cpp \|
83	\| OCR and text recognition \| Vision framework \|
84	\| Sentiment analysis, NER, tokenization \| Natural Language framework \|
85	\| Training custom classifiers on device \| Create ML \|
86
87	## Apple Foundation Models Overview
88
89	Use the system language model for short generation, summarization, tagging,
90	structured output, and tool-augmented tasks on Apple Intelligence devices. Gate
91	every entry point before creating a session:
92
93	```swift
94	import FoundationModels
95
96	switch SystemLanguageModel.default.availability {
97	case .available:
98	guard SystemLanguageModel.default.supportsLocale(Locale.current) else {
99	// Use locale fallback before generating
100	break
101	}
102	// Proceed with model usage
103	case .unavailable(.appleIntelligenceNotEnabled):
104	// Guide user to enable Apple Intelligence in Settings
105	case .unavailable(.modelNotReady):
106	// System model assets are not ready; show loading state
107	case .unavailable(.deviceNotEligible):
108	// Device cannot run Apple Intelligence; use fallback
109	case .unavailable(let reason):
110	// Unknown or future unavailable reason; use fallback and log reason
111	}
112	```
113
114	Then create a session and keep its shared context budget small:
115
116	```swift
117	let session = LanguageModelSession {
118	"You are a helpful cooking assistant."
119	}
120	session.prewarm()
121	let response = try await session.respond(to: "Suggest a quick pasta recipe")
122	```
123
124	Required guardrails:
125
126	- Sessions are stateful and accept one request at a time; serialize access and
127	check `isResponding` before issuing another response.

People who liked this also viewed

6 picks

Type