coreml

Name: coreml
Author: dpearson2699

915stars45forks

How to install coreml?

dpearson2699/swift-ios-skills/coreml

$npx -y skills add dpearson2699/swift-ios-skills --skill coreml

Installs into the current project.

Summary

Integrate Core ML models in iOS apps for on-device machine learning inference. Covers model loading (.mlmodel, .mlpackage, .mlmodelc), predictions with auto-generated classes and MLFeatureProvider, compute unit configuration (CPU, GPU, Neural Engine), MLTensor, VNCoreMLRequest, M

Files · 1

View on GitHub

SKILL.md

1	# Core ML Swift Integration
2
3	Load, configure, and run Core ML models in iOS apps. This skill covers the
4	Swift side: model loading, prediction, MLTensor, profiling, and deployment.
5
6	> Scope boundary: Python-side model conversion, optimization (quantization,
7	> palettization, pruning), and framework selection live in the `apple-on-device-ai`
8	> skill. This skill owns Swift integration only.
9
10	See [references/coreml-swift-integration.md](references/coreml-swift-integration.md) for complete code patterns including
11	actor-based caching, batch inference, image preprocessing, and testing.
12
13	## Contents
14
15	- [Loading Models](#loading-models)
16	- [Model Configuration](#model-configuration)
17	- [Making Predictions](#making-predictions)
18	- [MLTensor (iOS 18+)](#mltensor-ios-18)
19	- [Working with MLMultiArray](#working-with-mlmultiarray)
20	- [Image Preprocessing](#image-preprocessing)
21	- [Multi-Model Pipelines](#multi-model-pipelines)
22	- [Vision Integration](#vision-integration)
23	- [Performance Profiling](#performance-profiling)
24	- [Model Deployment](#model-deployment)
25	- [Memory Management](#memory-management)
26	- [Common Mistakes](#common-mistakes)
27	- [Review Checklist](#review-checklist)
28	- [References](#references)
29
30	## Loading Models
31
32	### Auto-Generated Classes
33
34	When you add a `.mlmodel` or `.mlpackage` to an app target, Xcode generates a Swift
35	class with typed input/output. Use this whenever possible.
36
37	```swift
38	import CoreML
39
40	let config = MLModelConfiguration()
41	config.computeUnits = .all
42
43	let model = try MyImageClassifier(configuration: config)
44	```
45
46	### Manual Loading
47
48	Load from a URL when the model is downloaded at runtime or stored outside the
49	bundle.
50
51	```swift
52	let modelURL = Bundle.main.url(
53	forResource: "MyModel", withExtension: "mlmodelc"
54	)!
55	let model = try MLModel(contentsOf: modelURL, configuration: config)
56	```
57
58	### Async Loading (iOS 15+)
59
60	Load models without blocking the main thread. Prefer this for large models.
61
62	```swift
63	let model = try await MLModel.load(
64	contentsOf: modelURL,
65	configuration: config
66	)
67	```
68
69	### Compile at Runtime (iOS 16+)
70
71	Compile a `.mlpackage` or `.mlmodel` to `.mlmodelc` on device. Useful for
72	models downloaded from a server. Do this once per model version, not on every
73	launch.
74
75	```swift
76	let compiledURL = try await MLModel.compileModel(at: packageURL)
77	let model = try await MLModel.load(contentsOf: compiledURL, configuration: config)
78	```
79
80	Cache the compiled URL -- recompiling on every launch is a bug. Copy
81	`compiledURL` to a persistent location (e.g., Application Support). When
82	reviewing runtime-loaded models, call out both facts together: async
83	`MLModel.compileModel(at:)` is iOS 16+, and compiled models must be cached so the
84	app does not recompile on every launch.
85
86	## Model Configuration
87
88	`MLModelConfiguration` controls compute units, GPU access, and model parameters.
89
90	### Compute Units Decision Table
91
92	\| Value \| Uses \| When to Choose \|
93	\|---\|---\|---\|
94	\| `.all` \| CPU + GPU + Neural Engine \| Default. Let the system decide. \|
95	\| `.cpuOnly` \| CPU \| Deterministic tests, CPU-only fallbacks, or constrained work after profiling shows accelerator policy, contention, thermal state, or energy budget is the limiting factor. \|
96	\| `.cpuAndGPU` \| CPU + GPU \| Need GPU but model has ops unsupported by ANE. \|
97	\| `.cpuAndNeuralEngine` (iOS 16+) \| CPU + Neural Engine \| Best energy efficiency for compatible models. \|
98
99	```swift
100	let config = MLModelConfiguration()
101	config.computeUnits = .cpuAndNeuralEngine
102
103	// Optional fallback for constrained work after profiling and policy review
104	config.computeUnits = .cpuOnly
105	```
106
107	### Configuration Properties
108
109	```swift
110	let config = MLModelConfiguration()
111	config.computeUnits = .all
112	config.allowLowPrecisionAccumulationOnGPU = true // faster, slight precision loss
113	```
114
115	## Making Predictions
116
117	### With Auto-Generated Classes
118
119	The generated class provides typed input/output structs.
120
121	```swift
122	let model = try MyImageClassifier(configuration: config)
123	let input = MyImageClassifierInput(image: pixelBuffer)
124	let output = try model.prediction(input: input)
125	print(output.classLabel) // "golden_retriever"
126	print(output.classLabelProbs) // ["golden_retriever": 0.95, ...]
127	```
128
129	### With MLDictionaryFeatureProvider
130
131	Use when inputs are dynamic or not known at compile time.
132
133	```swift
134	let inputFeatures = try MLDictionaryFeatureProvider(dictionary: [
135	"image": MLFeatureValue(pixelBuffer: pixelBuffer),
136	"confidence_threshold": MLFeatureValue(double: 0.5),
137	])
138	let output = try model.prediction(from: inputFeatures)

People who liked this also viewed

6 picks

Type

coreml

915stars45forks

View on GitHub

How to install coreml?

dpearson2699/swift-ios-skills/coreml

$npx -y skills add dpearson2699/swift-ios-skills --skill coreml

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Core ML Swift Integration
2
3	Load, configure, and run Core ML models in iOS apps. This skill covers the
4	Swift side: model loading, prediction, MLTensor, profiling, and deployment.
5
6	> Scope boundary: Python-side model conversion, optimization (quantization,
7	> palettization, pruning), and framework selection live in the `apple-on-device-ai`
8	> skill. This skill owns Swift integration only.
9
10	See [references/coreml-swift-integration.md](references/coreml-swift-integration.md) for complete code patterns including
11	actor-based caching, batch inference, image preprocessing, and testing.
12
13	## Contents
14
15	- [Loading Models](#loading-models)
16	- [Model Configuration](#model-configuration)
17	- [Making Predictions](#making-predictions)
18	- [MLTensor (iOS 18+)](#mltensor-ios-18)
19	- [Working with MLMultiArray](#working-with-mlmultiarray)
20	- [Image Preprocessing](#image-preprocessing)
21	- [Multi-Model Pipelines](#multi-model-pipelines)
22	- [Vision Integration](#vision-integration)
23	- [Performance Profiling](#performance-profiling)
24	- [Model Deployment](#model-deployment)
25	- [Memory Management](#memory-management)
26	- [Common Mistakes](#common-mistakes)
27	- [Review Checklist](#review-checklist)
28	- [References](#references)
29
30	## Loading Models
31
32	### Auto-Generated Classes
33
34	When you add a `.mlmodel` or `.mlpackage` to an app target, Xcode generates a Swift
35	class with typed input/output. Use this whenever possible.
36
37	```swift
38	import CoreML
39
40	let config = MLModelConfiguration()
41	config.computeUnits = .all
42
43	let model = try MyImageClassifier(configuration: config)
44	```
45
46	### Manual Loading
47
48	Load from a URL when the model is downloaded at runtime or stored outside the
49	bundle.
50
51	```swift
52	let modelURL = Bundle.main.url(
53	forResource: "MyModel", withExtension: "mlmodelc"
54	)!
55	let model = try MLModel(contentsOf: modelURL, configuration: config)
56	```
57
58	### Async Loading (iOS 15+)
59
60	Load models without blocking the main thread. Prefer this for large models.
61
62	```swift
63	let model = try await MLModel.load(
64	contentsOf: modelURL,
65	configuration: config
66	)
67	```
68
69	### Compile at Runtime (iOS 16+)
70
71	Compile a `.mlpackage` or `.mlmodel` to `.mlmodelc` on device. Useful for
72	models downloaded from a server. Do this once per model version, not on every
73	launch.
74
75	```swift
76	let compiledURL = try await MLModel.compileModel(at: packageURL)
77	let model = try await MLModel.load(contentsOf: compiledURL, configuration: config)
78	```
79
80	Cache the compiled URL -- recompiling on every launch is a bug. Copy
81	`compiledURL` to a persistent location (e.g., Application Support). When
82	reviewing runtime-loaded models, call out both facts together: async
83	`MLModel.compileModel(at:)` is iOS 16+, and compiled models must be cached so the
84	app does not recompile on every launch.
85
86	## Model Configuration
87
88	`MLModelConfiguration` controls compute units, GPU access, and model parameters.
89
90	### Compute Units Decision Table
91
92	\| Value \| Uses \| When to Choose \|
93	\|---\|---\|---\|
94	\| `.all` \| CPU + GPU + Neural Engine \| Default. Let the system decide. \|
95	\| `.cpuOnly` \| CPU \| Deterministic tests, CPU-only fallbacks, or constrained work after profiling shows accelerator policy, contention, thermal state, or energy budget is the limiting factor. \|
96	\| `.cpuAndGPU` \| CPU + GPU \| Need GPU but model has ops unsupported by ANE. \|
97	\| `.cpuAndNeuralEngine` (iOS 16+) \| CPU + Neural Engine \| Best energy efficiency for compatible models. \|
98
99	```swift
100	let config = MLModelConfiguration()
101	config.computeUnits = .cpuAndNeuralEngine
102
103	// Optional fallback for constrained work after profiling and policy review
104	config.computeUnits = .cpuOnly
105	```
106
107	### Configuration Properties
108
109	```swift
110	let config = MLModelConfiguration()
111	config.computeUnits = .all
112	config.allowLowPrecisionAccumulationOnGPU = true // faster, slight precision loss
113	```
114
115	## Making Predictions
116
117	### With Auto-Generated Classes
118
119	The generated class provides typed input/output structs.
120
121	```swift
122	let model = try MyImageClassifier(configuration: config)
123	let input = MyImageClassifierInput(image: pixelBuffer)
124	let output = try model.prediction(input: input)
125	print(output.classLabel) // "golden_retriever"
126	print(output.classLabelProbs) // ["golden_retriever": 0.95, ...]
127	```
128
129	### With MLDictionaryFeatureProvider
130
131	Use when inputs are dynamic or not known at compile time.
132
133	```swift
134	let inputFeatures = try MLDictionaryFeatureProvider(dictionary: [
135	"image": MLFeatureValue(pixelBuffer: pixelBuffer),
136	"confidence_threshold": MLFeatureValue(double: 0.5),
137	])
138	let output = try model.prediction(from: inputFeatures)

People who liked this also viewed

6 picks

Type