Navigating Apple AI/ML Resources
YAML--- name: navigating-apple-ai-ml-resources description: Maps AI/ML development needs to the correct Apple developer framework, tool, or resource (Core AI, Foundation Models, Core ML, Vision, Natural Language, Speech, Sound, Translation). Use when deciding which Apple AI/ML technology fits a task, integrating on-device models into an Apple platform app, or converting third-party models to Core ML. ---
A developer says: "I want to add [capability] to my iOS/macOS app."
- Identify the capability category (see mapping table below).
- Point to the specific framework/tool.
- Note the on-device/privacy angle if relevant (nearly everything here runs on-device).
Example prompt → answer: "I have a PyTorch LLM I want to run locally on iPhone" → Use Core AI (PyTorch → production pipeline for on-device LLMs on Apple Silicon), then convert with Core ML Tools if targeting Core ML runtime, or use Core AI's own model/optimization/debugger toolchain directly.
Progress:
- Clarify the task: generation, prediction, perception (vision/audio/text), or model conversion?
- Match task to framework using the decision table
- Check if a pre-built API already exists (Vision/NL/Speech/Sound/Translation) before reaching for custom ML
- If custom/foundation model needed, choose Foundation Models framework (built-in Apple Intelligence model or any provider via Language Model protocol) vs Core AI (bring-your-own LLM/generative model from PyTorch)
- If deploying a trained model, convert with Core ML Tools, then integrate via Core ML framework
- Reference HIG for Generative AI or general ML HIG for UX decisions
- Point to docs/videos/research/support channel as needed for depth
Decision table
| Need | Resource |
|---|---|
| Run a full LLM/generative model on-device from PyTorch | Core AI (framework, models, PyTorch extensions, optimization, debugger — all on GitHub) |
| Access Apple's on-device foundation model, or swap in any LLM provider via a common Swift protocol | Foundation Models framework |
| Design guidance for chat/gen-AI features | Human Interface Guidelines for Generative AI |
| Integrate a trained model (classification, regression, etc.) into an app, on-device inference/training | Core ML framework + Core ML models gallery |
| Convert a TensorFlow/PyTorch/other model to Apple's format | Core ML Tools (converters) |
| Detect objects/faces/text in images or video | Vision |
| Analyze/classify text, sentiment, language ID, tagging | Natural Language |
| Speech-to-text or text-to-speech | Speech |
| Classify ambient/environmental sounds | Sound |
| Translate text/UI between languages | Translation |
| Deep dive / see it in action | Videos (WWDC-style sessions) |
| API details, code samples | Documentation |
| Design best practices generally | Human Interface Guidelines |
| Latest Apple ML research/publications | Apple Machine Learning Research |
| Live help, bug reports, account issues | Meet with Apple, Developer Forums, Feedback Assistant, Support |
Example 1: Input: "I want my app to classify what a photo contains." Output: Use Vision framework for built-in object/scene detection; if a custom model is needed, train/convert it to Core ML (via Core ML Tools) and combine with Vision's request/observation APIs.
Example 2: Input: "I have a fine-tuned Llama model in PyTorch and want it running fully offline on an iPhone." Output: Use Core AI — it's purpose-built for taking PyTorch LLMs/generative models to production on Apple Silicon, including its own optimization and debugging tools. Consider Core ML conversion only if you specifically need the Core ML runtime/API surface instead of Core AI's pipeline.
Example 3: Input: "I want a chat feature using Apple's own on-device model, or optionally let users pick another provider." Output: Use Foundation Models framework — native Swift API, direct access to the Apple Intelligence on-device model, and supports swapping providers via the Language Model protocol. Follow the HIG for Generative AI for UX.
Example 4: Input: "My app needs to detect distinct real-world sounds like glass breaking or a dog barking." Output: Use Sound classification API.
- Prefer high-level task APIs (Vision, Natural Language, Speech, Sound, Translation) before building/integrating custom models — less code, maintained by Apple, on-device by default.
- Use Core ML as the integration layer once you have a trained model; use Core ML Tools for conversion rather than hand-rolling format translation.
- Reserve Core AI for cases specifically involving large-scale/generative PyTorch models that need a dedicated on-device production pipeline.
- Use Foundation Models framework when you want to leverage Apple's built-in model or need a provider-agnostic Swift interface, rather than bundling/shipping your own weights.
- Always cross-check UX decisions against the relevant HIG (general ML or generative AI specific).
- For anything unclear or edge-case, direct users to Documentation, Videos, or Developer Forums rather than guessing.
- Don't recommend building a custom model pipeline when an existing task API (Vision/NL/Speech/Sound/Translation) already solves the problem.
- Don't confuse Core ML (integration/runtime framework) with Core ML Tools (conversion utility) — they solve different steps.
- Don't conflate Core AI (bring-your-own PyTorch LLM to production) with Foundation Models framework (access to Apple's/other providers' ready-made models) — they serve different use cases.
- Don't ignore the HIG when suggesting generative AI features; design guidance is part of the resource set for a reason.
- Don't point to generic support channels first — surface the specific framework/tool before escalating to forums/support.