ARM Adaptive Intelligence Engine: A Heterogeneous ARM Integrated Toolchain for Edge AI Performance Tuning Utilizing Multi-Agent and RAG Pipeline
A heterogeneous ARM integrated toolchain for edge AI performance tuning, enabling developers to compress, optimize, debug and deploy AI models on ARM-based mobile devices beneficial to ARM architecture developers and users.
- Model Compressor: Reduces model size through 4-bit/3-bit/2-bit quantization, structured pruning, and knowledge distillation. Benchmarks latency, memory usage, and power consumption on ARM devices.
- Runtime Inspector: Multi-agent profiling system that monitors CPU/GPU/NPU utilization, tracks thermal conditions, analyzes computation graph bottlenecks, and generates natural language diagnostics explaining performance issues.
- Memory Engine: On-device Retrieval-Augmented Generation (RAG) system that indexes text documents, PDFs, images, and audio files locally. Creates embeddings, stores vectors, and enables semantic search without network connectivity.
- Battery Scheduler: ML-based predictor estimates battery drain for AI tasks. Schedules model inference during optimal power windows and defers tasks when battery is below threshold or device is overheating.
- IoT Layer: Connects ARM Cortex-M/A IoT devices via BLE, Thread, and Matter protocols. Fuses sensor data from accelerometers, gyroscopes, microphones, and biosignal sensors. Runs TinyML models on microcontrollers.
- Privacy Firewall: Sandboxes AI model execution to prevent data leakage. Enforces local-only processing policies, monitors data flow for sensitive information, and validates operations against permission rules.
- Live Site Demo: https://samirasamrose.github.io/arm-adaptive-intelligence/
- Source Code: https://github.com/SamiraSamrose/arm-adaptive-intelligence
- Video Demo: https://youtube.com/shorts/_--l0sK-NQQ
- Programming Languages: Python, Kotlin, Java, Swift, Objective-C, Shell Script
- Mobile Frameworks: Android SDK, iOS SDK, Jetpack Compose, SwiftUI, UIKit
- ML/AI Frameworks: TensorFlow Lite, PyTorch, ONNX Runtime, CoreML, ARM Compute Library
- Quantization Libraries: QLoRA, GPTQ, AWQ, bitsandbytes, PEFT
- Embedding & NLP: Transformers, Sentence-Transformers, Hugging Face Hub
- Computer Vision: OpenCV, Pillow, MobileViT, MobileSAM
- Audio Processing: Librosa, Soundfile, Whisper-tiny
- Vector Database: ChromaDB, Milvus Lite
- IoT Protocols: Bluetooth Low Energy (BLE), Thread, Matter, WiFi
- Build Tools: Gradle, CocoaPods, Xcode, CMake
- Testing: pytest, JUnit, XCTest
- Containerization: Docker, docker-compose
- Web Framework (Bridge): Flask, Flask-CORS
- System Monitoring: psutil, Android BatteryManager, iOS UIDevice
- Scientific Computing: NumPy, SciPy, Pandas, scikit-learn
- Configuration: YAML, JSON
- Data Formats: ONNX, TFLite, CoreML, PyTorch (.pt/.pth)
- Hardware APIs: ARM NEON, ARM SVE, ARM Ethos NPU, Mali GPU, Adreno GPU, Apple Neural Engine
- ML Models: MobileNetV2, ResNet18, BERT-Base, TinyBERT, YOLOv5, Whisper
- Data Integrations: Local file system, Mobile storage, Encrypted cache
- ML Model Compression / Distillation Benchmarks: HuggingFace Open LLM Leaderboard Models, TinyML Perf Dataset (MIT)
- Multimodal Personal Memory (OCR, Speech, Vision) Training: COCO Images, CommonVoice Speech, ArXiv Papers Corpus, DocVQA / IIT-CDIP PDFs
- Performance & Thermal Profiling- MLPerf Mobile, ARM Compute Library Samples
- Battery & Power Modeling- Smartphone Power Consumption Dataset (UMass Trace), WESAD Bio-signal Set (for wearable IoT sensor behavior)
pip install -r requirements.txt
python setup.py installfrom src.model_compressor import ModelCompressor
from src.runtime_inspector import RuntimeInspector
# Compress a model
compressor = ModelCompressor()
compressed_model = compressor.compress_model("path/to/model")
# Profile runtime
inspector = RuntimeInspector()
metrics = inspector.profile_inference(compressed_model)- Clone the repository
git clone https://github.com/samirasamrose/arm-adaptive-intelligence.git
cd arm-adaptive-intelligence- Create virtual environment
python3 -m venv venv
source venv/bin/activate- Install dependencies
pip install -r requirements.txt
pip install -e .- Run tests
python -m pytest tests/- Follow PEP 8 for Python code
- Use meaningful variable names
- Add docstrings to all functions
- Comment complex logic
- Write unit tests for new features
- Ensure all tests pass before submitting PR
- Aim for high code coverage
- Fork the repository
- Create a feature branch
- Make your changes
- Run tests
- Submit pull request