Overview

Shift is a comprehensive platform that connects signal ingestion, AI-powered enrichment, and structured workflow management. It enables innovation teams to systematically move from discovery to validated opportunities with complete traceability and audit trails.

Core Capabilities:

  • Real-time monitoring and AI-enriched signal scoring
  • Structured innovation pipeline with Kanban-style workflow management
  • Knowledge base integration for institutional learning
  • Research discovery through OpenAlex and competitor intelligence
  • Automated technical specification and requirements generation

Key Features

Monitoring

Ingests signals from ClickHouse, applies AI-powered scoring algorithms, and exposes enriched data through a REST API for downstream consumption.

Pipeline Management

Transforms raw signals into actionable workflows (evaluation → POC → deployment) with persistent knowledge storage in Qdrant vector database for cross-project reuse.

Studio

Provides an interactive workspace for deep-dive signal analysis and automated generation of technical specifications.

Research & Discovery

  • OpenAlex Integration: Access to research metadata and academic publications
  • Competitor Intelligence: Automated discovery and scoring of competitive landscape

Documentation Generation

  • Cahier des Charges: AI-assisted requirement specification documents
  • PDF Summaries: Automated summary generation with structured outputs

Architecture

System Workflow

Shift's end-to-end pipeline transforms raw signals into deployed applications through six interconnected phases with complete governance and auditability.

Phase 1: Data Ingestion & AI Enrichment

Flow: Scraper → AI Filter ↔ Qdrant → ClickHouse

  • Cron Scraper: Automated data collection from tech news, research papers, and patents
  • AI Filter: Scores and enriches signals using ML models, checks against knowledge base
  • Qdrant: Vector database for semantic search and deduplication
  • ClickHouse: Time-series storage for all signals

Phase 2: Monitoring & Real-Time Processing

Flow: ClickHouse → Change Watcher → Data Processor → PostgreSQL/API → Frontend

  • Change Watcher: Event-driven listener for new/updated signals
  • Data Processor: Transforms and validates signal data
  • PostgreSQL: Stores processed signals and pipeline metadata
  • Frontend API: Exposes monitoring endpoints to UI

Phase 3: Signal Analysis & Requirements Generation

Flow: User Selection → Signal Translator → Redis Queue → Cahier Generator → Frontend

  • Signal Translator: Converts selected signals into structured format
  • Redis Queue 1: Async processing queue
  • Cahier de Charge Generator:
    • Creates technical requirement documents using AI (Mistral)
    • Recommends tech stack and architecture
    • Outputs "Cahier des Charges" specification

Phase 4: Code Generation & Dockerfile Creation

Flow: Redis Queue → Docker Module → File System ↔ TinyKit → Redis Queue

  • Docker Generation Module: Creates Dockerfiles and compose configs from requirements
  • File System: Stores generated artifacts in /projects/{id}/ structure
  • TinyKit IDE:
    • Generates application scaffolding
    • Validates and fixes code using AI
    • Runs automated tests and linting
    • Outputs production-ready Dockerfile

Phase 5: Deployment Pipeline & Orchestration

Flow: Redis Queue → Go Service → Docker Build → K8s Deploy → Running Pods

  • Deployment Service (Go): Orchestrates entire deployment lifecycle
  • Docker API Client: Builds, tags, and pushes container images
  • K8s API Client: Generates manifests and deploys to cluster
  • Kubernetes: Runs pods with auto-scaling, load balancing, and health checks
  • Feedback Loop: Returns subdomain URL and server stats to frontend

Phase 6: Governance, Authentication & Audit Trail

Flow: User → Frontend → Auth/RBAC → ImmuDB (Audit Log)

Authentication:

  • JWT-based auth with role-based access control (Admin, Developer, Viewer)
  • Endpoints: /auth/register, /auth/login, /auth/me

Governance Workflow:

User initiates deployment
  ↓
ImmuDB: INTENT_TO_DEPLOY (decision_id)
  ↓
Deployment Service processes
  ↓
Go Notary → Frontend API: /go/notary/confirmation
  ↓
ImmuDB: TECHNICAL_CONFIRMATION (linked to decision_id)

Benefits: Immutable audit trail, compliance-ready, full traceability, non-repudiation

Built With

Share this project:

Updates