NeuralRot classifies webcam gestures using pose + hand landmarks, then serves live predictions to a web frontend.
- Data collection:
main.pycaptures webcam frames.- MediaPipe extracts
pose + left hand + right handlandmarks. - Features are normalized and appended to
gesture_data.csv.
- Training:
- Reads
gesture_data.csv. - Trains a
RandomForestClassifier. - Saves model to
gesture_model.pkl.
- Reads
- Final running:
- Frontend captures webcam frames.
- Backend (
/predict) extracts the same features and predicts label. - Frontend displays the predicted gesture live.
NeuralRot/
|-- main.py
|-- gesture_classifier.py
|-- backend_api.py
|-- meme_engine.py
|-- gesture_data.csv
|-- gesture_model.pkl
|-- requirements.txt
|-- frontend/
| |-- index.html
| |-- script.js
| `-- styles.css
`-- memes/
python -m venv venv
venv\Scripts\activate
python -m pip install -r requirements.txtRecommended Python: 3.10 to 3.12.
python main.py --labels dab,neutral --samples-per-label 100python main.py --skip-collectpython gesture_classifier.py --csv gesture_data.csv --model gesture_model.pkl --augment-factor 3python main.py --run-appOpen:
http://127.0.0.1:5500
Health check:
http://127.0.0.1:8000/health
- If predictions do not work, confirm
gesture_model.pklexists. - If model is missing, train first with
python main.py --skip-collect. - Keep head/shoulders/elbows/hands visible during collection and inference.