Rock-Paper-Scissors Mechanical Arm

Context & Evolution

This project grew directly out of my work on the flame thrower glove. While researching and planning that system, I became deeply interested in mechatronics—specifically how sensing, decision-making, and motion come together. I wanted a lower-risk environment to explore those ideas further, and computer vision felt like the natural next step.

After working on the flame thrower glove, I wanted to test how far perception could be pushed. Could a machine see what a human does and convincingly replicate it through robotics?

My Role

I designed the vision system, servo actuation logic, and control software. I pulled the mechanical hand design from an open source project since I wanted to focus more on the software for this project. I treated this as an opportunity to study how machines can observe human motion and physically respond to it in a legible way.

How It Works

A camera feeds a vision pipeline that detects and classifies hand gestures in real time. The system analyzes the video stream, identifies the human player's gesture (rock, paper, or scissors), and processes that classification through game logic. The software then coordinates multiple servos to articulate the mechanical fingers into the corresponding hand shape.

The emphasis is on embodiment—software decisions are expressed through motion rather than screens. The mechanical hand physically performs its response, making the interaction tangible and immediate.

Game Logic & Flow

Game State Machine

The system operates through a defined sequence of states:

• Waiting - Hand in neutral position, vision system monitoring for player gesture
• Detection - Valid gesture detected with sufficient confidence
• Decision - System randomly selects its own gesture (rock, paper, or scissors)
• Execution - Servos articulate fingers to form chosen gesture
• Display - Both gestures held momentarily for clear comparison
• Reset - Return to neutral position for next round

Randomization

The mechanical hand's gesture is randomly generated to ensure fair gameplay. The randomization occurs after the player's gesture is detected, preventing any appearance of the machine "cheating" by waiting to see the human choice.

Software Architecture

Vision Processing Module

Built using OpenCV for Python, the vision module handles all image processing and gesture classification. It runs in a continuous loop, processing frames and maintaining detection state.

Control Interface Module

Manages communication with the servo controller, translating high-level gesture commands into specific servo position arrays. Handles error checking and ensures commands are properly formatted.

Game Logic Module

Coordinates the overall game flow, managing state transitions, randomization, and timing. Ensures proper sequencing between gesture detection, decision-making, and mechanical response.

Calibration Tools

Separate utilities for tuning vision parameters (color thresholds, detection sensitivity) and servo positions (finger curl angles, speed profiles) without modifying core code.

Tools & Technologies

• OpenCV - Computer vision library for image processing and gesture recognition
• Python - Primary programming language for vision and control logic
• NumPy - Numerical computing for image array manipulation
• Serial Communication - Protocol for vision computer to servo controller communication
• Arduino/Servo Controller - Hardware interface for servo control
• 3D Printing - Fabrication of mechanical hand components