Javanese Script Recognition

An AI application in the field of Computer Vision, powered by a Deep Learning algorithm (CNN) which is a subset of Machine Learning. Draw a Hanacaraka character and let the AI identify it.

Draw here

Draw a character above, then click Predict

Behind the Screen: How the AI Works

This project is an end-to-end Deep Learning implementation that integrates a custom Convolutional Neural Network (CNN) built with PyTorch, a containerized FastAPI backend, and an interactive Next.js frontend canvas. Here is the technical breakdown of how the model was trained and deployed.

Personal Motivation

“Why Javanese Script? As a descendant of Javanese heritage who deeply loves our culture, I find the Hanacaraka script highly aesthetic and rich in meaning. In a rapidly modernizing world, it is vital to preserve our cultural roots. This project is my way of bridging traditional heritage with modern technology, contributing to the preservation and digitalization of Javanese script so it remains alive for future generations.”

1

The Dataset & Architecture

Dataset Origin

The model was trained using a Javanese Script dataset sourced from Kaggle, containing handwritten samples of the 20 core characters (Aksara Nglegena): HA, NA, CA, RA, KA, DA, TA, SA, WA, LA, PA, DHA, JA, YA, NYA, MA, GA, BA, THA, NGA.

The Challenge

Javanese script has extreme inter-class similarity (e.g., the visual structures of PA and YA, or HA and GA are incredibly similar). Traditional Multi-Layer Perceptrons (MLP) heavily struggle here, achieving barely ~11% accuracy.

The Solution

We designed a Custom Deeper CNN Architecture using PyTorch:

  • 3 Convolutional Blocks: Each block consists of a Conv2d layer (extracting spatial features like curves and strokes) followed by BatchNorm2d (for stabilizing training) and ReLU activation.
  • Pooling Layers: MaxPool2d downsamples the spatial dimensions from 64x64 → 32x32 → 16x16 → 8x8.
  • Fully Connected Classifier: Maps the extracted features into a 256-node hidden layer, protected by a Dropout(0.4) layer to prevent overfitting, before outputting raw logits for the 20 classes.
  • Performance: This optimization successfully boosted the model's test validation accuracy to a highly robust ~88%.
2

Advanced Real-Time Image Preprocessing Pipeline

When you click "Predict", the raw binary image blob from your canvas goes through an identical preprocessing pipeline matching the model's training data constraints:

  1. Grayscale Conversion: The RGB/RGBA canvas data is flattened into a single-channel grayscale matrix using OpenCV.
  2. Inversion & Thresholding: The image is inverted (bitwise_not) and binarized via cv2.threshold so that the strokes become a solid white foreground against a pure black background.
  3. Bounding Box Centering: Using cv2.findNonZero and cv2.boundingRect, the script dynamically crops the active drawing area, isolates the character, scales it proportionally to fit within a 48x48 bounding box, and pads it perfectly into the center of a strict 64x64 tensor. This eliminates position bias entirely.
3

Containerized Microservice Deployment

Backend Service

The trained PyTorch model weights (.pth) are hosted as a high-performance REST API using FastAPI and Uvicorn.

Dockerization & Mapping

Containerized using a multi-stage Dockerfile running lightweight python:3.9-slim. The container exposes services mapped to port 8928.

Frontend Interactivity

The frontend captures vectors from the canvas on-demand, packages them into a multipart/form-data payload under the file key, executes an asynchronous POST request, and decodes the Softmax confidence probability array returned by the API container.

4

How to Train the Model Locally

For transparency and reproducibility, the entire training pipeline can be executed locally via CLI. Here is the workflow to retrain or fine-tune the model:

Step 1: Clone and Setup Environment

Ensure you have Python 3.9+ installed, then clone the repository and install the dependencies:

pip install torch torchvision opencv-python numpy
Step 2: Prepare Dataset Structure

Organize your raw handwritten images folder inside a dataset/ directory. The loader dynamically reads directory names as class targets:

📂 dataset/
├── 📂 ha/ (contains ha_1.png, ha_2.png...)
├── 📂 na/
└── ... [up to 20 classes]
Step 3: Run Training Script

Execute the training script. The training pipeline automatically applies our custom crop_and_center_64 image normalization on the fly, splits the data 80:20 (Train/Val), sets up the Adam Optimizer ($lr=0.0005$), and runs for 45 Epochs:

python train.py --dataset ./dataset --epochs 45 --batch_size 16

Once completed, it will validate the weights and export a production-ready hanacaraka_cnn_20_full.pth matrix file.

5

Public API Integration Guide (Developer API Docs)

We are opening our containerized Deep Learning inference core as a public microservice. Developers can integrate Javanese script recognition into their own apps (Web, Mobile, or Desktop) by consuming our public endpoint.

📌 Base Endpoint Specification

Method: POST
Content-Type: multipart/form-data

📥 Request Payload Parameters

ParameterTypeRequiredDescription
filebinary (File)YesThe handwritten character image from a canvas or upload source (PNG/JPG).

📤 Response Schema

Success Response (HTTP 200 OK)

{
  "status": "success",
  "prediction": "NA",
  "confidence": 0.8523
}

Error Response (HTTP 400 / 500)

{
  "status": "error",
  "message": "Invalid image file or processing error description."
}

💻 Code Snippets for Integration

1. JavaScript / Next.js Fetch API

const uploadCanvasImage = async (imageBlob) => {
  const formData = new FormData();
  formData.append("file", imageBlob, "hanacaraka.png");

  try {
    const response = await fetch("https://api.adrianajisepta.my.id/predict", {
      method: "POST",
      body: formData,
    });
    const result = await response.json();
    console.log(`Predicted Class: ${result.prediction} (${(result.confidence * 100).toFixed(2)}%)`);
  } catch (error) {
    console.error("API Fetch Error:", error);
  }
};

2. Python (Requests Library)

import requests

url = "https://api.adrianajisepta.my.id/predict"
file_path = "path_to_your_drawing.png"

with open(file_path, "rb") as f:
    files = {"file": f}
    response = requests.post(url, files=files)
    
print(response.json())

3. cURL (Terminal CLI)

curl -X POST "https://api.adrianajisepta.my.id/predict" \
     -H "accept: application/json" \
     -H "Content-Type: multipart/form-data" \
     -F "file=@your_drawing.png;type=image/png"