Hanxie - Handwriting App

HanXie: Handwriting App is an application for Mandarin learners to learn and practice writing Mandarin characters with the help of Machine Learning. This application is supported on both major mobile platforms: Android and iOS.

This application is submitted as a part of a research paper for my Bachelor of Information Systems degree at Universitas Internasional Batam.

👨‍🎓

The research article has been published and can be read through this link.

Problem Statement:

The diversity of languages from one country to another requires the establishment of common languages to facilitate idea exchange, organizational collaboration, and overseas administration, such as Mandarin.

Learning Mandarin involves four essential components: listening (听力 tīnglì), speaking (口语 kŏuyŭ), reading (阅读 yuèdú) and writing (写作 xiĕzuò). Among these skills, writing Mandarin characters (漢字 hánzì) is considered one of the most challenging journey for beginners.

This complexity arises from the nature of character formation and the rules of writing Mandarin characters. Thus, tools are needed to support learners in this aspect.

Application Overview:

This application consists of 99 levels: 82 HSK 1 singled characters and 17 tests. It communicates with an API server to predict the user's handwriting corresponds to the current level they're on.

📘

You can learn more about the application through this guidebook.

Flask API Server

The first iteration of the API server is developed using Flask. It is hosted on Google Cloud Run with similar configurations as ParuParuKu.

Due to Google Cloud Run being a container-based serverless platform, the server will be shut down after a few minutes of inactivity and require around 7-10 seconds to start again.

Several volunteers expressed their concern about this issue as they need to wait for the server to start before they can use the application. Therefore, a new server choice is explored after the research process is completed.

Express API Server

As of 23 November 2023, ExpressJS is chosen as the framework due to its simplicity and flexibility. The server is hosted on Vercel as serverless functions with the same endpoints as the Flask server.

The model was converted into ONNX format to be used in the server using tf2onnx. The model is then loaded into the server using onnxruntime-node.

Screenshots:

Technologies:

Languages

1. C#
2. Python (Flask API Server)
3. Javascript (Express API Server)

Frameworks / Libraries / Packages

Machine Learning Models

1. Tensorflow Keras
2. Scikit-learn
3. Numpy
4. Seaborn (visualization purpose)
5. Matplotlib (visualization purpose)
6. Gdown (download sorted large datasets from google drives)
7. Tf2onnx converter

Flask Server

1. Flask
2. Flask Babel
3. Tensorflow-CPU
4. Pillow
5. Gunicorn
6. Werkzeug
7. Scipy
8. OpenCV

Express Server

1. ExpressJS
2. ONNX Runtime (onnxruntime-node)
3. Multer
4. Canvas (node-canvas)
5. ndarray

Mobile Application

1. Simple scroll snap
2. Image with rounded corners
3. Unity Firebase extensions

Tools

1. Unity Engine
2. Figma
3. Visual Studio Community
4. Visual Studio Code
5. Notion
6. Microsoft Excel
7. Google Colaboratory

Takeaways:

🔭

Due to some "dumb" decisions, this application is initially removed from my local GitHub folder without any backup. Hence, it requires me three full days to rebuild while being chased by deadline.

Just a quick reminder to always commit your changes and push in on cloud server in case of something wrong on your local devices.

Credits/Shoutouts:

I'd like to express my gratitude to my family, partner, and friends for their support during my entire thesis journey.

I want to give a special acknowledgment to Tony Wibowo for his support during my data collection process. Additionally, I extend my appreciation to Suwarno Liang for guiding me as my thesis lecturer.