LatihIsyarat

About LatihIsyarat

LatihIsyarat is a Flutter application for learning the ASL alphabet. Users can practise through the camera, photos or uploads, read letter guidance, and view practice history and progress. Recognition runs offline using TensorFlow Lite, while history is stored locally in SQLite.

I worked on hand detection, image preprocessing, camera recognition integration, J/Z motion development, learning progress, and testing. The default path covers 24 static letters; I developed J/Z on the rama branch.

My contributions

Hand detection in photos and uploads

I developed hand detection for still images and connected it to the photo/upload workflow. I handled hand region selection, input that is not yet suitable for processing, and the connection between detection results and the classifier. This sends user-selected images through the appropriate recognition stages.

Preprocessing and the TensorFlow Lite classifier

I worked on image preprocessing to match the model’s expected input, invoked the TensorFlow Lite classifier, and mapped prediction labels to application results. I also improved model identifier validation and classifier tests so model or label mapping errors can be handled at the recognition layer.

J/Z training package and model contract

On the rama branch, I built a Python J/Z training/evaluation package using sequence features and a GRU model. I prepared labels, metadata, and preprocessing contracts, grouped data by source video, and recorded split manifests. I also checked conversion to TensorFlow Lite, including float16 variants, and archived static model sources.

Hand landmarks and native Android integration

I connected MediaPipe components, detectors/landmarks, frame encoders, sequence preprocessing, and the J/Z classifier. I worked on the native Android/Kotlin bridge and isolate inference so camera frames can become sequence features consumed by the motion recognition service.

J/Z target practice and free mode

I integrated J/Z target practice and a free mode combining static pose and motion recognition. J/Z clips use user-controlled start/end actions. I configured timeouts, rejected results arriving from old sessions, and handled cancellation to keep predictions associated with the active session.

Camera lifecycle and session storage

I fixed camera release when users change pages or the application enters the background. I also handled history storage for interrupted sessions so saving and camera shutdown do not interfere. This work supports camera stability during practice and when stopping a session.

History, 26-letter progress, and guidance

I extended history models and progress to cover 26 letters in the J/Z development, fixed weekly practice markers based on activity dates, and added explanations of J/Z movements to the guide. I also refined practice controls, guidance pages, and progress displays connected to local session data.

Service and application integration tests

I wrote tests for classifiers, hand detectors, J/Z services, camera integration, history, gamification, and guidance. The focus includes input/output contracts, session handling, and connecting recognition results to learning workflows. This development remains distinct from the default branch’s 24-static-letter path.