Controlling Computer Device Cursors Using Hand Gestures by Utilizing OpenCV, MediaPipe, and PyAutoGui
Keywords:
OpenCV, Python, MediaPipe, Pyautogui, controlling the cursorAbstract
This research presents an innovative interactive solution for controlling computer device cursors by utilizing hand movements using OpenCV, MediaPipe, and PyAutoGui. By combining these three libraries, this program is able to detect and analyze hand movements in real-time, allowing users to control the cursor and click with hand movements. Through image processing methods and hand landmark analysis, this program succeeded in achieving its designed goals. The main goal of the research is to create interactive solutions that are responsive and efficient in controlling computer device cursors. Test results on computer devices with certain specifications show success in detecting hand movements and controlling the cursor well. However, this research still has the potential for further development, such as adding complex hand gesture recognition features and integration with certain applications to improve program functionality and performance.
References
Jubilee Enterprise. (2019). Python for Beginner Programmers. Jakarta: PT Elex Media Komputindo.
Lubis, M. S. (2021). Implementation of Artificial Intelligence in Integrated Manufacturing Systems.Posing of the UISU National Engineering Seminar, 1-7. Retrieved January 26, 2024, from https://jurnal.uisu.ac.id/index.php/semnastek/article/download/4134/2966
PyAutoGui. (2019).Installation. Retrieved January 25, 2024, from PyAutoGUI: https://pyautogui.readthedocs.io/en/latest/install.html
Wali, M., Nengsih, T. A., Hts, D. I. G., Choirina, P., Awaludin, A. A. R., Yusuf, M., ... & Baradja, A. (2023). INTRODUCTION TO THE 15 BEST PROGRAMMING LANGUAGES OF THE FUTURE (Reference & Coding For Beginners). PT. Sonpedia Publishing Indonesia.
Dea, G. (2020). January 2).Get to know the Code Editor, and the Three Code Editors. https://vantura.id/blog/mengenal-code-editor-dan-tiga-code-editor-terpopular
Dr. Aneu Yulianeu, S., & Oktamala, R. (2022). Web-Based Public Transport Route Information System in Tasikmalaya City.Journal of Information Engineering (pp. 125–134). http://jurnal.stmik-dci.ac.id/index.php/jutekin/
Enterprise, J. (2019). Python for Beginner Programmers. PT Elex Media Komputindo.
fattachulhudacom. (2023). Image Processing Using OpenCV Integrated with Python. https://www.fattachulhuda.com/2023/11/pengolahan-citra-using-opencv.html
Han, J.-S., Lee, C.-I., Youn, Y.-H., & Kim, S.-J. (2022). A Study on Real-time Hand Gesture Recognition Technology by Machine Learning-based MediaPipe. Journal of System and Management Sciences, 12(2), 468–482. https://doi.org/10.33168/JSMS.2022.0225
Harahap, H. R. (2023). Concepts and Functions of Programming Algorithms.Journal of Mathematics and Natural Science (pp. 254–257). https://doi.org/10.59581/konsanta-widyakarya.v1i4.1875
Jawas, N. (2017). Hand Movement Tracking for Gesture Recognition.Informatic Technique Journal (pp. 13–23). https://www.e-journal.potensi-utama.ac.id/ojs/index.php/ITJournal/article/download/364/337
Kim, J.-W., Choi, J.-Y., Ha, E.-J., & Choi, J.-H. (2023). Human Pose Estimation Using MediaPipe Pose and Optimization Method Based on a Humanoid Model. Applied Sciences (Switzerland), 13(4). https://doi.org/10.3390/app13042700
Lemonaki, D. (2022). Loops in Python - While True Loop Statement Example.
Ma’Arif, A. (2020). Programming Language Advanced Programming Textbook. https://eprints.uad.ac.id/32743/1/buku%20python.pdf
MediaPipe. (2023). September 13).Project Mediapipe. https://pypi.org/project/mediapipe/
Miftah, S. (2021). Mei 17).Python Library Get to know the differences between modules, packages and libraries in Python (A. W. Davita, Ed.). https://dqlab.id/library-python-kenali-perbedaan-module-package-dan-library-pada-python
Parashar, D., Mishra, O., Sharma, K., & Kukker, A. (2023). Improved Yoga Pose Detection Using MediaPipe and MoveNet in a Deep Learning Model. Revue d’Intelligence Artificielle, 37(5), 1197–1202. https://doi.org/10.18280/ria.370511
Patria, R. (2023). Studio Code Complete Definition, Features. Advantages! https://www.domainesia.com/berita/visual-studio-code/#Visual_Studio_Code_dengan_Code_Editor
Pradono, K. A., Safitri, Y. D., Adhiatma, B. S., Hestrio, Y. F., Soleh, M., Gunawan, H., & Sunarmodo, W. (2020). Development of Automatic Landsat Data Download Engine.IOP Conf. Materials Science and Engineering. https://doi.org/10.1088/1757-899X/1007/1/012109
Prihatiningsih, S., M, N. S., Andriani, F., & Nugraha, N. (2019). Performance Analysis of Handwritten Number Recognition Based on the Number of Iterations Using the Convolutional Neural Network. Method.Scientific Journal of Technology and Engineering, 58–66. https://ejournal.gunadarma.ac.id/index.php/tekno/article/download/1934/1644
Rahman, M. F., & Bambang. (2021). Trash Detection in Real-time Video using the Faster R-CNN. Method.Applied Technology and Computing Science Journal, 117–125. https://doi.org/10.33086/atcsj.v3i2.1846
Rasyid, M. F., Mustafa, M. S., & Suradi, A. A. (2022). Eye Detection in Smartphone Videos Using Mediapipe Python.JOINTECS (pp. 49–56). Journal of Information Technology and Computer Sciense. https://www.researchgate.net/publication/371968761_Eye
Romzi, M., & Kurniawan, B. (2020a). Implementation of Python programming using Visual Studio Code.Journal of Informatics and Computers, 1-9. https://jurnal.unmaha.ac.id/index.php/jik/article/view/11
Romzi, M., & Kurniawan, B. (2020b). Learning Python Programming with an Algorithmic Logic Approach.JTIM. Mahakarya Information Engineering Journal, 37–44. https://journal.unmaha.ac.id/index.php/jtim/article/view/6
Samaan, G. H., Wadie, A. R., Attia, A. K., Asaad, A. M., Kamel, A. E., Slim, S. O., Abdallah, M. S., & Cho, Y.-I. (2022). MediaPipe’s Landmarks with RNN for Dynamic Sign Language Recognition. Electronics (Switzerland), 11(19). https://doi.org/10.3390/electronics11193228
Subramanian, B., Olimov, B., Naik, S. M., Kim, S., Park, K.-H., & Kim, J. (2022). An integrated mediapipe-optimized GRU model for Indian sign language recognition. Scientific Reports, 12(1). https://doi.org/10.1038/s41598-022-15998-7
Suyudil, I., Sudadio, S., & Suherman, S. (2022). Introduction to Indonesian Sign Language using Mediapipe with Random Forest and Multinomial Logistic Regression Models.Jurnal Ilmu Siber dan Teknologi Digital (JISTED (pp. 65–80). https://doi.org/10.35912/jisted.v1i1.1899
Zulkhadi, T. C.-S., Maria, E., & Yulianto. (2019). Facial Shape Pattern Recognition with OpenCV.JURTI, 181-186. https://e-journals.unmul.ac.id/index.php/INF/article/view/4033
