Free and paid large language model performance evaluation in answering academic technical questions

Authors

  • Ilyas Nur Rokhim STMIK AMIKOM Surakarta, Indonesia
  • Fandy Dwi Putra STMIK AMIKOM Surakarta, Indonesia
  • Raditya Duta Pratama STMIK AMIKOM Surakarta, Indonesia
  • Sayudha Hanif Saputra STMIK AMIKOM Surakarta, Indonesia

Keywords:

Artificial Intelligence, ChatGPT, GPT-4o, Large Language Models

Abstract

The rapid advancement of Large Language Models (LLMs) has increased the adoption of artificial intelligence in higher education, particularly in supporting students with fundamental computer science tasks. Although ChatGPT is available in both Free (GPT-4o-mini) and Premium (GPT-4o) versions, empirical evidence comparing their performance in academic contexts remains limited. This study aims to compare the performance of both versions based on answer accuracy, information completeness, response clarity, and response time. A comparative experimental method was employed using ten computer science task scenarios covering programming, databases, and system visualization. The outputs were evaluated using a five-point Likert scale, while response time was measured quantitatively. The results indicate that both models achieved high answer accuracy. However, ChatGPT Premium produced more comprehensive, well-structured responses and demonstrated superior visualization capabilities. In contrast, ChatGPT Free delivered faster response times, making it more suitable for tasks requiring efficiency. These findings reveal a trade-off between response quality and processing speed. This study provides a comparative evaluation framework that can support students and educators in selecting the most appropriate ChatGPT service according to their academic needs.

References

[1] A. Alvin, R. Robet, and A. Tarigan, “Implementasi chatbot Otomatis Akademik Berbasis Web Menggunakan LLM dan Rule-Based System Studi Kasus : STMIK Time Implementation of Web-Based Academic Automated chatbot Using LLM and Rule-Based System Case Study : STMIK Time,” no. 3, pp. 651–665, 2025, doi: https://doi.org/10.26798/jiko.v9i3.2209.

[2] M. R. Kusuma and L. Ternado, “Evaluasi Kinerja Model LLM Terkuantisasi pada Tugas Bahasa Indonesia,” vol. 01, no. 01, pp. 25–38, 2025.

[3] C. K. Wigayha and U. Bunda, “Analisis Sistematis Penggunaan Large Language Models ( Llms ) Dan Artificial Intelligence ( Ai ) Untuk Peningkatan Literasi Digital Pada Jenjang Pendidikan Tinggi,” vol. 1, no. 1, pp. 1–7, 2025.

[4] A. S. Susanto and L. Lisana, “Jurnal Locus : Penelitian & Pengabdian Chat AI LLM ( Large Language Model ) di Universitas : Tinjauan Sistematis terhadap Variabel Pendorong Adopsi , Kerangka Teoritis, dan Tren Global,” vol. 4, no. 10, 2025. https://doi.org/10.58344/locus.v4i10.4324

[5] P. Studi, P. Jarak, J. Ilmu, U. P. Harapan, M. I. Komunikasi, and U. P. Harapan, “Penggunaan ChatGPT di Kalangan Mahasiswa dan Dosen Perguruan Tinggi Indonesia,” vol. 14, no. 1, pp. 130–145, 2024. https://doi.org/10.35814/coverage.v14i2.6058

[6] A. Yuanda et al., “Analisis Pemanfaatan Fitur Gratis Dan Premium Chatgpt Sebagai Materi Presentasi Mahasiswa Uinsu,” vol. 4, no. April, pp. 36–47, 2026.

[7] A. N. Annas, G. Wijayanto, D. Cahyono, and M. Safar, “Pelatihan Teknis Penggunaan Aplikasi Artificial Intelligences ( AI ) Chat Gpt Dan Bard AI Sebagai Alat Bantu Bagi Mahasiswa Dalam Mengerjakan Tugas Perkuliahan,” vol. 4, no. 1, pp. 332–340, 2024.

[8] M. B. Ibrahim, I. Sari, and L. Lubis, “Optimalisasi Riset Berbasis Artificial Intelligence dalam Meminimalisir Prokrastinasi Akademik Mahasiswa,” vol. 5, no. 2, pp. 100–112, 2025, doi: https://doi.org/10.59395/jitp.v5i2.127.

[9] S. Yin et al., “A survey on multimodal large language models,” no. October, 2024, doi: https://doi.org/10.1093/nsr/nwae403.

[10] K. Buatan, P. Personal, and S. D. Pembelajaran, “Education Journal : Journal Education Research and Development,” no. November 2022, pp. 158–166.

[11] S. Berbasis, K. Tpack, and D. A. N. Tam, “Generative ai,” vol. 11, 2026.

[12] A. J. Nabila, W. Pebrianti, and H. Setiawan, “Determinants of Continuance Intention Toward ChatGPT Premium Among Indonesian University Students : An Information Systems Success Model Perspective for Higher Education Management,” vol. 05, no. 02, pp. 1409–1427, 2026. https://doi.org/10.61987/jemr.v5i2.1892

[13] N. Luh, C. Pendiasari, F. D. Acarya, U. Hindu, N. I. Gusti, and B. Sugriwa, “Pengaruh Artificial Intelligence ( AI ) terhadap Efektivitas Proses Pembelajaran di Era Digital,” vol. 4, no. 1, pp. 1–11, 2026. https://doi.org/10.61292/cognoscere.302

[14] D. A. N. E. Program, “Jurnal Riset Ilmiah,” vol. 3, no. 1, pp. 67–76, 2026. https://doi.org/10.62335/sinergi.v3i1.2228

[15] I. Yunita, D. Rahmadiyanti, A. Aulia, N. Intan, D. Pandini, and I. Fauzi, “JPK : Jurnal Pendidikan dan Kebudayaan. Analisis Komparatif Laporan Penelitian Tindakan Kelas dan Penelitian JPK : Jurnal Pendidikan dan Kebudayaan,” vol. 03, no. 02, pp. 103–114, 2026. https://doi.org/10.56842/jpk.v3i2.1051

[16] A. Manan and U. Nasri, “Tantangan dan Peluang Pendidikan Bahasa Arab : Perspektif Global,” vol. 9, pp. 256–265, 2024. https://doi.org/10.29303/jipp.v9i1.2042

Downloads

Published

2026-08-31

Issue

Vol. 1 No. 3: August 2026

Section

Articles

License

Copyright (c) 2026 Machine Intelligence for Societal Advancement

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.