Open access peer-reviewed chapter - ONLINE FIRST
Software- and hardware-related parameters.
Abstract
Artificial intelligence technology is becoming increasingly mature and is being widely implemented and applied in academia and industry to solve practical problems. Due to the gradual growth and high development of AI, a large number of universities lack artificial intelligence literacy teaching for students and faculty members, especially the lack of educational innovation and exploration of algorithmic thinking and sentiment analysis for researchers. AI literacy is an emerging field that aims to equip individuals with the knowledge and skills to understand, interact with, and make informed decisions about AI technologies. This study aims to practice and reform the BERT-based model to develop an algorithmic thinking education innovation exploration curriculum and conduct sentiment analysis. To improve AI literacy and the efficiency of undergraduate students’ programming ability, reduce the burden of data retrieval work on students, and enhance the efficiency of students’ programming learning, this research has developed a series of algorithmic thinking courses based on the BERT model and aims to develop a knowledge graph-based question-answer system to enable undergraduate students to understand the operating rules and basic syntax of programming. The study finds that the system collects data from student and teacher portals and uses these data for sentiment analysis to optimize the system and allow researchers to efficiently obtain information for self-directed learning.
Keywords
- AI literacy
- algorithmic thinking
- sentiment analysis
- machine learning
- knowledge engineering
1. Introduction
In the higher education context, universities are encouraged to provide specialized AI workshops and broader computer science courses that include hands-on practice of AI tools, ethical discussions, and interdisciplinary applications. This research aims to practice and reform the algorithmic thinking education innovation exploration courses based on the BERT model. It aims to create a question and answer system based on the BERT model and collect relevant data from students and teachers to perform sentiment analysis on the system. According to research by scholars and university strategies, developing undergraduate AI literacy is necessary for the research and learning cycle [1]. The teaching team should provide practical and direct teaching, systematically improve the learning efficiency of researchers, and regard the artificial intelligence literacy of students and researchers as the goal of lifelong learning [2]. Teaching content could include ethical considerations for AI, or more workshops could be held to raise public awareness. In the study, a BERT-based system has been proposed and applied to practical teaching for undergraduates in Chinese-style education and UK-style education. This research also focuses on reforming a series of teaching activities for undergraduates, developing more algorithmic thinking courses, artificial intelligence literacy courses, and roundtable discussions. This study provides an in-depth exploration of approaches to improving undergraduate students’ AI literacy in higher education settings based on different teaching and research styles, applying the “classroom dialog” [3] as the basic idea and also as a guide for developing AI literacy for students and researchers.
The main research objectives of this study are as follows. The study will develop a series of algorithmic thinking innovation courses for undergraduate students based on the BERT model, aiming to improve artificial intelligence literacy from different schools and majors to better adapt to the digital age. Secondly, this study plans to design an algorithm model for knowledge base question and answer, divide the question and answer module into two subtasks, and propose two corresponding models (kmBert twin tower model and ken-KGQA model). In addition, this study aims to conduct experiments and analysis on a self-built question and answer data set in the programming field to test and confirm whether the above two models are effective compared to the classic model. Finally, this research aims to design a knowledge question and answer system to improve undergraduates’ artificial intelligence literacy and programming learning efficiency, collect relevant data from teachers and students for sentiment analysis, and optimize the system. The system allows students and researchers to ask questions to the system independently when they encounter programming problems, query similar information that occurred in history, and obtain corresponding answers. It improves the independent learning efficiency and enthusiasm of undergraduates, thereby achieving the goal of popularizing and cultivating undergraduates’ artificial intelligence literacy.
2. Related work
AI literacy has become more indispensable in all aspects of life and work with the continuous development of AI technology, and AI literacy has gradually transformed into an increasingly important ability in the field of education. AI-driven tools such as BERT-based question-answering systems provide opportunities to achieve the goal of lifelong learning and improve students’ proficiency in learning effectiveness. This study aims to explore how BERT-based systems can be used to improve the AI literacy of undergraduate students. Some scholars believe that BERT from Transformers has revolutionized the approach to natural language processing (NLP) tasks. BERT models are pre-trained on a large corpus of text and then fine-tuned for specific tasks. In question-answering systems, BERTs excel at understanding the context of queries and deriving the most relevant answers from a corpus of text [4]. A significant advantage of BERT-based question-answering systems is their ability to provide a precise answer to the question by understanding subtle differences in the meaning of words due to context. This feature makes BERT useful as an advanced educational tool and helps to help undergraduates understand complex material in greater depth.
This study aims to propose a new BERT model to improve AI literacy and conduct sentiment analysis among undergraduate students. By interacting with a BERT-based question-answering system, learners can actively participate in AI technology, demystifying how it works and what it can do. Some scholars believe that using AI systems for inquiry-based, hands-on learning can help deepen understanding and memory. The use of a question and answer system can enhance critical thinking and problem-solving skills to a certain extent. For example, the learner asks a question that leads to the best answer. It encourages them to think critically about the structure of questions and answers and how AI interprets queries, leading to better problem-solving skills [5]. In addition, BERT-based systems can be tailored to the learner’s level of knowledge, providing a more personalized educational experience. By analyzing the responses to questions, the system is adapted to provide more targeted educational content that appropriately challenges learners [6]. The system serves as a guide to lifelong learning goals. In the context of lifelong learning, the system can collect the learners’ feedbacks over a period of time to help learners develop better search and prompting skills, which are essential for effective information literacy. Users learn to navigate through large amounts of information quickly and efficiently, identifying high-quality sources [7]. As for academic use, the system can accelerate the development of abstract terms, concepts, or theories by quickly and efficiently extracting relevant information and summarizing findings from large paper datasets. This ability strengthens the learning memory of students and researchers, not only saving time but also enhancing the breadth and depth of reviewable meaning [8]. The system collects direct feedback from both the teacher and the students’ sides to construct the model of sentiment analysis [9].
2.1 Named entity recognition
With the rise of machine learning in recent years, statistical machine learning-based methods have gradually replaced dictionary- and rule-based methods. Different from the previous method, which has poor portability and high cost, the method based on statistical machine learning is to manually label the corpus and then train the annotated corpus to obtain a language model suitable for this task. The annotated corpus does not need to have rich knowledge, the cost is not high, and it only needs to replace the training corpus when transplanting to a new field, so it is portable. There are four main named entity recognition methods based on statistical machine learning: Support Vector Machine (SVM), Hidden Markov Model (HMM), Maximum Entropy (ME), and Conditional Random Fields (CRF) [10]. Among these methods, ME is compact and versatile, but it is expensive due to the high complexity of the training time. SVM should be used as a supervised learning method to effectively handle the classification of high-dimensional feature spaces and to solve the problem of small samples by relying only on a small number of samples when making decisions.
2.2 Pre-trained language models
BERT is composed of a multi-layer Transformer Encoder, which adopts the attention mechanism [11], which abandons the traditional RNN and CNN and converts the distance between two words at any position into 1 through the attention mechanism, which effectively solves the thorny long-term dependency problem in NLP. BERT jointly adjusts the context in all layers to train a depth bidirectional representation, which can be fine-tuned with an additional output layer. The success of the model is mainly due to the two training methods it proposes: MLM (Masked Language Model) and NSP (Next Sentence Prediction), which can help BERT learn vector representations of words from the vast linguistic corpus available. The notation learned by BERT has been shown to be well generalized to downstream tasks.
3. Methodology
3.1 Theoretical framework
This study applies the Dialog Framework [3] as a teaching method for the curriculum design of the AI literacy workshop and a basic principle for engineering pedagogy. The framework is often applied to the design or evaluation of instructional activities, particularly in advanced technology-enhanced learning environments. The framework serves as a tool to guide the development of teaching and learning processes, emphasizing the importance of dialog (interaction between students and teachers) and interaction in knowledge construction. It analyzes the different stages of the learning process. Each stage aims to connect the dialogs in order to bridge the gap between theory and practice. It shares a feedback loop throughout the learning process to enhance undergraduates’ understanding of subject knowledge, and teachers can use feedback to refine their teaching methods and fill gaps in student understanding. The framework provides students with the role of active learners. In order to strengthen teaching, some of the teaching courses will be relocated to the Wonder Lab. Each workshop consists of 60 minutes of lectures and 30 minutes of exercises, with small class sizes in practice and large classes in the teaching of the basics, and instruct students to write and publish in engineering pedagogy-related journals and conferences every year and attend some of the top conferences in the field of AI literacy so that they can keep up with the latest research trends.
3.2 Rationale
The widespread adoption of AI is reshaping the academic landscape of research and learning, transforming them from traditional to innovative approaches. This shift has had a significant impact on learning and scientific inquiry methods in higher education institutions. More and more universities around the world are involved in AI research. In order to improve researchers’ AI literacy and use AI technology to transform academic support, Github has established and designed an AI tutorial website to cultivate researchers’ AI fundamentals, including AI coding assistants and BERT-based systems, designed to help researchers start self-directed learning.
Since AI is a multidisciplinary field of study that requires a variety of skills, it is possible to establish cross-university and interdisciplinary AI working groups and conduct more AI literacy workshops or digital job visits so that academics, administrators, and students from different backgrounds can discuss and progress together.
3.3 Key research problems
How to make good use of existing resources to conduct the sentiment analysis, integrate the BERT model into the existing digital curriculum for undergraduates, and cultivate their algorithmic thinking?
What is the performance of the proposed system?
To what extent can the system improve the AI literacy or programming learning efficiency of undergraduate students?
Based on the construction of a data collaborative representation model in complex data space, the vertical island problem of data knowledge is solved, and the knowledge model of multi-source heterogeneous data is constructed.
Realize the extraction, association, and integration from local knowledge to global knowledge.
The deep learning algorithm is used to solve the situation recognition of complex product collaborative workflows, and the adaptive service recommendation in complex scenarios is realized through the service recommendation algorithm.
Research on collaborative decision-making methods and construct collaborative interactive decision-making models based on BERT.
4. Overall pipeline
The dictionary and rule-based method is an early method used in the named entity recognition task, which often requires experts to construct a rule template in the corresponding field, select specific features including position words, central words, direction words, and keywords, and make corresponding matches according to the specified matching rules and entity dictionaries to identify entities. Dictionary- and rule-based methods have a better recognition effect than machine learning-based methods when the rules and dictionaries are more detailed and accurate, but the formulation of these rules often requires domain experts to make corresponding control of domain knowledge and text style, and the compilation process is time-consuming and costly, and when the language style and domain involved are different, the method performance will be greatly reduced. Therefore, this method is not only time-consuming and labor-intensive but also has poor portability and a low recognition rate of new words. This model solves the problem that LSTM only obtains information in one direction and can obtain forward and reverse features at the same time, making the acquisition of contextual information more complete. However, the above models require a large number of datasets and have requirements for dataset quality, so people have begun to think about how to obtain a priori semantic information from a large number of unlabeled texts and apply it to subsequent training. Based on this idea, the Transformer model was born. Since then, a variety of pre-trained models derived from Transformer have emerged, and BERT is one of them (Figure 1).
Figure 1.
Overall working principle of the BERT model.
4.1 Sentiment analysis
The system review data serves as learners’ direct feedback on the system experience and contains rich information value. The study uses Python crawler technology to collect comment data, stores the collected data results in the SQLite database, uses a Python third-party library to process and analyze the comment data, and uses Python’s Flask framework to build a Web application to achieve dynamic visual display of comment data. Through in-depth mining of student and teacher comment data, we can understand students’ evaluations of different knowledge points, emotional tendencies, popular questions, popular knowledge points, etc., providing a scientific basis for the optimization of the question and answer system and public opinion monitoring. In terms of data visualization (Figure 2), the systematic review data analysis system uses a variety of chart forms, such as bar charts, pie charts, and word cloud charts, to intuitively display the analysis results. Through these charts, students and teachers can easily understand the distribution of popular issues and knowledge point data, the trend changes of new knowledge points, and other information. In this study, students’ and teachers’ feedbacks were collected as the data, and some typical algorithms (e.g., KNN Classifier, Naive Bayes, and SVM Classifier) were imported and applied in the experimental process. Feature vectorization and the Naive Bayes classification are applied during the data dealing process, and the accuracy of the model during training is around 83%.
Figure 2.
Key aspects of students’ feedbacks.
4.2 Experimental setup and result analysis
The entity recognition experiment in this article is constructed using the PyTorch deep learning framework and the Python 3.6.5 programming language. The specific software- and hardware-related parameters are shown in Table 1.
| Software and hardware | Related parameters |
|---|---|
| GPU | NVIDIA 3080TI 32 GB |
| RAM | 32 GB |
| Deep learning framework | PyTorch |
| Programming language | Python 3.6.5 |
Table 1.
The settings of hyperparameters have a great impact on the performance of the model. The hyperparameter settings of our model are as follows. For the knowledge graph embedding stage, the pre-trained model loaded by BERT is Chinese-BERT-base, and the output dimension is 770. Dropout processing is performed on BERT’s parameter learning, and the dropout probability is set to 29%. The dimension of the embedding vector output after passing through a fully connected layer is 66 dimensions. The Adam optimizer with a learning rate of 0.0001 is used for optimization. When the batch size is set to 130, the training epoch is 50 rounds. The specific parameter settings of the knowledge graph embedding model are shown in Table 2.
| Name | Related parameters |
|---|---|
| BERT pre-training model | Chinese-BERT-base |
| BERT maximum input sequence length | 50 |
| BERT output dimension | 770 |
| BERT dropout rate | 0.29 |
| Fusion layer output dimension | 66 |
| Learning rate | 1e − 4 |
| Optimizer | Adam |
| Batch size | 130 |
| Epoch | 50 |
Table 2.
Related parameters knowledge graph embedding model (former).
In the text encoding stage, the dimension of the output vector of each character in the BERT model is 770, the pre-trained model loaded by BERT is Chinese-BERT-base, and the dropout rate is set to 50%. The dimension of the embedding vector output after passing through the fusion layer is still 64 dimensions. In the label encoding stage, the BERT model used by the label encoder and the BERT model used by the text encoder are the same model and share parameters. The embedding vector of the label is the 768-dimensional vector output by [CLS] at the top level of the BERT model. The dimension of the embedding vector output after passing through the fusion layer is still 66 dimensions. Using the Adam optimizer with a learning rate of 0.00005, the training phase of named entity recognition was trained for 60 epochs with a batch size of 65. The specific parameter settings of the model are shown in Table 3.
| Name | Related parameters |
|---|---|
| BERT pre-training model | Chinese-BERT-base |
| BERT maximum input sequence length | 50 |
| BERT output dimension | 770 |
| BERT dropout rate | 0.5 |
| Fusion layer output dimension | 66 |
| Learning rate | 5e − 5 |
| Optimizer | Adam |
| Batch size | 65 |
| Epoch | 60 |
Table 3.
Related parameters knowledge graph embedding model (later).
4.3 Result analysis
For the model in this article, we conducted experiments on a self-built data set. We conducted comparative experiments between the model in this article and several classic models in the field of named entity recognition to verify the effectiveness of the model: (1) the LSTM-CRF model. The embedding layer uses pre-trained Word2Vec for word vector embedding, and then LSTM-CRF performs entity extraction. (2) The Bi-LSTM-CRF model is a classic model in the NER field. It uses pre-trained Word2Vec as the output of the embedding layer and then inputs it into the Bi-LSTM-CRF network for encoding and predicting labels. (3) BERT-CRF model, using the BERT pre-trained language model for word vector embedding and then decoding by the CRF layer to calculate the optimal annotation sequence.
The results are shown in Table 4. From the experimental results of each model, it can be found that the accuracy, recall rate, and F1 value of the model used in this article all achieve the best results compared with the other three comparison models. The F1 value of the model in this article reached 95.9%, while the F1 values of the other three baseline models, LSTM-CRF, Bi-LSTM-CRF, and BERT-CRF, were 82.6%, 85.1%, and 95.4%, respectively. It can be seen that the recognition effects of the first two models are far behind BERT-CRF. The reason may be that the BERT pre-trained language model can better learn the universal features of language and has better generalization effects. The recognition effect of the KLBERT model in this article is slightly better than the BERT-CRF model. The reason may be that the KLBERT model incorporates lexical information, learns more language features, and produces better performance. As shown in the table, our model performs better than other models, which proves the validity of the idea in this paper.
| Model | Precision | Recall | F1-score |
|---|---|---|---|
| LSTM-CRF Bi-LSTM-CRF BERT-CRF | 85.1 87.2 95.6 | 80.3 83.2 95.2 | 82.6 85.1 95.4 |
| KLBERT (our) KLBERT-t KLBERT-l | 96.1 94.8 95.2 | 95.8 94.3 94.7 | 95.9 94.5 94.9 |
Table 4.
Results on the students’ dataset.
To further analyze our model, this paper conducts ablation experiments to systematically study the impact and contribution of different components. KLBERT-t and KLBERT-l are variants of our full model KEm-KGQA. Here we briefly introduce these two model variants.
KLBERT-t: In the label encoding stage, the label is not expanded or converted into a description.
KLBERT-l: In the text encoding stage and label encoding stage, the encoding of text and labels does not fuse lexical information. After the text and labels pass through the BERT model, they no longer enter the fusion layer but directly perform similarity calculations.
The results of the ablation experiment are shown in Table 4. Based on the comparison of results between the model in this paper and its variants, we can clearly conclude that the semantic expansion of labels and the incorporation of lexical information have an impact on the results. The comparison between KLBERT-t and our complete model shows that if the labels are semantically expanded during the label encoding stage, the F1 value of the named entity recognition model can be improved by 1.4%. This shows that the semantic expansion of tags plays a role in named entity recognition. When our full model is compared with KLBERT-l, we can see that the F1 value of the named entity recognition model can be improved by 1.0% if the corresponding lexical information is incorporated into the encoding of text and tags, which proves the importance of incorporating lexical information. Within the 2 years, the system witnessed nearly 1000 students’ growth. Figure 3 shows the students’ satisfaction rate in three terms. It demonstrates that the satisfaction rate has successfully increased by 29% compared to before, and convinced of the effectiveness to some degree.
Figure 3.
Satisfaction rate of students.
5. Future development
The dictionary and rule-based method is an early method used in the named entity recognition task, which often requires experts to construct a rule template in the corresponding field, select specific features including position words, central words, direction words, and keywords, and make corresponding matches according to the specified matching rules and entity dictionaries to identify entities. Dictionary- and rule-based methods have a better recognition effect than machine learning-based methods when the rules and dictionaries are used. This study conducts in-depth research on knowledge graphs and question and answer systems based on the field of higher education. With the advent of the era of artificial intelligence and big data, the demand for question and answer systems based on knowledge graphs will only continue to increase. Although the current research on question and answer systems combined with knowledge graphs is still in its preliminary stages, with the increase in market demand, research progress will inevitably advance by leaps and bounds, and the related technologies of question and answer systems combined with knowledge graphs will also continue to innovate. The question and answer model proposed in this article can currently only answer simple questions; it is still difficult to solve complex relational questions.
Future work can be carried out from the following aspects:
Improve the system’s ability to understand complex relational questions. With the development of the times, the amount of data will inevitably skyrocket in the future, and the relationship between information will become more complex.
Therefore, research on the understanding of complex relationship questions is imperative.
Complete the knowledge graph in the field of computer science in colleges and universities. At present, the knowledge graph mainly focuses on students’ basic knowledge questions and answers, and running event information. For questions that are beyond the scope of the knowledge graph, the system will be overwhelmed. In the future, we can integrate more information into the knowledge graph, develop an automatic data collection function, regularly search for data in open source websites and databases, perform automatic collection and processing, and import it into the knowledge graph.
Enrich the functions of the question and answer system. Most of the question and answer systems currently on the market have some recommendation function. Therefore, we can also make personalized recommendations based on the historical data of user operations. In addition, we can provide voice recognition and image recognition functions, and users can also recognize and answer input voice.
6. Conclusion
This study designs and implements a question and answer system in the field of higher education with the purpose of improving the collaboration and interactivity of teachers and students in universities. This system can be used to ask basic information about students’ programming abilities as well as information about abnormal events in the system’s operation. This study introduces the overall architecture of the question and answer system in universities and the design and implementation of each functional module. The algorithm model of the question and answer system is integrated through the Python back-end framework Flask; the front-end display is completed using the React front-end language and front-end frameworks such as Echarts and Ant Design to implement functional modules such as entity search, entity recognition, and question and answer in the field of university computer science. The question and answer system is based on the knowledge graph in the information field of universities constructed in this article, and its back-end algorithm model is processed by the model in this article. The system will collect basic information from teachers and students during use for sentiment analysis, thereby optimizing existing scenarios to a certain extent and allowing students and teachers to understand the system’s operating information in real time. When encountering problematic events, students and teachers can ask questions to the Q&A system and query information about similar operating events that have occurred in history, including basic computer knowledge, operating status, event resolution measures, etc. The information can be used for self-study and to improve the programming efficiency of students and teachers.
Acknowledgments
We would like to thank the anonymous reviewers for their constructive suggestions and feedback. This research is supported by the research project JGBA2024536, in part by a grant from the Zhejiang Provincial Department of Education.
References
- 1.
Southworth J, Migliaccio K, Glover J, Reed D, McCarty C, Brendemuhl J, et al. Developing a model for AI across the curriculum: Transforming the higher education landscape via innovation in AI literacy. Computers and Education: Artificial Intelligence. 2023; 4 :100127 - 2.
Alamäki A, Nyberg C, Kimberley A, Salonen AO. Artificial intelligence literacy in sustainable development: A learning experiment in higher education. Frontiers in Education. 2024; 9 :1343406 - 3.
Gregory MR. A framework for facilitating classroom dialogue. Teaching Philosophy. 2007; 30 (1):59-84 - 4.
Sultani M, Daneshpour N. Extracting urgent questions from MOOC discussions: A BERT-based multi-output classification approach. Arabian Journal for Science and Engineering. 2024; 50 (2):1169-1190 - 5.
Xu Y, Shieh CH, van Esch P, Ling IL. AI customer service: Task complexity, problem-solving ability, and usage intention. Australasian Marketing Journal. 2020; 28 (4):189-199 - 6.
Suryanarayana KS, Kandi VP, Pavani G, Rao AS, Rout S, Krishna TSR. Artificial intelligence enhanced digital learning for the sustainability of education management system. The Journal of High Technology Management Research. 2024; 35 (2):100495 - 7.
Zheng J, Qiu S, Shi C, Ma Q. Towards lifelong learning of large language models: A survey. ACM Computing Surveys. 2024; 57 (8):1-35 - 8.
Rani N, Chu SL, Williamson YG. Supporting lifelong learning. Educational Technology & Society. 2024; 27 (2):256-269 - 9.
Yao L, Zheng N. Sentiment analysis based on improved transformer model and conditional random fields. IEEE Access. 2024; 12 (1):90145-90157 - 10.
Pradhan A, Yajnik A. Parts-of-speech tagging of Nepali texts with bidirectional LSTM, conditional random fields and HMM. Multimedia Tools and Applications. 2024; 83 (4):9893-9909 - 11.
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Advances in Neural Information Processing Systems. 2017; 30 (1):1-15