Developing an electronic module based on mathematical literacy to enhance students' mathematical reasoning

This study aims to produce an e-module based on mathematical literacy with a valid, effective, and feasible problem-based learning approach to improve students' mathematical reasoning in multivariable calculus material. This research used a development method with the stages: analysis, design, development, and evaluation. The research subjects involved fifth-semester students' mathematics education department. Small group trials with 17 students and large group trials with 35 students. The research instruments included validation sheets, student worksheets, questionnaires, and mathematical reasoning ability test questions. Data collection used the results of the validation as validity, test results as effectiveness, and the results of the questionnaire as feasibility. Data analysis was descriptively and qualitatively. The research produced an e-module based on mathematical literacy with a problem-based learning approach that is valid, very effective, and very feasible for improving students' mathematical reasoning abilities in multivariable calculus material. The developed e-module has the potential for significant differences and increases mathematical reasoning abilities. In addition, e-module can be used as an alternative solution to electronic teaching.


Introduction
teaching material. Chen et al. (2015) and Letchumanan and Tarmizi (2010) stated that technological developments need to be developed in electronic teaching materials and online learning so that students from anywhere can attend the lecture. The advantages of online learning are that students can study anywhere and anytime with an internet connection, and students are more interested in using e-books to help complete their assignments (Fernández et al., 2020;Owston et al., 2008). In addition, various information on the internet which aims to improve the teaching and learning of mathematics is continuous (Bozkurt & Ruthven, 2017;Mailizar & Fan, 2020).
The research results by Keengwe and Georgina (2012) stated that technological developments change teaching and learning implementation. In this regard, an e-module development innovation was carried out on multivariable calculus. The developed problembased learning and mathematical literacy e-modules with multivariable calculus material use sigil applications. Sigil is a means of electronic books that can be accessed through Electronic Publications (ePub) (Park et al., 2019). Sigil is an application that has almost the same function as a word in word processing, but the text is digital in an e-pub format that can be used and accessed at any time using a computer, laptop, or smartphone.
Several supporting studies, such as research by Brew and Saunders (2020) and Tan (2003), show that problem-based learning contributes to developing higher-order thinking skills, including mathematical reasoning abilities. The research results by Napitupulu et al. (2016) show that problem-based learning impacts increasing mathematical reasoning abilities. In addition, Genc and Erbas (2019) and Pugalee (1999) research results show that mathematical literacy can improve higher-order mathematical thinking skills, including mathematical reasoning abilities. Aulia and Prahmana (2022) and Nursakiah et al. (2022) show that e-module with mathematical literacy can improve students' abilities.
However, the novelty of this research needs to be studied more on mathematical literacy with the problem-based learning approach to foster mathematical reasoning abilities in calculus material. In addition, no one has linked it to the use of teaching materials in the form of electronic modules with e-Pub. It is a gap for researching to develop an e-module based on mathematical literacy with a problem-based learning approach that is valid, effective, and feasible to improve students' mathematical reasoning in multivariable calculus material. In addition, the urgency of the research is related to the importance of conducting studies focused on problem-based learning with mathematical literacy and mathematical reasoning abilities. It is following digital developments in education with independent learning, which focuses on cognitive abilities that are individually unique to students with 4C of 21st-century abilities, one of which is mathematical reasoning ability. This research is expected to contribute as an alternative solution for teaching materials to improve mathematical reasoning abilities.

Methods
This research used the development design of Richey et al. (2011): analysis, design, development, and evaluation. The development model of Dick et al. (2009) includes identifying the learning objectives, analysis of learning, implementation time, developing assessment instruments, developing learning strategies, developing and selecting materials, revising learning materials, and designing and carrying out evaluations.
In the analysis stage, researchers did two steps. The first step was identifying, and the second was learning analysis on multivariable calculus material. In addition, in the design stage, researchers determined the implementation time. The research was conducted at PGRI Kanjuruhan University in Malang, East Java Province, during the even semester of the 2021/2022 academic year. The subjects for the research were fifth-semester students, with the small group product test of 17 students and the large group product test with a total of 35 students. Then, the fourth step is to compile instruments which include material expert validation sheets, learning and design, student worksheets, questionnaires as responses from both lecturers and students, and test questions to measure mathematical reasoning abilities (pretest and post-test questions). Next, the fifth step is designing the product, product specifications, and e-content structures module.
In the development stage, researchers developed and selected materials with developing e-module components, producing e-module drafts, testing e-modules, and producing e-module drafts with final revisions. E-module developed in the multivariable calculus course. Next is the seventh step by developing materials and conducting validation, then the eighth step by designing valid, effective, and feasible products for the developed e-module. The e-module was based on mathematical literacy with a problem-based learning approach, which can be seen in Table 1. Developed e-module based on mathematical literacy with problem-based learning also utilizes technology using the sigil application. Converting module files in word to module files uses the sigil application, with the first step using word processing, and then using sigil with file formats were HTML and ePub. The developed e-module can be accessed via a computer, laptop, or smartphone.
The developed e-module product was validated by a validator which includes material assessment by material experts, learning assessment by learning experts, product design assessment by design experts, and research instrument validation assessment by research instrument experts. Qualifications for validation are shown in Table 2. In the evaluation stage, researchers evaluated and analyzed data on the effectiveness and feasibility of e-modules based on mathematical literacy with a problem-based learning approach on multivariable calculus material. After the developed e-module is declared valid, it is tested on small and large groups. Then, students' mathematical reasoning tests were carried out, and questionnaires were distributed. The mathematical reasoning test is in the form of three description questions on multivariable calculus material and a questionnaire in the form of 15 questions for responses to the e-module being developed. Mathematical reasoning ability test results with the indicators shown in Table 3. Table 3. Mathematical reasoning Indicator Sub Indicator

Making inferences in problems
Identifying problems Mentioning the question in the problem Providing mathematical manipulation Determining steps/strategies for solving problems Giving an argument or reason as the truth of the answer/solution Using the steps/strategy set out to solve the problem Providing an explanation of the relationship between the steps/strategies used in solving problems Drawing conclusions/generalizing Obtaining answers/completion of steps/strategies that have been applied in solving problems Provide conclusions from the answers / solutions that have been obtained in solving problems The data collection was the reference e-module in revising and analysis to achieve validity, effectiveness, and feasibility. Validation results were used for validity, test results were used for effectiveness, and questionnaire response results were used for eligibility. Then, the data from the pretest and post-test assessments were used to measure the potential for increasing students' mathematical reasoning abilities, which were analyzed based on the results of the T-Test and N-Gain Test using IBM SPSS statistics version 24. The T-test in this study used the Paired Sample T-Test. Students' mathematical reasoning abilities are said to increase if the value in the Paired T-Test table is less than 0.05, so Ho is rejected, which means that there is a significant difference between the pretest and post-test scores (Morgan et al., 2019). In addition, improving mathematical reasoning ability uses N-Gain with a minimum N-Gain of 0.3 to a maximum of 0.7 with moderate qualifications (Riyanto & Hatmawan, 2020).

Results
This research has four stages, namely analysis, design, development, and evaluation.

Analysis
The analysis results show that the curriculum used was the independent learning curriculum. In addition, students' mathematical reasoning abilities still need to be improved, and the teaching materials used were still teaching materials with face-to-face learning, yet to be using electronic teaching materials. Therefore, teaching materials in the form of e-modules were needed, which can be accessed anywhere via computers, laptops, or smartphones to facilitate the learning process, especially in multivariable calculus material.

Design
The results at the analysis stage determined that the research involved fifth-semester students in the mathematics education department as research subjects, with a small group product test of 17 students and a large group product test with a total of 35 students. Furthermore, designing e-modules based on mathematical literacy with a problem-based learning approach focused on mathematical reasoning abilities and material developed by multivariable calculus. The emodule design developed has specifications, namely material systematics with simple and clear sentences, learning objectives, student worksheets, mathematical reasoning questions, and learning outcomes according to the self-learning curriculum.
In addition, it also designs research instruments which include material expert validation sheets, learning and design, student worksheets, questionnaires as responses from both lecturers and students, and test questions to measure mathematical reasoning abilities (pretest and posttest questions). The mathematical reasoning test instruments for the pretest and post-test with three description questions on the multivariable calculus material. While the questionnaire instrument with 15 questions related to responses to e-modules based on mathematical literacy with a problem-based learning approach on multivariable calculus material.

Development
The development stage was carried out by validating the developed e-module based on mathematical literacy with a problem-based learning approach. The validator's suggestions for improvements to the material with examples and exercises that need attention so that the questions are easier to understand, and the appearance of the e-module is less attractive in explaining formulas/concepts. Researchers revise the e-module according to the suggestions given by the validator. The results of the revised e-module were re-validated and declared valid by the validator. In addition, research instruments that have been validated are declared valid.
In addition, the validator's suggestions for improving the instrument of the mathematical reasoning test on the pretest and post-test questions, include correcting the language of the questions, so there is no confusion with the questions. The validator's suggestion for improving the lecturer and student questionnaires needs to pay attention to the sentences in the questionnaire statements because there are statements that have the same meaning. The researcher revised the pretest and post-test mathematical reasoning test instruments and the questionnaire according to the suggestions the validator gave. The revision results were revalidated and declared valid by the validator.
The validated product for the e-module based on mathematical literacy with a problembased learning approach shows that it is valid. The validation results of the e-module are in Table 4. The validation results of validated student worksheets, mathematical reasoning tests, and student and lecturer response questionnaires are in Table 5. The results of the e-module product validation include material was 77%, learning was 73%, and product design was 75%. It shows that the e-module based on mathematical literacy with Problem-Based Learning approach is valid and can be applied to research. In addition, the validation results of research instruments, including student worksheets, were 77%, mathematical reasoning tests were 73%, student response questionnaires were 75%, and lecturers were 75%. It shows that t the research instrument is valid. The results of e-modules developed in ePub format and accessible with smartphones can be seen in Figure 1.

Evaluation
The results at the evaluation stage are the results of the e-module trials in small and large groups. In the small group trial, the first meeting gave pretest questions. The second meeting was with learning process activities using e-modules that were developed and valid, then given a posttest of mathematical reasoning abilities and questionnaires as student and lecturer responses to the e-module. The results of the trial of the e-module product developed in a small group of 17 mathematics education students are shown in Table 6. The results of the small group trials showed that the results of the mathematical reasoning test were 63.37%, which is likely effective. The questionnaire on the e-module obtained the results of a questionnaire response for students of 74.25% and lecturers of 73.37%. The average result of achieving a response questionnaire to the e-module is 73.81% which is feasible. The results of testing e-module products in small groups show that e-modules based on mathematical literacy with a problem-based learning approach can be declared valid, effective, and feasible.
In the large group trial, the first meeting provided pretest questions. The second meeting was with learning process activities using e-modules that were developed and valid, then given a post-test of mathematical reasoning abilities and questionnaires as student and lecturer responses to the e-module. The trial results of the e-module product developed in a large group of 35 mathematics education students are shown in Table 7. The results of the large group trials showed that the results of the mathematical reasoning test at 81.77% are very effective. The questionnaire on the e-module obtained the results of a questionnaire response for students of 83.55% and lecturers of 82.25%. The average result of the questionnaire response to the e-module is 82.9% which is very feasible. The results of testing e-module products in large groups show that e-modules based on mathematical literacy with a problem-based learning approach can be declared valid, very effective, and very feasible to apply to learning activities, especially in multivariable calculus material.
In addition, an analysis of students' mathematical reasoning abilities was also carried out from the results of the pretest and post-test scores using the T-Test and N-Gain Test. T-Test with Paired Sample T-Test using IBM SPSS statistics version 24 with a significance level of 0.05 and normality requirements using Shapiro-Wilk. The normality test results on the pretest of mathematical reasoning ability obtained a significance value of 0.164 (more than 0.05), and the post-test of mathematical reasoning ability obtained a significance value of 0.297 (more than 0.05) with a normal distribution. Next, a T-Test was carried out with the Paired Sample T-Test with the results of the pretest and post-test of mathematical reasoning ability with a significance of 0.000 (less than 0.05), which means that Ho is rejected (Morgan et al., 2019). It shows a significant difference between the pretest and post-test of mathematical reasoning abilities Analysis of studewere' mathematical reasoning abilities on the results of the pretest and post-test scores was also carried out based on the N-Gain test. The results of the analysis of students' mathematical reasoning abilities with the N-Gain test with an average of 0.39 is moderate qualifications (Riyanto & Hatmawan, 2020). It shows an increase of 39% in students' mathematical reasoning abilities. The results of the T-Test and N-Gain tests on the pretest and post-test on students' mathematical reasoning abilities show that an e-module based on mathematical literacy with a problem-based learning approach can improve mathematical reasoning abilities in multivariable calculus material.

Discussion
The development of an e-module based on mathematical literacy with the problem-based learning approach to improve students' mathematical reasoning abilities was carried out in four stages of development, including analysis, design, development, and evaluation. The analysis results show that students need electronic teaching materials like e-modules to provide learning motivation and assistance in reasoning in multivariable calculus so that they can answer questions correctly. It shows that it is necessary to innovate teaching materials like e-modules to improve students' mathematical reasoning and academic abilities. It is in line with the research by Tezer et al. (2019), who explained that prospective teachers significantly increased their success in teaching with online learning. It also agrees with Keengwe and Georgina (2012) and Mailizar and Fan (2020), that explain that technology, with its development, provides innovative changes to the implementation of learning.
The results of the development with the validation results of the e-module product on problem-based learning with mathematical literacy obtained an average validity of 75%, with valid qualifications, and the results of research instrument validation obtained an average validity of 75% with valid qualifications. In addition, the evaluation results on the large group mathematical reasoning test obtained the effectiveness of the e-module in multivariable calculus material at 81.77% with a very effective qualification. The questionnaire results in response to student and lecturer responses to the e-module obtained an average of 82.9% with very feasible qualifications. It is in line with Mooij et al. (2020) that responses in a course are indirectly related to changes in mathematical mindset and confidence. It shows that e-modules based on mathematical literacy with a problem-based learning approach in many calculus materials can be declared valid, very effective, and very feasible. It is in line with Choi and Walters (2018) that online learning with e-module can improve math performance. Mamolo (2022) argue that the type of online learning tools is very high depending on network connectivity.
In addition, the results of the mathematical literacy-based e-module with the developed problem-based learning approach show that discussing student worksheets using internet technology can shape student interaction and develop mindsets. It is in line with the opinion of Juandi and Tamur (2021) and Junianto and Wijaya (2019) that problem-based learning with mathematical literacy helps motivate students to develop a mindset. Çetinkaya (2019) and Juandi and Tamur (2021) argue that problem-based learning with mathematical literacy combines learning with technology.
Furthermore, the results of the analysis of pretest and post-test scores on students' mathematical reasoning abilities used the T-Test and N-Gain Test. The results of the T-test with the Paired Sample T-Test for the pretest and post-test of mathematical reasoning ability obtained a significance of 0.000 (less than 0.05), which means Ho is rejected. It shows a significant difference between the pretest and post-test of mathematical reasoning ability. The results of the N-Gain test for the pretest and post-test of mathematical reasoning ability obtained an average of 0.39 with moderate qualifications. It shows an increase of 39% in students' mathematical reasoning abilities. It is in line with the opinion of Kalobo (2016), Norvaiša (2021), and Rizki and Priatna (2019) that learning with mathematical literacy can improve students' mathematical reasoning. The research results support that literacy skills have the potential to improve mathematical reasoning abilities. The e-module product developed as an alternative solution to electronic teaching materials is the form of e-modules.

Conclusion
The research produced an e-module based on mathematical literacy with a problem-based learning approach that is valid, very effective, and very feasible for improving students' mathematical reasoning abilities in multivariable calculus material. The results showed that the developed e-module was declared valid based on the assessment of the material, learning and design validators, as well as suggestions for improvement from validators with valid qualifications. In addition, the developed e-module was stated to be very effective based on the results of mathematical reasoning ability tests in trials with very effective qualifications. The developed e-module was also declared very feasible based on the results of a questionnaire on the responses of students and lecturers to the e-module in trials with very plausible qualifications.
Furthermore, the developed e-module has the potential for significant differences in students' mathematical reasoning abilities based on the results of the pretest and post-test, after using the e-module. Mathematical reasoning ability after using the e-module has increased by 39% with moderate qualifications. Thus, the e-module based on mathematical literacy with a problem-based learning approach to multivariable calculus material can improve students' mathematical reasoning abilities. In addition, it is recommended for smooth internet signal readiness when implementing teaching materials in the form of an e-module. It is because the implementation of the e-module requires a stable signal.

Conflicts of Interest
The authors state that there is no conflict of interest regarding the publication of this research paper. Ethical issues, including in relation to plagiarism, infringement, falsification and/or falsification of data, publication and/or double submission, and redundancies have been fully borne by the authors.