Student errors in solving HOTS based-match story problems with Newman's theory

Students' ability for HOTS is critical to acquire. However, the PISA survey in 2018 showed that Indonesian participants were at level 1, and the national exam in 2018 indicated that 40% of students had an issue answering HOTS questions. The problem suggests that students solve questions requiring higher-order thinking skills, causing errors. Therefore, the study aimed to analyze student errors based on Newman's Theory in solving HOTS-based math story problems. The method applied for qualitative research with a descriptive approach. Data collection involved 126 grade IX students using HOTS-based math story tests, interviews, and documentation. Students were grouped into three categories from the HOTS-based math story test results: students with good, medium, and low abilities. This step was to find the average and standard deviation of the scores obtained by respondents when completing the given test. Then three students were selected from each category as subjects. The results showed that as many as 50% of students misunderstood the questions, 20% made transformation errors, and 10% errors in reading, processing skills, and encoding. The high error rate of these students shows the poor ability of students to solve HOTS-based math story problems.

Introduction by correcting mistakes made by previous students so as not to cause new errors (Chusnul et al., 2020;Mansur & Subanji, 2021;Pranitasari & Ratu, 2020;Yusnia & Fitriyani, 2010). Given the importance of HOTS abilities for students, and seeing that many students still find it challenging to solve HOTS questions, this study aims to analyze student errors in solving HOTS-based story questions using NEA.

Methods
The method used in this research was a qualitative research method with a descriptive approach. We collected data using a HOTS-based story test, interviews, and documentation. First, the researcher prepared an instrument for a test, a scoring guide, and an interview guide. The HOTS-based story test questions were as many as three questions about building spaces in the form of descriptions. In question number 1, students distinguished data that correlates with conclusions, compared data about a problem, and chose correct or incorrect data. Problem number 2: Students took measurements in various ways and evaluated the results.
Moreover, for question number 3, students formulated problem solutions and designed plans to answer problems in everyday life. Furthermore, we validated the instruments, and the results obtained that the instruments used were valid and could be used to collect data. After that, the researcher gave the questions to Class IX students, with 126 students consisting of six male students and 57 female students. These respondents were carefully chosen because we considered that they provided the information needed by researchers based on the objectives of this study. Furthermore, student test results were checked and given a score. The researchers grouped students into three categories based on these results: high, medium, and low groups. Grouping was done by calculating the mean and standard deviation of the test results. The calculation results showed that students were categorized as a high group if the score is 81.57, medium group if the score is in the range of 60.03 <x<81.57, and low group if the score is 60.03. Based on the grouping results, the researcher selected three students, S5, S6, and S7, who represented each group as subjects in the study using the purposive sampling technique. S5 illustrated the high group, S6 represented the medium group, and S7 represented the low group. Furthermore, the three students were interviewed. Based on the work of the three students, the researchers analyzed the work and results of student interviews using error indicators based on Newman's Theory which was modified from Widiawati et al. (2020)

Results
The followings are students' errors when answering HOTS-based story types of questions. These errors will be based on Newman's theory.

Reading Error
Reading error was performed by S5 when answering question number 1. The following figure shows how S5 answered question number 1.  Figure 1 above shows that a reading error occurred because the positions of symbols were incorrectly put. S5 wrote the numbers' signs of length, width, and height. These three symbols should be written before the numbers and followed by an equation mark (=). These symbols should also be separated into units. The interview excerpts with S2 below support this finding: P : What is known in the question? S5 : The volume of the gift box is 350 cm 3 . The wallet is 14 cm long, 9 cm wide, and 3 cm high. Furthermore, the dining table is 12 cm long, 10 cm wide, and 5 cm tall. The book is 15 cm long, 10 cm wide, and 5 cm high. P : What is known here is written 14 cm long, 9 cm wide, and 3 cm height. Why did you write them like that? S5 : The teacher has taught us like that. The formula of volume is V = l x w x h.
You said that l is for length, w is for width, and h is for height. I applied this way to shorten the time.
The interview excerpts above show that S5 wrote the symbols of length, width, and height as l, w, and h behind the numbers because she intended to shorten her writing. Based on the result of the written test and interview above, it can be assumed that S5 recognized how to use the symbols but wrote them in the wrong positions. She wrote the signs behind the numbers mainly because she wanted to make them short and easy to answer.

Comprehension Error
S6 and S7 performed comprehension errors. While S6 made a comprehension error when answering questions 1 and 2, S7 made a comprehension error when answering all of the questions. The following figure shows how S6 answered question number 1. Figure 2. the answer to question number 1 by S6 Figure 2 above shows that comprehension error occurred because all of the known data were not written. S6 only registered the box's volume but did not write the three items' length, width, and height. She did that because she forgot to write them. The interview excerpts with S6 below support this finding: P : What is known in the question? S6 : It is known that the volume of the gift box is 350 cm 3 . It is also known that the length x width x height of the wallet is 14 x 9 x 3, the length x width x height of the dining table is 12 x 10 x 5, and the length x width x height of the book is 15 x 10 x 2. P : Then, why did you write all of them? S6 : I initially did not understand well. When I looked back at the question, I forgot to write the wallet's length x width and height, dining table, and book.
The interview excerpts above show that S6 recognized what is known and asked. Based on the written test and interview result above, it can be assumed that S6 admitted what was known and what was asked, but he needed to read repeatedly to understand the question. Comprehension error was also performed by S7 when answering question number 1. In this case, a comprehension error occurred because S7 did not write what was known and what was asked. During the interview session, S7 reflected that she forgot to write what is known and asked and was not used to writing those when answering such questions. Instead, she was used to writing the main point only.
Comprehension error was also performed by S6 when answering question number 2. A comprehension error occurred because S6 did not wholly write what was known and asked. Based on the result of the written test and interview above, it can be assumed that S7 recognized what is known and what is asked in question number 2, but she was not used to writing them. As for question number 3, comprehension error was only performed by S7. Based on the written test and interview result above, it can be assumed that S7 recognized what is known and what is asked in question number 2, but forgot to write them when answering question number 3.
Taken together, comprehension errors occurred because the students did not write partially or completely what is known and what is asked. It is because the students forgot to write them and were not used to writing them. In addition, the students needed to read repeatedly to understand the questions, although they had already finished answering the questions.

Transformation Error
Transformation error was performed by S7 when answering questions number 1 and 3, as shown by the figures below: Figure 3. The answer to question number 1 by S7 Figure 3 above shows that a transformation error occurred because S7 could not write the correct formula when answering the question. However, when asked during the interview session, S7 could mention the right formula. This can be seen in the following excerpt with S7: P : What formula did you apply when answering this question? S7 : The formula I applied was volume = l x w x h, length, width, and height.
The interview excerpt above shows that S7 could mention the correct formula. Based on the written test and interview result, it can be assumed that S7 recognized the proper procedure, but he did not write it when answering question number 1. The interview excerpts above show that S7 mentioned the formulas that need to apply. The procedures he used were the box volume = l x w x h, and the volume of dice = s x s x s. Based on the result of the written test and interview, it can be assumed that S7 recognized what formulas needed to apply but did not write them when answering question number 3. In sum, formulation errors occurred because the students did not write the necessary procedures to use.

Process Skill Error
Process skill error was performed by S6 when answering question number 1. The following figure shows how S6 answered question number 1. Figure 5. The answer to question number 1 by S6 Figure 5 above shows that process skill error occurred because S6 miscalculated when answering question number 1. In this regard, S6 wrote the volume of the wallet was 14 x 9 x 3 = 369 cm 3 . The correct answer should be 378 cm 3 . During the interview session, S6 mentioned how he got 369 cm 3 . He firstly multiplied 14 and 9, which equaled 126. But, when he multiplied 126 and 3, he wrote 6 + 3 = 9, 2 x 3 = 6, and 1 x 3 = 3 which equalled 369. The interview excerpt supported this finding with S6 below: P : How did you answer this question? S6 : I first found the formula volume = length x width x height. I then determined that the volume of the gift box, 350 cm 3 . After that, I added up the volume of three items, wallet 369 cm 3 , dining table 600 cm 3 , and book 300 cm 3 . P : Can you count this one more time 14 x 9 x 3? S6 : I was miscalculated. The correct answer is 378 cm 3 . P : Then why did you answer incorrectly? S6 : I firstly multiplied 14 and 9, which equaled 126. But, when I multiplied 126 and 3, I wrote 6 + 3 = 9, 2 x 3 = 6, and 1 x 3 = 3 which equalled 369.
The interview excerpts above show that S6 could explain his miscalculation and mention the correct calculation. Based on the result of the written test and interview, it can be assumed that S6 recognized how to calculate a multiplication but made miscalculations correctly.

Encoding Error
Encoding error was performed by S6 when answering question number 3. The following figure shows how S6 answered question number 3. Figure 6. The answer to question number 3 by S6 Figure 6 shows that encoding error occurred because S6 did not write a conclusion and only wrote the volume of two items, dice and toy box, and the volume difference. During the interview session, S6 mentioned that he did not write a conclusion because he was not used to writing it. The interview excerpt supported this finding with S6 below: P : What conclusion did you draw from this question? S6 : The conclusion of question number 3 is the remaining volume of the toys box is 716 cm 3 P : Then why didn't you write it? S6 : I am not used to writing a conclusion when answering the story type of questions.
The interview excerpt above shows that S6 could conclude question number 3. Based on the result of the written test and interview, it can be assumed that S6 recognized the conclusion of the question but was not used to writing a conclusion when answering story-type of questions.

Discussion
This study shows five types of errors randomly performed by three participants when answering HOTS-based story types of mathematical questions regarding geometry or cuboids. These five types of errors included 10% reading errors, 50% comprehension errors, 20% transformation errors, 10% process skill errors, and 10% encoding errors. Reading mistakes occurred because the students incorrectly identified and wrote the symbols. This finding is in line with Humaerah (2017), who reported that students made reading errors because they could not read words and symbols in questions, such as incorrectly reading the symbols p, l, and t. Nuryadin and Lidinillah (2014) further argued that students tend to reread questions in the final step. Students were not able to immediately find the keywords or symbols. With this regard, Karnasih (2015) mentioned that one of the influential factors in answering story-type questions was reading skills.
Comprehension errors are the most one performed by students in this study. Such errors occurred because students did not write what was known and asked. It happened because of two main things; (1) students could not identify what is known and what is asked, and (2) students were not used to writing what is known and what is asked. This finding aligns with Fatahillah et al. (2017), which reported that students usually made comprehension errors because they could not express what a question wanted in writing. In this sense, students found it challenging to describe what a question wants with their own words.
Additionally, comprehension errors in this study occurred because students were less careful and not used to it. Further, Nuryadin and Lidinillah (2014) argued that most students made comprehension errors because they could not immediately identify math problems. In other words, students found it difficult to express what a question meant with their terms. It shows that students fail to comprehend math problems before immediately solving the issues. In line with this, Rohmah and Sutiarso (2018) and Alhassora et al. (2017) said that students could not completely comprehend what a question wants, resulting in comprehension errors.
Transformation errors are the second most performed by students in the present study. Such errors occurred because students did not write a formula. This finding aligns with Kurniawan (2017), who reported that students could not appropriately transform questions. In this regard, transformation refers to turning story-type math questions into mathematical models. It was found that this happened because of three main reasons; (1) students could not make mathematical models, (2) students did not know the correct formula, and (3) students did not know the correct arithmetic operation. Rohmah and Sutiarso (2018) argued that transformation errors occurred due to the absence of mathematical models resulting in students failing to solve math problems.
Process skill errors occurred because students were not able to complete calculations accurately. This finding supports Alhassora et al. (2017) reported that students found it difficult to solve math problems accurately. It shows that students could not accurately provide the procedure of math problem-solving. Further, Mahmudah (2018) argued that process skill errors occurred because students could not carry out the calculation, were stuck on analysis, and were less careful on measuring. Amalia (2017) found that students became less cautious and tended to answer questions hurriedly regarding students' carelessness. It leads to students' inability to write correct encodings.
Encoding errors occurred because students did not write a conclusion correctly. This finding aligns with that of Rusdi (2020), which revealed that encoding errors happened because students tended to answer questions briefly and were not used to writing a conclusion. Alhassora et al. (2017) further argued that students found it difficult to find correct encodings. In this sense, students could not interpret, justify, and describe encodings they had seen.

Conclusion
The majority of students in this study made some errors when answering questions. Based on the data analysis, it was found that comprehension error was the most error made by students, which accounted for 50%, followed by transformation error 20%. Meanwhile, reading error, process skill error, and encoding error made up the small percentage of 10% each. It could be assumed that students could not solve math problems if they could not comprehend the issues. Besides, the findings show that students were only used to answering routine, recognized, and instant questions. As a result, students failed to answer non-routine questions that needed specific skills to link one competency with another. This finding corroborates with the consequence of PISA 2018, which reported that Indonesian participants were at level 1. Nevertheless, the results of this study were only gained from grade IX students from one school due to limited access during the Covid-19 pandemic. Future research should be conducted on larger scales to identify students' errors further when solving HOTS-based mathematical problems.