Development of Pisa Model Math Problems with Central Java Cultural Context

Catur Yustika Melati^1*, Heri Retnowati²

^1*.2 Mathematics Education, Faculty of Mathematics and Natural Sciences, Yogyakarta State University, Indonesia

Email: [email protected]

Keywords

ABSTRACT

PISA Type Mathematics Questions, Central Java Culture, Design Research

This research aims to develop PISA model mathematics questions with a Central Javanese cultural context that are valid, reliable and have a potential effect on students' mathematical literacy abilities. The research method used in this research is design research type development studies, which consists of two stages: preliminary and formative evaluation. The resulting test instrument was then validated by experts consisting of experienced Mathematics Education lecturers and Mathematics Teachers. The respondents in this research were 29 students at SMP IT Harapan Ummat Purbalingga. The product of this research is PISA model Mathematics test questions with a Central Javanese cultural context that are valid, reliable and have a potential effect on students' literacy abilities based on the PISA framework. The validation results were analyzed using the Aiken V index by calculating the correlation coefficient ( ) and the Aiken V coefficient value was between 0.583 to 0.917, which means that for all questions, 4 questions had high criteria and 3 questions had medium criteria. Then reliability was analyzed using IBM SPSS Statistics 2022 by determining the Cronbach Alpha coefficient and obtained a value of 0.904. Then, from the results of the analysis at the field test stage, it can be seen that several basic mathematical abilities are emerging which are based on mathematical literacy abilities such as reasoning and argumentation abilities of 44.3%, communication skills of 34%, representation abilities of 41.25%, ability to formulate strategies to solve problems. Problems amounted to 70.3%, the ability to use symbolic language, formats, and techniques and operations amounted to 57.67%, and the ability to use mathematical tools amounted to 23.5%.

INTRODUCTION

The results of Indonesia's PISA study from 2000 to 2018 show that Indonesia's average score is below the international average score in the field of mathematical literacy (OECD, 2003; OECD, 2004; OECD, 2007; OECD, 2010; OECD, 2014; OECD, 2018; OECD, 2019). The results of the PISA study in 2003 � 2009 showed that almost 80% of Indonesian students were below level 2 (Widjaja, 2011). Furthermore, the results of the PISA study in 2015 also revealed that most Indonesian students were only able to solve problems at level 2, namely the level where students only carried out low-level thinking processes. The results of the 2018 PISA study Hendroanto et al., 2018; OECD, 2016)showed that only 1% of Indonesian students got a score. at Level 5 or above in mathematics, while the OECD average was 11% for that year(Markus, 2019; OECD, 2019).

These data show that Indonesian students' mathematics mastery skills still need to be improved. Efforts to increase these abilities require synergy from several parties, one of which is the role of teachers and the availability of PISA model mathematics questions. If students often encounter the same type of problem, then students will use the same way or method of solving it. So students will not find it difficult to solve these questions. And conversely, if students get a new type of problem that has never been done before, then students will be confused in solving the problem and this will become a problem for students (Sugiman et al., 2009).

A study stated that the results of an analysis of junior high school mathematics textbooks in terms of mathematical literacy showed that junior high school mathematics books were dominated by PISA level 2 mathematics competency at 52.74% and level 1 at 39.80%. With less than 10% at levels 3 to 6. This gives us information that the questions in the textbooks used by junior high school students are only able to train students at a low level. So other sources are still needed to be able to complete these deficiencies (Suharyono & Rosnawati, 2020). So students can practice more questions at different levels. Therefore, the development of mathematics questions adapted to the PISA 2022 model can be a solution for teachers in schools in order to improve the mathematical literacy skills of Indonesian students.

The PISA study requires students to use the skills and abilities they have acquired at school which can ultimately be applied to solve problems in various real life situations. This can be achieved through literacy skills with a mathematical context or in other words, mathematical literacy helps someone to recognize the role of mathematics in the world and make judgments and decisions. Mathematical literacy (OECD, 2010; OECD, 2019)is defined as the ability to formulate, use and interpret various mathematical problems in everyday life. (OECD, 2014).In the PISA study, all mathematical processes are integrated with four mathematical contents such as change and relationship, space and shape, quantity, and uncertainty and data. (Hendroanto et al., 2018). Familiarizing and training students to solve PISA model questions can improve Indonesian students' mathematical literacy skills (Mansur, 2018).

Apart from getting students used to solving PISA model questions, the use of context in developing questions is also considered very important. The use of context in mathematics learning will increase students' learning motivation. When students are motivated to learn, it is hoped that student learning outcomes will also increase (Charmila et al., 2016; Ilma Indra Putri et al., 2013; Widjaja, 2013). Familiarizing students with contextual problems will help students to utilize mathematics as a human activity in everyday life. This cannot be separated from the philosophy of Hans Freudenthal who considers �mathematics as a human activity " (Hans Freudenthal, 1971). The use of local context in developing mathematics problems can make it easier for students to understand mathematical phenomena with activities that students usually carry out in everyday life. Another opinion also stated that the use of local context will help students to understand mathematical phenomena from the perspective of the students' own life experiences. This makes mathematics much more interesting and useful for students (UNESCO, 2008).

However, the use of local context has not been widely used, especially in mathematics learning at SMP IT Harapan Ummat Purbalingga, which is known from teacher interviews and monitoring mathematics learning activities in class. Talking about local contexts, many researchers pay attention to the problem of developing mathematics questions with specific contexts. The existence of this research shows that many parties want to develop PISA model questions with various contexts, however, there is still very little development of PISA model questions with the cultural context of Central Java.

Central Java is a province that is rich in culture. Central Java itself has six world-recognized cultural heritages, three of which are material while the other three are non-material, such as Batik, Keris and Wayang. For this reason, experts consider that Central Java Province is the center of Javanese culture. (Punto Hendro, 2018).This potential, if combined and developed, will have a good impact on students' mastery of mathematical literacy skills through questions in the local cultural context. By looking at the various local cultures in Central Java, there are various things that contain mathematical concepts including mathematical content in PISA such as geometry, number patterns, algebra, as well as probability and uncertainty and data.

METHODS

The research model that will be carried out is design research type development studies. The Plomp development model consists of three steps including preliminary research , prototype stage (self-evaluation, expert review, one-to-one evaluation, small group and field test) and assessment phase . The product trial subjects were class IX students of SMP IT Harapan Ummat Purbalingga. The number of sample members in this study was 29 students. Data collection techniques include walkthroughs, interviews, validation sheets, tests, documentation. Data analysis techniques include content validation tests, validity of question items, reliability of question items, discrimination tests, difficulty level tests, analysis of mathematical literacy abilities.

RESULTS

The question development process is carried out in two stages, namely preliminary and formative evaluation. Researchers revised the evaluation results of each stage so that they obtained a set of PISA model mathematics questions with a Central Javanese cultural context that were valid and reliable and had a potential effect on the mathematical literacy skills of class IX students at SMP IT Harapan Ummat Purbalingga. This set of PISA model mathematics questions consists of 7 questions using the cultural context of Central Java, including Arjuna Temple, Monument to the Birthplace (MTL) of Commander-in-Chief General Soedirman, Papringan Market Temanggung, Getuk Goreng Sokaraja, Rumah Joglo and Batik Kawung Solo. With an average score of 12.2 with a total score of 14 points. Furthermore, the percentage of achievement on level 2 questions was 44.83%, level 3 was 56.90%, level 4 was 20.69% and level 5 was 40%.

Preliminary stage is an important initial stage. At this stage the researcher started by carrying out analytical activities, namely analyzing the 2013 curriculum which was then connected to the PISA 2022 framework and several literature about this research. Then determine the subject and location of the research. Next, we enter the design stage which includes designing the question grid, questions, answer keys and scoring guidelines by paying attention to three main criteria including content, construct and language.(Depdiknas, 2008; Fitria Alika et al., 2018).

formative evaluation stage which includes self - evaluation , expert review , one -to- one assessment , small group assessment and large group assessment ( field test). ). At the self-evaluation stage, the researcher evaluates and examines the questions that have been created independently. The revised results from the self-evaluation stage of the questions are referred to as prototype 1. The next stage is expert review and one-to-one which are carried out simultaneously to determine the validity of the questions that have been developed in prototype 1.

From the results of the content validity test, it can be seen that all the questions are valid with the information that there are 4 questions with high criteria and there are 3 questions with medium criteria. Based on the validation results at the expert review stage of prototype 1, it was found that all question items were valid and acceptable with several comments and suggestions from the validators. These comments and suggestions were then taken into consideration by the researcher in revising prototype 1 questions. The validation results at the expert review stage and one-to-one stage are evaluated and revised, which is then referred to as prototype 2. Prototype 2 can then be used at the small group stage.

At the small group stage, researchers conducted trials in a small class with 9 students. Before giving the question set, the researcher first ensures that students have received all the prerequisite material included in the question set. After that, students work on the questions individually. The results of the answers from this small group stage were then analyzed and tested for the validity of the questions, reliability tests, discrimination tests and difficulty level tests for the questions.

Based on the calculation of the validity of the question items, it was found that all the questions were declared valid. This can be known based on value and value where for all questions > . This means that all of these questions can be used in the field test stage. The same criteria and decisions are also found in research carried out by (Kartimi, 2013)those who carry out validity tests on questions that are being developed after carrying out the trial phase in their research.

Next, reliability testing of the questions is carried out with the aim of assessing the stability of the questions so that they can be used (Heri retnawati, 2016). Based on calculations using SPSS, the question reliability coefficient value was 0.904. This value is in the interval 0.80< ≤1.00 and it can be said that the questions are reliable with very high criteria.

Then a differentiating power test was carried out. From the results of the discriminating power test, it can be seen that questions number 1, 2, 3, 4,5 and 6 have good discriminating power criteria and question number 7 has very good discriminating power criteria. Thus, all questions can be used in the field test stage. This differentiating power means that the questions can differentiate students' abilities. The differentiating power test carried out during the development of the test questions was also carried out Kartimi (2013)in his research entitled "Development of a Tool for Measuring Critical Thinking Skills in Chemistry for High School Students". In this research, questions with good and very good criteria were then included in the next testing stage.

Finally, a test of the difficulty of the questions was carried out. From the results of the question difficulty test, it was found that the questions developed had very easy criteria. The decision is that these questions will continue to be used, subject to revision first. Robert L. Ebel & David A. Frisbie (1991)said that questions that have a low level of difficulty can still be included provided that the questions are corrected first. Apart from looking at the level of difficulty, the decision that the questions will continue to be used also takes into account the validity of the questions.

In prototype 2, the researcher made revisions based on the results of observations in the field (small group trials) with the aim of minimizing misunderstandings that occurred in making questions so that they were easier for students to understand. Based on the results of evaluation and improvements after the small group stage, prototype 3 was produced.

In the final stage, prototype 3, which has been said to be valid and practical, was tested at the field test stage with the research subjects of class IX C students at SMP IT Harapan Ummat Purbalingga. The aim of the field test stage is to see the potential effects that arise from the results of developing PISA model mathematics questions in the cultural context of Central Java based on students' mathematical literacy abilities.

The results of the analysis show that the majority of students are not yet able to involve reasoning and argumentation skills. In question number 5 regarding surface area, the problem that arises in the question is calculating the roof area (zinc) needed to cover the entire roof of the building. It turned out that many students calculated the inside of the roof, while the surface area asked for in the question was only the visible outside, the inside did not need to be counted, so many students had difficulty completing the question request. Most students when working on questions about surface area only need to calculate flat shapes or models that have been provided by the teacher, whereas in this problem students are expected to be able to think and reason about which parts need to be calculated and which do not need to be calculated. Cases like this can occur if students are not used to encountering contextual problems. If they are used to it, it will be easy to solve problems like this. (Sugiman et al., 2009)Familiarizing students with questions like this will hone their reasoning skills. In this case, (Pamungkas, 2017)it states that educators are also one of the factors causing students' low understanding in solving PISA model questions because teachers do not provide questions that are specifically designed to train reasoning skills that are directed at real life contexts, which they hope will help develop students' reasoning abilities to the maximum.

CONCLUSION

This research succeeded in producing seven PISA model mathematics questions with a Central Javanese cultural context, which covered three of the four mathematical contents tested in PISA, namely Space and Shape (four questions), Quantity (two questions), Uncertainty and Data (one question). The cultural context of Central Java is represented by elements such as Arjuna Temple, Papringan Market, Solo Kawung Batik, Getuk Goreng Sokaraja, Joglo Traditional House, and Liamasan. In the research conclusions, it can be explained that: First, the PISA model mathematics questions can be considered valid because they correspond to the PISA question indicators contained in the PISA 2022 framework, with Aiken V coefficient values ranging from 0.583 to 0.917. Second, these questions have very high reliability with a reliability value of 0.904, which indicates that retesting will produce similar results. Third, these questions also have potential effects, with the majority of students showing good mathematical literacy skills in solving these questions, with the ability to formulate problem-solving strategies being one of the most dominant aspects with a percentage of 70.3%. Thus, PISA model mathematics questions with the cultural context of Central Java can be used effectively to measure students' mathematical literacy abilities.

BIBLIOGRAPHY

Charmila, N., Zulkardi, Z., & Darmawijoyo, D. (2016). Development of PISA model mathematics questions using the Jambi context. Journal of Educational Research and Evaluation, 20 (2), 198�207. https://doi.org/10.21831/pep.v20i2.7444

Ministry of National Education. (2008). Guide to Writing Question Items. Directorate General of Primary and Secondary Education Management. https://pdf.wecabrio.com/depdiknas-2008-panduan-tulisan-modul.pdf

Fitria Alika, M., Darsono, T., Linuwih Department of Physics, S., & Mathematics and Natural Sciences, F. (2018). Development of Pisa Model Questions to Measure Middle School Students' Higher Level Thinking Abilities on Global Warming Material. UPEJ, 7 (3). http://journal.unnes.ac.id/sju/index.php/upej

Hans Freudenthal. (1971). Geometry between The Devil And The Deep Sea Philosophies.

Hendroanto, A., Istiandaru, A., Syakrina, N., Setyawan, F., Prahmana, RCI, & Hidayat, ASE (2018). How Students Solve PISA Tasks: An Overview of Students' Mathematical Literacy. International Journal on Emerging Mathematics Education, 2 (2), 129. https://doi.org/10.12928/ijeme.v2i2.10713

Heri retnawati. (2016). Quantitative Analysis of Research Instruments (Guide to Researchers, Students, and Psychometrians) (1st Ed.). parama Pblishing. www.nuhamedika.gu.ma

Ilma Indra Putri, R., Raya Palembang Indralaya, J., & Ilir, O. (2013). Evaluation of the Indonesian Realistic Mathematics Education Training Program (PMRI) for Mathematics Teachers in South Sumatra. In National Seminar on Curriculum Implementation.

Kartimi. (2013). Development of a Tool for Measuring Chemical Critical Thinking Skills for Senior High Schools. SPSS UPI.

Mansur, N. (2018). Training Students' Mathematical Literacy with PISA Questions. https://journal.unnes.ac.id/sju/index.php/prisma/

Markus, S. (2019). The Program for International Student Assessment (PISA) Results from PISA 2018.

OECD. (2003). Literacy skills for the world of tomorrow: further results from PISA 2000. Organization for Economic Co-operation and Development.

OECD. (2004). Learning for Tomorrow's World First Results from PISA 2003 Program for International Student Assessment OECD Organization For Economic Co-Operation And development . www.copyright.com.

OECD. (2007). PISA 2006 Science competencies for tomorrow's world. www.sourceoecd.org/9789264040007

OECD. (2010). PISA 2009 Results: What Students Know and Can Do. OECD. https://doi.org/10.1787/9789264091450-en

OECD. (2014). PISA 2012 Results in Focus. 3�39.

OECD. (2016). PISA 2015 Results Policies and Practices For Successful Schools: Vol. II. OECD. https://doi.org/10.1787/9789264267510-en

OECD. (2018). PISA 2015 Results in Focus.

OECD. (2019). PISA 2018 Results What Students Know and Can Do: Vol. I. OECD. https://doi.org/10.1787/5f07c754-en

Pamungkas, USA (2017). Development of literacy-based teaching materials on number material for prospective elementary school teachers. Journal of Elementary School Education, 3 (2), 228�240. https://doi.org/10.30870/jpsd.v3i2.2142

Punto Hendro, E. (2018). Building a Society with Personality in the Cultural Sector in Strengthening Central Java as a Javanese Cultural Center. Endogamy: Scientific Journal of Anthropological Studies, 1�17.

Robert L. Ebel, & David A. Frisbie. (1991). Essentials of Educational Measurement (5th ed.). Prentice Hall.

Sugiman, Kusumah, Y., & Sabandar, J. (2009). Mathematical Problem Solving in Realistic Mathematics. 1�8.

Suharyono, E., & Rosnawati, DR (2020). Journal of Mathematics Education Analysis of Middle School Mathematics Textbooks in terms of Mathematical Literacy. Mosharafa , 9 (3), 451�462. http://journal.institutpendidikan.ac.id/index.php/mosharafa

Widjaja, W. (2011). Towards Mathematical Literacy in the 21st Century: Perspectives from Indonesia. In Southeast Asian Mathematics Education Journal (Vol. 1, Issue 1). http://pisaindonesia.wordpress.com/angkat-pisa-indonesia.

Widjaja, W. (2013). The Use Of Contextual Problems To Support Mathematical Learning.