RESEARCH RELATED TO UNDERSTANDING

This page describes research related to the Assessing Earth Systems Science Understanding and the Geoscience Concept Inventory.

ASSESSING EARTH SYSTEMS SCIENCE UNDERSTANDING

A new initiative to use Multidimensional Item Response Theory to build easily implemented and interconnected measures of understanding across Earth’s four major systems: Geosphere, Hydrosphere, Atmosphere, and Biosphere.

DESCRIPTION: In this ongoing project, short assessments of 5-10 questions will target understanding of processes occurring within and between Earth’s spheres. Initial data were collected summer 2016 and analysis is ongoing. Contact libarkin@msu.edu if you would be interested in having your course or program participate in large scale data collection in 2016-18.

THE GEOSCIENCE CONCEPT INVENTORY (GCI)

A valid and reliable assessment instrument designed for diagnosis of alternative conceptions and assessment of learning in entry-level earth science courses. Rasch analysis was used to generate a bank of items aligned with ability.

The online testing system for the GCI is no longer active. A word document containing GCI items and answers is available here: GCIv.4_2014_Questions&Answers. A spreadsheet containing answers only is here: GCI MASTER answers 2014. Instructors and researchers are encouraged to use these items freely and without restriction. Item numbers correlate to numbers in paper reporting on GCI Rasch analysis: Libarkin, J.C., Anderson, S.W., 2006, The Geoscience Concept Inventory: Application of Rasch Analysis to Concept Inventory Development in Higher Education: in Applications of Rasch Measurement in Science Education, ed. X. Liu and W. Boone: JAM Publishers, p. 45-73: LibarkinandAnderson2006

DESCRIPTION: The Geoscience Concept Inventory (GCI) is a multiple-choice assessment instrument for use in the Earth sciences classroom. The GCI v.1.0 consisted of 69 validated questions that could be selected by an instructor to create a customized 15-question GCI subtest for use in their course. These test items cover topics related to general physical geology concepts, as well as underlying fundamental ideas in physics and chemistry, such as gravity and radioactivity, that are integral to understanding the conceptual Earth. Each question has gone through rigorous reliability and validation studies. Over TWENTY colleagues have contributed new questions to the item bank, bringing the number of available, high quality questions to almost 200.

We built the the GCI using the most rigorous methodologies available, including scale development theory, grounded theory, and item response theory (IRT). To ensure inventory validity we incorporated a mixed methods approach using advanced psychometric techniques not commonly used in developing content-specific assessment instruments. We conducted ~75 interviews with college students, collected nearly 1000 open-ended questionnaires, grounded test content in these qualitative data, and piloted test items at over 40 institutions nationwide, with ~5000 student participants.

In brief, the development of the GCI involved interviewing students, collecting open-ended questionnaires, generating test items based upon student responses, soliciting external review of items by both scientists and educators, pilot testing of items, analysis of items via standard factor analysis and item response theory, “Think Aloud” interviews with students during test piloting, revision, re-piloting, and re-analysis of items iteratively. Although time consuming, the resulting statistical rigor of the items on an IRT scale suggest that the methods we have used constitute highly valid practice for assessment test development.

Learn more about geocognition and geoscience education research.