The Geoscience Concept Inventory

Many people ask me for access to questions that have been developed over time as part of the bank of items that evaluate geoscience understanding. Here are item sets, including links to papers, that have been evaluated using item response theory approaches. This space will be updated as new items sets become available:

  1. Geoscience Concept Inventory Item Bank
  2. Climate Change Concept Inventory Item Set
  3. Earth Systems Science Item Bank

Geoscience Concept Inventory Item Bank
A valid and reliable bank of items designed for diagnosis of alternative conceptions and assessment of learning in entry-level earth science courses. Rasch analysis was used to generate a bank of items aligned with ability.

The online testing system for the GCI is no longer active. A word document containing original GCI items is available here: GCI_v3.April2011_origGCI. Instructors and researchers are encouraged to use these items freely and without restriction. Item numbers correlate to numbers in paper reporting on GCI Rasch analysis: Libarkin, J.C., Anderson, S.W., 2006, The Geoscience Concept Inventory: Application of Rasch Analysis to Concept Inventory Development in Higher Education: in Applications of Rasch Measurement in Science Education, ed. X. Liu and W. Boone: JAM Publishers, p. 45-73: LibarkinandAnderson2006

DESCRIPTION: The Geoscience Concept Inventory (GCI) is a multiple-choice assessment instrument for use in the Earth sciences classroom. The GCI v.1.0 consisted of 69 validated questions that could be selected by an instructor to create a customized 15-question GCI subtest for use in their course. These test items cover topics related to general physical geology concepts, as well as underlying fundamental ideas in physics and chemistry, such as gravity and radioactivity, that are integral to understanding the conceptual Earth. Each question has gone through rigorous reliability and validation studies. Over TWENTY colleagues have contributed new questions to the item bank, bringing the number of available, high quality questions to almost 200.

We built the the GCI using the most rigorous methodologies available, including scale development theory, grounded theory, and item response theory (IRT). To ensure inventory validity we incorporated a mixed methods approach using advanced psychometric techniques not commonly used in developing content-specific assessment instruments. We conducted ~75 interviews with college students, collected nearly 1000 open-ended questionnaires, grounded test content in these qualitative data, and piloted test items at over 40 institutions nationwide, with ~5000 student participants.

In brief, the development of the GCI involved interviewing students, collecting open-ended questionnaires, generating test items based upon student responses, soliciting external review of items by both scientists and educators, pilot testing of items, analysis of items via standard factor analysis and item response theory, “Think Aloud” interviews with students during test piloting, revision, re-piloting, and re-analysis of items iteratively. Although time consuming, the resulting statistical rigor of the items on an IRT scale suggest that the methods we have used constitute highly valid practice for assessment test development.

Climate Change Concept Inventory Item Set

A valid and reliable assessment instrument designed for diagnosis of alternative conceptions and assessment of learning around climate change conceptions. Rasch analysis was used to validate the alignment of the item set with ability.

Two publications document the utility of this measure with respect to the general public and college students. Both studies considered the impact of conceptual understanding, affect and world views on risk perception.

a) College students: Aksit, O., McNeal, K., Gold, A., Libarkin, J., Harris, S., 2018, The influence of instruction, prior knowledge, and values on climate change risk perception among undergraduates: Journal of Research in Science Teaching, v. 55, p. 550–572.

b) General public: Libarkin, J.C., Gold, A., Harris, S., McNeal, K., Bowles, R., 2018, A new, valid measure of climate change understanding: Associations with risk perception: Climatic Change., v. 150(3), p. 403-416.

Earth Systems Science Item Bank
A valid and reliable bank of items designed for diagnosis of alternative conceptions and assessment of learning around Earth’s spheres. Rasch analysis was used to evaluate the relationship of ability to items and to allow comparison of understanding within one sphere to another.

Publication of results and items is ongoing.

Learn more about geocognition and geoscience education research.

New paper published in Journal of Geoscience Education

The Geocognition Research Lab is happy to announce a new paper published in the February issue of the Journal of Geoscience Education:

by Steven W. Anderson and Julie C. Libarkin

ABSTRACT: Nationwide pre- and post-testing of introductory courses with the Geoscience Concept Inventory (GCI) shows little gain for many of its questions. Analysis of more than 3,500 tests shows that 22 of the 73 GCI questions had gains of <0.03, and nearly half of these focused on basic physics and chemistry. We also discovered through an assessment of nearly 500 matched pre- and posttests that students were less likely to change answers on basic physics and chemistry questions than they were on those for the geosciences, with many of the low-gain geoscience questions showing switch rates that were similar to that expected for guessing. These results also pertain to the high-scoring pretest students, suggesting that little geoscience conceptual entrenchment occurs for many students enrolled in entry-level courses. Switching rates for physics and chemistry questions were well below the rates associated with geosciences questions, suggesting greater entrenchment. We suggest that students may have difficulty settling on a correct geoscience conception because of the shaky, more entrenched supporting science underpinnings upon which Earth Science ideas are built. These results prompt the following questions: (1) When do our geology majors learn fundamental science concepts if little learning occurs in the introductory courses? (2) What role does the introductory course play in this eventual learning? (3) What strategies can be employed in introductory courses to enhance learning for those students who will only take one college-level geosciences course? We suggest that longitudinal studies of geosciences majors are needed for periods longer than a semester and that more attention be paid to when conceptual change occurs for our majors.

Announcing a new paper in International Journal of Science Education

A new paper that blends science and art (STEM to STEAM) and evaluates a new technique for analyzing drawings has been published in International Journal of Science Education by GRL Director Libarkin and colleagues:

Factor Analysis of Drawings: Application to College Student Models of the Greenhouse Effect, co-authored by Libarkin, Thomas, and Ording

Abstract:
Exploratory factor analysis was used to identify models underlying drawings of the greenhouse effect made by over 200 entering university freshmen. Initial content analysis allowed deconstruction of drawings into salient features, with grouping of these features via factor analysis. A resulting 4-factor solution explains 62% of the data variance, suggesting that 4 archetype models of the greenhouse effect dominate thinking within this population. Factor scores, indicating the extent to which each student’s drawing aligned with representative models, were compared to performance on conceptual understanding and attitudes measures, demographics, and non-cognitive features of drawings. Student drawings were also compared to drawings made by scientists to ascertain the extent to which models reflect more sophisticated and accurate models. Results indicate that student and scientist drawings share some similarities, most notably the presence of some features of the most sophisticated non-scientific model held among the study population. Prior knowledge, prior attitudes, gender, and non-cognitive components are also predictive of an individual student’s model. This work presents a new technique for analyzing drawings, with general implications for the use of drawings in investigating student conceptions.

New Paper on Social Capital and Diversity in Science

The Geocognition Research Lab is pleased to announce a new publication on the role of social capital in the development of diverse scientists:

Using the Lens of Social Capital to Understand Diversity in the Earth System Sciences Workforce

This manuscript is a collaboration between researchers (Caitlin Callahan, Julie Libarkin) at Michigan State University, Carmen McCallum at Buffalo State University, and Chris Atchison at University of Cincinnati.

Where Should A DBER Scholar Publish?

A couple of years ago the question of where discipline-based education research (DBER) should be published came up in conversation, and I did an informal survey of DBER faculty at my institution to determine where they publish and what they read. In essence, which journals are part of their scholarly conversations? I compiled the list of unique journals that my colleagues suggested, and wanted to share it more broadly. I also added in a few additional and reputable publishing opportunities that have arisen in the meantime. Note that this is NOT a comprehensive list of all good journals. Rather, this is a list that originated from a consensus of top journals used by DBER scholars. Email me if you think a top or high quality new journal needs to be added!

*Interestingly, only a subset (65%) of these journals are indexed by Thomson Reuters – anyone else wish they would expand their offerings or abandon the practice of assigning impact altogether? My institution seems like it wants to discount my scholarship because my field has a non-ISI journal as its main journal. Whether indexed by Thomson Reuters or not, always check Beall’s list to make sure your work is published in the highest quality places!

DBER JOURNALS

Bioscience
Advances in Physiology Education
Biochemistry and Molecular Biology Education
CBE Life Science Education
Evolution: Education and Outreach
Journal of Microbiology and Biology Education
Journal of Natural Resources and Life Sciences Education (also listed in Earth System Science)

Chemistry
Chemical Education Research and Practice
Chemical Engineering Education (also listed in Engineering)
Journal of Chemical Education

Earth System Science/Environmental Science
Bulletin of the American Meteorological Society
Environmental Education Research
Geosphere (special theme)
International Journal of Environmental and Science Education
Journal of Environmental Education
Journal of Geography in Higher Education
Journal of Geoscience Education
Journal of Natural Resources and Life Sciences Education (also listed in Bioscience)

Engineering
Advances in Engineering Education
Chemical Engineering Education (also listed in Chemistry)
International Journal of Engineering Education
Journal of Engineering Education

Mathematics
The College Mathematics Journal
Journal for Research in Mathematics Education
Research in Mathematics Education
Notices of the American Mathematical Society

Physics
American Journal of Physics
The Physics Teacher
Physical Review Special Topics – Physics Education Research

HIGHER EDUCATION/EDUCATION/PSYCHOLOGY JOURNALS
Active Learning in Higher Education
American Educational Research Journal
Assessment and Evaluation in Higher Education
British Journal of Educational Psychology
Cognition and Instruction
Educational Researcher
International Journal of Higher Education
International Journal of Science Education
Journal of Educational Psychology
Journal of Research in Science Teaching
Journal of Science Education and Technology
Journal of Science Teacher Education
Journal of the Learning Sciences
Research in Science Education
Review of Educational Research
Science Education
Teaching and Teacher Education

GEOSPHERE: Special Theme Issue on Human Dimensions in Geoscience

GEOSPHERE is an online journal published by the Geological Society of America (ISI impact = 2.7). The Human Dimensions in Geoscience theme is intended to bring together research that sits at the boundary between geoscience, broadly construed, and social science. This offers an opportunity for communication, education, sociology, anthropology, or similar scholars to interact with each other and reach mainstream scientists. I would personally love to see work from many different communities come together in GEOSPHERE to help build connections across different, yet very similar, research fields.

INVITING SUBMISSIONS TO GEOSPHERE THEME: HUMAN DIMENSIONS IN THE GEOSCIENCE
GEOSPHERE – a journal of the Geological Society of America – periodically runs theme-specific issues. These issues contain collections of articles devoted to the same topic or region and span multiple issues of the journal. Papers are published in regular Geosphere issues as they are accepted, and then each themed issue appears on a separate web page where all themed-issue papers are grouped. Theme issues remain open for two or more years and submissions are accepted on a rolling basis, allowing authors to submit manuscripts as work is completed rather than to meet a specific deadline.

The Call for Papers : http://geosphere.gsapubs.org/site/misc/smargin.xhtml#Human%20Dimensions%20in%20Geoscience

HUMAN DIMENSIONS IN THE GEOSCIENCE
Guest Editors:
Julie Libarkin
Renee Clary
Suzanne O’Connell

This themed issue will focus on the research that occurs at the interface between geoscience, broadly construed, and social science. Political science, education, history, philosophy, communication, information science, diversity studies, and similar fields can help illuminate some of the most vexing issues facing the geosciences. Best practices for communicating climate science, for example, emerge when deep understanding of geoscience intersects graphic design. Similarly, the solutions to the immediate and future need to train more geoscience students may lie in lessons already learned by diversity and access scholars. This special issue will provide a venue for researchers investigating human dimensions in geoscience to share research findings with each other and the broader geoscience community. We encourage submission of high quality research that sits at the interface between geoscience and social science, including science communication, science policy, history and philosophy of science, learning in formal and informal settings, diversity in science, and similar fields.

To submit a paper for this issue, go to www.editorialmanager.com/geosphere/ and be sure to note in your cover letter that this submission is for the “Human Dimensions in Geoscience” themed issue. This special issue will remain open for two years and submissions will be accepted on a rolling basis.

Recent Geocognition Research Lab Publications

Here are a few publications that rolled out just as 2014 was ending – the last few months have been a blur!

  1. McNeal, K.S., Libarkin, J.C., Shapiro-Ledley, T., Bardar, E., Haddad, N., Ellins, K., and Dutta, S., 2014, The role of research in on-line curriculum development: The case of EarthLabs climate change and Earth System modules: Journal of Geoscience Education, v. 62, p. 560-577.
  2. Ellins, K.K., Shapiro-Ledley, T., Haddad, N., McNeal, K., Gold, A., Lynds, S., and Libarkin, J., 2014, EarthLabs: Supporting teacher professional development to facilitate effective teaching of climate science: Journal of Geoscience Education, v. 62 (4), p. 330-342.
  3. Libarkin, J.C., 2014, Evaluation and Assessment of Civic Understanding of Planet Earth. In Roehrig, D. Dalbotten, & P. Hamilton (Eds.) Future Earth: Advancing Civic Understanding of the Anthropocene, p. 41-52.