The Geoscience Concept Inventory

Many people ask me for access to questions that have been developed over time as part of the bank of items that evaluate geoscience understanding. Here are item sets, including links to papers, that have been evaluated using item response theory approaches. This space will be updated as new items sets become available:

  1. Geoscience Concept Inventory Item Bank
  2. Climate Change Concept Inventory Item Set
  3. Earth Systems Science Item Bank

Geoscience Concept Inventory Item Bank
A valid and reliable bank of items designed for diagnosis of alternative conceptions and assessment of learning in entry-level earth science courses. Rasch analysis was used to generate a bank of items aligned with ability.

The online testing system for the GCI is no longer active. A word document containing original GCI items is available here: GCI_v3.April2011_origGCI. Instructors and researchers are encouraged to use these items freely and without restriction. Item numbers correlate to numbers in paper reporting on GCI Rasch analysis: Libarkin, J.C., Anderson, S.W., 2006, The Geoscience Concept Inventory: Application of Rasch Analysis to Concept Inventory Development in Higher Education: in Applications of Rasch Measurement in Science Education, ed. X. Liu and W. Boone: JAM Publishers, p. 45-73: LibarkinandAnderson2006

DESCRIPTION: The Geoscience Concept Inventory (GCI) is a multiple-choice assessment instrument for use in the Earth sciences classroom. The GCI v.1.0 consisted of 69 validated questions that could be selected by an instructor to create a customized 15-question GCI subtest for use in their course. These test items cover topics related to general physical geology concepts, as well as underlying fundamental ideas in physics and chemistry, such as gravity and radioactivity, that are integral to understanding the conceptual Earth. Each question has gone through rigorous reliability and validation studies. Over TWENTY colleagues have contributed new questions to the item bank, bringing the number of available, high quality questions to almost 200.

We built the the GCI using the most rigorous methodologies available, including scale development theory, grounded theory, and item response theory (IRT). To ensure inventory validity we incorporated a mixed methods approach using advanced psychometric techniques not commonly used in developing content-specific assessment instruments. We conducted ~75 interviews with college students, collected nearly 1000 open-ended questionnaires, grounded test content in these qualitative data, and piloted test items at over 40 institutions nationwide, with ~5000 student participants.

In brief, the development of the GCI involved interviewing students, collecting open-ended questionnaires, generating test items based upon student responses, soliciting external review of items by both scientists and educators, pilot testing of items, analysis of items via standard factor analysis and item response theory, “Think Aloud” interviews with students during test piloting, revision, re-piloting, and re-analysis of items iteratively. Although time consuming, the resulting statistical rigor of the items on an IRT scale suggest that the methods we have used constitute highly valid practice for assessment test development.

Climate Change Concept Inventory Item Set

A valid and reliable assessment instrument designed for diagnosis of alternative conceptions and assessment of learning around climate change conceptions. Rasch analysis was used to validate the alignment of the item set with ability.

Two publications document the utility of this measure with respect to the general public and college students. Both studies considered the impact of conceptual understanding, affect and world views on risk perception.

a) College students: Aksit, O., McNeal, K., Gold, A., Libarkin, J., Harris, S., 2018, The influence of instruction, prior knowledge, and values on climate change risk perception among undergraduates: Journal of Research in Science Teaching, v. 55, p. 550–572.

b) General public: Libarkin, J.C., Gold, A., Harris, S., McNeal, K., Bowles, R., 2018, A new, valid measure of climate change understanding: Associations with risk perception: Climatic Change., v. 150(3), p. 403-416.

Earth Systems Science Item Bank
A valid and reliable bank of items designed for diagnosis of alternative conceptions and assessment of learning around Earth’s spheres. Rasch analysis was used to evaluate the relationship of ability to items and to allow comparison of understanding within one sphere to another.

Publication of results and items is ongoing.

Learn more about geocognition and geoscience education research.

Research to Publication: Materials to Guide the Manuscript-Writing Process

I am teaching a new course this semester (Fall 2018) called Research to Publication. Hopefully, the students in the course will all write a manuscript by the end of the semester and submit it to either their co-authors or a journal! Stay tuned…I have promised a party if the course works…

I would like to share the materials I create for this course since writing academic manuscripts is honestly not something we are taught very well. I didn’t learn until well into my postdoc, and mostly by accident.

I am a big fan of open collaboration. If you have any suggestions for additions/changes, let me know! I have pasted the course outline below and will add materials as I teach them.

*Assignments are completed before the course meets that week.
The role of publishing; academic “currency” – who decides? READ (in general): Klingner et al., 2005
ASSIGNMENT: Try answering the following questions now, and revisit this discussion multiple times over the course of the semester.

  1. Why are you interested in learning how to write an academic manuscript?
  2. What do you feel like you still need to learn?
  3. Where else can you turn for assistance with your writing?
Identifying an appropriate journal READ: Where to Publish; Beall’s list
ASSIGNMENT 1: Take time to review the Where To Publish and Beall’s List websites. Consider what you have learned in the context of your own research.

  1. What, if anything, surprised you about Where to Publish and/or Beall’s List?
  2. Describe your area of research for your classmates.
  3. What will be important for you to consider as you choose a journal for your work?
  4. What questions do you have about publishing that your course colleagues might be able to help you answer?

ASSIGNMENT 2: Preparing to write…

  1. What is your manuscript topic? Be as specific as you can.
  2. Identify THREE journals that would be appropriate venues for your work. Explain why you have chosen these journals.
  3. Using Beall’s list, identify ONE journal that seems like it would be a good fit. DO NOT PUBLISH in these journals. I need this information for the WEEK 2 in-class activity!
Basic structure of an article DUE: Deconstruction of THREE articles in your target journal

READ/DISCUSS: Turbek et al., 2016; Perneger and Hudelson, 2004

Visualizing your manuscript! DUE: 12 VISUAL slides of your manuscript

READ: Rougier et al., 2014; Durbin, 2004

Writing research questions and methods DUE: Your research question(s) and methods

READ/DISCUSS: Kallet, 2004

Ethics and bias in the publication process DUE: Write up of THREE cases from

READ/DISCUSS: Gastel and Ray, 2016

Writing and visualizing results DUE: Your results in BOTH text and tables/figures aligned with questions, methods

READ/DISCUSS: Monash University guide

Authorship DUE: Authorship agreement and discussion with your advisor/collaborators

READ/DISCUSS: Shewan and Coats, 2010

Writing discussions and background DUE: Your discussion and background in parallel


Data Management and Publication DUE: Your Data Management Plan

READ/DISCUSS: NSF guide to data management plans; Gil et al., 2016; Costello, 2009

Editor Q&A (Janice Beecher) DUE: THREE questions for the Editor Q&A; Draft manuscript!

READ/DISCUSS: Kostic, 2016

Reviewing manuscripts DUE: Review of TWO manuscripts from classmates

READ/DISCUSS: Wiley guide; Scrimgeour and Prus, 2016

Preparing manuscripts for publication DUE: Revised and formatted manuscript


Reflecting on the process DUE: Submit that paper (to journal or your co-authors)!


The Academic Sexual Misconduct and Violations of Relationship Policies Database

Updated 9/15/18 to be crystal clear that I include violations of relationship policies within the database.

Over two years ago, I started tracking academic sexual misconduct and violations of relationship policies. I started the project in an attempt to address what I saw as a missing piece of media reporting on misconduct in academia. In general, the media reports on one case at a time, occasionally mentioning other cases although without any sense of the broader culture of sexual harassment within some areas of academia. Since I began tracking cases, the database has grown from a few dozen to over 700 evidenced cases.

Several people have used these data to conduct studies. I will list these here as they get published:

  1. Cantalupo, N. C., & Kidder, W. C. (2018). A Systematic Look at a Serial Problem: Sexual Harassment of Students by University Faculty. Utah Law Review, 2018(3), 4.

I have migrated the database from a list to a google sheet to facilitate broader use of these data. You can view the google sheet below, or view it in google by clicking on this link ACADEMIC SEXUAL MISCONDUCT DATABASE.

The database is very likely incomplete. If you would like to share an evidenced sexual misconduct case against a faculty, administrator, researcher, or similar university or research institute employee, you can submit it HERE. Please note that only cases with evidence of sexual misconduct can be included. PLEASE CHECK THE DATABASE CAREFULLY BEFORE SUBMITTING A NEW CASE.

  • Academic includes individuals who are employed in any setting where college or university students are working or studying.
  • Sexual misconduct includes: sexual harassment, sexual assault, sexual misconduct, stalking, violations of dating policies, violations of campus pornography policies, and similar violations
  • Evidence includes: institutional finding; admission on part of the accused, accused resigned/retired/died before an institution completed an investigation; a settlement by either the accused or the institution was reached with the victim/survivor; documented evidence (usually in form of texts or emails) of sexual misconduct exists; a legal finding of fact was made by a court, with or without legal punishment.

How (and When) to Write the Background Section of a Manuscript

The related posts: How to Review a Manuscript, How to Write a Manuscript, How (and When) to Write the Background Section of a Manuscript.

The background of a paper may have different names depending on your field. It might be called the introduction, or the literature review, or the theoretical framework. Certainly, you should be familiar enough with existing research BEFORE you start a study to know that your work aligns with accepted studies and offers something new. However, WRITING the background should be your LAST step in writing a paper.

  1. As always, look at example papers from the journal you are targeting.
    • How many paragraphs are typical for backgrounds in this journal?
    • What is the purpose of each paragraph?
    • How many sentences are each paragraph?
    • Do backgrounds have subsections?
    • What is the purpose of each subsection? How long? ETC.
  2. Write the literature review backwards. Many people recommend the hourglass method for writing articles. I recommend it for the literature review and discussion. The hourglass starts broad, becomes narrow, and then broadens again. For the literature review, you are writing a broad and narrow piece. For the discussion you are writing narrow and then getting broad.
    • Write the narrowest part of the introduction first. What is your research question? This is the last paragraph of your background (or a separate section depending on the journal)
    • Who in your community cares about this specific research question and your specific findings? Focus in on work that speaks to your specific findings. Someone WITHIN your community has done something similar. Discuss that similar work. This is towards the end of your background.
      • For example, imagine you studied the role that the availability of LEGOS plays in shaping girls’ notions of gender roles. Someone in your community (the TOY-GENDER community) has likely studied LEGOS, or other types of toys, and their impact on gender role notions among girls and boys.
    • What other communities might care about this specific research question and findings? Focus in on work that speaks to your question and findings. Someone OUTSIDE OF your community has done something similar. Discuss that similar work. This is in the middle of your background.
      • Someone outside of your community (in the CAR-GENDER community or the TOY-ETHNICITY community) has likely studied different types of objects (e.g., cars) and their impact on gender role notions among girls and boys; or, the impact of toys on societal role notions among different ethnic groups.
    • In your discussion points, you should have thought beyond your specific question and findings to discuss the broader implications of your work. Other communities studying other concepts related to your work will be interested in the implications of your work. This is near the beginning of your background.
      • People who study TOYS or GENDER (but not necessarily both) or WORKFORCE or PLAY or IDENTITY-BASED PERCEPTIONS (but not necessarily of gender) would be interested in your TOY-GENDER findings in a broad sense. Describe this prior work.
    • The very first paragraph of your paper should set up why your work is so vital. This should be written like this: FACT, FACT, FACT, PROBLEM, HEY – I CAN SOLVE THAT PROBLEM! Like this (I am totally making this up, but in real life there would be references. And don’t judge me – again, I made this up as an example of a first paragragph): Many careers are outside of the reach of young girls, not because of ability, but because of societal expectations. Often, these expectations are transmitted when we are quite young, in the objects we encounter. For example, the toys we play with can instill a sense of how we – as gendered individuals – are expected to engage with the world as we age and begin to work. In the western world, toys are often labeled by the gender group we expect to use them, up to and including use of specific colors (pink, purple) for toys deemed appropriate for girls. This labeling can encourage unconscious bias among parents and children alike, and can shape how young girls view themselves as workers. Reshaping how children play, and what they play with, can have far reaching implications for women in the workforce.

How to Write a Manuscript

The related posts: How to Review a Manuscript, How to Write a Manuscript, How (and When) to Write the Background Section of a Manuscript.

My personal opinion is that writing – especially technical writing – hinges on planning. Set very specific goals. Here is what I would do for an empirical paper; this can be adapted for theory/argument. Let’s say you need to write a journal article. Set a daily goal. Take about a week to dig into article structure and make your blueprint, and each subsequent day do one paragraph/table/figure. You’ll be submitting in 3 months.

  • Pick a journal.
    • Review all of the possible journals.
    • Check to make sure they are legitimate publication venues.
    • Look at metrics that other people might care about (impact factor, who is the publisher).
    • PICK ONE.
  • Examine the structure of articles in that journal.
    • Pick three example articles that you really like and which are similar to the kind of work you will be presenting.
    • Tear each article apart
      • How many sections?
      • How many paragraphs in each section?
        • What is the form and function of each paragraph?
        • How many sentences in each paragraph? ETC.
        • Tear the articles apart down to sentence structure if needed.
  • Based on what you have learned about articles published in this journal, develop a blueprint for the components of your paper. You can model the blueprint in as much detail based on example articles as you are comfortable with. Notice – this should be generic. You are going to drop the story of your research into this blueprint.
  • Once you have that blueprint, I recommend writing in this order:
    • Research question/hypotheses
    • Methods
    • Figures/tables.
    • Bullet points for each figure/table that articulate the information in the figures/table. If a figure/table is in the results section, also write bullet points for the SO WHAT? of each figure/table.
    • Write out bullet points for results not shown in figures/tables.
    • Bullet point discussion themes – these should convey the SO WHAT? of your research and connect to the literature (which you haven’t written up yet). Nail down discussion points – this is the hardest part.
    • The content of the background should be aligned with your discussion. The discussion should consist of:
      • 2-4 very specific findings linked to the work that most closely aligns with your study.
      • 1-3 broader implications that align with work that is linked with some, but not all aspects of your work.
      • 1-3 very broad implications that align with the most general community.
      • Then, and only then, do you write the background (this is the introduction or literature review or theoretical framework or whatever your field calls the background). As you write the background, your discussion points might adjust to accommodate any new literature that you discover. See this post for suggestions about how to write the background.
    • After finish your background, flesh out the discussion by weaving the literature you have cited into the discussion.

How to Review a Manuscript

The related posts: How to Review a Manuscript, How to Write a Manuscript, How (and When) to Write the Background Section of a Manuscript.

Reviewing a manuscript for a professional journal can seem like an overwhelming task, and can be time-consuming. I use this technique to help me dig into a paper quickly.

  1. Read the abstract and the title.
  2. Make a note of the central theme of the paper.
  3. Flip to the research question (usually at end of intro).
  4. Make a note of the connection or lack thereof between the research question and the theme as depicted by title and abstract.
  5. Flip to discussion.
  6. Does the discussion address the theme and question? Often, this is where a paper will fall apart first. Don’t read in great detail yet, just skim for alignment.
  7. If discussion, theme, and question do not align, then your review should focus on this. Do the next steps, but the authors either made a mistake, or didn’t actually do the work as they thought they had.
  8. If discussion, theme, and question all align, then go to methods. As above, check to make sure methods align with theme/question. Now, however, you get to decide if the methods are adequate. Not perfect. Adequate. You can note imperfections, but new reviewers are often too focused on minor details and then miss major errors. Look up any methods that you feel weak on – I do it all the time, especially for stats and even for things I’m an expert on. If I have any questions, I always double-check that my concern is legit.
  9. Dig into the results.
    1. Do the results align with the theme/question/method?
    2. Do the results need to push things further?
    3. Are figures/tables telling the story of the paper or are they difficult to follow?
  10. Dig into the discussion. The biggest mistake I see is a discussion that simply repeats results. A good discussion will tell a reader the SO WHAT of the paper. Seriously, why should anyone care enough to read it?
  11. Remember, you still shouldn’t have read much of the introduction or literature review. Read it now.
    1. Does the introduction link into the discussion?
    2. Does the discussion adequately reference both the literature they cite and the existing literature, in general? Often, people will have a misalignment between the literature they reference and their discussion. You might not know the literature well enough to know what’s missing, but GoogleScholar does.