314-5 Students Know How to Use AI. Do They Know How to Think With It?

Abstract:

When I began integrating generative AI into undergraduate geoscience courses, many students already considered themselves experienced AI users. In practice, familiarity often appeared to mean knowing how to ask a chatbot a question and obtain an answer. That raised a more interesting teaching question: what does it mean for a student to actually know how to think with AI?

I have explored that question while teaching a variety of asynchronous, online, introductory geoscience courses. Students work through AI-Guided Scientific Inquiries designed around explanation, follow-up questioning, simulation, verification, uncertainty, and scientific reasoning while remaining responsible for interpretation and conclusions. Pre-, mid-, and post-course surveys and reflections provide de-identified, aggregate evidence that complements my classroom experience. A separate non-AI-integrated course provides a baseline view of entering student attitudes.

Preliminary evaluation has complicated some of my own assumptions. Students generally report becoming more confident using AI for learning, but the more interesting development is how they describe that confidence. Over time, students increasingly report checking claims against course materials, supplying scientific context, asking follow-up questions, identifying unsupported assumptions, correcting errors, recognizing uncertainty, and deciding when an AI response should not be trusted.

At the same time, increased capability does not necessarily produce increased enthusiasm. Some students remain concerned about dependence, employment, environmental impacts, academic integrity, or weakened independent thinking. Skeptical students can nevertheless become quite deliberate about questioning, constraining, and evaluating AI.

Self-reported confidence at the beginning of a course cannot establish actual competence or overconfidence. The longitudinal evidence does suggest, however, that familiarity and well-calibrated confidence are not necessarily the same thing.

My experience suggests that AI acceptance is the wrong educational objective. A more useful goal for geoscience may be critical AI fluency: knowing when AI helps, when it needs to be challenged, when it should be rejected, and why the student must remain intellectually responsible for the science.

© Copyright 2026 The Geological Society of America (GSA), all rights reserved.