190-4 Evaluating Agentic AI for Reproducible Shale Gas Analytics: Independent Workflow Reconstruction with VibeResearch and Claude Code

Abstract:

Reproducing a published computational study provides a stringent test of agentic AI because success requires more than code generation: the system must interpret a scientific workflow, select compatible data, implement multiple algorithms, evaluate uncertainty, and explain geological differences. We evaluated two tool-using AI research environments—VibeResearch, developed by Zhejiang Lab, and Claude Code from Anthropic—by independently reproducing a hybrid shale-gas production analysis originally developed for the Fuling field, China. The target workflow combines random forest, gradient-boosted decision trees, XGBoost, fivefold stacking with a Ridge meta-learner, Tree-SHAP feature attribution, individual conditional expectation analysis, and particle swarm, differential evolution, and Bayesian optimization. Because the original industrial dataset was unavailable, both systems used the same public dataset of 506 North American shale-gas wells. Each independently generated a full analytical pipeline and completed all eight major methodological components. On a common single train-test split, Claude Code achieved a test R² of 0.9122 for the stacked model, whereas VibeResearch achieved 0.9006; the VibeResearch gradient-boosted model reached 0.9202. Differences among corresponding models were below 0.02. Repeated-split results converged further: stacked-model R² was 0.864 ± 0.064 for Claude Code (30 runs across all models) and 0.870 ± 0.055 for VibeResearch (100 runs for the stacked model). Both systems identified proppant loading and lateral length as the dominant controls in the North American data and independently reproduced the optimizer ranking differential evolution ≥ particle swarm optimization > Bayesian optimization, also observed in the original Fuling study. In contrast, the original Fuling analysis was dominated by geological variables, demonstrating that the workflow adapted to regional data rather than mechanically reproducing prior rankings. VibeResearch additionally generated narrative interpretation reports, whereas Claude Code produced more modular and reusable code. These results show that agentic research environments can reconstruct complex, end-to-end geoscience machine-learning workflows with comparable predictive performance and methodological completeness. Expert oversight nevertheless remains essential for dataset comparability, statistical design, geological interpretation, and code-quality assessment.

Geological Society of America Abstracts with Programs. Vol. 58, No. 5, 2026

doi: 10.1130/abs/2026AM-19249

© Copyright 2026 The Geological Society of America (GSA), all rights reserved.