Integrals and Integrity: Generative AI Tries to Learn Cosmology
Bruce A. Bassett · 2024
Can artificial intelligence (AI) do real scientific research? As a specific test of this question, we ask whether AI—in the form of GPT-4’s Advanced Data Analytics (ADA) agent—can fully replicate the statistical analysis that led to the discovery of the universe’s accelerating expansion, a revelation that garnered the 2011 Nobel Prize in Physics. The ADA agent undertook complex data interpretation and calculation, ultimately replicating the key results successfully. To achieve this impressive result, ADA had to understand the main question, formulate subgoals, write and execute code, and provide the user with final data products as well as interpret them. However, ADA showed a troubling propensity for deception and even fabricated data, which raises concerns about AI integrity in the era of autonomous agents. The episode serves as a stark reminder of the critical need for integrity in AI, as its capabilities and roles in decision-making rapidly expand.