Emerging Capabilities of LLMs

This study evaluates LLMs in generating novel research ideas, comparing LLM outputs directly with ideas from over 100 expert NLP researchers. Findings reveal that LLM-generated ideas are rated as more novel (p < 0.05) but slightly lower in feasibility. The study highlights challenges in LLM self-evaluation and idea diversity, proposing a comprehensive design for future

This study introduces AlphaGeometry, a neuro-symbolic system for solving complex Euclidean geometry theorems at the olympiad level without human-provided proofs. Using a neural model trained on synthesized theorem-proving data, AlphaGeometry guides symbolic deduction to solve 25 of 30 recent olympiad problems, surpassing previous methods. Notably, it generates human-readable proofs, solves historical IMO problems, and generalizes

This study assesses GPT-4’s effectiveness in providing scientific feedback on research papers. Comparing GPT-4 feedback with human peer reviewers across over 4,800 papers, researchers found comparable overlap between GPT-4 and human reviews, especially for lower-quality submissions. In a user study of 308 researchers, 57.4% rated GPT-4 feedback as helpful, and 82.4% found it more beneficial

This study evaluates large language models (LLMs) for generating new product ideas targeting college students. Comparing human-generated ideas with those produced by GPT-4 (zero-shot and few-shot prompts), results show AI-generated ideas have higher purchase intent but lower novelty and diversity. Few-shot prompting slightly outperforms zero-shot, and AI-generated ideas are seven times more likely to rank