ExecutiveGov

NIST Seeks Public Input on Draft Best Practices for Automated AI Benchmark Testing

NIST said Friday that its Center for AI Standards and Innovation, or CAISI, released an initial public draft of NIST AI 800-2 ...
VentureBeat

MLPerf 3.1 adds large language model benchmarks for inference

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More MLCommons is growing its suite of MLPerf AI benchmarks with the addition ...
Semiconductor Engineering

Benchmark For AI-Aided Chip Design That Evaluates LLMs Across 3 Critical Tasks (UCSD, Columbia)

Researchers at UCSD and Columbia University published “ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design.” Abstract “While Large Language Models (LLMs) show ...
VentureBeat

MLPerf 3.0 benchmark adds LLMs and shows dramatic rise in AI training performance

As the hype and momentum behind generative AI continue to grow, so too does the performance of the underlying systems that enable machine learning (ML) training. The big new inclusion with MLPerf ...
The Lancet

CARDBiomedBench: a benchmark for evaluating the performance of large language models in biomedical research

Although large language models (LLMs) have the potential to transform biomedical research, their ability to reason accurately across complex, data-rich domains remains unproven. To address this ...
insideHPC

MLPerf Training and HPC Benchmark Show 49X Performance Gains in 5 Years

Today, MLCommons announced new results from two MLPerf benchmark suites: the MLPerf Training v3.1 suite, which measures the performance of training machine learning models; and the MLPerf HPC v.3.0 ...