Analytics India Magazine

Cursor Achieves 1.8x Inference Speedup on NVIDIA B200 GPUs

In testing on NVIDIA B200 hardware, the pipeline sustained 3.95 TB/s of memory bandwidth at a batch size of 32, or about 58% ...

MLCommons Releases New MLPerf Inference v6.0 Benchmark Results

Today, MLCommons ® announced new results for its industry-standard MLPerf ® Inference v6.0 benchmark suite. This release includes several important advances that ensure the benchmark suite tests ...
InfoQ

New Technique Speeds up Deep-Learning Inference on TensorFlow by 2x

Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with content, and download exclusive resources. Ludi Akue discusses how the tech sector’s ...
Geeky Gadgets

SteerLM a simple technique to customize LLMs during inference introduced by NVIDIA

Large language models (LLMs) have made significant strides in artificial intelligence (AI) natural language generation. Models such as GPT-3, Megatron-Turing, Chinchilla, PaLM-2, Falcon, and Llama 2 ...
Semiconductor Engineering

Review of Tools & Techniques for DL Edge Inference

A new technical paper titled “Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review” was published in “Proceedings of the IEEE” by researchers at University ...

IndexCache, a new sparse attention optimizer, delivers 1.82x faster inference on long-context AI models

Researchers at Tsinghua University and Z.ai built IndexCache to eliminate redundant computation in sparse attention models ...
TechRepublic

DeepSeek-GRM: Introducing an Enhanced AI Reasoning Technique

Researchers from DeepSeek and Tsinghua University say combining two techniques improves the answers the large language model creates with computer reasoning techniques. Image: Envato/DC_Studio ...
EurekAlert!

Newly funded research to develop defenses against wireless inference threats

NORMAN, Okla. – Song Fang, a researcher with the University of Oklahoma, has been awarded funding from the U.S. National Science Foundation to create training-free detection methods and novel ...