Article: Researchers open-source benchmarks measuring quality of AI-generated code

Language, Open Data

Researchers open-source benchmarks measuring quality of AI-generated code

The applications of computer programming are vast in scope. And as computers become ubiquitous, the demand for quality code draws an ever-growing number of aspiring programmers to the profession. After years of study to become proficient at coding, experts learn to convert abstracts into concrete, executable programs. But what if AI could do the same?

In recent years, large-scale AI language models have shown promise in generalizing to tasks including writing code, implying that humans’ work may be one day supplemented by AI systems. But while some studies show that language models can translate code and fix compilation issues, there’s been little work on rigorously testing the coding ability of models given general coding problems.

Read Full Article