ChatGPT can write smart contracts; just don’t use it as a security auditor

Tristan Greene11 hours agoChatGPT can write smart contracts; just don’t use it as a security auditorResearchers from Salus Security tested GPT-4 and other artificial intelligence systems’ ability to detect seven common security vulnerabilities.1718 Total views8 Total sharesListen to article 0:00NewsOwn this piece of crypto historyCollect this article as NFTJoin us on social networksA pair of researchers from Salus Security, a blockchain security company with offices in North America, Europe and Asia, recently published research showcasing GPT-4’s talents when it comes to parsing and auditing smart contracts.

As it turns out, artificial intelligence (AI) is pretty good at generating and parsing code, but you wouldn’t want to use it as a security auditor.

Per the paper:“GPT-4 can be a useful tool in assisting with smart contract auditing, especially in code parsing and providing vulnerability hints. However, given its limitations in vulnerability detection, it cannot fully replace professional auditing tools and experienced auditors at this time.”

The Salus researchers used a data set of 35 smart contracts (called the SolidiFI-benchmark vulnerability library), which contained a total of 732 vulnerabilities, to judge the AI’s ability to detect potential security weaknesses across seven common types of vulnerabilities.

Related:Crypto lost in BNB Chain heists down by 85% in 2023: Report

According to their findings, ChatGPT is good at detecting true positives — actual vulnerabilities that, outside of a testing environment, would be worth investigating. It reached greater than 80% precision in testing.

However, it has an apparent problem with generating false negatives. This is expressed through a statistic called “recall rate,” and in the Salus team’s experiments, GPT-4’s recall rate was as low as only 11% (higher is better).

This indicates, as the researchers concluded, “that GPT-4’s vulnerability detection capabilities are lacking, with the highest accuracy being only 33%.” As such, the researchers recommend using dedicated auditing tools and good old-fashioned human know-how for auditing smart contracts until AI systems such as GPT-4 can be brought up to speed.“In summary, GPT-4 can be a useful tool in assisting with smart contract auditing, especially in code parsing and providing vulnerability hints. ... When using GPT-4, it should be combined with other auditing methods and tools to enhance the overall accuracy and efficiency of the audit.”# Blockchain# Smart Contracts# AI# ChatGPT# OpenAIAdd reactionAdd reactionRead moreCircle to cease minting USDC on Tron Network effective immediatelyStarknet airdrop largely successful despite controversies$56M moved from defunct exchange after executives fled: Report