Thanks to Raja and Gracious for their help on this article.

Smart contracts are essential to DeFi, and web3 builders must ensure their accuracy and security. Because smart contracts manage large amounts of funds, powerful auditing tools are more necessary than ever. Even though typical auditing approaches entail manual code review, recent breakthroughs in AI, specifically OpenAI's ChatGPT 3 and 4, have raised the question, "Can AI replace human auditors?"

At AuditOne, we analyzed a few snips of code from a prior audit we performed on UniCrow using ChatGPT4. The contract code and findings are all publicly available if you are interested in reading our audit findings at Unicrow Audit Report | AuditOne. Here are some of ChatGPT4's results.

The function below is supposed to allow the safe transfer of ether or ERC20 tokens from a treasury contract. Only the Unicrow Claim contract is authorized to call the sendEscrowShare function.

Summary of Auditor Findings

The problem is that any party can keep the escrow from ending by refusing to accept payments. This can occur if any party is dissatisfied with the arbitrator's decision in a disagreement, and they can utilize this to block all parties from getting payments. When settling escrow, the system employs a "Push" pattern for transferring funds, which means the escrow cannot be settled if any transfer fails. Any party can take advantage of this by refusing to accept money (whether in native token or ERC777) and blocking the escrow from closing. As a result, none of the parties involved can receive the funds.

‍

Related Article: Evolving AI Audits: Adapting to the Generative AI Era

‍

Prompt

Are there any potential security vulnerabilities or weaknesses in the code?

ChatGPT Results

Summary of ChatCPT Results

ChatGPT could not identify the specific vulnerability without being prompted with the specific request. The first three vulnerabilities identified are false positives. The `onlyUnicrowClaim` modifier and the SafeERC20 library have been properly implemented and audited, while the third vulnerability is already covered by the access control mechanism of the `onlyUnicrowClaim` modifier. The issues discovered by ChatGPT were mostly superficial and informational, but to a novice auditor, being aware of these vulnerabilities or their potential might be helpful.

Code 2

This code snippet facilitates the management of an escrow between parties, enabling the release of funds to the buyer's address when executed.

Summary of Auditor Findings

The code currently utilizes the non-reentrant modifier to prevent re-entrancy attacks and also performs a check to ensure that escrow.claimed is 0 to avoid re-claim. However, a potential vulnerability exists where malicious attackers can repeatedly claim funds from a single escrow and ultimately take all the funds in the contract. This can be achieved when the buyer and seller create a contract, and the buyer (acting maliciously) executes a function to claim payment before the state changes. Although both the refund and claim functions have the nonReentrant modifier, the fact that they are in different contracts means that the state does not synchronize. Attackers can exploit this loophole to obtain all ETH in the contract and even steal tokens if they follow ERC777.

Prompt

Are there any potential security vulnerabilities or weaknesses in the code?

ChatGPT Results

Summary of ChatCPT Results

ChattGPT identified the risk of a reentrance attack and provided solutions, but the response was not tailored to the specific situation. The `safeTransfer` function in the SafeERC20 library has been extensively tested and is considered secure. Contrary to initial concerns, the vulnerability associated with the `escrow.buyer` address is not a genuine issue. By examining the code, it is apparent that `escrow.buyer` is an externally owned account (EOA). This is illustrated by the `refund()` function, which allows a seller to refund a buyer and withdraw the funds to `escrow.buyer`. This approach assumes the buyer is usually an EOA (Externally Owned Account).

Code 3

Unicrow is a secure payment and escrow protocol for trading goods, assets, or services without exposing or holding custody of a user's funds. It reduces operational costs, offers 3rd party arbitration, and introduces a unique challenge mechanism for recourse.

The complete contract pre-audit can be found here (https://github.com/unicrowio/contracts/blob/main/contracts/Unicrow.sol)

Summary of Auditor Findings

(https://docsend.com/view/52h69q2tf3p789jq)

Prompt

Find all security vulnerabilities or weaknesses in this solidity code and list them in a table with descriptions and assign severity levels to each from low to high:

ChatGPT Results

‍

Summary of ChatCPT Results

During the evaluation of the contract, ChatGPT made some assumptions. One of these assumptions was about the Reentrancy vulnerability, identified using OpenZeppelin's ReentrancyGuard. However, ChatGPT did not specify any particular function that could lead to a reentrancy attack. A human auditor would have understood the purpose of using OpenZeppelin's ReentrancyGuard and noted any instance of vulnerability in the code.

‍

ChatCPT noted issues with Incorrect Function Access, but to successfully execute the `refund` function, the caller must be the seller as indicated by the line `require(sender == escrow.seller, "1–011");`. If another account attempts to execute this function, it will fail, producing the "1–011" error message. Similarly, to execute the `release` function, the caller must be the buyer as specified by the line `require(sender == escrow.buyer, "1–025");`.

‍

Under the integer overflow/ underflow vulnerability, the `challenge` function can only be called by the disputed contract, which is ensured by the `onlyUnicrowDispute` modifier. Likewise, the `settle` function can only be called by the arbitration or dispute contract, which is checked by the `onlyUnicrowArbitratorOrDispute` modifier. Similarly, ChatGPT also evaluated the use of the SafeMath function, but the developers had already taken precautions and used it where necessary. It's worth noting that Solidity 0.8 has addressed most integer overflow and underflow issues.

Prompt

Audit this solidity smart contract and provide a findings table ranking and describing the bugs found: Find all security vulnerabilities or weaknesses in this solidity code and list them in a table with descriptions and assign severity levels to each from low to high:

ChatGPT Results

‍

Lack of validation for the marketplace fee

‍

‍

An issue was raised regarding validating the marketplace fee, but this is a false positive. The function has a built-in validation that confirms the sum of the arbitrator fee, marketplace fee, and protocol fee is less than a specified number. This automatically ensures that the marketplace fee is less than or equal to that number. Therefore, there is no need to worry about the marketplace fee exceeding the allowed limit.

‍

Lack of validation for the currency address

‍

‍

It has been suggested that there is no validation for the currency address, but this statement is not entirely accurate. The `pay` function includes a validation check that ensures the payment currency is not the zero address. This check is performed in the following code line: `if(msg.value > 0) { require(input.currency == address(0), "0–010"); }`. It ensures that if the payment is made in ETH, the `input.currency` field is set to the zero address. However, it only applies to ETH payments and does not cover other ERC20 token payments. There is no explicit check for these payments to ensure that the payment currency address does not equal the zero address.

‍

Lack of validation for the amount sent with the `pay` function

‍

There is an error in the statement that the `pay` function does not validate the amount sent with it. The function does validate that the amount sent with the function call is greater than zero. This validation is done through the following line of code: "`require(amount > 0, "0–011");```

Lack of validation for the seller address

‍

There is no lack of validation for the seller address. The `pay` function includes a line of code that ensures the seller address is not the zero address: `require(input.seller != address(0), "0–002");.`

Lack of validation for the `msg.value` in the `pay` function

‍

‍

The `pay` function in the code ensures that the amount transferred matches the specified amount in the `EscrowInput` parameter. Still, it fails to verify that the `msg.value` equals the specified amount. If the payment is not made in ETH, the function requires that `msg.value` is 0, and the `EscrowInput` parameter is greater than 0. As a result, if `msg.value` is greater than 0, it will not fail validation, but it will not be considered as the payment amount.

Lack of validation for the arbitrator address in the `pay` function

‍

‍

There is an error in the statement that there is no validation for the arbitrator address in the `pay` function. The `pay` function includes validation to ensure that the arbitrator's address is not the same as the buyer's or seller's address. This is confirmed by the following line of code: ``` require(arbitrator != buyer && arbitrator != input.seller, “1–027”);

Summary of ChatGPT Notes

‍

When it comes to smart contract security, there are several key points that you should keep in mind to help prevent vulnerabilities. One way to reduce the risks of vulnerabilities is to use widely-used and tested libraries, such as those from OpenZeppelin.

Additionally, you can use SafeERC20 and SafeMath to safely handle ERC20 tokens and safe mathematical calculations, respectively. It's also important to implement reentrancy guards to prevent reentrancy attacks and to ensure that your contracts are well-structured and well-commented so that they are easier to read and understand. While these notes may not provide specific, actionable solutions, they offer valuable insights into best practices for smart contract development. They can serve as a starting point for smart contract audits and implementation.

Summary of ChatGPT Findings Compared to Auditors

‍

Conclusion

AI has great potential in smart contract audits, despite certain limitations. GPT-4 has demonstrated some ability to detect simple vulnerabilities and provide clear explanations.

Continued model training and improvement will make auditing big and complex contracts faster, smarter, and more thorough. However, AI cannot perform comprehensive analysis or identify chain attack paths. It should supplement human auditors, e.g., auto general documentation, rather than replacing auditors. In the future, advancements in AI may enable more effective automated auditing and certification. However, the challenge of identifying complex vulnerabilities remains, as current findings are often generic and based on a standard list of vulnerabilities.

With the rapid advancement of technology, we are on the verge of a paradigm shift that will significantly enhance human efficiency. The potential benefits of AI in improving blockchain security are encouraging, and we will continue to assess the impact of new AI technologies on this crucial subject. Inevitably, we will integrate with AI to some extent in the near future. The possibilities for contract analysis and auditing tools are limitless, with the potential of AI and blockchain working in conjunction.

Try our free AI EU Compliance Checker Tool