I actually tried letting ChatGPT-4o write some tests the other day.
Easily 50% of the tests were wrong. They ignored DB uniqueness constrains or even datatypes. In a few cases, they just hallucinated field names that didn’t exist.
I ended up spending just as much time cleaning up the cruft as writing them. I could easily see someone just starting out letting the code go through.
I actually tried letting ChatGPT-4o write some tests the other day.
Easily 50% of the tests were wrong. They ignored DB uniqueness constrains or even datatypes. In a few cases, they just hallucinated field names that didn’t exist.
I ended up spending just as much time cleaning up the cruft as writing them. I could easily see someone just starting out letting the code go through.