ekZepp@lemmy.world to TechnologyEnglish · 7 months agoStudy Finds That 52 Percent of ChatGPT Answers to Programming Questions Are Wrongfuturism.comexternal-linkmessage-square131fedilinkarrow-up1632arrow-down118
arrow-up1614arrow-down1external-linkStudy Finds That 52 Percent of ChatGPT Answers to Programming Questions Are Wrongfuturism.comekZepp@lemmy.world to TechnologyEnglish · 7 months agomessage-square131fedilink
minus-squareslacktoidlinkfedilinkEnglisharrow-up2arrow-down4·7 months agoWe need a comparison against an average coder. Some fucking baseline ffs.
minus-squareanachronist@midwest.sociallinkfedilinkEnglisharrow-up9·7 months ago“Self driving cars will make the roads safer. They won’t be drunk or tired or make a mistake.” Self driving cars start killing people. “Yeah but how do they compare to the average human driver?” Goal post moving.
minus-squareslacktoidlinkfedilinkEnglisharrow-up1·7 months agoThe issue is the powers that be arent comparing chatgpt to a reference manual. Rather another human being.
minus-squarehayes_@sh.itjust.workslinkfedilinkarrow-up6·7 months agoWhy would we compare it against an average coder? ChatGPT wants to be a coding aid/reference material. A better baseline would be the top rated answer for the question on stackoverflow or whether the answer exists on the first 3 Google search results.
minus-squareslacktoidlinkfedilinkEnglisharrow-up1·7 months agoCause people arent looking at chatgpt as an accurate simple code generator, rather a junior dev.
We need a comparison against an average coder. Some fucking baseline ffs.
“Self driving cars will make the roads safer. They won’t be drunk or tired or make a mistake.”
Self driving cars start killing people.
“Yeah but how do they compare to the average human driver?”
Goal post moving.
The issue is the powers that be arent comparing chatgpt to a reference manual. Rather another human being.
Why would we compare it against an average coder?
ChatGPT wants to be a coding aid/reference material. A better baseline would be the top rated answer for the question on stackoverflow or whether the answer exists on the first 3 Google search results.
Or a textbook’s explanation
Cause people arent looking at chatgpt as an accurate simple code generator, rather a junior dev.