If the researcher had spent as much time auditing the code as he did having to evaluate the merit of 100s of incorrect LLM reports then he would have found the second vulnerability himself, no doubt.
Problem is motivation. As someone with ADHD I definitely understand that having an interesting project makes tedious stuff much more likely to get done. LOL
I agree not brilliant, but It’s early days. If one is looking to mechanise a process like finding bugs, you have to start somewhere. Determine how to measure success, set performance baselines and all that.
You’re right, probably better put as: if he’d spent his time writing instead of working on that contraption, he’d have produced more books in the first month.
If the researcher had spent as much time auditing the code as he did having to evaluate the merit of 100s of incorrect LLM reports then he would have found the second vulnerability himself, no doubt.
this confirms what i just said in reply to a different comment: most cases of ai “success” are actually curated by real people from a sea of bullshit
Problem is motivation. As someone with ADHD I definitely understand that having an interesting project makes tedious stuff much more likely to get done. LOL
And if Gutenberg had just written faster, he would’ve produced more books in the first week?
I’m not sure if the Gutenberg Press had only produced one readable copy for every 100 printed it would have been the literary revolution that it was.
I agree not brilliant, but It’s early days. If one is looking to mechanise a process like finding bugs, you have to start somewhere. Determine how to measure success, set performance baselines and all that.
I get your point, but your comparison is a little… off. Wasn’t Gutenberg “printing”, not “writing”?
You’re right, probably better put as: if he’d spent his time writing instead of working on that contraption, he’d have produced more books in the first month.