Simple questions to an LLM like: “review this procedure, is it internally self consistent” will find issues, real issues - usually minor - but undeniable when you read the review, the problem it describes is absolutely present in the document, page 14 says “X” and page 23 says “not X” and the context is basically identical - so which are we really supposed to do: “X” or “not X”? Well, that depends. Depends on what? Additional context and detail that the already too long 50 page procedure doesn’t address.
lol and I find issues like this in LLM output of only 1 or 2 pages. LLMs can’t even be consistent across paragraphs, much less hundreds of pages of context window. what an insane claim.
Asking LLMs to find flaws, then verifying those flaws are real, is more powerful than just looking for flaws without the tool.
No it isn’t. Your example is so contrived. What was happening before these documents were reviewed by LLMs? Were the lines that were incorrect causing issues? If so then a human would have found it years ago. If not then they weren’t causing problems and all the LLM has done is invent work that a human now needs to clean up. The LLM didn’t solve a problem, it created a new one!
Now you need to pay a human to make sure that the LLM isn’t actually making things worse, which wasn’t even a job before! Your company that won’t pay a human to review the document in the first place is now needing more humans to clean up a mess that wasn’t a problem in the first place.
lol and I find issues like this in LLM output of only 1 or 2 pages. LLMs can’t even be consistent across paragraphs, much less hundreds of pages of context window. what an insane claim.
Removed by mod
No it isn’t. Your example is so contrived. What was happening before these documents were reviewed by LLMs? Were the lines that were incorrect causing issues? If so then a human would have found it years ago. If not then they weren’t causing problems and all the LLM has done is invent work that a human now needs to clean up. The LLM didn’t solve a problem, it created a new one!
Now you need to pay a human to make sure that the LLM isn’t actually making things worse, which wasn’t even a job before! Your company that won’t pay a human to review the document in the first place is now needing more humans to clean up a mess that wasn’t a problem in the first place.
Removed by mod