A price tag shows $10 and then crossed out to $4. Your parser outputs both prices as plain texts next to each other. Now the agent thinks there are two valid prices.Which one should it be using. I'm Simon from Llama Index. Most document OCR benchmarks completely ignore text formatting.They strip it before evaluation treating it as cosmetic. But in parsbench, the first document OCR benchmark for AI agents, we introduced a semantic formatting score because formatting carries meaning. Striketh through marks de ...

If a partial swaps two column headers in a financial table, every single row gets misread. But the industry standard metric, TEDs, barely notices. That's a problem.Hi, I'm Preston from Llama Index. We just released Parsbench, the first document OCR benchmark designed specifically for AI agents. In Parsebench, we introduced GTRM metric that changes how we evaluate table extraction.TED's treeedit distance similarity has been the go-to table metric for years. It compares the HTML tree structure of a predicted ...