Why AI Search Metrics Are Failing Procurement Departments

The discrepancy stems from how different platforms define their denominators. One vendor may calculate visibility based on the frequency a brand is named across a fixed set of questions, while another measures the proportion of cited sources belonging to a specific domain. Without clear definitions, these percentages become interchangeable labels for entirely different data sets. This ambiguity often triggers friction during contract acceptance, as a metric promised in a proposal may shift significantly depending on how the underlying question sets are edited.

Dean Luo, Chief Technology Officer at XstraStar, warns that buyers must scrutinize the methodology behind every percentage point presented in a pitch deck. To address this, XstraStar has released a 219-page reference library detailing its measurement definitions, which explicitly outline what each metric counts and—crucially—what it cannot prove. By making these standards public in both English and Chinese, the company aims to move the industry toward transparent reporting, ensuring that procurement teams can compare vendor performance based on consistent, verifiable denominators rather than opaque marketing claims.