Practical AI Experiments

The same AI agent scored 0%, 79% and 93% on the same 14 business questions. Here is the evaluation method I use to see what the score hides.

PRACTICAL AI EXPERIMENT A semantic layer AI setup improved Databricks Genie’s performance on the same 14 business questions from 79% with prompt-based rules to 93% with governed metrics. My manual review found all 14 final answers business-correct. More importantly, the semantic layer fixed a return-rate calculation that had previously produced 56.89% instead of the verified

If you want to learn how to build an AI agent, the easiest place to start is not with code. Start with one recurring business task, give ChatGPT the instructions and files required to perform it, define where it must stop, and test whether it behaves consistently. I used that approach to build an accounts-payable

AI GOVERNANCE FRAMEWORK FOR BUSINESS LEADERS An AI governance framework becomes real when an output can change a payment, a contract decision, an inventory order or an executive metric. Before that point, governance can look like policy work happening somewhere outside the project. After that point, it becomes part of whether the system is safe

BEGINNER DATABRICKS TUTORIAL Databricks Genie gives business users a familiar chat interface for asking questions about company data. Instead of starting with SQL, they can ask a question such as “What were gross sales last quarter?” and receive a written answer, result table, chart, and the SQL used to calculate it. If you have used

PRACTICAL AI EXPERIMENT This Databricks Genie tutorial tests a simple question that matters far more than whether an AI tool can write SQL: does the agent understand what the business means? I gave the same Genie Agent the same six tables and fourteen benchmark questions twice. The data stayed fixed. The questions stayed fixed. The

PRACTICAL AI EXPERIMENT Inventory reports have a habit of looking simple until you actually have to decide what deserves attention. A product can be below its reorder point and still be fine because stock is already arriving. Another can look healthy today while a long supplier lead time means it will run out before replacement

PRACTICAL AI EXPERIMENT Vendor contract review can involve a surprising amount of repetitive comparison work. A reviewer reads a clause, checks the company’s contract playbook, decides whether the position is acceptable, and then moves on to the next clause. Multiply that across a long agreement and a steady stream of vendors, and a fairly straightforward

PRACTICAL AI EXPERIMENT Accounts payable has a lot of work that is individually quite simple and collectively quite annoying. A vendor sends an invoice, someone finds the purchase order, and then the checking begins: quantity, unit price, freight, payment terms, total, and whatever special instructions happened to be written into the PO. I wanted to