AI for Accounts Payable: I Tested ChatGPT on Invoice and PO Matching

PRACTICAL AI EXPERIMENT

Accounts payable has a lot of work that is individually quite simple and collectively quite annoying. A vendor sends an invoice, someone finds the purchase order, and then the checking begins: quantity, unit price, freight, payment terms, total, and whatever special instructions happened to be written into the PO.

I wanted to see how much of that review ChatGPT could handle without building an AP application or connecting anything to an ERP. For this experiment I created synthetic purchase orders in Excel, generated a set of vendor invoices as PDFs, deliberately introduced a few problems, and gave the files to ChatGPT without telling it where the problems were.

The first test went well enough that I ran a second round with partial shipments, price tolerances, inconsistent descriptions and a harder duplicate scenario. I have included the files and the exact prompts below, so you can run the same experiment yourself while you read.

Problem Statement

Invoice matching looks like a straightforward comparison problem until the documents stop agreeing perfectly. If a PO says 100 units at $18.50 and the invoice says the same thing, ordinary software can verify that without involving generative AI.

The more interesting situations are the ones AP teams already deal with. A supplier might invoice only part of an order because partial shipments are allowed. The unit price might be two cents higher even though the PO permits a five-cent tolerance. The invoice description might use different wording while carrying the correct SKU. Freight can legitimately be zero because the PO says it is included. Two invoices can also look suspiciously similar without giving you enough information to decide which one is valid.

I wanted ChatGPT to read both the values and the business instructions surrounding those values, then prepare a compact review table showing what deserved attention and the evidence behind the finding. I also explicitly told it not to approve or reject payments. For anything involving company money, I want the person or system authorized to make that decision to remain responsible for it.

Plan of Approach

There were twelve invoice records across the two rounds. I looked for missed exceptions, false positives and cases where ChatGPT made an assumption that the documents did not support.

Round 1: Can ChatGPT catch straightforward invoice problems?

The first workbook contained five purchase orders, while the PDF contained six invoices. I included one clean transaction along with a unit-price problem, an over-billed quantity, unexpected freight and a pair of invoices that looked like duplicates.

I opened a fresh ChatGPT conversation, attached the spreadsheet and the PDF, and gave it the prompt below. There was no special setup beyond uploading the files.

I work in accounts payable. Compare these vendor invoices against the purchase orders.

Match each invoice to its PO and check the quantity, unit price, freight, payment terms and total.

Do not approve or reject any payment.

Return a simple table with:

- Invoice
- PO
- Vendor
- Status: Match, Review, or Possible Duplicate
- What is wrong
- Evidence from the PO and invoice

Show me only what a human reviewer would need to know. Do not invent missing information.

I kept the prompt fairly plain on purpose. A useful business experiment should be something a person can reproduce without spending an afternoon learning prompt engineering.

What happened

ChatGPT marked the clean invoice as a Match. It found that BluePeak Technology had invoiced $156.50 per unit when the PO specified $149.00, and it calculated the resulting $150 difference. It caught Harbor Office Furniture billing 14 chairs against a PO for 12, and it flagged the $275 freight charge from Summit Packaging because the corresponding PO stated that freight was included.

The final two invoices were more interesting because the invoice numbers were different. Everything else about the transactions matched, including the vendor, PO, item, quantity, date, unit price and total. ChatGPT classified both as Possible Duplicate and stopped short of deciding which invoice, if either, was the correct one.

One part of the response that I found particularly useful was the evidence column. Instead of giving the reviewer a generic message such as “price mismatch,” ChatGPT showed the PO value and invoice value together. A reviewer can see why the invoice was flagged without immediately reopening both source documents just to reconstruct the comparison.

Round 1 was intentionally clean, though, and most of the errors were fairly obvious. I wanted to know what would happen when a difference was acceptable because of a business rule written elsewhere in the document.

Round 2: What happens when the business rules matter?

For the second round I created another five purchase orders and six invoices. This time I added situations where simply comparing two fields would produce the wrong answer.

One invoice described an item differently from the PO, although the SKU matched and the PO specifically instructed the reviewer to use the SKU when descriptions differed. Another PO ordered 100 monitors but allowed partial invoicing, while the invoice billed only 40. A third invoice charged $12.52 against a PO price of $12.50, but the PO allowed a variance of plus or minus five cents per unit.

I also included two situations that genuinely needed attention: an invoice referencing a PO number that was not in the supplied workbook and a duplicate scenario involving two service invoices with different invoice numbers and different dates.

I work in accounts payable. Compare these vendor invoices against the purchase orders.

Match each invoice to its PO and check the PO number, vendor, SKU or item, quantity, unit price, freight, payment terms and total.

Read the PO notes and invoice notes carefully before deciding that a difference is an error.

Do not approve or reject any payment.

Return a simple table with:

- Invoice
- PO
- Vendor
- Status: Match, Review, or Possible Duplicate
- What needs attention
- Evidence from the PO and invoice

If something cannot be verified from the supplied documents, say so rather than guessing.

Show me only what a human AP reviewer would need to know.

The legitimate differences

The Metro Safety Supply invoice used noticeably different wording for the product description, but it carried the correct SKU. ChatGPT read the PO note instructing the reviewer to match by SKU and classified the invoice as a Match.

NorthStar Displays was more revealing. The PO ordered 100 monitors and the invoice contained only 40, which would be a quantity mismatch if you ignored the surrounding instructions. The PO allowed partial shipments and up to two invoices, while the vendor note identified this as partial shipment one of two. ChatGPT treated the invoice as a valid Match.

Evergreen Packaging tested a small price tolerance. The PO specified $12.50 per unit and the invoice used $12.52. Because the PO allowed a variance of plus or minus $0.05, ChatGPT correctly left it alone.

This was the part of the experiment that interested me most. Comparing numbers is easy. In actual business processes, the rule that explains whether a number is acceptable may be written in a note, contract term, tolerance policy or shipping instruction. The model handled the examples I gave it surprisingly well.

The exceptions that still needed a person

The Redwood Facilities invoice referenced PO-261099, which was not present in the workbook. ChatGPT noticed that PO-261004 had the same vendor, SKU, quantity, price, terms and amount, and sent the invoice for Review.

I agreed with the Review classification, although this example exposed an instruction I would tighten before using the workflow in a real process. I would want the system to state that PO-261099 could not be found and separately show PO-261004 as a possible related PO. Similar transaction details are useful evidence for an AP reviewer, but they do not prove that the vendor simply typed the wrong PO number.

The final pair came from NovaTech Services. Both invoices covered the same July to September service period, referenced the same PO and SKU, and billed $2,400. They had different invoice numbers and different dates, while the PO said that only one service invoice was expected.

ChatGPT classified both as Possible Duplicate and stated that the supplied documents did not establish which invoice was valid. That is about as far as I would want the automated review to go.

Results from the two tests

Across the two rounds I supplied twelve invoice records. All twelve received the classification I expected in this controlled experiment. ChatGPT caught the planted price, quantity, freight, PO-reference and duplicate issues, while leaving the legitimate partial invoice, allowed price variance and description difference alone.

I would be very careful about turning that result into an accuracy claim. These were synthetic documents that I created specifically for the experiment. They were readable, every invoice had one line item, and I knew exactly what conditions I wanted to test.

I did not test poor scans, handwritten adjustments, taxes, foreign currencies, credit memos, goods receipts, invoices covering multiple POs, complicated service contracts or twenty-page invoices. A real AP team will eventually encounter all of those and several things I have probably never seen.

How I would use this in an actual AP process

The practical opportunity I see is reducing the amount of routine review a person has to perform. If an AP team receives a large batch of invoices, an AI-assisted workflow could read the invoices and purchasing information, prepare the comparisons and present the reviewer with a queue containing the unusual transactions and the evidence behind each flag.

I would still use deterministic software for calculations and formal tolerance rules wherever possible. If $149 multiplied by 20 needs to equal $2,980, ordinary code can verify that very reliably. The language model becomes more useful when the documents contain context that affects the interpretation, such as partial-shipment instructions, differently worded product descriptions, vendor notes or other business language.

I would also keep the status vocabulary small. Match, Review and Possible Duplicate worked well for this experiment because each status tells the reviewer what kind of attention is needed without pretending that the system has authority it does not have.

This also fits the broader approach I use when thinking about AI projects: start with a bounded business problem where the outcome can be checked, rather than giving an AI system a huge mandate on day one. I wrote more about that idea in my
AI pilot with guardrails
article.

Could this be turned into an AP agent?

Yes, although I deliberately stopped before doing that for this experiment. Once the basic comparison works, the next version could retrieve a newly received invoice, find the corresponding PO, run the same review and send only the exceptions to an AP queue.

The technical workflow is easy to sketch: invoice arrives, the system retrieves the purchasing record, the comparison runs, and a reviewer receives the result. The moment this connects to live financial systems, however, permissions, audit history, data retention, ERP access, failure handling and business controls become part of the design. That is enough material for a separate experiment.

I would rather keep this first article reproducible for someone who simply wants to know whether the idea can help. If I build the agent version later, I can compare what changed once the simple experiment became an operational workflow.

Key Learnings

For a test involving two file types and an ordinary ChatGPT conversation, I got more value out of this than I expected. The result is good enough that I would try the same approach with a larger and messier synthetic dataset next, then decide whether there is enough benefit to build a repeatable workflow around it.

If you are thinking about a broader AI initiative rather than this particular AP example, my
AI ROI roadmap
covers how I think about moving from an interesting use case to something that produces measurable business value.