The data foundation for AI in product configuration

Your customers' configuration choices are the richest signal you have about your products. Here's how to turn them from buried SAP codes into an AI-ready dataset, and why it's the foundation everything else builds on.

By Sachin Shinde · SAP Variant Configuration · Data engineering & AI readiness

TL;DR

  • Most AI projects stall on the data foundation, not the model. Gartner expects 60% of AI projects to be abandoned through 2026 for that reason.
  • If you sell configurable products, your best signal (what customers actually chose) sits locked across about ten SAP tables, in internal codes, versioned by date.
  • Flatten it into one point-in-time table and you get AI-ready data for guided selling, feature-level demand analytics, and trustworthy AI assistants.
  • Characteristics and their value descriptions are date-dependent, changing through the change-management process your organization follows, so every record is read as of the order's pricing date.
  • The technical companion (table map, sample SQL, replication guide) is attached.

Everyone's buying models. Are we ready for them ?.

The hard part of enterprise AI turned out not to be the model. It's the data underneath.

Gartner has projected that 60% of AI projects will be abandoned through 2026 because the data foundation isn't there. In Informatica's 2025 survey of data leaders, data quality and readiness was the most-cited obstacle to AI success, ahead of skills or tooling. Models have become interchangeable. Whether your data is AI-ready is what separates the pilots that ship from the ones that quietly get shelved.

"AI-ready" means something specific: data that's structured, consistent across history and the present, documented, governed, and trusted enough to point a model or an agent at. Most enterprise data doesn't clear that bar yet.

The product-configuration space has this problem in a sharp form

If you sell configurable products, your richest signal is what customers actually chose: every option, on every order, over years. It's the closest thing you have to a record of what your products really are in the market and how demand moves.

In raw form, it's almost unusable. SAP spreads it across about ten tables, written in internal codes and versioned by date. High value and completely un-AI-ready at the same time.

What it looks like once it's ready

The target is one clean table where each row reads: this order line, this characteristic, the value the customer picked, in plain English, valid as of the order's pricing date. One configured order becomes dozens of rows, one per option.

That shape is what "AI-ready" looks like in practice: flattened, labeled, and tied to a point in time.

How the data fits together

SAP keeps configuration in three layers, and you have to walk all three: the sales order (what was ordered), the configuration instance (the choices tied to each order line), and the characteristic dictionary (what each option and value actually means).

One key holds the whole thing together. Nearly every dictionary join runs on a single internal characteristic number, which SAP calls ATINN. Follow it and the model reads cleanly, from symbol to characteristic to value to description.

Time matters just as much. Characteristics and their value descriptions are date-dependent: they change through the change-management process your organization follows, and SAP keeps every version. To know how a two-year-old order was configured, you read each record as of that order's pricing date. Skip that and you describe old orders with today's option list.

The table-by-table map, the join logic, and a simplified version of the query are in the attached technical companion.

Getting the data flowing is no longer the hard part

Replicating SAP tables into a lakehouse like Databricks is a solved problem: SAP's own zero-copy sharing went GA in 2025, and proven CDC (change data capture) routes exist for every landscape and licensing situation. The attachment walks through the options. For decision-makers, the replication method is a detail you can swap later. The modeling is the part that makes the data trustworthy, and the part worth investing in.

Key takeaways

  • AI readiness is a data decision before it's a model decision.
  • In configurable products, the highest-value signal (actual customer choices) is locked in about ten SAP tables.
  • Flattened plus point-in-time equals AI-ready: one key spine, one date rule.
  • Treat it as a foundation: one dataset, reused across sales, product, and AI assistants.
  • Getting SAP into the lakehouse is no longer the hard part. The modeling is.

Your turn

If you work with SAP Variant Configuration, how are you making that data AI-ready today? Full CDC into a lakehouse, the new zero-copy sharing, or still exporting extracts? I'd like to hear what's working.

For data engineers: the attached technical companion covers the table map, the join spine, the point-in-time validity logic, a simplified version of the query, and replication options for getting the SAP tables into your lakehouse.

References

  • Dataversity: Why AI Projects Fail at Scale, The Data Foundation Enterprise Leaders Overlook - dataversity.net
  • Techment: Data Quality for AI (2026 enterprise guide) - techment.com
  • IBM: What Is AI-Ready Data? - ibm.com
  • Databricks: Announcing the GA of SAP Business Data Cloud Connect to Databricks - databricks.com
  • SAP News: SAP and Databricks, A Bold New Era of Data and AI - news.sap.com
  • Zoovu: CPQ for Complex Products - zoovu.com
  • K2view: Grounding LLMs with enterprise data - k2view.com
technical_companion.html