Lima Backbone reads Oracle PeopleSoft through partitioned JDBC reads, types the data correctly on landing, writes it into Amazon S3 as parquet files partitioned by date and key under raw/ and history/ prefixes, and keeps it current from the source’s changes. Every run ends with a row count and a control total per table.
Choosing partitions that suit both the archive and the queries that will read it.
The first load is assembled from these objects through partitioned JDBC reads. Change capture (by last-update fields) keeps the daily volume small afterwards.
| Planning object | Source objects | Destination in Amazon S3 |
|---|---|---|
| Items | PS_MASTER_ITEM_TBL · PS_BU_ITEMS_INV | raw/ Parquet, partitioned → history/ |
| Bills of material | PS_EN_BOM_COMPS | raw/ Parquet, partitioned → history/ |
| Routings | PS_EN_RTG_OPER | raw/ Parquet, partitioned → history/ |
| Stock | PS_BU_ITEMS_INV · PS_PHYSICAL_INV | raw/ Parquet, partitioned → history/ |
| Open purchase orders | PS_PO_HDR · PS_PO_LINE · PS_PO_LINE_SHIP | raw/ Parquet, partitioned → history/ |
| Open sales orders | PS_ORD_HEADER · PS_ORD_LINE | raw/ Parquet, partitioned → history/ |
| Production | PS_SF_PRDNID_HEADR | raw/ Parquet, partitioned → history/ |
| Forecasts | PS_DP_FCST | raw/ Parquet, partitioned → history/ |
| Customers and suppliers | PS_CUSTOMER · PS_VENDOR | raw/ Parquet, partitioned → history/ |
| Calendars | PS_CAL_DETP_TBL | raw/ Parquet, partitioned → history/ |
Every check either passes a row or turns it into an exception record with an owner. Nothing is dropped silently.
| Check | Kind | What it catches, with an example |
|---|---|---|
| Types set from the source | typing | SAP dates and times stored as text become dates and times; currency decimals shifted by the currency's rule |
| Keys complete and unique | keys | client or organisation columns kept in the key; zero-padded keys preserved |
| Deletes applied | consistency | rows deleted in the source are marked or removed in the copy |
| Late changes land in order | consistency | a change that arrives after a later one is applied by its source timestamp |
| Wide and cluster tables complete | completeness | every column of a wide table lands; cluster tables read through the released module |
| Partitions cover the range | completeness | every date partition present; gaps reported |
| Counts and control totals match | reconciliation | rows and amounts per table compared between source and target |
| Schema changes noticed | drift | a new or changed column is flagged before it breaks a downstream query |
After the first landing, Amazon S3 receives the source's changes: by last-update fields. A migration month costs the same as a quiet month on the same tier.
The tables you name, landed in Amazon S3 with types set, keys complete and a count and control total per table, then kept current. Scoped in one call.
Offers and principles →Two weeks on your largest tables: terabytes moved, rows per minute, reconciliation variance and cost per terabyte against your current tool.
The offer →Days for the first tables; a full-history migration is sized by table count and volume, and read in partitions inside the run windows your administrators allow.
Counts per step, the check that stopped a row, the owner and the replay point, and the lineage of any number back to its source column.
Your team, on the schedule you set, with the same report after every run; or your systems integrator; or Lima's team under the subscription.
One call scopes the dataset, the run window and the tier. The reconciliation report at the end is the acceptance test.
