Oracle AIDP Migrator for Databricks

Plugin oracle-ai-data-platform-workbench-databricks-migrator in oracle-samples/oracle-aidp-samples. Commands read from the repo's manifest — today.

Skills (16)

• aidp-acceptance-contract

  YAML-driven acceptance contract for batch and streaming pipelines that need convergence verification, not just "no exceptions". Declares PASS only after K consecutive zero-result windows (consecutive-zero-window convergence). Use for streaming jobs, slowly-draining backfills, or any pipeline whose s

• aidp-bucket-mapping

  Configure the s3:// → oci:// bucket / namespace mapping the migrator uses to rewrite paths during notebook + catalog migration. Use when (a) the user has external tables / files at s3:// paths that need to land on OCI Object Storage, OR (b) check_data_availability reports "S3 bucket X not found in O

• aidp-build-dag

  Build a migration manifest (the execution DAG) from a Databricks workspace path or workflow ID. Walks %run dependency chains, captures dbutils.notebook.run invocations, and emits reports/<job>_manifest.json — the input every other execute-skill consumes. Use when the user wants to see what would mig

• aidp-check-data

  Pre-migration data-availability scan. Reads every notebook in a migration manifest, extracts every spark.read.table / spark.read.parquet / saveAsTable reference, and probes whether each target schema/table/path exists on the AIDP cluster BEFORE you spend Pass-2 cluster time. Use after aidp-build-dag

• aidp-fixup-cell

  Targeted rewind of a migrated notebook. Re-executes cells from history index N onwards (or a specific cell range) through the execute+verify+fix loop, with a 'why' reason injected so the model knows what to fix. Use when aidp-migrate-job left a notebook at RESULT=PARTIAL or the user identifies a spe

• aidp-migrate-catalog

  Migrate Databricks Unity Catalog / HMS schemas + tables onto AIDP. Two-stage extract→rewrite→replay pipeline. The rewriter applies 18 DDL rules (3-part→2-part flatten, s3://→oci:// via bucket-map, source-format preserved (Delta stays Delta) with delta.* property scrub, MV/streaming rejection, CREATE

• aidp-migrate-job

  Run the full Databricks→AIDP migration against a manifest. Pass-1 walks the %run dep tree and rewrites Databricks APIs in each dep notebook code-only. Pass-2 executes each task cell-by-cell on a live AIDP cluster, runs 4-way verify (exec error / stderr patterns / Spark logs / model eval), and re-att

• aidp-migrator-bootstrap

  One-shot environment readiness check for the Oracle AIDP Databricks migrator. Verifies Python deps, OCI auth, AIDP cluster reachability, and the customer's env-coords file. Use the first time the user invokes the migrator on a workstation, or when any other skill fails with an auth / connectivity er

• aidp-migrator-overview

  Router skill. Read this first whenever the user mentions migrating a Databricks workload (notebooks, jobs, catalogs, schedules) onto Oracle AI Data Platform (AIDP). Lays out the toolkit, the two-pass migration architecture, and which of the other aidp-* skills handles each phase. Compose those skill

• aidp-resume-migration

  Resume an interrupted migration. The migrator caches already-migrated notebooks (in-memory + on-cluster) so subsequent runs skip them. Use when a prior aidp-migrate-job run was killed mid-flight (Ctrl-C, cluster restart, network drop) or when resuming after a manual fix to a specific dep notebook.

• check-data-flow

  Pre-migration data-availability scan. Reads a migration manifest, probes every spark.read.* / saveAsTable target on the AIDP cluster, reports OK / MISSING / EMPTY with remediation tips per category. Use before any aidp-migrate-job run, or whenever the user asks "is the data ready?", "pre-migration c

• databricks-notebook-analyzer

  Read a single Databricks notebook (.ipynb / .py / .scala source format) and produce a migration-readiness report — what it does, what it depends on, what risks the migrator will hit, which gotchas (from references/gotchas.md) will apply. Run before manually authoring an entry in a migration manifest

• migrate-catalog-flow

  Guided catalog migration flow. Extracts Databricks Unity Catalog / HMS metadata, previews the 18-rule DDL rewrite, then batched replay on AIDP. Asks before the destructive replay step. Use when the user wants to migrate the catalog layer (schemas + tables) with checkpoints, rather than running aidp-

• migrate-job-flow

  Guided full-job migration flow. Walks the user from "I want to migrate this Databricks job" through DAG → data check → migrate → status, asking before each phase. Composes aidp-build-dag, aidp-check-data, aidp-migrate-job, and migration-status. Use when the user wants the end-to-end guided experienc

• migration-reviewer

  Review a migrated .ipynb for correctness — NOT just "did it run". Catches latent issues the cell-execute loop missed — wrong write-mode, lost rows, dropped columns, hardcoded paths, dead Databricks-isms, builtins shadowing, format downgrades. Outputs a structured Blockers / Important / Style / Good

• migration-status

  Parse and summarize the JOB_REPORT.md produced by a prior migration run. Surfaces cell pass/fail/fix counts per task, lists the failing cells, and recommends a next-step skill (fixup-cell, resume, or all-clear). Use after any aidp-migrate-job run, or when the user asks "how did the migration go", "w

Hooks (1)

• SessionStart

  python "${PLUGIN_ROOT}/hooks/session_start.py"

Similar plugins