John Paul Castro

Production Azure Databricks Pipeline — Financial Accounting Data (GL, AR, AP)

• → Re-engineered approximately 50 sequential production notebooks into a modular, source-controlled DAB project with parameterized dev/prod deployment targets across separate Unity Catalog catalogs, reducing end-to-end pipeline runtime from 1 hour 38 minutes to approximately 2 minutes
• → Built the full Bronze/Silver/Gold medallion architecture using Lakeflow Declarative Pipelines (DLT) with Auto Loader ingestion from ADLS Gen2 and declarative data quality enforcement, processing GL, AR, and AP financial transaction data
• → Designed and implemented a 26-column Gold-layer Metric Mart (materialized view) computing revenue, expense, vendor cost, AR/AP balances, aging buckets, DSO, DPO, and cash metrics with inline enrichment: trailing 8-week baselines, deviation scoring (delta_abs, delta_pct), and materiality calculations, all computed within the DLT pipeline refresh
• → Built a Silver-layer Class B evidence framework: 14 event, profile, and baseline views providing audit-grade traceability for every financial transformation, plus a mapping stability tracker detecting account classification drift across pipeline runs
• → Implemented a pytest-based validation suite automating full legacy-versus-new equivalence checks across all 223 shared reporting keys, asserting zero accounting-logic differences as a repeatable regression gate prior to production cutover
• → Diagnosed and resolved Unity Catalog storage infrastructure gaps blocking all pipeline execution: provisioned an Azure Databricks access connector with managed identity, created storage credentials, and repointed external locations, restoring ADLS Gen2 ingestion
• → Conducted a comprehensive legacy codebase audit, remediating critical defects including silent data corruption in shared null-handling utilities, GROUP BY aggregation errors, canonical account mapping defects, and revenue/expense sign-convention bugs
• → Built a State Memory baseline engine computing statistical baselines (mean, median, std dev, percentiles, trend slope, seasonality, cadence, structural stability) from the Metric Mart for anomaly scoring

Independent Portfolio Projects — jpcenterprises.com

• → Built an end-to-end Databricks data lakehouse for JDE Aerospace ERP data using DLT with a full medallion architecture, SCD Type 2 change data capture via APPLY CHANGES INTO, DLT Expectations for data quality enforcement, and Unity Catalog governance with role-based access control; achieved 96% Photon engine execution with zero disk spill
• → Architected and deployed a full open-source data platform: Python/Node.js extraction into PostgreSQL, dbt Core modeling with Kimball dimensional techniques, Apache Airflow orchestration, and a Next.js/Fastify reporting stack, live in production
• → Built LLM-powered data pipelines using n8n and PostgreSQL for automated SEC EDGAR filing analysis and an AI-driven RFQ intake system with master data validation
• → Engineered an MDM foundation using Splink probabilistic record linkage in Python, resolving cross-ERP customer entities into golden records with API endpoints and dashboard pages

Azure Databricks & Cloud Platform

• → Served as Chief Data Architect for a global aerospace distributor with 3 ERP systems (JD Edwards/DB2, tcmIS/Oracle, GEAC/DB2), leading cloud migration of heterogeneous financial transaction data (GL, AR, AP, sales orders, inventory) into a unified Azure Databricks data lakehouse with Delta Lake
• → Authored Spark SQL transformations migrating legacy SSIS and Cognos ETL into Databricks medallion pipelines, processing billions of financial transaction rows into canonical Bronze/Silver/Gold schemas
• → Designed medallion architecture data pipelines using Spark SQL and Databricks Workflows, transforming raw ERP financial objects into canonical schemas supporting Finance, SIOP, and compliance reporting at enterprise scale

Data Architecture, Financial Normalization & Governance

• → Led a full Kimball dimensional model redesign during the 2014 Haas merger migration from Cognos to QlikView, normalizing financial data from acquired ERP systems into a unified data warehouse schema that remains the backbone of Finance and SIOP reporting more than a decade later
• → Built cross-system reconciliation frameworks validating transactional integrity between ERP sources: row count reconciliation, referential integrity checks, primary key uniqueness validation, and GL-to-subledger balancing across JDE and tcmIS
• → Established enterprise data governance standards including data dictionaries, process/data flow documentation, and naming conventions

MS SQL Server, ETL & Pipeline Engineering

• → Hands-on SQL Server developer and administrator across a 20-year tenure, managing multiple production instances through five major version migrations (2005 through 2022); authored thousands of stored procedures, functions, and complex T-SQL transformations
• → Re-engineered a legacy Cognos ETL pipeline running 8+ hours daily into an optimized SSIS solution completing in under 30 minutes (94% reduction), then migrated to Azure Databricks for scalable processing
• → Lead technical resource for PwC and external audit/consulting engagements, delivering audit-grade financial data with full traceability and provenance

BI, Reporting & Applications

• → Led enterprise migration from Cognos to QlikView/Qlik Sense, designing underlying dimensional models and establishing self-service reporting standards for Finance, SIOP, and executive leadership
• → Designed and built a mission-critical .NET C# / JavaScript web application for Boeing's 787 Dreamliner tooling program, replacing disparate booth interfaces with a unified JDE DB2-integrated system
• → Engineered real-time warehouse StatusBoard applications deployed across worldwide distribution facilities; developed a multi-customer web portal providing real-time visibility into order status, inventory, and consignment data
• → Recovered millions of zeroed F4211 pricing records in approximately two hours by reconstructing correct values from F42199 history; built bidirectional JDE integrations for post-acquisition systems

• → Contributed to a $50M+ enterprise PowerBuilder/Sybase-to-Java/DB2 migration, responsible for data transformation logic using T-SQL and VB.NET, and development of views and stored procedures supporting re-platforming of mission-critical business systems

• → Built a multi-tenant Rights Management platform serving NBA, WNBA, MLB, Hallmark, National Geographic, NBC Enterprises, MGM, Lionsgate, and others, handling inventory, royalties, and accounting via a robust 3-tier architecture using VB.NET and T-SQL

• → Taught Microsoft Visual C++, C Programming (Basics and Advanced), and software engineering principles at the collegiate level

• → Managed IT operations across two California telecenters (400+ workstations, team of 12), overseeing mission-critical applications for 30+ clients and modernizing legacy Clipper systems to Visual Basic 6.0, SQL Server, and web-based architectures