Skip to content
Call Today800-600-5822
Programmers.io
Call Today800-600-5822
Menu
  • Hire An Expert
        • We are programming experts in both legacy and modern platforms.

          We extend your existing team so you can accomplish your goals faster. We offer an industry unique happiness guarantee on our work.

        • Legacy
          Programming
          • IBM i/AS400
          • EDI(Electronic Data Interchange)
          • IBM Cognos
          • C++
          • COBOL
        • Cloud Data
          Warehouse
          • Salesforce
          • Matillion
          • Snowflake
          • SharePoint
          • MS Azure
        • Frontend
          Development
          • Vue.Js
          • React.Js
          • Ext Js
          • Angular
          • Full Stack
        • Backend
          Development
          • .Net
          • Java
          • Python
          • Node Js
          • PHP
        • QA
          Testing
          • API Testing
          • Manual Testing
          • Automation Testing
          • Performance Testing
          • Quality Engineering
        • Tech
          stack
          • React.Js with Node JS
          • Vue.Js with Laravel
          • Vue.Js with Python
          • Angular with C#
          • React with C#
        • Data Engineering
          & Analytics
          • Business Intelligence
          • Data Stack Modernization
          • Data Warehouse
          • Power BI Services
        • Mobile App
          Development
          • React Native
          • Android
          • IOS
          • Xamarin
          • Ionic
        • ECommerce
          & CMS
          • Magento
          • WordPress
          • Drupal
          • Laravel
          • Joomla
        • Other
          Technologies
          • ERP
          • Ruby on Rails
          • IT Support and Services
          • Offshore Development
          • Delphi
  • Industries
        • We are programming experts in both legacy and modern platforms.

          We extend your existing team so you can accomplish your goals faster. We offer an industry unique “Don’t pay us if you are not satisfied” guarantee on our work.

        • Industries
          • Insurance Industry
          • Manufacturing Industry
          • Education Industry
          • Healthcare Industry
          • Technology Industry
          • Retail and Consumer Goods
          • Transportation and Warehousing Industry
  • About Us
        • We are programming experts in both legacy and modern platforms.

          We extend your existing team so you can accomplish your goals faster. We offer an industry unique “Don’t pay us if you are not satisfied” guarantee on our work.

        • Our Work
          • Customers
          • Case Studies
          • Portfolio
          • Testimonials
          • Video Testimonials
        • Our Company
          • About PIO
          • Why PIO
          • Happiness Guarantee
          • Team & Leadership
          • Recognition
          • IBM i Rising Stars
          • Life @ Programmers.io
          • Privacy Policy
        • Referral Program
          • Client Referral
          • Colleague Referral
        • Our News
          • News
          • Blog
          • Job Openings
          • Hiring
  • Contact Us
  • Hire Us Today800-600-5822
  • 800-600-5822
Case Studies

Data Loading into Snowflake using Matillion with a Meta data driven approach

About Client

The client discussed in the case is one of the famous fashion retail chains of Women and girls’ attire. It is widely known for its shift dresses, active-wear, shoes, accessories and print dresses. Being in the business for 60 years, the client has been a designer, marketer, and distributor of upscale collections for women. It has emerged as an affluent brand with a heritage and aesthetic based on the Palm Beach resort lifestyle.

The brand has demonstrated multi-generational appeal. Its products can be found in the company’s owned signature stores, e-commerce site, certain department stores, and a variety of independent specialty stores. Programmers.io (PIO) is supporting this fashion retail client in the field of Data analysis using business intelligence, EDI transactions and IBM i projects.

catIcon

Retail and Consumer Goods

industry_challenge

The Challenge

The client being into fashion retail needs real-time data of the customers to analyze and create its business strategy.
Following challenges were faced by the client:

  • As the customer data was stored on various data sources such as IBM i DB2, Oracle, SAP and Salesforce (SFDC, SFMC, SFSC), the history of the data could not be stored
  • The data had to be loaded in a centralised data warehouse for accessing and analysing real time data using generated visual reports, along with maintenance of data history
  • Data overload needed immediate integration with the Data Lake along with proper QA and validation of the data
  • The client expected the data loading and validation task duration to be around 6 months with a separate team of QA engineers and a greater number of resources that increased the overall cost of the project

Technology Stack

  • SQL SQL
  • Snowflake Snowflake
  • Matillion Matillion
  • Aws Services Aws Services
  • Metadata Driven Metadata Driven
  • Looker BI report tool Looker BI report tool

The Solution

diagram-flow
Programmers.io team handled the project immediately and optimized the turn-around time to just 1 week! We adopted integrating Matillion (as an engine) with our MetaData driven approach – which improved our time value.

Data experts at Programmers.io suggested to build a Data Lake first. The team performed end to end integration of the data sources such as Salesforce, IBM i DB2, Oracle, Flat files, and SAP with the data lake

Our team used the Metadata driven approach/configuration to run Matillion in loading 200+ SFDC tables into Snowflake using single job

The Data was exported from the source (Salesforce, IBM i DB2, Oracle and SAP) and staged in AWS S3. This data was ingested into Snowflake (Landing layer) where it was validated by the automated QA suite and transferred to Data Lake for comparison and history analysis

Using a metadata-driven approach this robust solution made the onboarding changes faster, streamlined development cycle and supported quick maintenance of data tables

Audit framework captured the error logs, data lineage etc. For each table with solution and helped in generating the health report of the entire process.

Cleaned and real time collected data was then transformed into visual reports for in-depth business insights and proper decision making for the client

Business Impact

Programmers.io team delivered the project from expected time frame of 6 months to 1 week with end-to-end development, integration, audit log mechanism, data validation, QA validation (generic Test case) etc.

The meta data driven approach ensured accuracy, flexibility and hold on optimized real time data. This saved a lot of time of the client and helped it focus on designing the right business strategy

The process involved just one job hence leading to low maintenance cost

The visual representation made the right information accessible and easy to understand, empowering the client to generate more leads and revenue.

Read More Case Studies

Post navigation

PreviousPrevious Prev
NextNext

Testimonials

"We find your team to be true business partners. Exceptionally responsive, strong domain knowledge, and outstanding at execution."

Chief Integration Officer

"Happy with the performance. The team is self-driving, which is something I’ve had struggles with in the past. Very knowledgeable on the IBMi and requires minimal guidance."

Director of Information Technology

"Every developer we have worked with has been great! They are great at analyzing and making great suggestions, in addition to coding all that we ask for. Even with the time difference, they are always willing to work on "our hours" when needed for deadlines and projects as needed. Thanks for everything!"

Chief Information Security Officer

"The programmers I have worked with have all been great! They think out of the box and make suggestions, and they research and figure things out on their own, which is a tremendous help to me. I would recommend them to any prospective clients you have."

Project Manager

"Great team and still one of the best I’ve worked with on the IBMi/RPG environment"

VP, Commercial Excellence & Digitization

close
video-testimonials

Video Testimonials

Effectively Extending
Your Team

Ken Stewart, PMP, CSM

Vice president of operations & Engineering

PSigen Software Inc.

play-btn

Play Video

video-testimonials

Video Testimonials

Onboarding The Programmers.io Team

Brandy Churchill

Software Development Manager

Western National Insurance

play-btn

Play Video

video-testimonials

Video Testimonials

Effectively Extending
Your Team

Jackie Flummer

Director, Technical Business Systems

AmerCareRoyal

play-btn

Play Video

video-testimonials

Video Testimonials

Effectively Extending
Your Team

Luis Gonzalez

President

SalesScoreKeeper

play-btn

Play Video

View All

Contact Us Today!

Find out how Programmers.io can help your company
get to the next level.

Risk Free! We guarantee you’ll be happy after 30 days or we won’t invoice you.

Awards and Certifications

  • company-logo
  • company-logo
  • company-logo
  • company-logo
  • company-logo
  • company-logo
  • company-logo
  • company-logo
footer-logo
  • facebook
  • twitter
  • linkedin
  • youtube
Legacy Platforms
  • EDI(TrustedLink)
  • IBM Cognos
  • Mainframe/Cobol
  • Delphi
  • C++
Modern Platforms
  • Python
  • Java
  • Dot Net
  • React.Js
  • Salesforce
  • iOS
  • Android
  • PHP
  • Cloud Engineering
  • Angular
About Programmers.io
  • About pio
  • Hire Us
  • We are hiring
  • Job Openings
  • Happiness Guarantee
  • Team & Leadership
  • Recognition
  • Blog
  • Privacy Policy
  • Hire Developers
Contact Us

8951 Cypress Waters Blvd
STE 160
Dallas, TX 75019-4784 USA

800-600-5822

India Offices

Jaipur Office:

C-60, Lalkothi Scheme, Behind Vidhansabha, Jaipur, Rajasthan 302015, Phone No.: +91 95295 66661

Hyderabad Office:

Hitech City. Vatika Business Center, Whitefields Kondapur, Hyderabad 500081

Pune Office:

Office no. 301, TRIOS Coworking space, Pune, Maharashtra 411014

  • facebook
  • twitter
  • linkedin
  • youtube
Copyright © 2023 programmers.io | All Rights Reserved.
chatbot
United States