ETL Strategy for Business Intelligence and Reporting Cloud Solutions
With modern Business Intelligence solutions focused so heavily on being "in the cloud," one aspect of the traditional data warehousing approach that often gets over looked or over shadowed is your organization's ETL processes .
ETL (Extract, Transform, and Load) processes typically serve two primary functions: moving data from a source to a target and transforming that data to be utilized in the best way by the reporting tools. Many folks I talk to often think of them primarily as means to move data to a place their reporting tools can consume them efficiently, doubly so when talking about cloud solutions . When groups are designing cloud solutions, they often get caught up in where the data can be stored and still accessed by all the interacting processes within a solution, but in many organizations, the transformation piece is just as, if not more, critical than just simply moving data.
It is rare to find a robust, heavily BI solution reporting on data exactly how it lives in a transactional system. Usually, the data needs to be re-modeled to build conforming dimensions, join semi-related data elements together, or to just shape the data in a way that answers specific analytical questions.
In the traditional BI solution , this would be handled by a tool like SSIS, Informatica, ODI, or a similar product that allows you to build out processes that map data from a source to target while performing any necessary transformations along the way. These can range from very basic copies of the data from the source system to a data mart for querying purposes to huge, complex processes that heavily transform the data to feed a system of interrelated processes to serve reporting and analytical needs.
In terms of Cloud BI solutions , the good news is that not much has to change if you aren't ready for it. In most cases, you can continue to use on-premise ETL solutions to load data into cloud-based storage and database solutions. Connectivity between on-premise and cloud-based architectures can become tricky due to private networks, firewalls, and other security infrastructure. In most solutions, the path is difficult to connect from a cloud-based server into your network but connecting from within your network into the cloud is typically painless. This means that if the server initiating any transaction is within your network, you shouldn't have to worry about opening any access points.
Here's an important caveat that seems to pop in anytime I discuss cloud-based tools: always involve your network and security teams when designing cloud integrated solutions. This ensures the design meets corporate policies and industry best practices for maintaining the integrity of your data, so it is not left susceptible to outside interference. If you don't involve these folks, they will eventually find out about your system and request (sometimes not so nicely) a review of your system and its interfacing points with the open internet.
I often advocate for iterative steps to the cloud for most of my large scale BI clients. This means identifying one star schema or one set of reports to migrate to the cloud. Then we go through the processes to build just the objects that support that relatively narrowly specified set of requirements. Once that is successful you can build on it over time, adding more schemas and reporting requirements based on their priority and resource availability. This keeps projects moving briskly with a small, focused team.
An iterative approach for ETL transitions can sometimes mean simply pointing existing processes to a new target, assuming the technology between the existing target and the new target doesn't require significant revamping/rewriting of the processes. For example, if you have an ETL process that loads tables in an Oracle DB and are moving to an Oracle Cloud DB, this cut over should be straightforward.
However, if you're looking for a cloud-based ETL tool , you still have plenty of options. Oracle's ODI has a cloud-based counterpart, as does Informatica, Microsoft, and many others. Which one you use will depend on a variety of factors including cost, existing licensing obligations, resource availability, and others, but the important takeaway is that there are now a variety of options to meet the needs of complex data transformations for cloud reporting tools that are both on premise and in the cloud.
We have worked with clients who have wanted to keep one foot on premise at times, making hybrid approaches ideal for them where the reporting tool is in the cloud, but the data is kept in their on-premise databases. We have also worked with newer clients, typically younger companies without a large history of IT systems to keep integrated and functioning, who do not want to host a server on their internal network for even development tools. They want to be 100% cloud-based, and these ETL solutions certainly exist for them as well.
Partner SpotLight
Oracle
Oracle has the most comprehensive suite of integrated, global business applications that enable organizations to make better decisions, reduce cost..
Our Company
MindStream Analytics' senior staff was there at the birth of Business Intelligence. We have been part of building Business Intelligence nationally from its humble niche product status to the ubiquitous analytic tool that it is today. MindStream consultants are well versed in reporting and information management and are ready to help you leverage the power of multiple tier-1vendors. From Oracle Hyperion to IBM Cognos, we can help you select and integrate the right tools for you to better understand your information. MindStream Analytics has experience across a wide variety of industries: Business Services, Consumer Products, Energy, Financial Services, Healthcare, Manufacturing, Transportation , and Telecommunication. We have the depth and breadth of experience to help you deliver actionable information to users.
Whether you need an enterprise wide Oracle BI Enterprise Edition (OBIEE) rollout, an Oracle BI Apps rollout, or an IBM Cognos ReportNet rollout, MindStream Analytics is here to help you succeed.
Case Studies
Accumen
Thanks to the intervention of MindStream Analytics, Accumen's Finance department can now model their business with a new, more organized structure that isn't conventionally available in NetSuite.
Acme Brick
Acme Brick turned to MindStream Analytics for help implementing OneStream to replace their outdated TM1 solution.
Alterra
Alterra sought the expertise of MindStream to address the challenges they faced in their Capital Planning process.
ATCO Group
Energy conglomerate ATCO operates worldwide in utilities, power generation, and related services.
Avalon
Working with MindStream Analytics, Avalon Healthcare Solutions adopts NetSuite Planning and Budgeting to accelerate budgeting and forecasting processes.
Bayer Health Care
Bayer Healthcare implemented Hyperion Planning and Workforce Planning in 10 weeks to dramatically streamline their Income Statement budget and Workforce Planning process..
BluEarth
MindStream Analytics' partnership with BluEarth Renewables epitomizes the power of technology and collaboration.
Celgene
An Oracle Hyperion Planning Upgrade provides multi-national organization Hyperion Application optimization and stabilization.
Cleaver Brooks
OneStream XF was chosen as the platform that would transform Cleaver-Brooks' Finance processes.
CoorsTek
The collaboration between CoorsTek and MindStream resulted in significant improvements in CoorsTek's financial consolidation and reporting processes.
Elite Body Sculpture
MindStream Analytics' collaboration with Elite Body Sculpture encapsulates the transformative potential of targeted tech solutions in streamlining administrative processes.
Enlyte
Enlyte, a merger of Mitchell, Genex, and Coventry, faced challenges with disparate financial solutions and the need for combined reporting.
Flanders
MindStream Analytics collaborated with Flanders to implement OneStream Consolidation and Reporting solution.
Foley Products
Foley Products was facing a significant challenge with its Excel-based actual management reporting system.
Harte Hanks
The collaboration between MindStream Analytics and Harte Hanks culminated in a highly customized, user-friendly NetSuite implementation.
Interface
Interface used a complex, manual, excel-based FP&A process for monthly review, and the summary data was loaded in OneStream.
Kymera International
Thanks to Mindstream Analytics' assistance, Kymera was able to load all of their data into OneStream and validate it successfully.
MacLean Fogg
MacLean-Fogg partnered with MindStream, a leading implementor specializing in modernizing and optimizing enterprise systems.
MEPPI
MindStream's expertise and experience were sought to conduct a vendor selection initiative focusing on MEPPI's F2023 planning process.
OUAI
MindStream Analytics and OUAI's collaboration showcases the transformative power of strategic technological intervention.
Plaskolite
By migrating to OneStream, Plaskolite has achieved a material reduction in consolidation time and overall Financial Close cycle, eliminated the hours spent compiling and verifying data in Excel, streamlined its Planning, Budgeting and Forecasting model and delivered flexible and timely reporting that enables more strategic analysis of their financial data.
Redwire
Understanding the nuances of Redwire's challenges, MindStream Analytics devised a holistic approach to overcome them. The implementation of NetSuite was just the beginning.
Simon
Simon's existing corporate Hyperion Financial Management (HFM) production application was consolidating at a rate of seven hours, a performance issue causing great headache to corporate Accounting.
Source Code
The successful transition to OneStream revolutionized Source Code's financial reporting.
Subway
Subway collaborated with MindStream Analytics for the NetSuite Analytics Warehouse implementation.
UPenn
MindStream Consulting and AppCare team members are proud be working side by side with UPenn university in accomplishing this implementation and along with continuing our AppCare services after go-live.
USG
USG was an Oracle Hyperion customer realizing that it needed more specialized support for its various Oracle Hyperion applications.
Vantiv
Dividing a hyperion planning application, expanding the hyperion footprint to forecast on the business? Customer categories.
Versant Health
Versant Health engaged MindStream to help resolve the challenges they were experiencing with their consolidation, close, and financial reporting processes.
Virginia Spaceport Authority
The MindStream team implemented the Standard + Workforce NetSuite Planning & Budgeting.
WeWork
MindStream Analytics determined that the best solution was to implement Oracle Essbase Cloud as part of the Oracle Analytics Cloud (OAC) platform-as-a-service
WindStream
Innovative use of essbase to streamline and connect hyperion financial management for enhanced financial analysis.
XY Planning
MindStream Analytics, well-versed in addressing such challenges, presented a comprehensive Netsuite solution for XY Planning.