Frequently Refreshing Data in NetSuite Analytics Warehouse
Over my 17 years of working in Business Intelligence and Analytics tools, one of the great white whales has often been achieving a perfect balance of data storage and data freshness. Data warehouses allow us to store and analyze billions of rows of data, but the processes used to load those warehouse tables takes time.
Long data loads that populate data warehouse tables are extremely common across different tools, and NetSuite Analytics Warehouse is no exception. When the tool was originally released in 2022, users were limited to refreshing the data warehouse once per day.
Since then, NetSuite has enhanced the capabilities available in the tool to allow what they call “Frequent Data Refreshes.” This means that you can select portions of the application to refresh on a more frequent basis. Since it doesn’t force a refresh of the entire warehouse structure, the refreshes are quicker and can be scheduled more often.
This definition can be controlled by functional area or specific warehouse table, so the recommendation is to only apply this setting where there is a tangible business benefit or value to doing so. Having data refreshes run unnecessarily will only cause more issues with maintenance and administration if they run long or are interrupted.
Options for the frequent data loads include running them as often as once per hour, although you will need to ensure your selection completes well within that window to allow it to routinely fire a new load so often.
You can define frequent refreshes by functional area…
...or by specific warehouse table.
NetSuite has also included the option to prioritize certain tables to be refreshed over others. This applies to the nightly refresh and ensures that these tables are loaded and made available first, which provides some flexibility for administrators with more active user groups that want the latest data as early as possible each day.
Finally, NetSuite has also added the ability to schedule full warehouse refreshes. These loads completely wipe the data warehouse of NetSuite ERP data and reloads with the current values from the ERP. These loads take longer than the default incremental loads that only load records that have been changed or created recently, so you should schedule them with care. They can be beneficial to run in certain cases, though, if data inconsistency has been found, or a new subsidiary has been added, as examples.
All of these new features provide a greater flexibility with customizations for how NSAW handles loading your ERP data. As the NSAW user base continues to expands, I only expect more and more flexibility with these and other aspects of the tool to become standard in the coming months and years.
Partner SpotLight
NetSuite SuiteSuccess
NetSuite SuiteSuccess offers a pathway for businesses to succeed by engaging with them continuously throughout their lifecycle. NetSuite delivers a unique set of processes, activities, and systems designed to deliver rapid value. Customers are equipped to make the most of their NetSuite investment quickly and are able to expand towards next-generation solutions to meet changing business needs.
Case Studies
Accumen
Thanks to the intervention of MindStream Analytics, Accumen's Finance department can now model their business with a new, more organized structure that isn't conventionally available in NetSuite.
Acme Brick
Acme Brick turned to MindStream Analytics for help implementing OneStream to replace their outdated TM1 solution.
Alterra
Alterra sought the expertise of MindStream to address the challenges they faced in their Capital Planning process.
ATCO Group
Energy conglomerate ATCO operates worldwide in utilities, power generation, and related services.
Avalon
Working with MindStream Analytics, Avalon Healthcare Solutions adopts NetSuite Planning and Budgeting to accelerate budgeting and forecasting processes.
Bayer Health Care
Bayer Healthcare implemented Hyperion Planning and Workforce Planning in 10 weeks to dramatically streamline their Income Statement budget and Workforce Planning process..
BluEarth
MindStream Analytics' partnership with BluEarth Renewables epitomizes the power of technology and collaboration.
Celgene
An Oracle Hyperion Planning Upgrade provides multi-national organization Hyperion Application optimization and stabilization.
Cleaver Brooks
OneStream XF was chosen as the platform that would transform Cleaver-Brooks' Finance processes.
CoorsTek
The collaboration between CoorsTek and MindStream resulted in significant improvements in CoorsTek's financial consolidation and reporting processes.
Elite Body Sculpture
MindStream Analytics' collaboration with Elite Body Sculpture encapsulates the transformative potential of targeted tech solutions in streamlining administrative processes.
Enlyte
Enlyte, a merger of Mitchell, Genex, and Coventry, faced challenges with disparate financial solutions and the need for combined reporting.
Flanders
MindStream Analytics collaborated with Flanders to implement OneStream Consolidation and Reporting solution.
Foley Products
Foley Products was facing a significant challenge with its Excel-based actual management reporting system.
Harte Hanks
The collaboration between MindStream Analytics and Harte Hanks culminated in a highly customized, user-friendly NetSuite implementation.
Interface
Interface used a complex, manual, excel-based FP&A process for monthly review, and the summary data was loaded in OneStream.
Kymera International
Thanks to Mindstream Analytics' assistance, Kymera was able to load all of their data into OneStream and validate it successfully.
MacLean Fogg
MacLean-Fogg partnered with MindStream, a leading implementor specializing in modernizing and optimizing enterprise systems.
MEPPI
MindStream's expertise and experience were sought to conduct a vendor selection initiative focusing on MEPPI's F2023 planning process.
OUAI
MindStream Analytics and OUAI's collaboration showcases the transformative power of strategic technological intervention.
Plaskolite
By migrating to OneStream, Plaskolite has achieved a material reduction in consolidation time and overall Financial Close cycle, eliminated the hours spent compiling and verifying data in Excel, streamlined its Planning, Budgeting and Forecasting model and delivered flexible and timely reporting that enables more strategic analysis of their financial data.
Redwire
Understanding the nuances of Redwire's challenges, MindStream Analytics devised a holistic approach to overcome them. The implementation of NetSuite was just the beginning.
Simon
Simon's existing corporate Hyperion Financial Management (HFM) production application was consolidating at a rate of seven hours, a performance issue causing great headache to corporate Accounting.
Source Code
The successful transition to OneStream revolutionized Source Code's financial reporting.
Subway
Subway collaborated with MindStream Analytics for the NetSuite Analytics Warehouse implementation.
UPenn
MindStream Consulting and AppCare team members are proud be working side by side with UPenn university in accomplishing this implementation and along with continuing our AppCare services after go-live.
USG
USG was an Oracle Hyperion customer realizing that it needed more specialized support for its various Oracle Hyperion applications.
Vantiv
Dividing a hyperion planning application, expanding the hyperion footprint to forecast on the business? Customer categories.
Versant Health
Versant Health engaged MindStream to help resolve the challenges they were experiencing with their consolidation, close, and financial reporting processes.
Virginia Spaceport Authority
The MindStream team implemented the Standard + Workforce NetSuite Planning & Budgeting.
WeWork
MindStream Analytics determined that the best solution was to implement Oracle Essbase Cloud as part of the Oracle Analytics Cloud (OAC) platform-as-a-service
WindStream
Innovative use of essbase to streamline and connect hyperion financial management for enhanced financial analysis.
XY Planning
MindStream Analytics, well-versed in addressing such challenges, presented a comprehensive Netsuite solution for XY Planning.