Scaling & Performance

This section discusses issues and options related to managing the scaling and performance of Omniscope-based solutions.  In a typical fully-automated Enterprise-based  implementation, very large server-based .IOK/.IOM files (with associated images and links) can be maintained and refreshed from data sources on the more powerful servers with abundant RAM. These Omniscope copies of the data warehouse views/tables can be refreshed automatically or on-demand using ODBC/JDBC protocols or a custom Connector.  Once in Omniscope format, the still relatively large, entity level files can be automatically exported from the master files and delivered to selected power users with more RAM for re-configuration into one or more smaller, more specialised reporting .IOK/.IOM  files for delivery/onward distribution for use on standard desktops. Using DataPlayer Edition, yet smaller, more personalised reporting subsets can also be generated from Omniscope as universally-accessible Flash DataPlayers for interactive web page display and embedding in documents.

Scaling Omniscope-based solutions

Omniscope is a locally-installed in-memory application with few inherent data volume limits in terms of record or cell counts (cells = rows x columns) in the application itself. If you have sufficient memory addressing capacity in your operating system (64-bit systems have much more than 32-bit systems), plus sufficient RAM and fast processors, the upper limits extend to multi-million record data sets on 64-bit systems running 64-bit Java with 2+ GB of RAM. However, given that reporting/publishing chains must anticipate less powerful 32-bit machines on user desktops, managing reporting/publishing solutions based on very large data sets can involve various optimisation techniques and tools. The sections below discuss various options and issues related to scaling and performance of Omniscope:

Managing scaling & performance in Omniscope

• Running on 64-bit platforms
• Pivot View Data Truncation
• Memory footprint optimisation tools
• Memory leak diagnosis

Scaling DataPlayers (Flash .SWF files)

Due to inherent limitations in Flash, .SWF DataPlayers will always have much lower capacity for data in terms of record or cell counts (cells = rows x columns) than does a highly-scalable, locally-installed data analysis, management and reporting solution like Omniscope. In general, DataPlayers can contain between 5-10,000 records (rows), but various factors specific to the data can affect performance and impose lower limits on record count. Extracts from larger Omniscope .IOK files can be automatically converted to DataPlayers according to a routine schedule, or on-demand using personalised XML data sets delivered from back-end repositories and/or analytical staging 'data marts'.

Managing .SWF DataPlayer scaling & performance