Demand forecasting with Prophecy - FAQ
Our FAQ is divided into 4 sections.
- Questions about Prophecy functionality
- Commercial questions
- Technical (IT) questions
- Questions about demand forecasting techniques
If there is a question we haven't answered here, please contact us, either by telephone or our web contact form.
Questions about Prophecy functionality
Yes, Prophecy can create an Excel spreadsheet from any Prophecy report. All formatting in the original Prophecy report is exported to the Excel spreadsheet. Each Prophecy report page is written to a separate Excel workbook tab.
In addition, you can copy and paste data through the Windows clipboard. Prophecy lets you copy non-contiguous cell selections from the Prophecy report into the target Excel or other application, passing only the cells and labels which apply to the original selection.
In addition, you can copy and paste data through the Windows clipboard. Prophecy lets you copy non-contiguous cell selections from the Prophecy report into the target Excel or other application, passing only the cells and labels which apply to the original selection.
Prophecy integrates seamlessly with the open source R Statistical Computing environment and allows expert and non-experts in R to generate statistical forecasts using the leading-edge forecasting routines in the R forecast library. Prophecy offers fully automatic forecasting from R, requiring no R or statistics knowledge, or 'to the metal' access to all the power of R, right from within Prophecy. This feature was added in February 2019.
Prophecy's original built-in statistical forecasting engine supports the following subset of statistical forecasting algorithms:
- linear regression, with and without seasonality
- exponential regression, with and without seasonality
- Holt-Winters
- last year + previous year over year percent
- Batch mode: apply statistical forecasting to a large number of product/customer combinations. (This mode is closest to what the majority of specialist statistical forecasting tools offer.)
- Wizard mode: select the target range of cells in your Prophecy report, choose a method from a simple dialog box and either paste the forecasts in individually or to all target cells with a single click.
- On demand view: generates a new statistical forecast view for the product, customer and measure at the current cell position in your Prophecy report. The view shows the time series as an interactive table, a graph, and also generates a commented description of the statistical control information from the forecast.
- Accuracy analysis: the final, 'human' forecast is compared to the forecast the statistics engine would have generated at t he time the forecast was made.
Finally, bear in mind that increased statistical sophistication does not necessarily equate to increased forecast accuracy. In fact, there is some evidence to suggest that there is often a low correlation between statistical sophistication and increased accuracy. demand forecasting software should empower the forecaster with the right tools for business forecasting, rather than overpower the forecaster with unnecessary statistical and mathematical complexity.
Definitely! Forecasting in a business context is more often a process that involves multiple individual forecasters, management review steps, sign off and publication. As well as generating a volume forecast it is often integrated into the overall planning and budgeting process. Therefore there is a definite functional divide between software for forecasting and software to support your demand forecasting process.
Business forecasting software should have the following properties:
Business forecasting software should have the following properties:
- It must support multi-user demand forecasting because business forecasting is a multi-user activity.
- It must enable a proper forecast/reforecast process. That is, it must support the ability to review / change an existing forecast in the light of the latest period's actuals. It must support forecast versioning through time so that you can track changes made. It must extend through to money as well as quantities, so that it can be integrated into the overall process of business planning and budgeting. 'Money', meaning sales revenue, gross margin etc..
- It must allow you to generate budgets by product, customer and time and to compare the progress of the current sales plan (actuals to date plus forecasts thereafter) versus the budget, as well as previous year(s) at all the varying levels of product, customer and time detail.
- Accuracy analysis. It must be simple for forecasters to analyse their forecast accuracy. If they cannot identify their mistakes they cannot learn from them and therefore improve next time! The start point for a forecasting cycle should be an analysis of the previous period's forecast accuracy. How well did the forecaster use the information they had to make the forecast and what can be learned from the forecast errors?
- The data must be viewable and capable of being forecasted across varying levels of product, customer and time detail. Bottom-up forecasting is generally the starting point, but top-down helps to give an overall perspective and 'sanity check' on the overall numbers, and allows review by exception.
- Business forecasting software must support a review and sign-off process. Therefore the forecaster must be able to submit his/her forecasts for review. A reviewer must then review and either request/make changes or approve. Multiple review levels may be necessary - e.g. sales manager to regional manager to sales director etc.. This feature ensures forecasts have ownership and visibility.
- Software for business demand forecasting must be easy to use, because business forecasters are not necessarily IT or statistics gurus.
- Mainly concerned with generating a forecast mathematically, given a time series history.
- Not multi-user.
- Forecasts the measure you provide it with, but does not (automatically) extend to revenue, margin etc..
- Limited ability to analyse or report on the data it generates.
- Limited or no ability to compare forecasts with budgets.
- Limited or no accuracy analysis on previous forecasts.
- Does not support a forecast review workflow process.
- Limited or no ability to report on or forecast at varying levels of product or customer detail.
- Offers numerous statistical options, which operate in a 'black box' manner.
- Not necessarily easy for non-statisticians to use or understand.
Yes - please see the section in our Changelog file or our knowledgebase article on Automatic Forecast Adjustment.
There are, of course, situations where the computer cannot second-guess. Does an oversale last month mean the start of an increasing trend or merely business brought forward from this month? Does an undersale last month mean the start of a decline, or merely that the downstream stock pipeline was reduced and that actually there will be larger than expected orders this month? Basically, it is impossible for the software to know.
Therefore, other than via the user-controlled automatic forecast adjustment options, Prophecy will never change forecasts except at the forecaster's instigation. That way the forecaster is always responsible and retains full ownership of all their numbers.
Yes! Prophecy's forecast and report screens are the exactly the same - you update your forecasts in Prophecy reports. Prophecy reports let you display just the products, customers, measures, time periods and comparisons you require and replay instantly, always pulling in the latest data from the Prophecy database.
You can display any dimension (i.e. product, customers, periods, years, measures or versions) on any report axis, just like (but better than!) an Excel pivot table.
You can drill up and down through the product and customer hierarchies.
You can conditionally format cells based on their numeric contents, to highlight particular conditions.
You can create your own, user-defined subtotals of products, customers or time period.
The data from any Prophecy report can be graphed and the graph always updates in real time as you change the forecasts or navigate the hierarchies.
All this functionality is implemented in a very visual way that non-technical users will find easy to use productively.
We have found that a lot of Prophecy users get a substantial part of their sales reporting from Prophecy. Sales managers in particular like the fact that all their data is in one place, is instantly available and that they only have one system to master.
Click here for a self-running demo of some of the features of Prophecy reports (opens in a new window).
You can display any dimension (i.e. product, customers, periods, years, measures or versions) on any report axis, just like (but better than!) an Excel pivot table.
You can drill up and down through the product and customer hierarchies.
You can conditionally format cells based on their numeric contents, to highlight particular conditions.
You can create your own, user-defined subtotals of products, customers or time period.
The data from any Prophecy report can be graphed and the graph always updates in real time as you change the forecasts or navigate the hierarchies.
All this functionality is implemented in a very visual way that non-technical users will find easy to use productively.
We have found that a lot of Prophecy users get a substantial part of their sales reporting from Prophecy. Sales managers in particular like the fact that all their data is in one place, is instantly available and that they only have one system to master.
Click here for a self-running demo of some of the features of Prophecy reports (opens in a new window).
If you are based in the UK, please call +44 (0)7460 109038. Outside the UK, please book an on-line demo at run a zero-install self-running demos in your web browser here.
Commercial questions about Prophecy
If you are based in the UK the best way to get to see Prophecy for the first time is in person, via a face to face meeting at your site. Call +44 (0)7460 109038 (UK) to speak to a Prophecy expert now and arrange your demo.
Alternatively, we can run a Prophecy demo over the internet using any web meeting option that works for you.
If Prophecy looks like the potential solution to your demand forecasting software requirement we will often agree to develop a trial system for you, containing a subset of your products and customers. There may or may not be a nominal cost associated with this activity, dependent primarily on how 'elegantly' you are able to supply the data required to build the trial database. Any cost incurred will normally be subtracted from the eventual cost of your Prophecy solution, in the happy event that you decide to proceed.
We are happy to put you in touch with an existing Prophecy customer in the event that you decide Prophecy is your chosen solution, subject to talking to an existing user. We are confident this will provide full reassurance of the quality of our software and our strong commitment to providing the very highest quality of support to our customers.
Please see here for a fuller description of the demo options or here for our full contact details or to contact us via our web form.
Alternatively, we can run a Prophecy demo over the internet using any web meeting option that works for you.
If Prophecy looks like the potential solution to your demand forecasting software requirement we will often agree to develop a trial system for you, containing a subset of your products and customers. There may or may not be a nominal cost associated with this activity, dependent primarily on how 'elegantly' you are able to supply the data required to build the trial database. Any cost incurred will normally be subtracted from the eventual cost of your Prophecy solution, in the happy event that you decide to proceed.
We are happy to put you in touch with an existing Prophecy customer in the event that you decide Prophecy is your chosen solution, subject to talking to an existing user. We are confident this will provide full reassurance of the quality of our software and our strong commitment to providing the very highest quality of support to our customers.
Please see here for a fuller description of the demo options or here for our full contact details or to contact us via our web form.
The Prophecy licence is a perpetual, non-expiring licence, which comes with 60 days of free support and maintenance.
Prophecy is licenced on a per-named-user basis. Therefore any individual, whether as a forecaster or a viewer, who uses the Prophecy client application is deemed a named user. Volume discounts apply from 5 users upwards.
Please contact us for a detailed pricing quotation.
Prophecy is licenced on a per-named-user basis. Therefore any individual, whether as a forecaster or a viewer, who uses the Prophecy client application is deemed a named user. Volume discounts apply from 5 users upwards.
Please contact us for a detailed pricing quotation.
Prophecy is priced on a named-user basis because named-user is simpler to apply to the type of system Prophecy is. We can spend lots of time 'dancing round the houses' on concurrency versus named users but ultimately what's important is agreeing a price that both parties find acceptable. Whether we talk concurrency or named users, you have a price in mind and we have a price in mind. Hopefully they will be similar!
Having said that, Data Perceptions would like to be 100% customer-focussed in all areas of our business. If you absolutely have to have a concurrent licensing model then we can discuss!
Having said that, Data Perceptions would like to be 100% customer-focussed in all areas of our business. If you absolutely have to have a concurrent licensing model then we can discuss!
We have Prophecy users in a number of countries and time zones. We provide support by telephone, Skype and email. Where possible we work with you on your support issue using our secure remote desktop support server. In addition, Prophecy databases typically compress quite well so you can email your database to us in a zip file if required. Time zone differences can work to our advantage too - we fix while you sleep!
Overall, Data Perceptions takes support very seriously indeed. If you pay us the compliment of using our software to help run your business we feel you are entitled to the very best quality of support that we can provide. We know that our existing customers speak very highly indeed of the quality of support they receive.
Overall, Data Perceptions takes support very seriously indeed. If you pay us the compliment of using our software to help run your business we feel you are entitled to the very best quality of support that we can provide. We know that our existing customers speak very highly indeed of the quality of support they receive.
Technical (IT) questions about Prophecy
Prophecy supports two different database formats:
- SQL Server hosted databases store their data in a Microsoft SQL Server database. The advantages of SQL Server databases are scalability and the ability to access Prophecy data from any SQL reporting tool. Performance is outstanding. Sparse data is fully handled, minimising unnecessary storage and network traffic. Most new implementations of Prophecy are SQL Server based these days. You can also host your Prophecy data in the cloud, via a Microsoft Azure SQL Server instance.
- 'Native mode' databases are proprietary, file-based databases. They reside in a shared folder on a server. Prophecy handles all contention issues associated with shared files via protected rights, queues etc.. The advantage of 'native mode' is simplicity. All the database files are held in a single, shared folder, which can be backed up or zipped easily and there are no third party software dependencies. 'Native mode' is more suitable for standalone use - e.g. on laptops.
A 'native-mode' database can easily be migrated to SQL Server at any time by running a Prophecy database maintenance script in our Prophecy Management Studio application. Both formats share the same real time consolidation and allow multiple users to concurrently 'hit' the database.
The IT elements of a Prophecy implementation are described in more detail here.
You can import data into Prophecy using SQL queries against your ERP database, any ODBC source (e.g. Excel) or by reading in flat files (i.e. comma or tab delimited).
You only need to import data at the base level of the product and customer hierarchies
A Prophecy Script automates running one or several import filters sequentially, solving any dependent measures and aggregating from the base levels in the product and customer hierarchy.
The typical Prophecy refresh is from one, pre-made script, which is simply run from Prophecy Management Studio.
Yes! The export is scripted using a 'write once / use many times' Prophecy script. You can export either a fixed time range or a specified time horizon of future periods. You can export either to a text delimited file (e.g. csv or tab) or write to any ODBC/OLEDB target, such as Sql Server, Access, Oracle etc..
Also, you can choose the product and customer levels in the export. So, for instance, to export the forecasts into a production planning system you would export the base level from products (i.e. skus) and the top level from customers (i.e. Total Customers). You can either use period/year fields in the export, or your own, custom-defined time labels which the export script extracts from a lookup table in the Prophecy database.
Also, you can choose the product and customer levels in the export. So, for instance, to export the forecasts into a production planning system you would export the base level from products (i.e. skus) and the top level from customers (i.e. Total Customers). You can either use period/year fields in the export, or your own, custom-defined time labels which the export script extracts from a lookup table in the Prophecy database.
Prophecy provides an exceptionally rich user experience - which we feel is required to productively assist demand forecasters develop and understand their forecasts more effectively.
Think of Prophecy as the forecasting application within Microsoft Office! Therefore the best way to deploy Prophecy to remote users is via a Remote Desktop connection to your server. This enables remote and office based Prophecy users to share the same database concurrently, with no additional consolidation or administration.
Think of Prophecy as the forecasting application within Microsoft Office! Therefore the best way to deploy Prophecy to remote users is via a Remote Desktop connection to your server. This enables remote and office based Prophecy users to share the same database concurrently, with no additional consolidation or administration.
Prophecy is written in 100% C++. We use Microsoft Visual Studio 2010 and the latest version of the Microsoft Foundation Class framework. The learning curve of C++ is steep, but it's still the language of choice for professional, top performance application development.
There are currently over 130,000 lines of hand-crafted C++ code in Prophecy. That's code written by Data Perceptions, and excludes comment lines (we have them too!), blank lines and code from the Microsoft class libraries that we use. It's tough, but we remain committed to C++ because it provides the best performance for our users.
There are currently over 130,000 lines of hand-crafted C++ code in Prophecy. That's code written by Data Perceptions, and excludes comment lines (we have them too!), blank lines and code from the Microsoft class libraries that we use. It's tough, but we remain committed to C++ because it provides the best performance for our users.
It will be easy and quick, providing you can efficiently source sales data and product/customer hierarchy data from your business systems. In time terms, that means weeks or days, not months or years! Getting data into Prophecy is simple, as described above.
However, Prophecy is not just a simple, single user spreadsheet. It is a multi-user business system. Therefore it should be implemented with the processes and disciplines you would apply to any other business system implementation. The hierarchies, data and database management processes must be robust and properly validated before users are allowed to start using the system.
Follow this link for more information: Prophecy implementation process. Follow this link for more on the recommended IT resource required to implement and support Prophecy.
Please contact us for more information on the implementation process. We are happy to send you additional documentation on implementation and the data import/export processes.
However, Prophecy is not just a simple, single user spreadsheet. It is a multi-user business system. Therefore it should be implemented with the processes and disciplines you would apply to any other business system implementation. The hierarchies, data and database management processes must be robust and properly validated before users are allowed to start using the system.
Follow this link for more information: Prophecy implementation process. Follow this link for more on the recommended IT resource required to implement and support Prophecy.
Please contact us for more information on the implementation process. We are happy to send you additional documentation on implementation and the data import/export processes.
Once the import filters and database management scripts have been prepared (during your implementation) a normal database refresh consists of:
Typically, database management is performed by a person from "the business" who has good administrative skills but is not necessarily an IT 'guru'. Refreshing the database each period should take a lot less than an hour each time.
- Reading in the latest product file (to add new products to the database).
- Reading in the latest customer file (to add new customers to the database).
- Reading in the latest sales actuals and (if included) cost etc. files to the database.
- Aggregating the new data over products and customers.
Typically, database management is performed by a person from "the business" who has good administrative skills but is not necessarily an IT 'guru'. Refreshing the database each period should take a lot less than an hour each time.
Client: The Prophecy applications run on any 32 bit and 64 versions of Windows from Windows Vista upwards and on the corresponding versions of Windows Server (running under Remote Desktop or RemoteApp). Any reasonably modern PC will run Prophecy very satisfyingly.
Server: Prophecy hosts its data in a Microsoft SQL Server database (2008R2 or later.) The Prophecy client software connects to SQL Server using the standard installed ODBC drivers. A Prophecy database on SQL Server is normally no more than ~4Gb, though obviously the number of products, customers, time periods, measures and forecast versions, plus the data sparsity, affects database size.
Server: Prophecy hosts its data in a Microsoft SQL Server database (2008R2 or later.) The Prophecy client software connects to SQL Server using the standard installed ODBC drivers. A Prophecy database on SQL Server is normally no more than ~4Gb, though obviously the number of products, customers, time periods, measures and forecast versions, plus the data sparsity, affects database size.
Demand forecasting techniques questions
We don't know your business and have not put your data to the test. However, our experience in implementing forecasting software in a wide range of markets is that the accuracy of statistical forecasts by product by time period tends to be unacceptable due to the high variability of promotions, competitor activity, marketing campaigns, customer logistics etc.. These factors generally do not happen at the same time each year and it requires judgement and interpretation to quantify their effects on sales. It is therefore very hard for a statistical algorithm to forecast them.
If forecasting the monthly peaks and troughs is the required output of your forecasting process then it is likely that statistics is going to be of limited use, other than possibly as a baseline forecast or a sanity check.
We believe in empowering your business managers to become better forecasters. These are the people who are most qualified to know and understand the effects of promotions, competitor activity etc. and when these factors are likely to apply. Prophecy is designed from the ground upwards to help business managers forecast productively and effectively by providing them with the flexible tools and graphical options which support 'judgemental' forecasting. It has statistical forecasting too, of course. Used carefully Prophecy's statistical forecasting algorithms can provide a start point or act as a sanity check on a forecast.
We believe that the forecaster should 'own' the demand forecast. This ownership comes from having carefully and objectively prepared it. Getting the overall (smoothed) medium or long term sales trend right is relatively easy - a year on year percent is likely to be virtually as accurate as a statistical forecast. What is hard, but extremely valuable to the business (and often the primary reason for forecasting), is predicting the individual peaks and troughs in product sales which is frequently typically very 'spiky'. That's where 'judgemental forecasting' and having the right people and tools to support 'judgemental forecasting' is key. Data Perceptions has the software and your business will have the right people!
One additional important point to underline. In the business process of demand forecasting the starting point for this month's forecast is normally last month's forecast for the out periods, updated with last month's actual sale. The process is then one of modifying the existing future forecasts in the light of the latest sale plus new information about promotional plans, competitor activity etc.. You will very rarely need to generate a completely new forecast from scratch, because this will overwrite all the promotional and other peaks or troughs that (should) have been crafted into the forecast. Therefore, again, demand forecasting software for business needs to offer features that help forecasters track and understand the patterns of sales and identify which forecasts they need to review rather than continually overwriting existing forecasts with new statistical ones.
At the very least, if you feel you want to use statistical forecasting, make sure the statisticians prove to you that their algorithms work on your data! We realise that there will be markets where statistical forecasts can be used - we just haven't found any yet! (Note: Prophecy does have statistical forecasting options!See above for more information.)
Please see ourwhitepapers section for more information about the thinking behind Prophecy's approach to business forecasting and the role of statistical algorithms.
If forecasting the monthly peaks and troughs is the required output of your forecasting process then it is likely that statistics is going to be of limited use, other than possibly as a baseline forecast or a sanity check.
We believe in empowering your business managers to become better forecasters. These are the people who are most qualified to know and understand the effects of promotions, competitor activity etc. and when these factors are likely to apply. Prophecy is designed from the ground upwards to help business managers forecast productively and effectively by providing them with the flexible tools and graphical options which support 'judgemental' forecasting. It has statistical forecasting too, of course. Used carefully Prophecy's statistical forecasting algorithms can provide a start point or act as a sanity check on a forecast.
We believe that the forecaster should 'own' the demand forecast. This ownership comes from having carefully and objectively prepared it. Getting the overall (smoothed) medium or long term sales trend right is relatively easy - a year on year percent is likely to be virtually as accurate as a statistical forecast. What is hard, but extremely valuable to the business (and often the primary reason for forecasting), is predicting the individual peaks and troughs in product sales which is frequently typically very 'spiky'. That's where 'judgemental forecasting' and having the right people and tools to support 'judgemental forecasting' is key. Data Perceptions has the software and your business will have the right people!
One additional important point to underline. In the business process of demand forecasting the starting point for this month's forecast is normally last month's forecast for the out periods, updated with last month's actual sale. The process is then one of modifying the existing future forecasts in the light of the latest sale plus new information about promotional plans, competitor activity etc.. You will very rarely need to generate a completely new forecast from scratch, because this will overwrite all the promotional and other peaks or troughs that (should) have been crafted into the forecast. Therefore, again, demand forecasting software for business needs to offer features that help forecasters track and understand the patterns of sales and identify which forecasts they need to review rather than continually overwriting existing forecasts with new statistical ones.
At the very least, if you feel you want to use statistical forecasting, make sure the statisticians prove to you that their algorithms work on your data! We realise that there will be markets where statistical forecasts can be used - we just haven't found any yet! (Note: Prophecy does have statistical forecasting options!See above for more information.)
Please see ourwhitepapers section for more information about the thinking behind Prophecy's approach to business forecasting and the role of statistical algorithms.