What's new in Prophecy?

• Added a new Custom Time Label Builder. This new feature allows you to create your own custom time labels using calendar data components from the calendar table in your database. There is a special briefing page with a demo video for this feature.

• Added Microsoft Machine Learning forecasting extension to 'Automatic mode' statistical forecasting options. Microsoft's machine learning solution implements the Singular Spectrum Analysis (SSA) model for univariate time-series forecasting. It generates excellent forecasts in many situations and runs significantly faster than any forecasting models implemented in R. You will find it in the list of models on the automatic forecasting's 'Methods' page.
• Added special handling for automatic (statistical) forecasting of 53 week years. 53 week years occur every fourth year and pose an issue for statistical modelling.  This enhancement lets you either feed all of the history, including the 53rd week, into the forecast models (as previously) or to feed only the first 52 weeks of each history year into the models, generate a 52 week forecast and then create the 53rd week forecast using a specified proportion of any other week (typically week 52).  This is simple, easily understood and has a reasonable chance of generating a sensible week 53 forecast

• Auto-size on grid rows now works correctly on multi-line cells. This helps with the display of forecast diary cells and any other grid in the Prophecy application that contains text cells that wrap text.

• History overrides are now visible on standard 'Quick Graph' screens:added to R 'Expert Mode' statistical forecasting. Now you can compare the statistical to the current forecast and decide whether you want to import it by checking the forecast in an instant graph:

  

• New 'Instant Preview' graph tab added to R 'Expert Mode' statistical forecasting. Now you can compare the statistical to the current forecast and decide whether you want to import it by checking the forecast in an instant graph:

  

• When you load a results file in R 'Expert Mode' you canpush the products and customer in the results file into your current Prophecy report, by clicking the corresponding button:

  

• New option added to R 'Export Mode' export to R tool. In addition to launching your data and the R code from your template into RStudio, you can alternativelylaunch the R script into a standalone process, where the R code runs in the background and you can continue using Prophecy. When the background process has finished you can import the results back into Prophecy using the 'Expert Mode' import tool as normal.

  

• New Seasonality Analyzer module added.  Key takeouts:
  • Analyze seasonality at any hierarchical level of product and customer.
  • Statistically determine whether history is seasonal using leading edge statistical methods.
  • Use a 'parent' item's seasonal profile or the 'child' product / customer combination itself to seasonalise a 'child' forecast.
  • View seasonality graphs to quickly understand seasonal strength / key periods.
  • Click the mini-video player below to view a very short video.  Use the video controls to show the video full-screen:
• New 'Find' option added to all Prophecy grid views.  Use the right-click context menu and select the 'Find dialog' option or simply pressCtrl +F:
  The Find tool supports 'find as you type' mode. A list of matches appears in the drop down list when you have typed at least 4 characters into the search field.
• Enhanced the functionality of the  button to support copying codes FROM the current selection TO the Windows clipboard.  The available permutations are now:
  • Click -> Parses the Windows clipboard for product or customer codes and adds the matching items to the right-hand ('Selected') listbox of the selector tool.
  • Shift + Click -> As Click, butreplaces the existing selection with the codes in the Windows clipboard.
  • NEW!
    Ctrl + Click -> Copies the codes of the current item selection in the right-hand ('Selected') listbox of the selector toolinto the Windows Clipboard.

• Added a new Excel export option to the 'Tools' tab on the main Prophecy toolbar.  It creates an Excel with time periods across and products/customers down for any range of years, for any Prophecy fact.  The Excel is 'live-linked' to the Prophecy database and can be refreshed with the latest data using Excel's 'Data' --> 'Refresh All' option.  The Excel format produced can be reimported into Prophecy relatively easily using a Prophecy filter/script because it contains Prophecy codes, as well as labels for the products and customers.
• The existing Excel export option which was previously labelled as 'Export to Excel' has been relabelled 'Report to Excel' because it better describes its function:
  

• The Year Selector tool has been redesigned to allow year on year comparisons to be created between any two years, rather than being restricted to comparison versus previous year as previously.

• Data driven product and customer selections already let you pull out products or customers based on any criteria that can be coded in SQL, against any database that Prophecy can query via ODBC database connectivity.  With this release, data-driven selections can now be created that use values (aka "parameters") specified by the Prophecy Client user at run time.
  
  For example, say you have an existing data-driven selection which returns products with a last forecast versus actual percent deviation greater than 25% on the fact Volume.  With around 5 minutes work by your database manager (or Data Perceptions) this selection can be changed to let you specify any percent deviation (rather than simply using the baked in threshold as previously) and can also let you choose the Prophecy fact to use (e.g. use Revenue rather than Volume).  A newEdit Parameters button has been added to the data driven selector (accessed from the Products or Customers selector tools):
  
  
  
  TheEdit Parameters button becomes enabled when you select a data driven option that has parameters. Click the button to view / edit the parameters. The changes you make to their values will be used next time you use the data-driven selection.
  
  If you save or update a Prophecy report that uses a parameterized data-driven selection, the choices in place when you ran the selection will be saved with the report and will apply next time the report is replayed. The choices replayed with the report become the active choices each time you run the data-driven selection during your Prophecy session, until you use the dialog box to change them or until you replay another report that uses the same data-driven selection but with different parameter choices.
  
  
  We have tried to make this functionality as 'light-touch' as possible, so you only need to bring up the Parameters editor if you want to view the current parameter values or change them.

• Added setting to Automatic forecast adjustment ('consumption rules') to allow database managers to separately permit / deny editing of periods in horizon affected by consumption rules execution and the extra, optional periods appended to the view on the 'Scope' page.

• Automatic forecast adjustment ('consumption rules') have been implemented. Prophecy can automatically adjust the near term horizon using a pre-defined ruleset. For example, you can add undersales from previous period(s) into current period(s) or subtract over-sales from the forecast until the full extent of the oversale has been consumed.

  

  (Please note that this feature requires configuration by your database manager before it can be used. But you can find out more information by pressing F1 for help in your Prophecy Client and locating the topic )

  See our article on 'Automatic Forecast Adjustment' for more information.

• Holding down theShift key and selecting an item in the product and customer selector now clears the current selectionand adds the selected item, saving at least one click for this type of selection operation! (OK, baby steps!)
• Prophecy has a shiny new icon!  And it appears in 'giant' size when pinned to your Windows 10 'Start' menu!
  
  
  
  
• Various assorted fixes and stability enhancements.

• Changing the number of displayed decimal places in percent comparison cells have now been implemented in Prophecy reports:

  -

  -Year on year comparisons

  

  (Fact on fact comparisons were already supported via the ‘Format’ button on the Facts selector.)

  
  
• Reliability fix on Data-driven selections, where selection of one dimension (e.g. products) is dependent on current selection of the other dimension (e.g. customers).  Now works correctly in all cases, including drill-downs and reselects.
  

• "Hide if not matched" option added to Label Rules feature: The 'Label Contents' option on the 'Format' tab of Prophecy has always allowed you to hide rows or columns based on text-matching against row or column labels.  This, for example, has allowed you to hide comparisons of last year against last year's budget (which you might not want in a this-year analysis).  The inverse has now been implemented - Prophecy will hide everything which does not match your search clues.  Combined with the options on the 'Specials' button on the rule dialog, you can format your report to show, for example, current period and the 3 before and after in your report.
  
  
• Pick any label text from report menus added to 'Specials' menu on 'Label Rules' dialog: There are sub-menus added to the 'Specials' menu on the 'Add or edit a rule' dialog that save you having to type into the search clue field:
  
  
• Made the Prophecy Reports Manager resizeable: Drag any edge or corner to resize the window.  See more reports without scrolling the list!  The new size persists between sessions.
  
• Added 'Save as web page' option to Accuracy Analysis: In addition to displaying the Accuracy Analysis in Prophecy this option will prompt you for a filename and display the file in your default web browser.  This makes it easy to save Accuracy Analysis reports for onward distribution, archiving purposes and to take advantage of the printing facilities of your web browser.       
  
  
• Added 'Fit columns to screen' option to report righ-click menus: You can resize the columns of a report to fit the width of your screen by bringing up the right-click context menu in your report (or 'Format' tab, 'Fit to screen' button).  This option retains the current font sizes and simply resizes the columns, so best practice is to get the font size right first (Ctrl+mouse wheel or 'Format' tab options) and then fit to screen.           
  
  
• ... and a number of ongoing fixes and minor improvements!             
  
  

• New option added for History Cleanup Manager: A button menu has been added to the right of the History Cleanup button on the main Prophecy toolbar ('R Open' tab) which lets you choose between loading all report items into the History Cleanup Manager or just the items in the currently selected cursor cell range. (And if just once cell is selected, you get just the item at that position.) Choosing to load just the item(s) at the cursor position speeds up the appearance of the History Cleanup Manager.
• History Cleanup Manager always defaults to the item at the report cursor position: If display the History Cleanup Manager with multiple products and customers, the item at the cursor position in your Prophecy report is automatically selected when the Cleanup Manager window appears.

• Time Series Graphs (aka 'Quick Graphs') now work on product or customer total cells.  (Previously they only graphed distinct product/customer combinations.)  Also, the cursor changes to a '+' (cross) when it hovers over a graph point which can be moved via drag and drop.
• Two newmulti-cell data entry options added:              
  • Apply the percent increase/decrease entered in the focus cell to all the selected cells.
  • Apply the unit increase/decrease entered in the focus cell to all the selected cells
  In addition, there is a new checkbox option which lets you round the numbers pushed into the target cells to the same number of decimal places as shown in the report cells.
  
  
• Several enhancements to theHistory Overrides Manager to make it (even!) more usable:          
  • Checkbox added to limit the tree view to items that have non-zero history.
  • Multi-cell data entry added : select a range of cells and enter a number in the focus cell. It gets pushed into all the cells you selected.
  • Enter overrides as absolute amounts or percents, using the new dropdown menu at the top of the History Overrides Manager.
• New 'Chameleon' application theme added to 'Style' menu at top right of Prophecy application window.  When this theme is selected, the main toolbar ribbon theme is dynamically adjusted to whatever colour the current Prophecy report's fixed cells are rendered in.  The toolbar theme changes automatically when you change the fixed cell background colour or replay a report with a different background colour.  Here's a little demo:
        

• Exclude filters added to Accuracy Explorer.  This lets you, for example, ignore items that sold but which had no forecast (or vice versa) from the summary statistics.  Or ignore low selling items from the summary data.
• The scaling and labelling of the x-axis on all Prophecy graphs has been improved.  (Using Ctrl+Mouse Wheel you can also control the font size on the graph.)
• Both R Automatic Mode statistical forecasting and Expert mode have been speeded up very significantly.
• The tree controls on the R Automatic mode screens are now sorted alphabetically.  In addition, only items with non-zero history are listed in the tree control on the R Automatic mode graph page.

Prophecy's R statistical forecasting already has the option of automatic history cleansing, using R's sophisticated tsclean method.  However, in addition, this release adds the option of creating manual history overrides to the sales history used as input into R to forecast generation.

Please note that the history you see in your Prophecy reports will always be the actual, as it happened history.  The overrides you create with the 'R Open' / 'History Cleanup' option can be viewed and edited within the History Cleanup Manager and are used only for improving (hopefully!) the quality of statistical forecasting.

• Major re-engineering of data-entry rules.  Rules can be used to identify, for example, forecasting a volume when there is no price, accidentally zeroising the price when there is a forecast and many other, preventing a forecast below minimum order quantity, any customer-specific tests.
• All existing rules work as before, but new SQL-based rules, which can use any logic (written in background T-SQL) can now optionally 'correct' the forecast - e.g. round up to pallet quantities for specified skus and customers.
• Much better handling of multiple warnings / errors : you now get a single, floating window listing the errors, and clicking on an error in the window navigates to the corresponding cell in your Prophecy report.
• Rules can now be specified to run on demand, rather than automatically.  There is a new main toolbar option on the 'Tools' tab allowing you to instigate a manual rules scan.
• Please read our November 2019 newsletter for more information on the major improvements to Prophecy rules in this release.  See  https://www.dataperceptions.co.uk/newsletters2019/rules-nov-2019/.
• Added 'holdout' period testing to R Automatic and Expert mode statistical forecasting.  Prophecy will generate a forecast for the most recentn periods using all of the remaining history.  The accuracy of this forecast, which can be compared with actual, lets you judge the likely reliability of the generated forecast much more effectively.  Prophecy therefore now provides two accuracy figures - the MAPE of the holdout period and the MAPE of the model in describing all of the history.
• Added the option to automatically override (cleanse) history in R Automatic Forecasting mode.  Prophecy's R templates will optionally replace outliers (e.g. promotional peaks) and missing periods, which would otherwise skew the forecasts, with interpolated 'normal' values.  This simplifies forecasting standard rate of sale, allowing promotions to be manually added to the statistically generated base forecast.  The graph view shows the 'as-is' plotted against the cleansed history, allowing you to see exactly what changes went into the history used to generate the forecasts.
• Added new TBATS, Neural Network (Autoregressive) and Multi-model Average R Templates.
• Conditional formatting of cells based on their numeric content has been improved.  You can now specify a range of "0.0" to "0.0" (helpful in identifying zero cells!).  You can also specify the same conditional background and text colours : so if you want zero cells to display as 'empty' just choose the same conditional text and background colours.
• Improvements to the logic used to determine font size on all Prophecy graph axes.  Hopefully the axis labels are more readable now!

• New 'Customer Summary' tab added to Accuracy Explorer, showing forecast accuracy for all selected products through time, split by selected customer.
• Added 'filter zero products' option to Automatic mode R forecasting 'Results' tab.
• Graph rendering on Automatic mode R forecasting 'Graphs' tab improved so that 'Current forecast' line does not hide 'R Forecast' line when they are already contain the same numbers.
• Shift+Home key combination restores default axis orientation on all multi-page, reorientable grids in Prophecy (other than the main Prophecy report, which can simply be replayed to restore its default axis orientation).

• Accuracy Explorer now lets you choose which periods you want to view in the grid and summary tabs.
• For users running multiple screens, Prophecy will now display all its windows on the screen running the Prophecy application.
• The Forecastability 'Average Demand Interval' statistic is now calculated from the first non-zero period in the history, rather than the first period in the Prophecy database time range.  This seems a more appropriate way of calculating it.
• Item search in the grid 'Navigator' tree and the Product/Customer selector tools is now case sensitive.  However, if the first character in your search clue is '/' the match will be insensitive, as previously.  In addition, search will only be made once the third character of the search clue field has been entered.
• Important fix on the top band of the 'Accuracy Bands' tab in Accuracy Explorer - it is now correctly accumulated.
• Assorted minor fixes and speedups.

Major new feature - Forecastability Analyser added to 'Views' tab.

The new Forecastability Analyser view auto-categorises product/customer combinations based on the variability of their sales level against the 'density' of sales by period. Based on plotting these two measures, an item can have demand that is 'Intermittent', 'Lumpy', 'Smooth' or 'Erratic'. Each classification has implications on how to forecast the items within it.

For example, items classed as 'Smooth' have regular sales at a similar level through time and are therefore excellent candidates for automatic, statistical forecasting. At the opposite end of the forecastability spectrum, items classed as 'Lumpy' have irregular and highly variable order quantities and it is unreasonable to expect accurate forecasts from statistics or human forecasters, unless the human forecasters can access additional information from the customers.

The new view adds value because it helps you decide how the different items can be most effectively forecasted.  There is much more information on this analysis in the new online help file.  Just press F1 when you have your first Forecastability Analyser view in the screen for a full description.

Performance Improvement on Product and Customer Selector tools

There is a many-fold speed-up on large selections resulting from data-driven selections, pasted in selections and 'children of..' selections from the right-click menu in the 'Selected items' list box.

Accuracy Explorer - editable accuracy bands.

The accuracy bands are now editable and we have added an extra band!

Vertical Axis range logic improved on Prophecy Graphs

The vertical axis on Prophecy graphs no longer starts at zero. Instead, it starts at (slightly below) the minimum graph series value. This makes the peaks and troughs in the graph series much clearer.

Major new feature - Accuracy Explorer added to 'Views' tab.

Accuracy Exporer lets you analyse demand forecasting accuracy over many time periods, for one or more pre-determined forecast horizons.  For example, in one analysis track the forecast made in November for January, December for February, January for March etc..

Separate results grids are shown for:

• the individual forecasts versus actuals
• summary statistics, such as MAPE etc.
• accuracy bands, over / under forecasts

The grids are fully multi-dimensional and the report axes can be freely re-oriented using right-click and drag, just like standard Prophecy reports.

Together, these analyses provide an unparalled view of overall forecast accuracy.

The new Accuracy Explorer option does not replace the existing 'Accuracy Report' option. Running an Accuracy Report on last period, using it to identify / understand the biggest forecast deviations, is a great starting point at the start of the forecasting process each period. Accuracy Explorer, on the other hand, is helpful in identifying longer term forecast accuracy characteristics and bias in the forecasts.

Note: this feature is enabled for Prophecy databases hosted in Microsoft SQL Server and is not available in 'file-mode' or 'Takeaway' databases.

New feature - 'Instant Graphs' anywhere

You can now generate an 'Instant' graph from any range of selected cells, in any Prophecy report, including standard Prophecy reports, Accuracy Explorer views, R Forecast results views and TPM views.  Just select the cells you want to graph and press Ctrl+Space on your keyboard (or right-click and choose 'Instant Graph' from the menu).

Instant Graph windows 'float' - i.e. you can generate an instant graph, move and resize its window whilst continuing to use Prophecy as usual.  Any number of instant graph windows can be created.

Fixes and Minor improvements - cumulative minor fixes and improvements.

• Access to a class-leading set of forecasting models, written by renowned statisticians and academics
• Fully automatic mode forecasting using R forecasting models with zero exposure to the complexities of R. Click here for demo!
• For R experts, a two-stage round trip through R. For stage 1, Prophecy generates a templated R script file and a sales history export file for R to read in. It then loads the script into RStudio, the open source R IDE. The R expert can then use all the facilities of R to analyse, graph or report on their data, as well as to generate a forecast. For stage 2, this forecast can be brought into Prophecy and written to the Prophecy database. Click here for demo!

1. A comprehensive Trade Promotions Planning module, which is fully integrated with Prophecy's standard demand forecasting features.
2. Ad-hoc forecast versions.  The Prophecy database manager can freely create additional forecast versions for numerous purposes, such as:
• To hold Budgets (replacing existing Budget type 'facts').  Holding Budgets in ad-hoc versions means you can have more than one Budget active and gives performance improvements to all Prophecy reports which do not display Budget data.
• For 'what-if' or scenario modelling.  No need to clone the database and compare the 'what if' with the live, side by side within Prophecy.
3. Multiple calendarisation transformations.  The forecast can be displayed in any alternative calendar - such as calendar years (for Prophecy databases which default to fiscal years), weeks from a Prophecy database which uses monthly time buckets or any customer's unique calendar.
4. Automatic 'My Customers' customer attributes.  Use the Prophecy Attribute Selector to instantly select your own customers (i.e. those you have read-write rights to) and create attribute subtotals of 'My Customers'.
5. In-cell formulae.
6. 9 new colourful themes for the Prophecy Client application.
7. Numerous performance improvements and fixes.

Please see the expanded version of this list, including screenshots, by following this link.

• New! Quick graphs can now display the +/- 1, 2 and 3 standard deviations from the mean of the actuals. (According to statistical theory, if your data is 'normally distributed' around the mean 68% of observations will lie between +/- 1 standard deviation of the mean, 95% will lie between +/- 2 standard deviations and 99.7% will lie between +/- 3 standard deviations of the mean.)
• New! The 'Magic Themer' tool on the 'Format' tab of Prophecy has been extensively redesigned. It now contains a slider control that provides unlimited control over the overall lighness shade of the theme that is generated. It also offers either different shades of the fixed cell colour as a theme variation (which may be more pleasing to the eye) or alternatively it uses colour theory to apply colours that are complimentary to the fixed cell colour. Please give it a try - it's the fastest way to turn a plainly formatted report into one which is a little more 'designed'!
• Assorted minor fixes and improvements.

• New! Create scripted subtotals of customer or product attributes on the Attribute Selector with a single click.
• New! Create scripted subtotals of ALL attribute values on the Attribute Selector with a single click.
• New! Product and Customer Subtotals Manager option. Just right-click on 'My Subtotals' or 'Database Subtotals' (if you have rights to manage 'Database Subtotals') or any individual subtotal item in the tree and select 'Manage Subtotals' from the popup menu. You can then multi-delete subtotals or move groups of subtotals between 'My Subtotals' and 'Database Subtotals'.
• New! The 'Search' product or customer dialog now also searches user-defined product and customer subtotals in 'My Subtotals' and 'Database Subtotals'.
• of the new features in action within your browser!
• Updated : You should notice an improvement in the speed with which Prophecy reports replay.
• Fixes : this release contains several fixes for unusual report definitions or reports containing over 32,000 rows.

• Products with non-zero history or forecasts in the selected customer (i.e. just listed items in my customer)
• Customers who buy the selected product
• Products with a forecast change greater than n% versus the previous forecast version
• Products with a forecasted volume but no forecast price or zero standard cost
• Products with margin percent less than n%
• Products with less than 3 weeks of stock

• SQL Nuggets now support 'Filter by Selection'. Keywords added which make providing a 'universal diary view' simple - i.e. to display a view of the diary entries for all the parents and children of the item at the current cursor position in the Prophecy report. (Note: this is only available in databases hosted on SQL Server.) Therefore, whilst viewing a 'brand' in Prophecy you can see all the diary notes for all the skus within that brand. Or viewing skus, you can see diary notes that were entered at the parent brand level.
• Scenario files now saved as zipped xml format files. These files are much smaller than Prophecy's original file format for scenarios.
• Export to Excel now handles Prophecy report pages where all data rows are filtered out (because they contain zero data and the appropriate dimension filters have been set).
• Export to Excel now correctly restores label rule formatting in report after export has completed.
• Added "Don't paste negatives" and "Paste as Using Displayed Decimal Precision" options to Forecast Wizard option. Statistics view also generates forecasts at displayed decimal precision of the fact being statistically forecasted.
• Time subtotals file format changed to XML (when saving as a file in Time Subtotals dialog box).
• 'Publish to File' option implemented for SQL Server hosted Prophecy databases.
• Conditional Format saved files now in XML file format.

1. Where your selection contains items from different levels in the hierarchy you can now specify whether the labels should be indented on hierarchy levels when shown on the report row axis. (Tip: Specifying non-indented labels means that spaces are not added to the start of the labels in the export to Excel feature, making lookups within Excel easier to create.)
2. You can 'paste' a selection of codes from another application into the product or customer selector by copying them to the Windows clipboard. Then, in Prophecy, you can hold down the Ctrl and Shift keys simultaneously and click the select button (the right arrow button). Any codes or other text in the clipboard contents which do not exist in Prophecy are simply ignored. The 'paste' will work with most common internal clipboard delimiter formats such as tab characters, commas, and carriage returns.

Fixed zero-day crash on Facts selector when changing label formats and then reordering facts.

• On the 'Open a Prophecy Database dialog box, click the 'Edit database' button.
• On the 'Common Settings' tab, specify or select the report personal file you want to use for this database.

• Batch Mode Statistical Forecasting now supports forecasting at any level of hierarchy rather than forcing everything to the base levels of customers and products. (Tip: If you use this option to generate forecasts for non-base level combinations you should ensure that the descendant items are correctly proportioned across the base level products and customers before generating statistical forecasts.)
• Added 'Very Long Label' button to Label Builder option on product and customer selector dialog boxes.
• Multi-cell data entry tool positions itself so part of it does not disappear off screen for some cells.
• Diary editor window positions itself so part of it does not disappear off screen for some cells.
• Public domain Libraries are now also copied to a Takeaway Database.
• Magic themer now remembers your preferences for always showing a white background and white text on fixed cells.
• Various bug fixes, performance tweaks and stability adjustments.