28 August 2013

SIP math and Monte Carlo Simulation

Thinking about the contrast between SIP math and MCS.

SIPmath has its roots in Monte Carlo Simulation, but the implementation, application architecture, and data architecture are different. MCS munges generation, data, and use together. SIPmath extracts the data part and puts it in a SIP where, being pure data, it can be cataloged, and passed around as easily as you attach a picture to an email. How it's generated and how it's used are separate concerns.

Under ideal conditions, where there's lots of data, each value in a SIP is valid because it has actually happened, and the frequencies in the SIP are the same as the reality. That is, a well-formed SIP is correct by construction. Since the rest is simple arithmetic, avoiding implementation errors and independent validation are both fairly simple.

MCS stratified sampling and SIPmath are the same except for where in the workflow the samples are taken.

On the other hand, MCS generating random values from a curve that approximates the data, is approximate by construction.  We can only hope to get close to the fidelity that comes effortlessly in a SIP composed from history.