However, within a year, pretty much everyone involved in the estimating process had come to ignore the new system and had gone back to the way things were done before the roll-out of the heralded new estimating system. What happened? What went wrong? Did the new system only look good on paper but was unworkable? Those are the kinds of questions that should have been considered and answered through the monitoring process.
One of the elements of an adequate estimating system (and any internal control system for that matter) is the need to provide for internal review of, and accountability for, the acceptability of the estimating system, including the budgetary data supporting indirect cost estimates and comparisons of projected results to actual results, and an analysis of any differences (DFARS 252.215-7002(d)(4)(xii)). So what does this really mean? It means that someone in the organization - management, internal auditors, peers - must perform some level of oversight to determine whether the controls put in place (e.g. the estimating system policies, procedures, and manual) are operating as intended. If they're not, the monitoring activity should also figure out how those control activities should be modified to make them work.
All contractors need to monitor their internal control systems. The sufficiency of the monitoring activity will depend upon a number of factors including the size and complexity of the contractor's operations. One would not expect an SBA 8(a) contractor to have the same controls as a Lockheed or Boeing or Northrup. In small or mid-sized companies, ongoing monitoring activities are likely to be informal, but there still needs to be some form of monitoring.
Here's a self-assessment that contractors can undertake to see if they might pass the the "monitoring" test.
- Do you have policies and procedures that require periodic monitoring (e.g. management reviews) of your estimating system?
- Do the timeframes and/or guidelines appear sufficient given the complexity and size of your operations to determine that controls are operating as intended and that they are modified as appropriate?
- Are you actually performing reviews in accordance with time frames and guidelines established in the policies and procedures?
- Is there documentation to support your internal reviews of, and accountability for, the acceptability of the estimating system, including comparisons of projected results to actual results, and an analysis of any differences.
If you have trouble answering these questions, you might need to assess the adequacy of your monitoring program.